about
FlowTSE: Target Speaker Extraction with Flow Matching (arxiv.org)
25 points by agold97 on May 28, 2025 | hide | past | pdf | 2 comments on HN

In plain words: It takes a short sample of the wanted voice plus the mixed recording and generates that person's clean speech directly, instead of the usual multi-part pipelines built from pretrained pieces. On standard tests it matched or beat strong baselines.

Abstract

Target speaker extraction (TSE) aims to isolate a specific speaker's speech from a mixture using speaker enrollment as a reference. While most existing approaches are discriminative, recent generative methods for TSE achieve strong results. However, generative methods for TSE remain underexplored, with most existing approaches relying on complex pipelines and pretrained components, leading to computational overhead. In this work, we present FlowTSE, a simple yet effective TSE approach based on conditional flow matching. Our model receives an enrollment audio sample and a mixed speech signal, both represented as mel-spectrograms, with the objective of extracting the target speaker's clean speech. Furthermore, for tasks where phase reconstruction is crucial, we propose a novel vocoder conditioned on the complex STFT of the mixed signal, enabling improved phase estimation. Experimental results on standard TSE benchmarks show that FlowTSE matches or outperforms strong baselines.

Aviv Navon, Aviv Shamsian, Yael Segal-Feldman, Neta Glazer, Gil Hetz, Joseph Keshet
arXiv:2505.14465 · eess.AS, cs.LG, cs.SD · submitted May 20, 2025
abstract · pdf · html · InterSpeech 2025

add comment on HN

Examples can be found here: https://aiola-lab.github.io/flow-tse/
This is brilliant.

It can allow calls be be made clear when people talking in noisy places.

Assuming this can be made to work in real time.