In plain words: A speech generator that builds the waveform one small frame at a time, so it can stream, turning compressed speech parameters sent at 1.6 kbit/s into sound. It beat the usual step-by-step generator and matched a standard codec at 5.9 kbit/s on clean speech.
Abstract · A Streamwise GAN Vocoder for Wideband Speech Coding at Very Low Bit Rate
Recently, GAN vocoders have seen rapid progress in speech synthesis, starting to outperform autoregressive models in perceptual quality with much higher generation speed. However, autoregressive vocoders are still the common choice for neural generation of speech signals coded at very low bit rates. In this paper, we present a GAN vocoder which is able to generate wideband speech waveforms from parameters coded at 1.6 kbit/s. The proposed model is a modified version of the StyleMelGAN vocoder that can run in frame-by-frame manner, making it suitable for streaming applications. The experimental results show that the proposed model significantly outperforms prior autoregressive vocoders like LPCNet for very low bit rate speech coding, with computational complexity of about 5 GMACs, providing a new state of the art in this domain. Moreover, this streamwise adversarial vocoder delivers quality competitive to advanced speech codecs such as EVS at 5.9 kbit/s on clean speech, which motivates further usage of feed-forward fully-convolutional models for low bit rate speech coding.
Ahmed Mustafa, Jan Büthe, Srikanth Korse, Kishan Gupta, Guillaume Fuchs, Nicola Pia
arXiv:2108.04051 · eess.AS, cs.LG, cs.SD, eess.SP · submitted Aug 9, 2021
abstract · pdf · html · Accepted to the 2021 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA 2021)