about
Multilingual Speech Recognition with Corpus Relatedness Sampling (arxiv.org)
1 point by sel1 on Aug 7, 2019 | hide | past | pdf | discuss on HN

In plain words: Instead of drawing training audio equally from many small speech collections, this system measures how similar each collection is to the target one and slowly shifts sampling toward the closest matches. It beat the equal-sampling approach on all 16 low-resource speech tasks.

Abstract

Multilingual acoustic models have been successfully applied to low-resource speech recognition. Most existing works have combined many small corpora together and pretrained a multilingual model by sampling from each corpus uniformly. The model is eventually fine-tuned on each target corpus. This approach, however, fails to exploit the relatedness and similarity among corpora in the training set. For example, the target corpus might benefit more from a corpus in the same domain or a corpus from a close language. In this work, we propose a simple but useful sampling strategy to take advantage of this relatedness. We first compute the corpus-level embeddings and estimate the similarity between each corpus. Next, we start training the multilingual model with uniform-sampling from each corpus at first, then we gradually increase the probability to sample from related corpora based on its similarity with the target corpus. Finally, the model would be fine-tuned automatically on the target corpus. Our sampling strategy outperforms the baseline multilingual model on 16 low-resource tasks. Additionally, we demonstrate that our corpus embeddings capture the language and domain information of each corpus.

Xinjian Li, Siddharth Dalmia, Alan W. Black, Florian Metze
arXiv:1908.01060 · cs.CL, cs.SD, eess.AS · submitted Aug 2, 2019
abstract · pdf · html · Interspeech 2019

add comment on HN