| 1 |
Knowledge Transfer from Large-scale Pretrained Language Models to End-to-end Speech Recognizers ...
|
|
|
|
Abstract:
End-to-end speech recognition is a promising technology for enabling compact automatic speech recognition (ASR) systems since it can unify the acoustic and language model into a single neural network. However, as a drawback, training of end-to-end speech recognizers always requires transcribed utterances. Since end-to-end models are also known to be severely data hungry, this constraint is crucial especially because obtaining transcribed utterances is costly and can possibly be impractical or impossible. This paper proposes a method for alleviating this issue by transferring knowledge from a language model neural network that can be pretrained with text-only data. Specifically, this paper attempts to transfer semantic knowledge acquired in embedding vectors of large-scale language models. Since embedding vectors can be assumed as implicit representations of linguistic information such as part-of-speech, intent, and so on, those are also expected to be useful modeling cues for ASR decoders. This paper extends ... : To be presented in ICASSP 2022 ...
|
|
Keyword:
Audio and Speech Processing eess.AS; Computation and Language cs.CL; FOS Computer and information sciences; FOS Electrical engineering, electronic engineering, information engineering; Sound cs.SD
|
|
URL: https://dx.doi.org/10.48550/arxiv.2202.07894
https://arxiv.org/abs/2202.07894
|
|
BASE
|
|
Hide details
|
|
|
| 2 |
Multi-Dialect Speech Recognition With A Single Sequence-To-Sequence Model ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 4 |
Improved Name-Recognition with Meta-data Dependent Name Networks
|
|
|
|
BASE
|
|
Show details
|
|
|
| 5 |
Improved Name-Recognition with Meta-data Dependent Name Networks ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|
|
