1 |
Multistream neural architectures for cued-speech recognition using a pre-trained visual feature extractor and constrained CTC decoding
|
|
|
|
In: ICASSP 2022 - IEEE International Conference on Acoustics, Speech and Signal Processing ; https://hal.archives-ouvertes.fr/hal-03578503 ; ICASSP 2022 - IEEE International Conference on Acoustics, Speech and Signal Processing, May 2022, Singapour, Singapore (2022)
|
|
BASE
|
|
Show details
|
|
2 |
Multistream neural architectures for cued-speech recognition using a pre-trained visual feature extractor and constrained CTC decoding
|
|
|
|
In: ICASSP 2022 - IEEE International Conference on Acoustics, Speech and Signal Processing ; https://hal.archives-ouvertes.fr/hal-03578503 ; ICASSP 2022 - IEEE International Conference on Acoustics, Speech and Signal Processing, May 2022, Singapour, Singapore (2022)
|
|
Abstract:
International audience ; This paper proposes a simple and effective approach for automatic recognition of Cued Speech (CS), a visual communication tool that helps people with hearing impairment to understand spoken language with the help of hand gestures that can uniquely identify the uttered phonemes in complement to lipreading. The proposed approach is based on a pre-trained hand and lips tracker used for visual feature extraction and a phonetic decoder based on a multistream recurrent neural network trained with connectionist temporal classification loss and combined with a pronunciation lexicon. The proposed system is evaluated on an updated version of the French CS dataset CSF18 for which the phonetic transcription has been manually checked and corrected. With a decoding accuracy at the phonetic level of 70.88%, the proposed system outperforms our previous CNN-HMM decoder and competes with more complex baselines.
|
|
Keyword:
[SHS.INFO]Humanities and Social Sciences/Library and information sciences; cued speech; hearing impairment; multi-modality; neural network; Visual speech
|
|
URL: https://hal.archives-ouvertes.fr/hal-03578503/file/sankar-ICASSP2022-VF.pdf https://hal.archives-ouvertes.fr/hal-03578503/document https://hal.archives-ouvertes.fr/hal-03578503
|
|
BASE
|
|
Hide details
|
|
3 |
Multistream neural architectures for cued-speech recognition using a pre-trained visual feature extractor and constrained CTC decoding ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
Re-synchronization using the Hand Preceding Model for Multi-modal Fusion in Automatic Continuous Cued Speech Recognition
|
|
|
|
In: ISSN: 1520-9210 ; IEEE Transactions on Multimedia ; https://hal.archives-ouvertes.fr/hal-02433830 ; IEEE Transactions on Multimedia, Institute of Electrical and Electronics Engineers, 2021, 23, pp.292-305. ⟨10.1109/TMM.2020.2976493⟩ (2021)
|
|
BASE
|
|
Show details
|
|
5 |
Auditory and Audiovisual Close-shadowing in Post-Lingually Deaf Cochlear-Implanted Patients and Normal-Hearing Elderly Adults
|
|
|
|
In: ISSN: 0196-0202 ; Ear and Hearing ; https://hal.archives-ouvertes.fr/hal-01546756 ; Ear and Hearing, Lippincott, Williams & Wilkins, 2018, 39 (1), pp.139-149. ⟨10.1097/AUD.0000000000000474⟩ (2018)
|
|
BASE
|
|
Show details
|
|
8 |
The shadow of a doubt? Evidence for perceptuo-motor linkage during auditory and audiovisual close-shadowing
|
|
|
|
BASE
|
|
Show details
|
|
9 |
Physical modeling of bilabial plosives production
|
|
|
|
In: ICA 2013 - Acoustics 2013 - 21st International Congress on Acoustics - 165th Meeting of the Acoustical Society of America ; https://hal.archives-ouvertes.fr/hal-00868345 ; ICA 2013 - Acoustics 2013 - 21st International Congress on Acoustics - 165th Meeting of the Acoustical Society of America, Jun 2013, Montréal, Canada. 035047 (9p.), ⟨10.1121/1.4799466⟩ (2013)
|
|
BASE
|
|
Show details
|
|
12 |
Image and Video for hearing impaired people
|
|
|
|
In: ISSN: 1687-5176 ; EISSN: 1687-5281 ; EURASIP Journal on Image and Video Processing ; https://hal.archives-ouvertes.fr/hal-00275029 ; EURASIP Journal on Image and Video Processing, Springer, 2007, 2007, pp.ID 45641. ⟨10.1155/2007/45641⟩ (2007)
|
|
BASE
|
|
Show details
|
|
13 |
Analysis and synthesis of the 3D movements of the head, face and hand of a speaker using cued speech
|
|
|
|
In: ISSN: 0001-4966 ; EISSN: 1520-8524 ; Journal of the Acoustical Society of America ; https://hal.archives-ouvertes.fr/hal-00143622 ; Journal of the Acoustical Society of America, Acoustical Society of America, 2005, 118 (2), pp.1144-1153 (2005)
|
|
BASE
|
|
Show details
|
|
17 |
Characterizing and classifying Cued Speech vowels from labial parameters
|
|
|
|
In: 8th International Conference on Spoken Language Processing (ICSLP'04 or InterSpeech'04) ; https://hal.archives-ouvertes.fr/hal-00328134 ; 8th International Conference on Spoken Language Processing (ICSLP'04 or InterSpeech'04), 2004, Jeju, South Korea (2004)
|
|
BASE
|
|
Show details
|
|
|
|