1 |
Towards a Perceptual Model for Estimating the Quality of Visual Speech ...
|
|
|
|
BASE
|
|
Show details
|
|
2 |
An error correction scheme for improved air-tissue boundary in real-time MRI video for speech production ...
|
|
|
|
BASE
|
|
Show details
|
|
3 |
Expression-preserving face frontalization improves visually assisted speech processing ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modeling ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Facetron: Multi-speaker Face-to-Speech Model based on Cross-modal Latent Representations ...
|
|
|
|
BASE
|
|
Show details
|
|
8 |
Silent Speech and Emotion Recognition from Vocal Tract Shape Dynamics in Real-Time MRI ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
Improving Ultrasound Tongue Image Reconstruction from Lip Images Using Self-supervised Learning and Attention Mechanism ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
Cascaded Multilingual Audio-Visual Learning from Videos ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
Speaker embeddings by modeling channel-wise correlations ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
Can phones, syllables, and words emerge as side-products of cross-situational audiovisual learning? -- A computational investigation ...
|
|
|
|
BASE
|
|
Show details
|
|
14 |
Large-scale multilingual audio visual dubbing ...
|
|
Yang, Yi; Shillingford, Brendan; Assael, Yannis; Wang, Miaosen; Liu, Wendi; Chen, Yutian; Zhang, Yu; Sezener, Eren; Cobo, Luis C.; Denil, Misha; Aytar, Yusuf; de Freitas, Nando. - : arXiv, 2020
|
|
Abstract:
We describe a system for large-scale audiovisual translation and dubbing, which translates videos from one language to another. The source language's speech content is transcribed to text, translated, and automatically synthesized into target language speech using the original speaker's voice. The visual content is translated by synthesizing lip movements for the speaker to match the translated audio, creating a seamless audiovisual experience in the target language. The audio and visual translation subsystems each contain a large-scale generic synthesis model trained on thousands of hours of data in the corresponding domain. These generic models are fine-tuned to a specific speaker before translation, either using an auxiliary corpus of data from the target speaker, or using the video to be translated itself as the input to the fine-tuning process. This report gives an architectural overview of the full system, as well as an in-depth discussion of the video dubbing component. The role of the audio and text ... : 26 pages, 8 figures ...
|
|
Keyword:
Audio and Speech Processing eess.AS; Computer Vision and Pattern Recognition cs.CV; FOS Computer and information sciences; FOS Electrical engineering, electronic engineering, information engineering; Sound cs.SD
|
|
URL: https://dx.doi.org/10.48550/arxiv.2011.03530 https://arxiv.org/abs/2011.03530
|
|
BASE
|
|
Hide details
|
|
15 |
Cross-modal Speaker Verification and Recognition: A Multilingual Perspective ...
|
|
|
|
BASE
|
|
Show details
|
|
16 |
Designing, Playing, and Performing with a Vision-based Mouth Interface ...
|
|
|
|
BASE
|
|
Show details
|
|
17 |
SLNSpeech: solving extended speech separation problem by the help of sign language ...
|
|
|
|
BASE
|
|
Show details
|
|
18 |
Unsupervised Audiovisual Synthesis via Exemplar Autoencoders ...
|
|
|
|
BASE
|
|
Show details
|
|
19 |
UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation ...
|
|
|
|
BASE
|
|
Show details
|
|
20 |
Disentangled Speech Embeddings using Cross-modal Self-supervision ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|