| 1 |
On Leveraging the Visual Modality for Neural Machine Translation ...
|
|
|
|
Abstract:
Leveraging the visual modality effectively for Neural Machine Translation (NMT) remains an open problem in computational linguistics. Recently, Caglayan et al. posit that the observed gains are limited mainly due to the very simple, short, repetitive sentences of the Multi30k dataset (the only multimodal MT dataset available at the time), which renders the source text sufficient for context. In this work, we further investigate this hypothesis on a new large scale multimodal Machine Translation (MMT) dataset, How2, which has 1.57 times longer mean sentence length than Multi30k and no repetition. We propose and evaluate three novel fusion techniques, each of which is designed to ensure the utilization of visual context at different stages of the Sequence-to-Sequence transduction pipeline, even under full linguistic context. However, we still obtain only marginal gains under full linguistic context and posit that visual embeddings extracted from deep vision models (ResNet for Multi30k, ResNext for How2) do not ... : Accepted to INLG 2019 ...
|
|
Keyword:
Computation and Language cs.CL; FOS Computer and information sciences; Machine Learning cs.LG
|
|
URL: https://arxiv.org/abs/1910.02754
https://dx.doi.org/10.48550/arxiv.1910.02754
|
|
BASE
|
|
Hide details
|
|
|
| 2 |
Economy of Effort or Maximum Rate of Information? Exploring Basic Principles of Articulatory Dynamics
|
|
|
|
BASE
|
|
Show details
|
|
|
|
|
