1 |
When Does Translation Require Context? A Data-driven, Multilingual Exploration ...
|
|
|
|
Abstract:
Although proper handling of discourse phenomena significantly contributes to the quality of machine translation (MT), common translation quality metrics do not adequately capture them. Recent works in context-aware MT attempt to target a small set of these phenomena during evaluation. In this paper, we propose a new metric, P-CXMI, which allows us to identify translations that require context systematically and confirm the difficulty of previously studied phenomena as well as uncover new ones that have not been addressed in previous work. We then develop the Multilingual Discourse-Aware (MuDA) benchmark, a series of taggers for these phenomena in 14 different language pairs, which we use to evaluate context-aware MT. We find that state-of-the-art context-aware MT models find marginal improvements over context-agnostic models on our benchmark, which suggests current models do not handle these ambiguities effectively. We release code and data to invite the MT research community to increase efforts on ...
|
|
Keyword:
Computation and Language cs.CL; FOS Computer and information sciences; Machine Learning cs.LG
|
|
URL: https://dx.doi.org/10.48550/arxiv.2109.07446 https://arxiv.org/abs/2109.07446
|
|
BASE
|
|
Hide details
|
|
2 |
Do Context-Aware Translation Models Pay the Right Attention? ...
|
|
|
|
BASE
|
|
Show details
|
|
3 |
MLQE-PE: A Multilingual Quality Estimation and Post-Editing Dataset ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
Understanding the Mechanics of SPIGOT: Surrogate Gradients for Latent Structure Learning ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
MLQE-PE: A multilingual quality estimation and post-editing dataset
|
|
|
|
BASE
|
|
Show details
|
|
6 |
OpenKiwi: An Open Source Framework for Quality Estimation ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Universal Dependencies 2.2
|
|
|
|
In: https://hal.archives-ouvertes.fr/hal-01930733 ; 2018 (2018)
|
|
BASE
|
|
Show details
|
|
8 |
Contextual Neural Model for Translating Bilingual Multi-Speaker Conversations ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
Universal Dependencies 2.1
|
|
|
|
In: https://hal.inria.fr/hal-01682188 ; 2017 (2017)
|
|
BASE
|
|
Show details
|
|
|
|