| 3 |
Pushing the right buttons: adversarial evaluation of quality estimation
|
|
|
|
In: Proceedings of the Sixth Conference on Machine Translation ; 625 ; 638 (2022)
|
|
BASE
|
|
Show details
|
|
|
| 4 |
Assessing Reference-Free Peer Evaluation for Machine Translation ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 7 |
BLEU might be Guilty but References are not Innocent ...
|
|
|
|
Abstract:
The quality of automatic metrics for machine translation has been increasingly called into question, especially for high-quality systems. This paper demonstrates that, while choice of metric is important, the nature of the references is also critical. We study different methods to collect references and compare their value in automated evaluation by reporting correlation with human evaluation for a variety of systems and metrics. Motivated by the finding that typical references exhibit poor diversity, concentrating around translationese language, we develop a paraphrasing task for linguists to perform on existing reference translations, which counteracts this bias. Our method yields higher correlation with human judgment not only for the submissions of WMT 2019 English to German, but also for Back-translation and APE augmented MT output, which have been shown to have low correlation with automatic metrics using standard references. We demonstrate that our methodology improves correlation with all modern ... : Accepted at EMNLP 2020 ...
|
|
Keyword:
Artificial Intelligence cs.AI; Computation and Language cs.CL; FOS Computer and information sciences; Machine Learning cs.LG
|
|
URL: https://dx.doi.org/10.48550/arxiv.2004.06063
https://arxiv.org/abs/2004.06063
|
|
BASE
|
|
Hide details
|
|
|
| 12 |
Making or breaking informal volunteering : welfare statism and social capital in a sub-national comparative perspective
|
|
|
|
In: European Societies ; 12 (2010), 5. - S. 627-652. - ISSN 1461-6696 (2010)
|
|
BASE
|
|
Show details
|
|
|
|
|
