Page: 1 2 3 4 5 6 7 8... 83
| 61 |
SummEval: Re-evaluating Summarization Evaluation
|
|
|
|
In: Transactions of the Association for Computational Linguistics, Vol 9, Pp 391-409 (2021) (2021)
|
|
Abstract:
AbstractThe scarcity of comprehensive up-to-date studies on evaluation metrics for text summarization and the lack of consensus regarding evaluation protocols continue to inhibit progress. We address the existing shortcomings of summarization evaluation methods along five dimensions: 1) we re-evaluate 14 automatic evaluation metrics in a comprehensive and consistent fashion using neural summarization model outputs along with expert and crowd-sourced human annotations; 2) we consistently benchmark 23 recent summarization models using the aforementioned automatic evaluation metrics; 3) we assemble the largest collection of summaries generated by models trained on the CNN/DailyMail news dataset and share it in a unified format; 4) we implement and share a toolkit that provides an extensible and unified API for evaluating summarization models across a broad range of automatic metrics; and 5) we assemble and share the largest and most diverse, in terms of model types, collection of human judgments of model-generated summaries on the CNN/Daily Mail dataset annotated by both expert judges and crowd-source workers. We hope that this work will help promote a more complete evaluation protocol for text summarization as well as advance research in developing evaluation metrics that better correlate with human judgments.
|
|
Keyword:
Computational linguistics. Natural language processing; P98-98.5
|
|
URL: https://doaj.org/article/3a815d182a8440e1ac02f25f1d9da002
https://doi.org/10.1162/tacl_a_00373
|
|
BASE
|
|
Hide details
|
|
|
| 62 |
Neural OCR Post-Hoc Correction of Historical Corpora
|
|
|
|
In: Transactions of the Association for Computational Linguistics, Vol 9, Pp 479-493 (2021) (2021)
|
|
BASE
|
|
Show details
|
|
|
| 63 |
Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation
|
|
|
|
In: Transactions of the Association for Computational Linguistics, Vol 9, Pp 1460-1474 (2021) (2021)
|
|
BASE
|
|
Show details
|
|
|
| 64 |
How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering
|
|
|
|
In: Transactions of the Association for Computational Linguistics, Vol 9, Pp 962-977 (2021) (2021)
|
|
BASE
|
|
Show details
|
|
|
| 65 |
Modeling Content and Context with Deep Relational Learning
|
|
|
|
In: Transactions of the Association for Computational Linguistics, Vol 9, Pp 100-119 (2021) (2021)
|
|
BASE
|
|
Show details
|
|
|
| 66 |
A Statistical Analysis of Summarization Evaluation Metrics Using Resampling Methods
|
|
|
|
In: Transactions of the Association for Computational Linguistics, Vol 9, Pp 1132-1146 (2021) (2021)
|
|
BASE
|
|
Show details
|
|
|
| 67 |
Optimizing over subsequences generates context-sensitive languages
|
|
|
|
In: Transactions of the Association for Computational Linguistics, Vol 9, Pp 528-537 (2021) (2021)
|
|
BASE
|
|
Show details
|
|
|
| 68 |
Morphology Matters: A Multilingual Language Modeling Analysis
|
|
|
|
In: Transactions of the Association for Computational Linguistics, Vol 9, Pp 261-276 (2021) (2021)
|
|
BASE
|
|
Show details
|
|
|
| 69 |
Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics
|
|
|
|
In: Transactions of the Association for Computational Linguistics, Vol 9, Pp 1249-1267 (2021) (2021)
|
|
BASE
|
|
Show details
|
|
|
| 70 |
Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognition
|
|
|
|
In: Transactions of the Association for Computational Linguistics, Vol 9, Pp 586-604 (2021) (2021)
|
|
BASE
|
|
Show details
|
|
|
| 71 |
Continual Learning for Grounded Instruction Generation by Observing Human Following Behavior
|
|
|
|
In: Transactions of the Association for Computational Linguistics, Vol 9, Pp 1303-1319 (2021) (2021)
|
|
BASE
|
|
Show details
|
|
|
| 72 |
Deciphering Undersegmented Ancient Scripts Using Phonetic Prior
|
|
|
|
In: Transactions of the Association for Computational Linguistics, Vol 9, Pp 69-81 (2021) (2021)
|
|
BASE
|
|
Show details
|
|
|
| 73 |
Sparse, Dense, and Attentional Representations for Text Retrieval
|
|
|
|
In: Transactions of the Association for Computational Linguistics, Vol 9, Pp 329-345 (2021) (2021)
|
|
BASE
|
|
Show details
|
|
|
| 74 |
Decoupling the Role of Data, Attention, and Losses in Multimodal Transformers
|
|
|
|
In: Transactions of the Association for Computational Linguistics, Vol 9, Pp 570-585 (2021) (2021)
|
|
BASE
|
|
Show details
|
|
|
| 75 |
Partially Supervised Named Entity Recognition via the Expected Entity Ratio Loss
|
|
|
|
In: Transactions of the Association for Computational Linguistics, Vol 9, Pp 1320-1335 (2021) (2021)
|
|
BASE
|
|
Show details
|
|
|
| 76 |
Formal Basis of a Language Universal
|
|
|
|
In: Computational Linguistics, Vol 47, Iss 1, Pp 9-42 (2021) (2021)
|
|
BASE
|
|
Show details
|
|
|
| 77 |
Multimodal Pretraining Unmasked: A Meta-Analysis and a Unified Framework of Vision-and-Language BERTs
|
|
|
|
In: Transactions of the Association for Computational Linguistics, Vol 9, Pp 978-994 (2021) (2021)
|
|
BASE
|
|
Show details
|
|
|
| 78 |
Revisiting Negation in Neural Machine Translation
|
|
|
|
In: Transactions of the Association for Computational Linguistics, Vol 9, Pp 740-755 (2021) (2021)
|
|
BASE
|
|
Show details
|
|
|
| 79 |
Quantifying Cognitive Factors in Lexical Decline
|
|
|
|
In: Transactions of the Association for Computational Linguistics, Vol 9, Pp 1529-1545 (2021) (2021)
|
|
BASE
|
|
Show details
|
|
|
| 80 |
Joint Universal Syntactic and Semantic Parsing
|
|
|
|
In: Transactions of the Association for Computational Linguistics, Vol 9, Pp 756-773 (2021) (2021)
|
|
BASE
|
|
Show details
|
|
|
Page: 1 2 3 4 5 6 7 8... 83 |
|
