1 |
The statistical advantage of automatic NLG metrics at the system level ...
|
|
|
|
BASE
|
|
Show details
|
|
2 |
Evaluation Examples are not Equally Informative: How should that change NLP Leaderboards? ...
|
|
|
|
BASE
|
|
Show details
|
|
3 |
Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
The statistical advantage of automatic NLG metrics at the system level ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
Swords: A Benchmark for Lexical Substitution with Improved Data Coverage and Quality ...
|
|
|
|
Abstract:
We release a new benchmark for lexical substitution, the task of finding appropriate substitutes for a target word in a context. To assist humans with writing, lexical substitution systems can suggest words that humans cannot easily think of. However, existing benchmarks depend on human recall as the only source of data, and therefore lack coverage of the substitutes that would be most helpful to humans. Furthermore, annotators often provide substitutes of low quality, which are not actually appropriate in the given context. We collect higher-coverage and higher-quality data by framing lexical substitution as a classification problem, guided by the intuition that it is easier for humans to judge the appropriateness of candidate substitutes than conjure them from memory. To this end, we use a context-free thesaurus to produce candidates and rely on human judgement to determine contextual appropriateness. Compared to the previous largest benchmark, our Swords benchmark has 4.1x more substitutes per target word ... : Published as a conference paper at NAACL 2021 ...
|
|
Keyword:
Computation and Language cs.CL; FOS Computer and information sciences
|
|
URL: https://dx.doi.org/10.48550/arxiv.2106.04102 https://arxiv.org/abs/2106.04102
|
|
BASE
|
|
Hide details
|
|
6 |
Swords: A Benchmark for Lexical Substitution with Improved Data Coverage and Quality ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|