1 |
Semantic Data Set Construction from Human Clustering and Spatial Arrangement ...
|
|
|
|
BASE
|
|
Show details
|
|
2 |
Context vs Target Word: Quantifying Biases in Lexical Semantic Datasets ...
|
|
|
|
Abstract:
State-of-the-art contextualized models such as BERT use tasks such as WiC and WSD to evaluate their word-in-context representations. This inherently assumes that performance in these tasks reflect how well a model represents the coupled word and context semantics. This study investigates this assumption by presenting the first quantitative analysis (using probing baselines) on the context-word interaction being tested in major contextual lexical semantic tasks. Specifically, based on the probing baseline performance, we propose measures to calculate the degree of context or word biases in a dataset, and plot existing datasets on a continuum. The analysis shows most existing datasets fall into the extreme ends of the continuum (i.e. they are either heavily context-biased or target-word-biased) while only AM$^2$iCo and Sense Retrieval challenge a model to represent both the context and target words. Our case study on WiC reveals that human subjects do not share models' strong context biases in the dataset ...
|
|
Keyword:
Computation and Language cs.CL; FOS Computer and information sciences
|
|
URL: https://dx.doi.org/10.48550/arxiv.2112.06733 https://arxiv.org/abs/2112.06733
|
|
BASE
|
|
Hide details
|
|
3 |
AM2iCo: Evaluating Word Meaning in Context across Low-Resource Languages with Adversarial Examples ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
AM2iCo: Evaluating Word Meaning in Context across Low-Resource Languages with Adversarial Examples ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
Quantifying lexical usage: vocabulary pertaining to ecosystems and the environment
|
|
|
|
BASE
|
|
Show details
|
|
6 |
Manual Clustering and Spatial Arrangement of Verbs for Multilingual Evaluation and Typology Analysis ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Manual Clustering and Spatial Arrangement of Verbs for Multilingual Evaluation and Typology Analysis ...
|
|
|
|
BASE
|
|
Show details
|
|
8 |
Manual Clustering and Spatial Arrangement of Verbs for Multilingual Evaluation and Typology Analysis
|
|
Majewska, Olga; Vulic, Ivan; McCarthy, Diana. - : International Committee on Computational Linguistics, 2020. : https://www.aclweb.org/anthology/2020.coling-main.423, 2020. : Proceedings of the 28th International Conference on Computational Linguistics (COLING 2020), 2020
|
|
BASE
|
|
Show details
|
|
9 |
Investigating the cross-lingual translatability of VerbNet-style classification. ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
Investigating the cross-lingual translatability of VerbNet-style classification.
|
|
|
|
BASE
|
|
Show details
|
|
13 |
Word Sense Clustering and Clusterability
|
|
|
|
In: ISSN: 0891-2017 ; EISSN: 1530-9312 ; Computational Linguistics ; https://hal.archives-ouvertes.fr/hal-01838502 ; Computational Linguistics, Massachusetts Institute of Technology Press (MIT Press), 2016, 42, pp.245-275. ⟨10.1162/COLI⟩ (2016)
|
|
BASE
|
|
Show details
|
|
14 |
Integrating character representations into Chinese word embedding
|
|
|
|
BASE
|
|
Show details
|
|
16 |
Semantic clustering of pivot paraphrases
|
|
|
|
In: International Conference on Language Resources and Evaluation ; https://hal.archives-ouvertes.fr/hal-01838559 ; International Conference on Language Resources and Evaluation, Jan 2014, Reykjavik, Iceland (2014)
|
|
BASE
|
|
Show details
|
|
19 |
Quantifying lexical usage: vocabulary pertaining to ecosystems and the environment
|
|
|
|
BASE
|
|
Show details
|
|
20 |
Finding Meaning in Context Using Graph Algorithms in Mono- and Cross-lingual Settings
|
|
|
|
BASE
|
|
Show details
|
|
|
|