2 |
SemEval-2021 Task 12: Learning with Disagreements
|
|
|
|
Abstract:
Disagreement between coders is ubiquitous in virtually all datasets annotated with human judgements in both natural language processing and computer vision. However, most supervised machine learning methods assume that a single preferred interpretation exists for each item, which is at best an idealization. The aim of the SemEval-2021 shared task on learning with disagreements (Le-Wi-Di) was to provide a unified testing framework for methods for learning from data containing multiple and possibly contradictory annotations covering the best-known datasets containing information about disagreements for interpreting language and classifying images. In this paper we describe the shared task and its results.
|
|
URL: http://repository.essex.ac.uk/31851/1/2021.semeval-1.41.pdf http://repository.essex.ac.uk/31851/ https://doi.org/10.18653/v1/2021.semeval-1.41
|
|
BASE
|
|
Hide details
|
|
5 |
Crowdsourcing and Aggregating Nested Markable Annotations ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
A Crowdsourced Corpus of Multiple Judgments and Disagreement on Anaphoric Interpretation
|
|
|
|
BASE
|
|
Show details
|
|
9 |
A Crowdsourced Corpus of Multiple Judgments and Disagreement on Anaphoric Interpretation
|
|
|
|
BASE
|
|
Show details
|
|
10 |
Exploring Language Style in Chatbots to Increase Perceived Product Value and User Engagement
|
|
|
|
BASE
|
|
Show details
|
|
11 |
A Probabilistic Annotation Model for Crowdsourcing Coreference
|
|
|
|
BASE
|
|
Show details
|
|
14 |
Markup Infrastructure for the Anaphoric Bank: Supporting Web Collaboration
|
|
|
|
BASE
|
|
Show details
|
|
|
|