| 1 |
Agreeing to Disagree: Annotating Offensive Language Datasets with Annotators' Disagreement ...
|
|
|
|
Abstract:
Anthology paper link: https://aclanthology.org/2021.emnlp-main.822/ Abstract: Since state-of-the-art approaches to offensive language detection rely on supervised learning, it is crucial to quickly adapt them to the continuously evolving scenario of social media. While several approaches have been proposed to tackle the problem from an algorithmic perspective, so to reduce the need for annotated data, less attention has been paid to the quality of these data. Following a trend that has emerged recently, we focus on the level of agreement among annotators while selecting data to create offensive language datasets, a task involving a high level of subjectivity. Our study comprises the creation of three novel datasets of English tweets covering different topics and having five crowd-sourced judgments each. We also present an extensive set of experiments showing that selecting training and test data according to different levels of annotators' agreement has a strong effect on classifiers performance and ...
|
|
Keyword:
Computational Linguistics; Machine Learning; Machine Learning and Data Mining; Natural Language Processing
|
|
URL: https://underline.io/lecture/37766-agreeing-to-disagree-annotating-offensive-language-datasets-with-annotators'-disagreement
https://dx.doi.org/10.48448/w49c-2969
|
|
BASE
|
|
Hide details
|
|
|
| 2 |
FrameNet-like Annotation of Olfactory Information in Texts ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|
|
