1 |
What do You Mean by Relation Extraction? A Survey on Datasets and Study on Scientific Relation Classification ...
|
|
|
|
Abstract:
Over the last five years, research on Relation Extraction (RE) witnessed extensive progress with many new dataset releases. At the same time, setup clarity has decreased, contributing to increased difficulty of reliable empirical evaluation (Taillé et al., 2020). In this paper, we provide a comprehensive survey of RE datasets, and revisit the task definition and its adoption by the community. We find that cross-dataset and cross-domain setups are particularly lacking. We present an empirical study on scientific Relation Classification across two datasets. Despite large data overlap, our analysis reveals substantial discrepancies in annotation. Annotation discrepancies strongly impact Relation Classification performance, explaining large drops in cross-dataset evaluations. Variation within further sub-domains exists but impacts Relation Classification only to limited degrees. Overall, our study calls for more rigour in reporting setups in RE and evaluation across multiple test sets. ... : Accepted at the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (2022) ...
|
|
Keyword:
Computation and Language cs.CL; FOS Computer and information sciences
|
|
URL: https://dx.doi.org/10.48550/arxiv.2204.13516 https://arxiv.org/abs/2204.13516
|
|
BASE
|
|
Hide details
|
|
3 |
On the Effectiveness of Dataset Embeddings in Mono-lingual,Multi-lingual and Zero-shot Conditions ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
Genre as Weak Supervision for Cross-lingual Dependency Parsing ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
DaN+: Danish Nested Named Entities and Lexical Normalization ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
From Masked Language Modeling to Translation: Non-English Auxiliary Tasks Improve Zero-shot Spoken Language Understanding ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Psycholinguistics meets Continual Learning: Measuring Catastrophic Forgetting in Visual Question Answering ...
|
|
|
|
BASE
|
|
Show details
|
|
8 |
Beyond task success: A closer look at jointly learning to see, ask, and GuessWhat ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
Distant Supervision from Disparate Sources for Low-Resource Part-of-Speech Tagging ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
The Best of Both Worlds: Lexical Resources To Improve Low-Resource Part-of-Speech Tagging ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
Bleaching Text: Abstract Features for Cross-lingual Gender Prediction ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
ALL-IN-1: Short Text Classification with One Model for All Languages ...
|
|
|
|
BASE
|
|
Show details
|
|
15 |
Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Loss ...
|
|
|
|
BASE
|
|
Show details
|
|
16 |
Keystroke dynamics as signal for shallow syntactic parsing ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|