4 |
Probing Multilingual Language Models for Discourse ...
|
|
|
|
Abstract:
Pre-trained multilingual language models have become an important building block in multilingual natural language processing. In the present paper, we investigate a range of such models to find out how well they transfer discourse-level knowledge across languages. This is done with a systematic evaluation on a broader set of discourse-level tasks than has been previously been assembled. We find that the XLM-RoBERTa family of models consistently show the best performance, by simultaneously being good monolingual models and degrading relatively little in a zero-shot setting. Our results also indicate that model distillation may hurt the ability of cross-lingual transfer of sentence representations, while language dissimilarity at most has a modest effect. We hope that our test suite, covering 5 tasks with a total of 22 languages in 10 distinct families, will serve as a useful evaluation platform for multilingual performance at and beyond the sentence level. ... : To be presented at RepL4NLP 2021 ...
|
|
Keyword:
Computation and Language cs.CL; FOS Computer and information sciences
|
|
URL: https://arxiv.org/abs/2106.04832 https://dx.doi.org/10.48550/arxiv.2106.04832
|
|
BASE
|
|
Hide details
|
|
7 |
A sentiment-annotated dataset of English causal connectives ...
|
|
|
|
BASE
|
|
Show details
|
|
8 |
Disambiguation of Potentially Idiomatic Expressions with Contextual Embeddings ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
What Do Language Representations Really Represent?
|
|
|
|
In: Bjerva, Johannes; Östling, Robert; Veiga, Maria Han; Tiedemann, Jörg; Augenstein, Isabelle (2019). What Do Language Representations Really Represent? Computational Linguistics, 45(2):381-389. (2019)
|
|
BASE
|
|
Show details
|
|
13 |
Universal Dependencies 2.2
|
|
|
|
In: https://hal.archives-ouvertes.fr/hal-01930733 ; 2018 (2018)
|
|
BASE
|
|
Show details
|
|
16 |
Visual Iconicity Across Sign Languages: Large-Scale Automated Video Analysis of Iconic Articulators and Locations
|
|
|
|
BASE
|
|
Show details
|
|
17 |
Universal Dependencies 2.1
|
|
|
|
In: https://hal.inria.fr/hal-01682188 ; 2017 (2017)
|
|
BASE
|
|
Show details
|
|
19 |
Articulation rate in Swedish child-directed speech increases as a function of the age of the child even when surprisal is controlled for ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|