1 |
THEaiTRobot 1.0
|
|
Rosa, Rudolf; Dušek, Ondřej; Kocmi, Tom. - : Charles University, Faculty of Mathematics and Physics, Institute of Formal and Applied Linguistics (UFAL), 2021. : The Švanda Theatre in Smíchov, 2021. : The Academy of Performing Arts in Prague, Theatre Faculty (DAMU), 2021
|
|
BASE
|
|
Show details
|
|
9 |
Examining Cross-lingual Contextual Embeddings with Orthogonal Structural Probes ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
Examining Cross-lingual Contextual Embeddings with Orthogonal Structural Probes ...
|
|
|
|
Abstract:
Anthology paper link: https://aclanthology.org/2021.emnlp-main.376/ Abstract: State-of-the-art contextual embeddings are obtained from large language models available only for a few languages. For others, we need to learn representations using a multilingual model. There is an ongoing debate on whether multilingual embeddings can be aligned in a space shared across many languages. The novel Orthogonal Structural Probe (Limisiewicz and Mareček, 2021) allows us to answer this question for specific linguistic features and learn a projection based only on mono-lingual annotated datasets. We evaluate syntactic (UD) and lexical (WordNet) structural information encoded inmBERT's contextual representations for nine diverse languages. We observe that for languages closely related to English, no transformation is needed. The evaluated information is encoded in a shared cross-lingual embedding space. For other languages, it is beneficial to apply orthogonal transformation learned separately for each language. We ...
|
|
Keyword:
Computational Linguistics; Language Models; Machine Learning; Machine Learning and Data Mining; Natural Language Processing
|
|
URL: https://underline.io/lecture/37732-examining-cross-lingual-contextual-embeddings-with-orthogonal-structural-probes https://dx.doi.org/10.48448/6ks5-gs79
|
|
BASE
|
|
Hide details
|
|
14 |
Syntax Representation in Word Embeddings and Neural Networks -- A Survey ...
|
|
|
|
BASE
|
|
Show details
|
|
15 |
Measuring Memorization Effect in Word-Level Neural Networks Probing ...
|
|
|
|
BASE
|
|
Show details
|
|
19 |
Derivational Morphological Relations in Word Embeddings ...
|
|
|
|
BASE
|
|
Show details
|
|
20 |
Universal Dependencies 2.2
|
|
|
|
In: https://hal.archives-ouvertes.fr/hal-01930733 ; 2018 (2018)
|
|
BASE
|
|
Show details
|
|
|
|