1 |
Survey of business sectors hiring linguists and language professionals ...
|
|
|
|
BASE
|
|
Show details
|
|
2 |
Survey of business sectors hiring linguists and language professionals ...
|
|
|
|
BASE
|
|
Show details
|
|
3 |
On the Language-specificity of Multilingual BERT and the Impact of Fine-tuning ...
|
|
|
|
Abstract:
Recent work has shown evidence that the knowledge acquired by multilingual BERT (mBERT) has two components: a language-specific and a language-neutral one. This paper analyses the relationship between them, in the context of fine-tuning on two tasks -- POS tagging and natural language inference -- which require the model to bring to bear different degrees of language-specific knowledge. Visualisations reveal that mBERT loses the ability to cluster representations by language after fine-tuning, a result that is supported by evidence from language identification experiments. However, further experiments on 'unlearning' language-specific representations using gradient reversal and iterative adversarial learning are shown not to add further improvement to the language-independent component over and above the effect of fine-tuning. The results presented here suggest that the process of fine-tuning causes a reorganisation of the model's limited representational capacity, enhancing language-independent ... : 14 pages, 6 figures, 5 tables, submitted in BlackBoxNLP 2021 (https://aclanthology.org/2021.blackboxnlp-1.15/) ...
|
|
Keyword:
Computation and Language cs.CL; FOS Computer and information sciences; Neural and Evolutionary Computing cs.NE
|
|
URL: https://dx.doi.org/10.48550/arxiv.2109.06935 https://arxiv.org/abs/2109.06935
|
|
BASE
|
|
Hide details
|
|
4 |
Data Augmentation for Speech Recognition in Maltese: A Low-Resource Perspective ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
On the Language-specificity of Multilingual BERT and the Impact of Fine-tuning ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
Automatic Removal of Identifying Information in Official EU Languages for Public Administrations: The MAPA Project
|
|
|
|
In: Legal Knowledge and Information Systems ; Frontiers in Artificial Intelligence and Applications ; International Conference on Legal Knowledge and Information Systems ; https://hal.archives-ouvertes.fr/hal-03058311 ; International Conference on Legal Knowledge and Information Systems, Dec 2020, Brno, Prague, Czech Republic. pp.223-226, ⟨10.3233/FAIA200869⟩ ; http://ebooks.iospress.nl/volume/legal-knowledge-and-information-systems-jurix-2020-the-thirty-third-annual-conference-brno-czech-republic-december-911-2020 (2020)
|
|
BASE
|
|
Show details
|
|
7 |
Annotating for Hate Speech: The MaNeCo Corpus and Some Input from Critical Discourse Analysis ...
|
|
|
|
BASE
|
|
Show details
|
|
8 |
Compositionality in English deverbal compounds: The role of the head ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
Compositionality in English deverbal compounds: The role of the head ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
Forthcoming: The role of constituents in multiword expressions: An interdisciplinary, cross-lingual perspective
|
|
|
|
In: Language Science Press; (2019)
|
|
BASE
|
|
Show details
|
|
11 |
Forthcoming: The role of constituents in multiword expressions: An interdisciplinary, cross-lingual perspective
|
|
|
|
In: Language Science Press; (2019)
|
|
BASE
|
|
Show details
|
|
12 |
Forthcoming: The role of constituents in multiword expressions: An interdisciplinary, cross-lingual perspective
|
|
|
|
In: Language Science Press; (2019)
|
|
BASE
|
|
Show details
|
|
13 |
Forthcoming: The role of constituents in multiword expressions: An interdisciplinary, cross-lingual perspective
|
|
|
|
In: Language Science Press; (2019)
|
|
BASE
|
|
Show details
|
|
14 |
Forthcoming: The role of constituents in multiword expressions: An interdisciplinary, cross-lingual perspective
|
|
|
|
In: Language Science Press; (2019)
|
|
BASE
|
|
Show details
|
|
15 |
Forthcoming: The role of constituents in multiword expressions: An interdisciplinary, cross-lingual perspective
|
|
|
|
In: Language Science Press; (2019)
|
|
BASE
|
|
Show details
|
|
16 |
Forthcoming: The role of constituents in multiword expressions: An interdisciplinary, cross-lingual perspective
|
|
|
|
In: Language Science Press; (2019)
|
|
BASE
|
|
Show details
|
|
17 |
Forthcoming: The role of constituents in multiword expressions: An interdisciplinary, cross-lingual perspective
|
|
|
|
In: Language Science Press; (2019)
|
|
BASE
|
|
Show details
|
|
18 |
Forthcoming: The role of constituents in multiword expressions: An interdisciplinary, cross-lingual perspective
|
|
|
|
In: Language Science Press; (2019)
|
|
BASE
|
|
Show details
|
|
19 |
Forthcoming: The role of constituents in multiword expressions: An interdisciplinary, cross-lingual perspective
|
|
|
|
In: Language Science Press; (2019)
|
|
BASE
|
|
Show details
|
|
20 |
Forthcoming: The role of constituents in multiword expressions: An interdisciplinary, cross-lingual perspective
|
|
|
|
In: Language Science Press; (2019)
|
|
BASE
|
|
Show details
|
|
|
|