DE eng

Search in the Catalogues and Directories

Hits 1 – 12 of 12

1
AUTOLEX: An Automatic Framework for Linguistic Exploration ...
BASE
Show details
2
Evaluating the Morphosyntactic Well-formedness of Generated Texts ...
BASE
Show details
3
Evaluating the Morphosyntactic Well-formedness of Generated Texts ...
BASE
Show details
4
Do Context-Aware Translation Models Pay the Right Attention? ...
BASE
Show details
5
When is Wall a Pared and when a Muro? -- Extracting Rules Governing Lexical Selection ...
BASE
Show details
6
When is Wall a Pared and when a Muro?: Extracting Rules Governing Lexical Selection ...
BASE
Show details
7
Do Context-Aware Translation Models Pay the Right Attention? ...
BASE
Show details
8
DICT-MLM: Improved Multilingual Pre-Training using Bilingual Dictionaries ...
Abstract: Pre-trained multilingual language models such as mBERT have shown immense gains for several natural language processing (NLP) tasks, especially in the zero-shot cross-lingual setting. Most, if not all, of these pre-trained models rely on the masked-language modeling (MLM) objective as the key language learning objective. The principle behind these approaches is that predicting the masked words with the help of the surrounding text helps learn potent contextualized representations. Despite the strong representation learning capability enabled by MLM, we demonstrate an inherent limitation of MLM for multilingual representation learning. In particular, by requiring the model to predict the language-specific token, the MLM objective disincentivizes learning a language-agnostic representation -- which is a key goal of multilingual pre-training. Therefore to encourage better cross-lingual representation learning we propose the DICT-MLM method. DICT-MLM works by incentivizing the model to be able to predict not ... : 13 pages ...
Keyword: Computation and Language cs.CL; FOS Computer and information sciences
URL: https://dx.doi.org/10.48550/arxiv.2010.12566
https://arxiv.org/abs/2010.12566
BASE
Hide details
9
SIGTYP 2020 Shared Task: Prediction of Typological Features ...
BASE
Show details
10
Automatic Extraction of Rules Governing Morphological Agreement ...
BASE
Show details
11
A Summary of the First Workshop on Language Technology for Language Documentation and Revitalization ...
BASE
Show details
12
Adapting Word Embeddings to New Languages with Morphological and Phonological Subword Representations ...
BASE
Show details

Catalogues
0
0
0
0
0
0
0
Bibliographies
0
0
0
0
0
0
0
0
0
Linked Open Data catalogues
0
Online resources
0
0
0
0
Open access documents
12
0
0
0
0
© 2013 - 2024 Lin|gu|is|tik | Imprint | Privacy Policy | Datenschutzeinstellungen ändern