42 |
Relational and Fine-Grained Argument Mining
|
|
|
|
In: Datenbank-Spektrum (2020)
|
|
BASE
|
|
Show details
|
|
44 |
SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings
|
|
|
|
In: Findings of ACL: EMNLP 2020 (2020)
|
|
BASE
|
|
Show details
|
|
45 |
Monolingual and Multilingual Reduction of Gender Bias in Contextualized Representations
|
|
|
|
BASE
|
|
Show details
|
|
46 |
Increasing Learning Efficiency of Self-Attention Networks through Direct Position Interactions, Learnable Temperature, and Convoluted Attention
|
|
|
|
BASE
|
|
Show details
|
|
47 |
SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings ...
|
|
|
|
BASE
|
|
Show details
|
|
48 |
Identifying Necessary Elements for BERT's Multilinguality ...
|
|
|
|
Abstract:
It has been shown that multilingual BERT (mBERT) yields high quality multilingual representations and enables effective zero-shot transfer. This is surprising given that mBERT does not use any crosslingual signal during training. While recent literature has studied this phenomenon, the reasons for the multilinguality are still somewhat obscure. We aim to identify architectural properties of BERT and linguistic properties of languages that are necessary for BERT to become multilingual. To allow for fast experimentation we propose an efficient setup with small BERT models trained on a mix of synthetic and natural data. Overall, we identify four architectural and two linguistic elements that influence multilinguality. Based on our insights, we experiment with a multilingual pretraining setup that modifies the masking strategy using VecMap, i.e., unsupervised embedding alignment. Experiments on XNLI with three languages indicate that our findings transfer from our small setup to larger scale settings. ... : EMNLP2020 CRV ...
|
|
Keyword:
Computation and Language cs.CL; FOS Computer and information sciences
|
|
URL: https://dx.doi.org/10.48550/arxiv.2005.00396 https://arxiv.org/abs/2005.00396
|
|
BASE
|
|
Hide details
|
|
49 |
Identifying Elements Essential for BERT’s Multilinguality ...
|
|
|
|
BASE
|
|
Show details
|
|
50 |
Identifying Necessary Elements for BERT’s Multilinguality ...
|
|
|
|
BASE
|
|
Show details
|
|
51 |
Monolingual and Multilingual Reduction of Gender Bias in Contextualized Representations ...
|
|
|
|
BASE
|
|
Show details
|
|
52 |
SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings ...
|
|
|
|
BASE
|
|
Show details
|
|
54 |
Predicting the Growth of Morphological Families from Social and Linguistic Factors ...
|
|
|
|
BASE
|
|
Show details
|
|
55 |
Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly ...
|
|
|
|
BASE
|
|
Show details
|
|
56 |
Quantifying the Contextualization of Word Representations with Semantic Class Probing ...
|
|
|
|
BASE
|
|
Show details
|
|
57 |
Sentence Meta-Embeddings for Unsupervised Semantic Textual Similarity ...
|
|
|
|
BASE
|
|
Show details
|
|
58 |
Monolingual and Multilingual Reduction of Gender Bias in Contextualized Representations ...
|
|
|
|
BASE
|
|
Show details
|
|
59 |
A Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matters ...
|
|
|
|
BASE
|
|
Show details
|
|
60 |
Unsupervised Embedding-based Detection of Lexical Semantic Changes ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|