1 |
Pathologies of Pre-trained Language Models in Few-shot Fine-tuning ...
|
|
|
|
BASE
|
|
Show details
|
|
2 |
MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learning ...
|
|
|
|
Abstract:
The combination of multilingual pre-trained representations and cross-lingual transfer learning is one of the most effective methods for building functional NLP systems for low-resource languages. However, for extremely low-resource languages without large-scale monolingual corpora for pre-training or sufficient annotated data for fine-tuning, transfer learning remains an under-studied and challenging task. Moreover, recent work shows that multilingual representations are surprisingly disjoint across languages, bringing additional challenges for transfer onto extremely low-resource languages. In this paper, we propose MetaXL, a meta-learning based framework that learns to transform representations judiciously from auxiliary languages to a target one and brings their representation spaces closer for effective transfer. Extensive experiments on real-world low-resource languages - without access to large-scale monolingual corpora or large amounts of labeled data - for tasks like cross-lingual sentiment analysis ... : 2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2021) ...
|
|
Keyword:
Computation and Language cs.CL; FOS Computer and information sciences; Machine Learning cs.LG
|
|
URL: https://arxiv.org/abs/2104.07908 https://dx.doi.org/10.48550/arxiv.2104.07908
|
|
BASE
|
|
Hide details
|
|
3 |
A Dataset and Baselines for Multilingual Reply Suggestion ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
A Conditional Generative Matching Model for Multi-lingual Reply Suggestion ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
XtremeDistilTransformers: Task Transfer for Task-agnostic Distillation ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Smart To-Do : Automatic Generation of To-Do Items from Emails ...
|
|
|
|
BASE
|
|
Show details
|
|
8 |
Distilling BERT into Simple Neural Networks with Unlabeled Transfer Data ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
Multi-Source Cross-Lingual Model Transfer: Learning What to Share ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|