2 |
XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation ...
|
|
|
|
BASE
|
|
Show details
|
|
3 |
nmT5 -- Is parallel data still relevant for pre-training massively multilingual language models? ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
The Power of Scale for Parameter-Efficient Prompt Tuning ...
|
|
|
|
Abstract:
Anthology paper link: https://aclanthology.org/2021.emnlp-main.243/ Abstract: In this work, we explore "prompt tuning", a simple yet effective mechanism for learning "soft prompts" to condition frozen language models to perform specific downstream tasks. Unlike the discrete text prompts used by GPT-3, soft prompts are learned through backpropagation and can be tuned to incorporate signal from any number of labeled examples. Our end-to-end learned approach outperforms GPT-3's "few-shot" learning by a large margin. More remarkably, through ablations on model size using T5, we show that prompt tuning becomes more competitive with scale: as models exceed billions of parameters, our method "closes the gap" and matches the strong performance of model tuning (where all model weights are tuned). This finding is especially relevant in that large models are costly to share and serve, and the ability to reuse one frozen model for multiple downstream tasks can ease this burden. Our method can be seen as a simplification ...
|
|
Keyword:
Computational Linguistics; Language Models; Machine Learning; Machine Learning and Data Mining; Natural Language Processing
|
|
URL: https://dx.doi.org/10.48448/hw3q-4598 https://underline.io/lecture/37409-the-power-of-scale-for-parameter-efficient-prompt-tuning
|
|
BASE
|
|
Hide details
|
|
6 |
TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer ...
|
|
|
|
BASE
|
|
Show details
|
|
8 |
Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
mT5: A massively multilingual pre-trained text-to-text transformer ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
Towards Zero-Shot Multilingual Synthetic Question and Answer Generation for Cross-Lingual Reading Comprehension ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
LAReQA: Language-agnostic answer retrieval from a multilingual pool ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
Multilingual Universal Sentence Encoder for Semantic Retrieval ...
|
|
|
|
BASE
|
|
Show details
|
|
13 |
Effective Parallel Corpus Mining using Bilingual Sentence Embeddings ...
|
|
|
|
BASE
|
|
Show details
|
|
15 |
Contrastive Topic: Meanings and Realizations
|
|
|
|
In: Doctoral Dissertations (2014)
|
|
BASE
|
|
Show details
|
|
17 |
Witnessable quantifiers license type-e meaning: Evidence from contrastive topic, equatives and supplements
|
|
|
|
In: Semantics and Linguistic Theory; Proceedings of SALT 22; 286-306 ; 2163-5951 (2012)
|
|
BASE
|
|
Show details
|
|
18 |
Mandarin 'even', `all' and the Trigger of Focus Movement
|
|
|
|
In: University of Pennsylvania Working Papers in Linguistics (2010)
|
|
BASE
|
|
Show details
|
|
|
|