1 |
Le modèle Transformer: un « couteau suisse » pour le traitement automatique des langues
|
|
|
|
In: Techniques de l'Ingenieur ; https://hal.archives-ouvertes.fr/hal-03619077 ; Techniques de l'Ingenieur, Techniques de l'ingénieur, 2022, ⟨10.51257/a-v1-in195⟩ ; https://www.techniques-ingenieur.fr/base-documentaire/innovation-th10/innovations-en-electronique-et-tic-42257210/transformer-des-reseaux-de-neurones-pour-le-traitement-automatique-des-langues-in195/ (2022)
|
|
BASE
|
|
Show details
|
|
3 |
Navigating the Kaleidoscope of COVID-19 Misinformation Using Deep Learning ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
HittER: Hierarchical Transformers for Knowledge Graph Embeddings ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classification ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
Unsupervised Multi-View Post-OCR Error Correction With Language Models ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
AttentionRank: Unsupervised Keyphrase Extraction using Self and Cross Attentions ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
Automatic Fact-Checking with Document-level Annotations using BERT and Multiple Instance Learning ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
Towards the Early Detection of Child Predators in Chat Rooms: A BERT-based Approach ...
|
|
|
|
BASE
|
|
Show details
|
|
13 |
Semantic Categorization of Social Knowledge for Commonsense Question Answering ...
|
|
|
|
BASE
|
|
Show details
|
|
14 |
Pre-train or Annotate? Domain Adaptation with a Constrained Budget ...
|
|
|
|
BASE
|
|
Show details
|
|
15 |
Stepmothers are mean and academics are pretentious: What do pretrained language models learn about you? ...
|
|
|
|
BASE
|
|
Show details
|
|
16 |
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization ...
|
|
|
|
BASE
|
|
Show details
|
|
17 |
Automatic Text Evaluation through the Lens of Wasserstein Barycenters ...
|
|
|
|
BASE
|
|
Show details
|
|
18 |
Combining sentence and table evidence to predict veracity of factual claims using TaPaS and RoBERTa ...
|
|
|
|
BASE
|
|
Show details
|
|
19 |
Meta Distant Transfer Learning for Pre-trained Language Models ...
|
|
|
|
Abstract:
Anthology paper link: https://aclanthology.org/2021.emnlp-main.768/ Abstract: With the wide availability of Pre-trained Language Models (PLMs), multi-task fine-tuning across domains has been extensively applied. For tasks related to distant domains with different class label sets, PLMs may memorize non- transferable knowledge for the target domain and suffer from negative transfer. Inspired by meta-learning, we propose the Meta Distant Transfer Learning (Meta-DTL) framework to learn the cross-task knowledge for PLM-based methods. Meta-DTL first employs task representation learning to mine implicit relations among multiple tasks and classes. Based on the results, it trains a PLM-based meta-learner to capture the transferable knowledge across tasks. The weighted maximum entropy regularizers are proposed to make meta-learner more task-agnostic and unbiased. Finally, the meta-learner can be fine-tuned to fit each task with better parameter initialization. We evaluate Meta-DTL using both BERT and ALBERT on seven ...
|
|
Keyword:
Computational Linguistics; Language Models; Machine Learning; Machine Learning and Data Mining; Natural Language Processing
|
|
URL: https://dx.doi.org/10.48448/xp16-y443 https://underline.io/lecture/37379-meta-distant-transfer-learning-for-pre-trained-language-models
|
|
BASE
|
|
Hide details
|
|
|
|