DE eng

Search in the Catalogues and Directories

Page: 1 2
Hits 1 – 20 of 28

1
How does the pre-training objective affect what large language models learn about linguistic properties? ...
Abstract: Several pre-training objectives, such as masked language modeling (MLM), have been proposed to pre-train language models (e.g. BERT) with the aim of learning better language representations. However, to the best of our knowledge, no previous work so far has investigated how different pre-training objectives affect what BERT learns about linguistics properties. We hypothesize that linguistically motivated objectives such as MLM should help BERT to acquire better linguistic knowledge compared to other non-linguistically motivated objectives that are not intuitive or hard for humans to guess the association between the input and the label to be predicted. To this end, we pre-train BERT with two linguistically motivated objectives and three non-linguistically motivated ones. We then probe for linguistic characteristics encoded in the representation of the resulting models. We find strong evidence that there are only small differences in probing performance between the representations learned by the two different ... : Accepted at ACL 2022 ...
Keyword: Computation and Language cs.CL; FOS Computer and information sciences
URL: https://dx.doi.org/10.48550/arxiv.2203.10415
https://arxiv.org/abs/2203.10415
BASE
Hide details
2
Automatic Identification and Classification of Bragging in Social Media ...
BASE
Show details
3
Analyzing Online Political Advertisements ...
BASE
Show details
4
Modeling the Severity of Complaints in Social Media ...
Jin, Mali; Aletras, Nikolaos. - : arXiv, 2021
BASE
Show details
5
Translation Error Detection as Rationale Extraction ...
BASE
Show details
6
Knowledge Distillation for Quality Estimation ...
BASE
Show details
7
Frustratingly Simple Pretraining Alternatives to Masked Language Modeling ...
BASE
Show details
8
Analyzing Online Political Advertisements ...
BASE
Show details
9
Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classification ...
BASE
Show details
10
Enjoy the Salience: Towards Better Transformer-based Faithful Explanations with Word Salience ...
BASE
Show details
11
Modeling the Severity of Complaints in Social Media ...
NAACL 2021 2021; Aletras, Nikolaos; Jin, Mali. - : Underline Science Inc., 2021
BASE
Show details
12
Active Learning by Acquiring Contrastive Examples ...
BASE
Show details
13
In Factuality: Efficient Integration of Relevant Facts for Visual Question Answering ...
BASE
Show details
14
Frustratingly Simple Pretraining Alternatives to Masked Language Modeling ...
BASE
Show details
15
Knowledge Distillation for Quality Estimation ...
BASE
Show details
16
Machine Extraction of Tax Laws from Legislative Texts
In: Proceedings of the Natural Legal Language Processing Workshop 2021 (2021)
BASE
Show details
17
Point-of-Interest Type Prediction using Text and Images ...
BASE
Show details
18
Point-of-Interest Type Prediction using Text and Images ...
BASE
Show details
19
An Empirical Study on Leveraging Position Embeddings for Target-oriented Opinion Words Extraction ...
BASE
Show details
20
Knowledge distillation for quality estimation
Gajbhiye, Amit; Fomicheva, Marina; Alva-Manchego, Fernando. - : Association for Computational Linguistics, 2021
BASE
Show details

Page: 1 2

Catalogues
0
0
0
0
0
0
0
Bibliographies
0
0
0
0
0
0
0
0
0
Linked Open Data catalogues
0
Online resources
0
0
0
0
Open access documents
28
0
0
0
0
© 2013 - 2024 Lin|gu|is|tik | Imprint | Privacy Policy | Datenschutzeinstellungen ändern