DE eng

Search in the Catalogues and Directories

Hits 1 – 6 of 6

1
Static Embeddings as Efficient Knowledge Bases? ...
BASE
Show details
2
Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models ...
Abstract: Recently, it has been found that monolingual English language models can be used as knowledge bases. Instead of structural knowledge base queries, masked sentences such as "Paris is the capital of [MASK]" are used as probes. We translate the established benchmarks TREx and GoogleRE into 53 languages. Working with mBERT, we investigate three questions. (i) Can mBERT be used as a multilingual knowledge base? Most prior work only considers English. Extending research to multiple languages is important for diversity and accessibility. (ii) Is mBERT's performance as knowledge base language-independent or does it vary from language to language? (iii) A multilingual model is trained on more text, e.g., mBERT is trained on 104 Wikipedias. Can mBERT leverage this for better performance? We find that using mBERT as a knowledge base yields varying performance across languages and pooling predictions across languages improves performance. Conversely, mBERT exhibits a language bias; e.g., when queried in Italian, it ... : Accepted to EACL 2021 ...
Keyword: Computation and Language cs.CL; FOS Computer and information sciences
URL: https://arxiv.org/abs/2102.00894
https://dx.doi.org/10.48550/arxiv.2102.00894
BASE
Hide details
3
Measuring and Improving Consistency in Pretrained Language Models ...
BASE
Show details
4
Static Embeddings as Efficient Knowledge Bases? ...
NAACL 2021 2021; Dufter, Philipp; Kassner, Nora. - : Underline Science Inc., 2021
BASE
Show details
5
Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly
Schütze, Hinrich; Kassner, Nora. - : Ludwig-Maximilians-Universität München, 2020
BASE
Show details
6
Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly ...
Kassner, Nora; Schütze, Hinrich. - : Association for Computational Linguistics, 2020
BASE
Show details

Catalogues
0
0
0
0
0
0
0
Bibliographies
0
0
0
0
0
0
0
0
0
Linked Open Data catalogues
0
Online resources
0
0
0
0
Open access documents
6
0
0
0
0
© 2013 - 2024 Lin|gu|is|tik | Imprint | Privacy Policy | Datenschutzeinstellungen ändern