DE eng

Search in the Catalogues and Directories

Hits 1 – 8 of 8

1
Examining the Inductive Bias of Neural Language Models with Artificial Languages ...
Abstract: Read paper: https://www.aclanthology.org/2021.acl-long.38 Abstract: Since language models are used to model a wide variety of languages, it is natural to ask whether the neural architectures used for the task have inductive biases towards modeling particular types of languages. Investigation of these biases has proved complicated due to the many variables that appear in the experimental setup. Languages vary in many typological dimensions, and it is difficult to single out one or two to investigate without the others acting as confounders. We propose a novel method for investigating the inductive biases of language models using artificial languages. These languages are constructed to allow us to create parallel corpora across languages that differ only in the typological feature being investigated, such as word order. We then use them to train and test language models. This constitutes a fully controlled causal framework, and demonstrates how grammar engineering can serve as a useful tool for analyzing ...
URL: https://dx.doi.org/10.48448/kt06-r736
https://underline.io/lecture/25392-examining-the-inductive-bias-of-neural-language-models-with-artificial-languages
BASE
Hide details
2
Modeling the Unigram Distribution ...
BASE
Show details
3
Language Model Evaluation Beyond Perplexity ...
BASE
Show details
4
Higher-order Derivatives of Weighted Finite-state Machines ...
BASE
Show details
5
On Finding the K-best Non-projective Dependency Trees ...
BASE
Show details
6
Determinantal Beam Search ...
BASE
Show details
7
Is Sparse Attention more Interpretable? ...
BASE
Show details
8
A Cognitive Regularizer for Language Modeling ...
BASE
Show details

Catalogues
0
0
0
0
0
0
0
Bibliographies
0
0
0
0
0
0
0
0
0
Linked Open Data catalogues
0
Online resources
0
0
0
0
Open access documents
8
0
0
0
0
© 2013 - 2024 Lin|gu|is|tik | Imprint | Privacy Policy | Datenschutzeinstellungen ändern