6 |
Searching for Search Errors in Neural Morphological Inflection ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Applying the Transformer to Character-level Transduction ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
Examining the Inductive Bias of Neural Language Models with Artificial Languages ...
|
|
|
|
Abstract:
Read paper: https://www.aclanthology.org/2021.acl-long.38 Abstract: Since language models are used to model a wide variety of languages, it is natural to ask whether the neural architectures used for the task have inductive biases towards modeling particular types of languages. Investigation of these biases has proved complicated due to the many variables that appear in the experimental setup. Languages vary in many typological dimensions, and it is difficult to single out one or two to investigate without the others acting as confounders. We propose a novel method for investigating the inductive biases of language models using artificial languages. These languages are constructed to allow us to create parallel corpora across languages that differ only in the typological feature being investigated, such as word order. We then use them to train and test language models. This constitutes a fully controlled causal framework, and demonstrates how grammar engineering can serve as a useful tool for analyzing ...
|
|
URL: https://dx.doi.org/10.48448/kt06-r736 https://underline.io/lecture/25392-examining-the-inductive-bias-of-neural-language-models-with-artificial-languages
|
|
BASE
|
|
Hide details
|
|
17 |
Higher-order Derivatives of Weighted Finite-state Machines ...
|
|
|
|
BASE
|
|
Show details
|
|
19 |
A surprisal--duration trade-off across and within the world's languages ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|