2 |
Language Models Use Monotonicity to Assess NPI Licensing ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
Causal Transformers Perform Below Chance on Recursive Nested Constructions, Unlike Humans ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
Sparse Interventions in Language Models with Differentiable Masking ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Mechanisms for Handling Nested Dependencies in Neural-Network Language Models and Humans ...
|
|
|
|
Abstract:
Recursive processing in sentence comprehension is considered a hallmark of human linguistic abilities. However, its underlying neural mechanisms remain largely unknown. We studied whether a modern artificial neural network trained with "deep learning" methods mimics a central aspect of human sentence processing, namely the storing of grammatical number and gender information in working memory and its use in long-distance agreement (e.g., capturing the correct number agreement between subject and verb when they are separated by other phrases). Although the network, a recurrent architecture with Long Short-Term Memory units, was solely trained to predict the next word in a large corpus, analysis showed the emergence of a very sparse set of specialized units that successfully handled local and long-distance syntactic agreement for grammatical number. However, the simulations also showed that this mechanism does not support full recursion and fails with some long-range embedded dependencies. We tested the ...
|
|
Keyword:
Computation and Language cs.CL; FOS Computer and information sciences
|
|
URL: https://arxiv.org/abs/2006.11098 https://dx.doi.org/10.48550/arxiv.2006.11098
|
|
BASE
|
|
Hide details
|
|
9 |
Compositionality decomposed: how do neural networks generalise? ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
Do Language Models Understand Anything? On the Ability of LSTMs to Understand Negative Polarity Items ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|