1 |
Coloring the Blank Slate: Pre-training Imparts a Hierarchical Inductive Bias to Sequence-to-sequence Models ...
|
|
|
|
BASE
|
|
Show details
|
|
2 |
Does Putting a Linguist in the Loop Improve NLU Data Collection? ...
|
|
|
|
BASE
|
|
Show details
|
|
3 |
NOPE: A Corpus of Naturally-Occurring Presuppositions in English ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
NOPE: A Corpus of Naturally-Occurring Presuppositions in English ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Prediction ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
The Language Model Understood the Prompt was Ambiguous: Probing Syntactic Uncertainty Through Generation ...
|
|
|
|
BASE
|
|
Show details
|
|
8 |
How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN ...
|
|
|
|
Abstract:
Current language models can generate high-quality text. Are they simply copying text they have seen before, or have they learned generalizable linguistic abstractions? To tease apart these possibilities, we introduce RAVEN, a suite of analyses for assessing the novelty of generated text, focusing on sequential structure (n-grams) and syntactic structure. We apply these analyses to four neural language models (an LSTM, a Transformer, Transformer-XL, and GPT-2). For local structure - e.g., individual dependencies - model-generated text is substantially less novel than our baseline of human-generated text from each model's test set. For larger-scale structure - e.g., overall sentence structure - model-generated text is as novel or even more novel than the human-generated baseline, but models still sometimes copy substantially, in some cases duplicating passages over 1,000 words long from the training set. We also perform extensive manual analysis showing that GPT-2's novel text is usually well-formed ... : 10 pages, plus 39 pages of appendices ...
|
|
Keyword:
Computation and Language cs.CL; FOS Computer and information sciences
|
|
URL: https://arxiv.org/abs/2111.09509 https://dx.doi.org/10.48550/arxiv.2111.09509
|
|
BASE
|
|
Hide details
|
|
9 |
Frequency Effects on Syntactic Rule Learning in Transformers ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Prediction ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
Frequency Effects on Syntactic Rule Learning in Transformers ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
Structure Here, Bias There: Hierarchical Generalization by Jointly Learning Syntactic Transformations
|
|
|
|
In: Proceedings of the Society for Computation in Linguistics (2021)
|
|
BASE
|
|
Show details
|
|
13 |
Priming syntactic ambiguity resolution in children and adults
|
|
|
|
In: ISSN: 2327-3798 ; EISSN: 2327-3801 ; Language, Cognition and Neuroscience ; https://hal.archives-ouvertes.fr/hal-03099573 ; Language, Cognition and Neuroscience, Taylor and Francis, 2020, 35 (10), pp.1445-1455. ⟨10.1080/23273798.2020.1797130⟩ (2020)
|
|
BASE
|
|
Show details
|
|
14 |
Priming syntactic ambiguity resolution in children and adults ...
|
|
|
|
BASE
|
|
Show details
|
|
15 |
Priming syntactic ambiguity resolution in children and adults ...
|
|
|
|
BASE
|
|
Show details
|
|
16 |
How Can We Accelerate Progress Towards Human-like Linguistic Generalization? ...
|
|
|
|
BASE
|
|
Show details
|
|
17 |
Universal linguistic inductive biases via meta-learning ...
|
|
|
|
BASE
|
|
Show details
|
|
18 |
The reliability of acceptability judgments across languages. Glossa: a journal of general linguistics, 3(1), 100. ...
|
|
|
|
BASE
|
|
Show details
|
|
19 |
The reliability of acceptability judgments across languages. Glossa: a journal of general linguistics, 3(1), 100. ...
|
|
|
|
BASE
|
|
Show details
|
|
20 |
Invited Talk: Neural networks as cognitive models of syntax - Tal Linzen ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|