1 |
Benchmarking Answer Verification Methods for Question Answering-Based Summarization Evaluation Metrics ...
|
|
|
|
BASE
|
|
Show details
|
|
2 |
Question-Based Salient Span Selection for More Controllable Text Summarization ...
|
|
|
|
BASE
|
|
Show details
|
|
3 |
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
ESTER: A Machine Reading Comprehension Dataset for Reasoning about Event Semantic Relations ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
What is Your Article Based On? Inferring Fine-grained Provenance ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
BabyBERTa: Learning More Grammar With Small-Scale Child-Directed Language ...
|
|
|
|
Abstract:
Transformer-based language models have taken the NLP world by storm. However, their potential for addressing important questions in language acquisition research has been largely ignored. In this work, we examined the grammatical knowledge of RoBERTa (Liu et al., 2019) when trained on a 5M word corpus of language acquisition data to simulate the input available to children between the ages 1 and 6. Using the behavioral probing paradigm, we found that a smaller version of RoBERTa-base that never predicts unmasked tokens, which we term BabyBERTa, acquires grammatical knowledge comparable to that of pre-trained RoBERTa-base - and does so with approximately 15X fewer parameters and 6,000X fewer words. We discuss implications for building more efficient models and the learnability of grammar from input available to children. Lastly, to support research on this front, we release our novel grammar test suite that is compatible with the small vocabulary of child-directed input. ...
|
|
Keyword:
Computational Linguistics; Language Models; Machine Learning; Machine Learning and Data Mining; Natural Language Processing
|
|
URL: https://dx.doi.org/10.48448/j36t-vf29 https://underline.io/lecture/39845-babyberta-learning-more-grammar-with-small-scale-child-directed-language
|
|
BASE
|
|
Hide details
|
|
7 |
{Z}ero-shot {L}abel-Aware {E}vent {T}rigger and {A}rgument {C}lassification ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
Zero-shot Event Extraction via Transfer Learning: Challenges and Insights ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
Do We Know What We Don't Know? Studying Unanswerable Questions beyond SQuAD 2.0 ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
Towards Question-Answering as an Automatic Metric for Evaluating the Content Quality of a Summary ...
|
|
|
|
BASE
|
|
Show details
|
|
13 |
Constrained Labeled Data Generation for Low-Resource Named Entity Recognition ...
|
|
|
|
BASE
|
|
Show details
|
|
15 |
Cross-lingual Entity Alignment with Incidental Supervision ...
|
|
|
|
BASE
|
|
Show details
|
|
17 |
TransOMCS: From Linguistic Graphs to Commonsense Knowledge ...
|
|
|
|
BASE
|
|
Show details
|
|
18 |
Is Killed More Significant than Fled? A Contextual Model for Salient Event Detection ...
|
|
|
|
BASE
|
|
Show details
|
|
19 |
Towards Question-Answering as an Automatic Metric for Evaluating the Content Quality of a Summary ...
|
|
|
|
BASE
|
|
Show details
|
|
20 |
Extending Wikification: Nominal discovery, nominal linking, and the grounding of nouns
|
|
|
|
BASE
|
|
Show details
|
|
|
|