| 1 |
When Classifying Arguments, BERT Doesn't Care About Word Order. Except When It Matters
|
|
|
|
In: Proceedings of the Society for Computation in Linguistics (2022)
|
|
Abstract:
We probe nouns in BERT contextual embedding space for grammatical role (subject vs. object of a clause), and examine how probing results vary between prototypical examples, where the role matches what we would expect from seeing that word in the context, and non-prototypical examples, where the role is mostly imparted by the context. In this way, engage with the contrast that has arisen in the literature, between studies that show contextual models as grammatically sensitive, and others that show that these models are robust to changes in word order. Our experiments yield three results: 1) Grammatical role is recovered in later layers for difficult non-prototypical cases, while prototypical cases are accurate without many layers of context 2) When we switch the subject and the object of a sentence around (eg, The chef cut the onion, The onion cut the chef), we see that the same word (eg, onion) can be fluently identified as both a subject and an object 3) Subjecthood probing breaks if we ablate local word order by shuffle words locally and break grammaticality.
|
|
Keyword:
BERT; Computational Linguistics; Contextual embeddings; grammatical role; prototype; subjecthood; verb arguments; word order
|
|
URL: https://scholarworks.umass.edu/scil/vol5/iss1/18
https://scholarworks.umass.edu/cgi/viewcontent.cgi?article=1244&context=scil
|
|
BASE
|
|
Hide details
|
|
|
| 4 |
Loose and tight languages: A typology based on associations between constructions and lexemes ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 5 |
Loose and tight languages: A typology based on associations between constructions and lexemes ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 6 |
Word order preferences and the effect of phrasal length in SOV languages: evidence from sentence production in Persian
|
|
|
|
In: Glossa: a journal of general linguistics; Vol 5, No 1 (2020); 86 ; 2397-1835 (2020)
|
|
BASE
|
|
Show details
|
|
|
| 7 |
Word order preferences and the effect of phrasal length in SOV languages: evidence from sentence production in Persian
|
|
|
|
In: Glossa: a journal of general linguistics (2016-2021) ; https://hal.archives-ouvertes.fr/hal-02559840 ; Glossa: a journal of general linguistics (2016-2021), Ubiquity Press, 2020, 5 (1), pp.86. ⟨10.5334/gjgl.1078⟩ (2020)
|
|
BASE
|
|
Show details
|
|
|
| 11 |
The relationship between sentence meaning and word order: Evidence from structural priming in German
|
|
|
|
BASE
|
|
Show details
|
|
|
| 18 |
Хіазм Як Мовностилістичний Прийом У Віршовій Оповіді (На Матеріалі Української Поезії Іі Половини – Початку Ххі Століття) ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 19 |
Хіазм Як Мовностилістичний Прийом У Віршовій Оповіді (На Матеріалі Української Поезії Іі Половини – Початку Ххі Століття) ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 20 |
Word order and sentence structure in Mandarin Chinese: new perspectives
|
|
Morbiato, Anna. - : The University of Sydney, 2017. : Ca' Foscari University of Venice, 2017. : Department of Linguistics, 2017. : Faculty of Arts and Social Sciences, School of Literature, Art and Media, 2017
|
|
BASE
|
|
Show details
|
|
|
|
|
