• Refine your search:
  • 
    Keyword
  • 
    Creator / Publisher:
    
    • Cotterell, Ryan (4)
    • White, Jennifer C. (4)
    • Pimentel, Tiago (2)
    • Saphra, Naomi (2)
    • Beltagy, Iz (1)
    • Bethard, Steven (1)
    • Chakraborty, Tanmoy (1)
    • Hakkani-Tur, Dilek (1)
    • Li, Wenjie (1)
    • Navigli, Roberto (1)
    •   more
  • 
    Year
  • 
    Medium
  • 
    Type:
    
    • Article (4)
  • 
    BLLDB-Access

Search in the Catalogues and Directories

All fields Title Creator / Publisher Keyword Year AND OR AND NOT All fields Title Creator / Publisher Keyword Year AND OR AND NOT All fields Title Creator / Publisher Keyword Year AND OR AND NOT All fields Title Creator / Publisher Keyword Year AND OR AND NOT All fields Title Creator / Publisher Keyword Year Sort by creator [A → Z]' creator [Z → A]' publishing year ↑ (asc)' publishing year ↓ (desc)' title [A → Z]' title [Z → A]' Simple Search Hits 1 – 4 of 4 1 Examining the Inductive Bias of Neural Language Models with Artificial Languages ... White, Jennifer C.; Cotterell, Ryan. - : ETH Zurich, 2021 BASE Show details 2 A Non-Linear Structural Probe Pimentel, Tiago; Cotterell, Ryan; Saphra, Naomi... In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (2021) BASE Show details 3 Examining the Inductive Bias of Neural Language Models with Artificial Languages White, Jennifer C.; Cotterell, Ryan In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (2021) Abstract: Since language models are used to model a wide variety of languages, it is natural to ask whether the neural architectures used for the task have inductive biases towards modeling particular types of languages. Investigation of these biases has proved complicated due to the many variables that appear in the experimental setup. Languages vary in many typological dimensions, and it is difficult to single out one or two to investigate without the others acting as confounders. We propose a novel method for investigating the inductive biases of language models using artificial languages. These languages are constructed to allow us to create parallel corpora across languages that differ only in the typological feature being investigated, such as word order. We then use them to train and test language models. This constitutes a fully controlled causal framework, and demonstrates how grammar engineering can serve as a useful tool for analyzing neural models. Using this method, we find that commonly used neural architectures exhibit different inductive biases: LSTMs display little preference with respect to word ordering, while transformers display a clear preference for some orderings over others. Further, we find that neither the inductive bias of the LSTM nor that of the transformer appear to reflect any tendencies that we see in attested natural languages. URL: https://hdl.handle.net/20.500.11850/521265 https://doi.org/10.3929/ethz-b-000519004 BASE Hide details 4 A Non-Linear Structural Probe ... White, Jennifer C.; Pimentel, Tiago; Saphra, Naomi. - : ETH Zurich, 2021 BASE Show details

Mobile view All Catalogues UB Frankfurt Linguistik 0 IDS Mannheim 0 OLC Linguistik 0 UB Frankfurt Retrokatalog 0 DNB Subject Category Language 0 Institut für Empirische Sprachwissenschaft 0 Leibniz-Centre General Linguistics (ZAS) 0 Bibliographies BLLDB 0 BDSL 0 IDS Bibliografie zur deutschen Grammatik 0 IDS Bibliografie zur Gesprächsforschung 0 IDS Konnektoren im Deutschen 0 IDS Präpositionen im Deutschen 0 IDS OBELEX meta 0 MPI-SHH Linguistics Collection 0 MPI for Psycholinguistics 0 Linked Open Data catalogues Annohub 0 Online resources Link directory 0 Journal directory 0 Database directory 0 Dictionary directory 0 Open access documents BASE 4 Linguistik-Repository 0 IDS Publikationsserver 0 Online dissertations 0 Language Description Heritage 0