• Refine your search:
  • 
    Keyword
  • 
    Creator / Publisher
  • 
    Year:
    
    • 2021 (18)
  • 
    Medium
  • 
    Type
  • 
    BLLDB-Access

Search in the Catalogues and Directories

All fields Title Creator / Publisher Keyword Year AND OR AND NOT All fields Title Creator / Publisher Keyword Year AND OR AND NOT All fields Title Creator / Publisher Keyword Year AND OR AND NOT All fields Title Creator / Publisher Keyword Year AND OR AND NOT All fields Title Creator / Publisher Keyword Year Sort by creator [A → Z]' creator [Z → A]' publishing year ↑ (asc)' publishing year ↓ (desc)' title [A → Z]' title [Z → A]' Simple Search Hits 1 – 18 of 18 1 Universal Dependencies 2.9 Zeman, Daniel; Nivre, Joakim; Abrams, Mitchell. - : Universal Dependencies Consortium, 2021 BASE Show details 2 Universal Dependencies 2.8.1 Zeman, Daniel; Nivre, Joakim; Abrams, Mitchell. - : Universal Dependencies Consortium, 2021 BASE Show details 3 Universal Dependencies 2.8 Zeman, Daniel; Nivre, Joakim; Abrams, Mitchell. - : Universal Dependencies Consortium, 2021 BASE Show details 4 BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models ... Zaken, Elad Ben; Ravfogel, Shauli; Goldberg, Yoav. - : arXiv, 2021 BASE Show details 5 Including Signed Languages in Natural Language Processing ... Yin, Kayo; Moryossef, Amit; Hochgesang, Julie. - : arXiv, 2021 BASE Show details 6 Including Signed Languages in Natural Language Processing ... The Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing 2021; Alikhani, Malihe; Goldberg, Yoav. - : Underline Science Inc., 2021 BASE Show details 7 Contrastive Explanations for Model Interpretability ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Choi, Yejin; Elazar, Yanai. - : Underline Science Inc., 2021 BASE Show details 8 Provable Limitations of Acquiring Meaning from Ungrounded Form: What will Future Language Models Understand? ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Goldberg, Yoav; Merrill, William. - : Underline Science Inc., 2021 BASE Show details 9 Measuring and Improving Consistency in Pretrained Language Models ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Elazar, Yanai; Goldberg, Yoav. - : Underline Science Inc., 2021 BASE Show details 10 Aligning Faithful Interpretations with their Social Attribution ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Goldberg, Yoav; Jacovi, Alon. - : Underline Science Inc., 2021 BASE Show details 11 Amnesic Probing: Behavioral Explanation With Amnesic Counterfactuals ... The Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing 2021; Elazar, Yanai; Goldberg, Yoav. - : Underline Science Inc., 2021 BASE Show details 12 Data Augmentation for Sign Language Gloss Translation ... Moryossef, Amit; Yin, Kayo; Neubig, Graham. - : arXiv, 2021 BASE Show details 13 Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descent ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Goldberg, Yoav; Merrill, William; Ramanujan, Vivek; Schwartz, Roy; Smith, Noah A.. - : Underline Science Inc., 2021 Abstract: Anthology paper link: https://aclanthology.org/2021.emnlp-main.133/ Abstract: The capacity of neural networks like the widely adopted transformer is known to be very high. Evidence is emerging that they learn successfully due to inductive bias in the training routine, typically a variant of gradient descent (GD). To better understand this bias, we study the tendency for transformer parameters to grow in magnitude ($\ell_2$ norm) during training, and its implications for the emergent representations within self attention layers. Empirically, we document norm growth in the training of transformer language models, including T5 during its pretraining. As the parameters grow in magnitude, we prove that the network approximates a discretized network with saturated activation functions. Such "saturated" networks are known to have a reduced capacity compared to the full network family that can be described in terms of formal languages and automata. Our results suggest saturation is a new characterization of an ... Keyword: Language Models; Natural Language Processing; Semantic Evaluation; Sociolinguistics URL: https://underline.io/lecture/37533-effects-of-parameter-norm-growth-during-transformer-training-inductive-bias-from-gradient-descent https://dx.doi.org/10.48448/2yr8-q466 BASE Hide details 14 Asking It All: Generating Contextualized Questions for any Semantic Role ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Dagan, Ido; Goldberg, Yoav. - : Underline Science Inc., 2021 BASE Show details 15 Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Prediction ... Ravfogel, Shauli; Prasad, Grusha; Linzen, Tal. - : arXiv, 2021 BASE Show details 16 Neural Extractive Search ... The Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing 2021; Goldberg, Yoav; Ravfogel, Shauli. - : Underline Science Inc., 2021 BASE Show details 17 Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Prediction ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Goldberg, Yoav; Linzen, Tal. - : Underline Science Inc., 2021 BASE Show details 18 Ab Antiquo: Neural Proto-language Reconstruction ... NAACL 2021 2021; Goldberg, Yoav; Meloni, Carlo. - : Underline Science Inc., 2021 BASE Show details

Mobile view All Catalogues UB Frankfurt Linguistik 0 IDS Mannheim 0 OLC Linguistik 0 UB Frankfurt Retrokatalog 0 DNB Subject Category Language 0 Institut für Empirische Sprachwissenschaft 0 Leibniz-Centre General Linguistics (ZAS) 0 Bibliographies BLLDB 0 BDSL 0 IDS Bibliografie zur deutschen Grammatik 0 IDS Bibliografie zur Gesprächsforschung 0 IDS Konnektoren im Deutschen 0 IDS Präpositionen im Deutschen 0 IDS OBELEX meta 0 MPI-SHH Linguistics Collection 0 MPI for Psycholinguistics 0 Linked Open Data catalogues Annohub 0 Online resources Link directory 0 Journal directory 0 Database directory 0 Dictionary directory 0 Open access documents BASE 18 Linguistik-Repository 0 IDS Publikationsserver 0 Online dissertations 0 Language Description Heritage 0