• Refine your search:
  • 
    Keyword:
    
    • Language Models (27)
    • Natural Language Processing (27)
    • Semantic Evaluation (27)
    • Sociolinguistics (27)
  • 
    Creator / Publisher
  • 
    Year:
    
    • 2021 (27)
  • 
    Medium
  • 
    Type
  • 
    BLLDB-Access

Search in the Catalogues and Directories

All fields Title Creator / Publisher Keyword Year AND OR AND NOT All fields Title Creator / Publisher Keyword Year AND OR AND NOT All fields Title Creator / Publisher Keyword Year AND OR AND NOT All fields Title Creator / Publisher Keyword Year AND OR AND NOT All fields Title Creator / Publisher Keyword Year Sort by creator [A → Z]' creator [Z → A]' publishing year ↑ (asc)' publishing year ↓ (desc)' title [A → Z]' title [Z → A]' Simple Search Page: 1 2 Hits 1 – 20 of 27 1 Minimal Supervision for Morphological Inflection ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Goldman, Omer; Tsarfaty, Reut. - : Underline Science Inc., 2021 BASE Show details 2 Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descent ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Goldberg, Yoav; Merrill, William; Ramanujan, Vivek; Schwartz, Roy; Smith, Noah A.. - : Underline Science Inc., 2021 Abstract: Anthology paper link: https://aclanthology.org/2021.emnlp-main.133/ Abstract: The capacity of neural networks like the widely adopted transformer is known to be very high. Evidence is emerging that they learn successfully due to inductive bias in the training routine, typically a variant of gradient descent (GD). To better understand this bias, we study the tendency for transformer parameters to grow in magnitude ($\ell_2$ norm) during training, and its implications for the emergent representations within self attention layers. Empirically, we document norm growth in the training of transformer language models, including T5 during its pretraining. As the parameters grow in magnitude, we prove that the network approximates a discretized network with saturated activation functions. Such "saturated" networks are known to have a reduced capacity compared to the full network family that can be described in terms of formal languages and automata. Our results suggest saturation is a new characterization of an ... Keyword: Language Models; Natural Language Processing; Semantic Evaluation; Sociolinguistics URL: https://underline.io/lecture/37533-effects-of-parameter-norm-growth-during-transformer-training-inductive-bias-from-gradient-descent https://dx.doi.org/10.48448/2yr8-q466 BASE Hide details 3 Softmax Tree: An Accurate, Fast Classifier When the Number of Classes Is Large ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Carreira-Perpinan, Miguel; Gabidolla, Magzhan. - : Underline Science Inc., 2021 BASE Show details 4 Multivalent Entailment Graphs for Question Answering ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; McKenna, Nick. - : Underline Science Inc., 2021 BASE Show details 5 GOLD: Improving Out-of-Scope Detection in Dialogues using Data Augmentation ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Chen, Derek; Yu, Zhou. - : Underline Science Inc., 2021 BASE Show details 6 RuleBERT: Teaching Soft Rules to Pre-Trained Language Models ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Ahmadi, Naser; Nakov, Preslav. - : Underline Science Inc., 2021 BASE Show details 7 Implicit Premise Generation with Discourse-aware Commonsense Knowledge Models ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Chakrabarty, Tuhin; Muresan, Smaranda. - : Underline Science Inc., 2021 BASE Show details 8 On the Challenges of Evaluating Compositional Explanations in Multi-Hop Inference: Relevance, Completeness, and Expert Ratings ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Jansen, Peter; Moreno, Dan. - : Underline Science Inc., 2021 BASE Show details 9 Is Everything in Order? A Simple Way to Order Sentences ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Basu Roy Chowdhury, Somnath; Brahman, Faeze. - : Underline Science Inc., 2021 BASE Show details 10 Cross-Domain Label-Adaptive Stance Detection ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Hardalov, Momchil. - : Underline Science Inc., 2021 BASE Show details 11 Enhanced Language Representation with Label Knowledge for Span Extraction ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Liu, Xingwu; Sun, Zhenyu. - : Underline Science Inc., 2021 BASE Show details 12 The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Csordás, Róbert; Irie, Kazuki. - : Underline Science Inc., 2021 BASE Show details 13 VeeAlign: Multifaceted Context Representation Using Dual Attention for Ontology Alignment ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Agarwal, Arvind; Iyer, Vivek. - : Underline Science Inc., 2021 BASE Show details 14 Shortcutted Commonsense: Data Spuriousness in Deep Learning of Commonsense Reasoning ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Branco, Ruben; Branco, António. - : Underline Science Inc., 2021 BASE Show details 15 On Classifying whether Two Texts are on the Same Side of an Argument ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Hakimi, Ahmad Dawar; Heyer, Gerhard. - : Underline Science Inc., 2021 BASE Show details 16 Causal Direction of Data Collection Matters: Implications of Causal and Anticausal Learning for NLP ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; ., Tejas; Jin, Zhijing. - : Underline Science Inc., 2021 BASE Show details 17 MTAdam: Automatic Balancing of Multiple Training Loss Terms ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Malkiel, Itzik; wolf, lior. - : Underline Science Inc., 2021 BASE Show details 18 Types of Out-of-Distribution Texts and How to Detect Them ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Arora, Udit; He, He. - : Underline Science Inc., 2021 BASE Show details 19 Asking It All: Generating Contextualized Questions for any Semantic Role ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Dagan, Ido; Goldberg, Yoav. - : Underline Science Inc., 2021 BASE Show details 20 Competency Problems: On Finding and Removing Artifacts in Language Data ... The 2021 Conference on Empirical Methods in Natural Language Processing 2021; Dodge, Jesse; Gardner, Matt. - : Underline Science Inc., 2021 BASE Show details Page: 1 2

Mobile view All Catalogues UB Frankfurt Linguistik 0 IDS Mannheim 0 OLC Linguistik 0 UB Frankfurt Retrokatalog 0 DNB Subject Category Language 0 Institut für Empirische Sprachwissenschaft 0 Leibniz-Centre General Linguistics (ZAS) 0 Bibliographies BLLDB 0 BDSL 0 IDS Bibliografie zur deutschen Grammatik 0 IDS Bibliografie zur Gesprächsforschung 0 IDS Konnektoren im Deutschen 0 IDS Präpositionen im Deutschen 0 IDS OBELEX meta 0 MPI-SHH Linguistics Collection 0 MPI for Psycholinguistics 0 Linked Open Data catalogues Annohub 0 Online resources Link directory 0 Journal directory 0 Database directory 0 Dictionary directory 0 Open access documents BASE 27 Linguistik-Repository 0 IDS Publikationsserver 0 Online dissertations 0 Language Description Heritage 0