1 |
Analyzing Gender Representation in Multilingual Models ...
|
|
|
|
Abstract:
Multilingual language models were shown to allow for nontrivial transfer across scripts and languages. In this work, we study the structure of the internal representations that enable this transfer. We focus on the representation of gender distinctions as a practical case study, and examine the extent to which the gender concept is encoded in shared subspaces across different languages. Our analysis shows that gender representations consist of several prominent components that are shared across languages, alongside language-specific components. The existence of language-independent and language-specific components provides an explanation for an intriguing empirical observation we make: while gender classification transfers well across languages, interventions for gender removal, trained on a single language, do not transfer easily to others. ... : Published at RepL4NLP 2022 ...
|
|
Keyword:
Computation and Language cs.CL; FOS Computer and information sciences
|
|
URL: https://dx.doi.org/10.48550/arxiv.2204.09168 https://arxiv.org/abs/2204.09168
|
|
BASE
|
|
Hide details
|
|
5 |
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
Including Signed Languages in Natural Language Processing ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Including Signed Languages in Natural Language Processing ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
Provable Limitations of Acquiring Meaning from Ungrounded Form: What will Future Language Models Understand? ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
Measuring and Improving Consistency in Pretrained Language Models ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
Aligning Faithful Interpretations with their Social Attribution ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
Amnesic Probing: Behavioral Explanation With Amnesic Counterfactuals ...
|
|
|
|
BASE
|
|
Show details
|
|
14 |
Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descent ...
|
|
|
|
BASE
|
|
Show details
|
|
15 |
Asking It All: Generating Contextualized Questions for any Semantic Role ...
|
|
|
|
BASE
|
|
Show details
|
|
16 |
Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Prediction ...
|
|
|
|
BASE
|
|
Show details
|
|
18 |
Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Prediction ...
|
|
|
|
BASE
|
|
Show details
|
|
20 |
Simple, Interpretable and Stable Method for Detecting Words with Usage Change across Corpora
|
|
|
|
In: ACL 2020 - 58th Annual Meeting of the Association for Computational Linguistics ; https://hal.inria.fr/hal-03161637 ; ACL 2020 - 58th Annual Meeting of the Association for Computational Linguistics, Jul 2020, Seattle / Virtual, United States. pp.538-555, ⟨10.18653/v1/2020.acl-main.51⟩ (2020)
|
|
BASE
|
|
Show details
|
|
|
|