1 |
The Mapping of Deep Language Models on Brain Responses Primarily Depends on their Performance
|
|
|
|
In: https://hal.archives-ouvertes.fr/hal-03361439 ; 2021 (2021)
|
|
Abstract:
Recent deep networks like transformers not only excel in several language tasks, but their activations linearly map onto the human brain during language processing. Is this functional similarity caused by specific factors, such as the language abilities and the architecture of the algorithms? To address this issue, we analyze the brain responses to isolated sentences in a large cohort of 102 subjects, each recorded with both functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG). We then compare the ability of 32,400 transformer embeddings to linearly map onto these brain responses. Finally, we evaluate how the architecture, training, and performance of the models independently account for this brain mapping. Our analyses reveal two main findings. First, the similarity between brain responses and the activations of language models primarily depends on their ability to predict words from the context. Second, this similarity allows us to decompose and precisely track the rise and maintenance of perceptual, lexical, and compositional representations within each cortical region. Overall, this study evidences a partial convergence of language transformers to brainlike solutions, and shows how this phenomenon helps unravel the brain bases of natural language processing.
|
|
Keyword:
[INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI]; [INFO.INFO-CL]Computer Science [cs]/Computation and Language [cs.CL]; [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG]; [SCCO.NEUR]Cognitive science/Neuroscience; Encoding; Functional Magnetic Resonance Imaging; Magneto-encephalography; Natural Language Processing
|
|
URL: https://hal.archives-ouvertes.fr/hal-03361439 https://hal.archives-ouvertes.fr/hal-03361439/file/Mous_mapping.pdf https://hal.archives-ouvertes.fr/hal-03361439/document
|
|
BASE
|
|
Hide details
|
|
2 |
Neural Substrates of the Interplay between Cognitive Loading and Emotional Involvement in Bilingual Decision Making ...
|
|
|
|
BASE
|
|
Show details
|
|
3 |
Neural Substrates of the Interplay between Cognitive Loading and Emotional Involvement in Bilingual Decision Making ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
Patterns and networks of language control in bilingual language production ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
Neural Substrates of the Interplay between Cognitive Loading and Emotional Involvement in Bilingual Decision Making ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
Data for: Language context modulates brain network for language control in bilingual language production ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Data for: Language context modulates brain network for language control in bilingual language production ...
|
|
|
|
BASE
|
|
Show details
|
|
8 |
Data for: Language context modulates brain network for language control in bilingual language production ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
Language learning experience and mastering the challenges of perceiving speech in noise
|
|
|
|
In: https://www.sciencedirect.com/journal/brain-and-language/vol/196/suppl/C (2019)
|
|
BASE
|
|
Show details
|
|
10 |
Sign and speech share partially overlapping conceptual representations ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
Sign and speech share partially overlapping conceptual representations ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
Mother of unification studies, a 204-subject multimodal neuroimaging dataset to study language processing ...
|
|
|
|
BASE
|
|
Show details
|
|
13 |
Simultaneous learning of two languages from birth positively impacts intrinsic functional connectivity and cognitive control
|
|
|
|
In: Brain and Cognition ; Volume 117 ; Pages 49-56. (2017)
|
|
BASE
|
|
Show details
|
|
14 |
Phonological processing in speech perception: What do sonority differences tell us?
|
|
|
|
In: http://www.sciencedirect.com/science/article/pii/S0093934X15001285?via%3Dihub (2015)
|
|
BASE
|
|
Show details
|
|
15 |
Visualizing and modeling vocal production dynamics ...
|
|
Bresch, Erik. - : University of Southern California Digital Library (USC.DL), 2015
|
|
BASE
|
|
Show details
|
|
17 |
Language processing, functional magnetic resonance imaging of [encyclopaedia entry]
|
|
|
|
BASE
|
|
Show details
|
|
18 |
Understanding Cortical Activation Associated with Second Language Acquisition
|
|
|
|
BASE
|
|
Show details
|
|
19 |
Language dominance assessment in a bilingual population: validity of fMRI in the second language.
|
|
|
|
In: In: Epilepsia. (pp. 1504 - 1511). (2014) (2014)
|
|
BASE
|
|
Show details
|
|
20 |
Investigating Lesion Size and Neuroimaging Methods with Magnetic Resonance Imaging in People With Aphasia
|
|
|
|
BASE
|
|
Show details
|
|
|
|