1 |
Decoding the information structure underlying the neural representation of concepts
|
|
|
|
In: Proc Natl Acad Sci U S A (2022)
|
|
BASE
|
|
Show details
|
|
2 |
Deep Artificial Neural Networks Reveal a Distributed Cortical Network Encoding Propositional Sentence-Level Meaning
|
|
|
|
In: J Neurosci (2021)
|
|
Abstract:
Understanding how and where in the brain sentence-level meaning is constructed from words presents a major scientific challenge. Recent advances have begun to explain brain activation elicited by sentences using vector models of word meaning derived from patterns of word co-occurrence in text corpora. These studies have helped map out semantic representation across a distributed brain network spanning temporal, parietal, and frontal cortex. However, it remains unclear whether activation patterns within regions reflect unified representations of sentence-level meaning, as opposed to superpositions of context-independent component words. This is because models have typically represented sentences as “bags-of-words” that neglect sentence-level structure. To address this issue, we interrogated fMRI activation elicited as 240 sentences were read by 14 participants (9 female, 5 male), using sentences encoded by a recurrent deep artificial neural-network trained on a sentence inference task (InferSent). Recurrent connections and nonlinear filters enable InferSent to transform sequences of word vectors into unified “propositional” sentence representations suitable for evaluating intersentence entailment relations. Using voxelwise encoding modeling, we demonstrate that InferSent predicts elements of fMRI activation that cannot be predicted by bag-of-words models and sentence models using grammatical rules to assemble word vectors. This effect occurs throughout a distributed network, which suggests that propositional sentence-level meaning is represented within and across multiple cortical regions rather than at any single site. In follow-up analyses, we place results in the context of other deep network approaches (ELMo and BERT) and estimate the degree of unpredicted neural signal using an “experiential” semantic model and cross-participant encoding. SIGNIFICANCE STATEMENT A modern-day scientific challenge is to understand how the human brain transforms word sequences into representations of sentence meaning. A recent approach, emerging from advances in functional neuroimaging, big data, and machine learning, is to computationally model meaning, and use models to predict brain activity. Such models have helped map a cortical semantic information-processing network. However, how unified sentence-level information, as opposed to word-level units, is represented throughout this network remains unclear. This is because models have typically represented sentences as unordered “bags-of-words.” Using a deep artificial neural network that recurrently and nonlinearly combines word representations into unified propositional sentence representations, we provide evidence that sentence-level information is encoded throughout a cortical network, rather than in a single region.
|
|
Keyword:
Research Articles
|
|
URL: https://doi.org/10.1523/JNEUROSCI.1152-20.2021 http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8176751/ http://www.ncbi.nlm.nih.gov/pubmed/33753548
|
|
BASE
|
|
Hide details
|
|
3 |
Differential Activation of the Visual Word Form Area During Auditory Phoneme Perception in Youth with Dyslexia
|
|
|
|
In: Neuropsychologia (2020)
|
|
BASE
|
|
Show details
|
|
4 |
An Integrated Neural Decoder of Linguistic and Experiential Meaning
|
|
|
|
BASE
|
|
Show details
|
|
6 |
Lesion localization of speech comprehension deficits in chronic aphasia
|
|
|
|
BASE
|
|
Show details
|
|
7 |
The Relationship Between Maternal Education and the Neural Substrates of Phoneme Perception in Children: Interactions Between Socioeconomic Status and Proficiency Level
|
|
|
|
BASE
|
|
Show details
|
|
8 |
Surface Errors Without Semantic Impairment in Acquired Dyslexia: A Voxel-Based Lesion-Symptom Mapping Study ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
Heteromodal Cortical Areas Encode Sensory-Motor Features of Word Meaning
|
|
|
|
BASE
|
|
Show details
|
|
10 |
Predicting brain activation patterns associated with individual lexical concepts based on five sensory-motor attributes
|
|
|
|
BASE
|
|
Show details
|
|
13 |
Cerebral Localization of Impaired Phonological Retrieval During Rhyme Judgment
|
|
|
|
BASE
|
|
Show details
|
|
14 |
Anatomy is strategy: Skilled reading differences associated with structural connectivity differences in the reading network
|
|
|
|
BASE
|
|
Show details
|
|
16 |
FMRI of Phonemic Perception and Its Relationship to Reading Development in Elementary- to Middle-School-Age Children
|
|
|
|
BASE
|
|
Show details
|
|
17 |
The Role of Left Occipitotemporal Cortex in Reading: Reconciling Stimulus, Task, and Lexicality Effects
|
|
|
|
BASE
|
|
Show details
|
|
18 |
Perceptual demand modulates activation of human auditory cortex in response to task-irrelevant sounds
|
|
|
|
BASE
|
|
Show details
|
|
20 |
The Role of Left Occipitotemporal Cortex in Reading: Reconciling Stimulus, Task, and Lexicality Effects
|
|
|
|
BASE
|
|
Show details
|
|
|
|