| 1 |
On-Line Processing of Grammatical Structure Using Reservoir Computing
|
|
|
|
In: In A. E. P. Villa, et al.: Artificial Neural Networks and Machine Learning - ICANN 2012 - 22nd International Conference on Artificial Neural Networks ; https://hal.inria.fr/hal-02561301 ; In A. E. P. Villa, et al.: Artificial Neural Networks and Machine Learning - ICANN 2012 - 22nd International Conference on Artificial Neural Networks, Sep 2012, Lausanne, Switzerland. pp.596-603, ⟨10.1007/978-3-642-33269-2_75⟩ (2012)
|
|
Abstract:
International audience ; Previous words in the sentence can influence the processing of the current word in the timescale of hundreds of milliseconds. The current research provides a possible explanation of how certain aspects of this on-line language processing can occur, based on the dynamics of recurrent cortical networks. We simulate prefrontal area BA47 as a recurrent network that receives on-line input of "grammatical" words during sentence processing, with plastic connections between cortex and striatum (homology with Reservoir Computing). The system is trained on sentence-meaning pairs, where meaning is coded as activation in the striatum corresponding to the roles that different "semantic words" play in the sentences. The model learns an extended set of grammatical constructions, and demonstrates the ability to generalize to novel constructions. This demonstrates that a RNN can decode grammatical structure from sentences in an on-line manner in order to generate a predictive representation of the meaning of the sentences.
|
|
Keyword:
[INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG]; [INFO.INFO-NE]Computer Science [cs]/Neural and Evolutionary Computing [cs.NE]; [SCCO.LING]Cognitive science/Linguistics; [SDV.NEU]Life Sciences [q-bio]/Neurons and Cognition [q-bio.NC]; Echo State Networks; Grammatical Structure; Language; On-line Language Processing; Prefrontal Cortex; Recurrent Neural Networks; Reservoir Computing; Striatum
|
|
URL: https://hal.inria.fr/hal-02561301/file/Hinaut2012_ICANN.pdf
https://hal.inria.fr/hal-02561301
https://doi.org/10.1007/978-3-642-33269-2_75
https://hal.inria.fr/hal-02561301/document
|
|
BASE
|
|
Hide details
|
|
|
|
|
