| 1 |
Which Hype for my New Task? Hints and Random Search for Reservoir Computing Hyperparameters
|
|
|
|
In: ICANN 2021 - 30th International Conference on Artificial Neural Networks ; https://hal.inria.fr/hal-03203318 ; ICANN 2021 - 30th International Conference on Artificial Neural Networks, Sep 2021, Bratislava, Slovakia (2021)
|
|
Abstract:
International audience ; In learning systems, hyperparameters are parameters that are not learned but need to be set a priori. In Reservoir Computing, there are several parameters that needs to be set a priori depending on the task. Newcomers to Reservoir Computing cannot have a good intuition on which hyperparameters to tune and how to tune them. For instance, beginners often explore the reservoir sparsity, but in practice this parameter is not of high influence on performance for ESNs. Most importantly, many authors keep doing suboptimal hyperparameter searches: using grid search as a tool to explore more than two hyperparameters, while restraining the spectral radius to be below unity. In this short paper, we give some suggestions, intuitions, and give a general method to find robust hyperparameters while understanding their influence on perfor- mance. We also provide a graphical interface (included in ReservoirPy) in order to make this hyperparameter search more intuitive. Finally, we discuss some potential refinements of the proposed method.
|
|
Keyword:
[INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI]; [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG]; [INFO.INFO-NE]Computer Science [cs]/Neural and Evolutionary Computing [cs.NE]; [SCCO.LING]Cognitive science/Linguistics; [SDV.NEU]Life Sciences [q-bio]/Neurons and Cognition [q-bio.NC]; Echo State Networks; Effective Spectral Radius; Grid Search; Hyperparameters; Random Search; Reservoir Computing
|
|
URL: https://hal.inria.fr/hal-03203318v2/document
https://hal.inria.fr/hal-03203318
https://hal.inria.fr/hal-03203318v2/file/Hinaut2021_ICANN_Reservoir-Random-Search_HAL-preprint-v2.pdf
|
|
BASE
|
|
Hide details
|
|
|
| 2 |
Canary Song Decoder: Transduction and Implicit Segmentation with ESNs and LTSMs
|
|
|
|
In: https://hal.inria.fr/hal-03203374 ; 2021 (2021)
|
|
BASE
|
|
Show details
|
|
|
| 3 |
Which Hype for my New Task? Hints and Random Search for Reservoir Computing Hyperparameters
|
|
|
|
In: https://hal.inria.fr/hal-03203318 ; 2021 (2021)
|
|
BASE
|
|
Show details
|
|
|
| 4 |
Canary Song Decoder: Transduction and Implicit Segmentation with ESNs and LTSMs
|
|
|
|
In: ICANN 2021 - 30th International Conference on Artificial Neural Networks ; https://hal.inria.fr/hal-03203374 ; ICANN 2021 - 30th International Conference on Artificial Neural Networks, Sep 2021, Bratislava, Slovakia. pp.71--82, ⟨10.1007/978-3-030-86383-8_6⟩ ; https://link.springer.com/chapter/10.1007/978-3-030-86383-8_6 (2021)
|
|
BASE
|
|
Show details
|
|
|
| 5 |
Cross-Situational Learning with Reservoir Computing for Language Acquisition Modelling
|
|
|
|
In: 2020 International Joint Conference on Neural Networks (IJCNN 2020) ; https://hal.inria.fr/hal-02594725 ; 2020 International Joint Conference on Neural Networks (IJCNN 2020), Jul 2020, Glasgow, Scotland, United Kingdom ; https://wcci2020.org/ (2020)
|
|
BASE
|
|
Show details
|
|
|
| 6 |
Language Acquisition with Echo State Networks: Towards Unsupervised Learning
|
|
|
|
In: ICDL 2020 - IEEE International Conference on Development and Learning ; https://hal.inria.fr/hal-02926613 ; ICDL 2020 - IEEE International Conference on Development and Learning, Oct 2020, Valparaiso / Virtual, Chile (2020)
|
|
BASE
|
|
Show details
|
|
|
| 7 |
Recurrent Neural Networks Models for Developmental Language Acquisition: Reservoirs Outperform LSTMs
|
|
|
|
In: SNL 2020 - 12th Annual Meeting of the Society for the Neurobiology of Language ; https://hal.inria.fr/hal-03146558 ; SNL 2020 - 12th Annual Meeting of the Society for the Neurobiology of Language, Oct 2020, Virtual Edition, Canada (2020)
|
|
BASE
|
|
Show details
|
|
|
| 8 |
A Reservoir Model for Intra-Sentential Code-Switching Comprehension in French and English
|
|
|
|
In: CogSci'19 - 41st Annual Meeting of the Cognitive Science Society ; https://hal.inria.fr/hal-02432831 ; CogSci'19 - 41st Annual Meeting of the Cognitive Science Society, Jul 2019, Montréal, Canada ; https://cognitivesciencesociety.org/cogsci-2019/ (2019)
|
|
BASE
|
|
Show details
|
|
|
| 9 |
On-Line Processing of Grammatical Structure Using Reservoir Computing
|
|
|
|
In: In A. E. P. Villa, et al.: Artificial Neural Networks and Machine Learning - ICANN 2012 - 22nd International Conference on Artificial Neural Networks ; https://hal.inria.fr/hal-02561301 ; In A. E. P. Villa, et al.: Artificial Neural Networks and Machine Learning - ICANN 2012 - 22nd International Conference on Artificial Neural Networks, Sep 2012, Lausanne, Switzerland. pp.596-603, ⟨10.1007/978-3-642-33269-2_75⟩ (2012)
|
|
BASE
|
|
Show details
|
|
|
|
|
