TY - JOUR
T1 - Interactive reservoir computing for chunking information streams
AU - Asabuki, Toshitake
AU - Hiratani, Naoki
AU - Fukai, Tomoki
N1 - Funding Information:
This work was partly supported by KAKENHI (nos. 16H01289 and 17H06036: https://www.jsps.go.jp/english/index.html) to TF, and TA was supported by Junior Research Associate program of RIKEN. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. We thank Vladimir Klinshov, Tomoki Kurikawa, Keita Watanabe and Takashi Takekawa for illuminating discussions about analysis methods for population neural dynamics.
Publisher Copyright:
© 2018 Asabuki et al. http://creativecommons.org/licenses/by/4.0/.
PY - 2018/10
Y1 - 2018/10
N2 - Chunking is the process by which frequently repeated segments of temporal inputs are concatenated into single units that are easy to process. Such a process is fundamental to time-series analysis in biological and artificial information processing systems. The brain efficiently acquires chunks from various information streams in an unsupervised manner; however, the underlying mechanisms of this process remain elusive. A widely-adopted statistical method for chunking consists of predicting frequently repeated contiguous elements in an input sequence based on unequal transition probabilities over sequence elements. However, recent experimental findings suggest that the brain is unlikely to adopt this method, as human subjects can chunk sequences with uniform transition probabilities. In this study, we propose a novel conceptual framework to overcome this limitation. In this process, neural networks learn to predict dynamical response patterns to sequence input rather than to directly learn transition patterns. Using a mutually supervising pair of reservoir computing modules, we demonstrate how this mechanism works in chunking sequences of letters or visual images with variable regularity and complexity. In addition, we demonstrate that background noise plays a crucial role in correctly learning chunks in this model. In particular, the model can successfully chunk sequences that conventional statistical approaches fail to chunk due to uniform transition probabilities. In addition, the neural responses of the model exhibit an interesting similarity to those of the basal ganglia observed after motor habit formation.
AB - Chunking is the process by which frequently repeated segments of temporal inputs are concatenated into single units that are easy to process. Such a process is fundamental to time-series analysis in biological and artificial information processing systems. The brain efficiently acquires chunks from various information streams in an unsupervised manner; however, the underlying mechanisms of this process remain elusive. A widely-adopted statistical method for chunking consists of predicting frequently repeated contiguous elements in an input sequence based on unequal transition probabilities over sequence elements. However, recent experimental findings suggest that the brain is unlikely to adopt this method, as human subjects can chunk sequences with uniform transition probabilities. In this study, we propose a novel conceptual framework to overcome this limitation. In this process, neural networks learn to predict dynamical response patterns to sequence input rather than to directly learn transition patterns. Using a mutually supervising pair of reservoir computing modules, we demonstrate how this mechanism works in chunking sequences of letters or visual images with variable regularity and complexity. In addition, we demonstrate that background noise plays a crucial role in correctly learning chunks in this model. In particular, the model can successfully chunk sequences that conventional statistical approaches fail to chunk due to uniform transition probabilities. In addition, the neural responses of the model exhibit an interesting similarity to those of the basal ganglia observed after motor habit formation.
UR - http://www.scopus.com/inward/record.url?scp=85055204907&partnerID=8YFLogxK
U2 - 10.1371/journal.pcbi.1006400
DO - 10.1371/journal.pcbi.1006400
M3 - Article
C2 - 30296262
AN - SCOPUS:85055204907
SN - 1553-734X
VL - 14
JO - PLoS computational biology
JF - PLoS computational biology
IS - 10
M1 - e1006400
ER -