Title: Impact of recurrent connectivity on off-line memory reprocessing in a hierarchical neural network formed by unsupervised learning
Abstract: Event Abstract Back to Event Impact of recurrent connectivity on off-line memory reprocessing in a hierarchical neural network formed by unsupervised learning Jenia Jitsev1* 1 Max Planck Institute for Neurological Research, Cortical Networks and Cognitive Functions, Germany Multiple experimental studies provide evidence for memory performance improvement following off-line regimes like restful waking or states of sleep, where brain is not actively involved in processing of external stimuli. It has been hypothesized that this improvement is due to off-line reprocessing and consolidation of initially labile memory traces formed during active waking. Although it is obvious that such improvement requires beneficial modifications to be done in the memory network during off-line regime, not much is known about what kind of changes are indeed induced in the memory network during off-line reprocessing. In our study we examine off-line memory reprocessing in a hierarchical recurrent neural network model that is able to self-generate ongoing sparse activity even in absence of external stimulation due to self-excitable but competitive neural dynamics that shapes winner-take-all like behavior of network units. In this regime, the network reactivates memory traces established during preceding on-line learning, where it has been exposed to natural face images of different persons in unsupervised fashion [1]. Remarkably, this off-line memory replay turns out to be highly beneficial for the network recognition performance [2, 3]. To our surprise, the positive effect was independent of synapse-specific plasticity, relying completely on a synapse-unspecific local mechanism of homeostatic activity regulation that equalizes unit excitabilities within network layers during off-line reprocessing. Moreover, fully recurrent network gets a much stronger performance boost after off-line reprocessing as compared to its purely feed-forward version. To clarify the contribution of different network connectivity types to the positive effect observed after the off-line reprocessing, we conducted experiments where a specific kind of long-range recurrent connectivity, lateral or top-down, was disabled following off-line regime. Disabling either lateral or top-down connectivity in the recognition test phase results in decrease of the performance boost achieved through preceding off-line reprocessing. Loss of top-down connectivity impairs network performance stronger than loss of lateral connectivity. Interestingly, even if both top-down and lateral connectivity are disabled, there is still a significant advantage over performance of a purely feed-forward network that employed neither lateral nor top-down connectivity during learning. This indicates that off-line reprocessing in the fully recurrent network is able to boost recognition performance even if the network has to rely on its established feed-forward structure only. These findings suggest that while long-range recurrent connectivity is crucial during on-line learning and may help to further enhance the performance boost following off-line reprocessing, it is not necessary for maintaining the major benefit obtained after the off-line regime. Acknowledgements This work was supported by the German Research Foundation (DFG, clinical research unit KFO 219) and by the Bernstein Network Computational Neuroscience (research grant 01GQ0840). References [1] J. Jitsev and C. von der Malsburg. Experience-driven formation of parts-based representations in a model of layered visual memory. Front. Comput. Neurosci., 3:15, Sep 2009 [2] Jitsev, J. and von der Malsburg, C. Off-line memory reprocessing following on-line unsupervised learning strongly improves recognition performance in a hierarchical visual memory. In: The International Joint Conference on Neural Networks (IJCNN), p. 3123-30, 2010, Barcelona, Spain [3] Jitsev J. and von der Malsburg, C. Off-line memory reprocessing in a recurrent neuronal network formed by unsupervised learning. In: Proc. Computational and Systems Neuroscience (COSYNE), 2011, Salt Lake City, USA, doi:10.1038/npre.2011.5776.1 Keywords: Intrinsic Plasticity, memory replay, off-line memory reprocessing, recurrent neural network, unsupervised learning, Visual Cortex, visual object recognition, winner-take-all computation Conference: Bernstein Conference 2012, Munich, Germany, 12 Sep - 14 Sep, 2012. Presentation Type: Poster Topic: Learning, plasticity, memory Citation: Jitsev J (2012). Impact of recurrent connectivity on off-line memory reprocessing in a hierarchical neural network formed by unsupervised learning. Front. Comput. Neurosci. Conference Abstract: Bernstein Conference 2012. doi: 10.3389/conf.fncom.2012.55.00235 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 09 May 2012; Published Online: 12 Sep 2012. * Correspondence: Mr. Jenia Jitsev, Max Planck Institute for Neurological Research, Cortical Networks and Cognitive Functions, Cologne, D-50931, Germany, [email protected] Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Jenia Jitsev Google Jenia Jitsev Google Scholar Jenia Jitsev PubMed Jenia Jitsev Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.