Title: Echo state networks with decoupled reservoir states
Abstract: Echo state networks (ESNs) are a novel form of recurrent neural networks that provide an efficient and powerful computational model to approximate dynamic nonlinear systems. Why a random, large, fixed recurrent neural network (reservoir) has such astonishing performance in approximating nonlinear systems remains a mystery. In this paper, we first compare two reservoir scenarios in ESNs, i.e. sparsely versus fully connected reservoirs, and show that the eigenvalues of these reservoir weight matrices have the same limit distribution in the complex plane. We discuss the link between the eigenvalues of the reservoir weight matrix and the ESN approximation ability in a simplified ESN case. We propose a new ESN with decoupled reservoir states, in which the neurons in the reservoir are decoupled into single or pairs of neurons. A reservoir state back-elimination strategy is presented, which not only reduces model complexity but also increases numerical stability when calculating the output weights. The proposed model is tested in a communication channel equalization problem and applied to gene expression time series modeling with very promising results.
Publication Year: 2008
Publication Date: 2008-10-01
Language: en
Type: article
Indexed In: ['crossref']
Access and Citation
Cited By Count: 8
AI Researcher Chatbot
Get quick answers to your questions about the article from our AI researcher chatbot