Title: Catastrophic Forgetting in Connectionist Networks: Causes, Consequences and Solutions
Abstract: All natural cognitive systems, and, in particular, our own, gradually forget previously learned information. Consequently, plausible models of human cognition should exhibit similar patterns of gradual forgetting old information as new information is acquired. Only rarely (see Box 3) does new learning in natural cognitive systems completely disrupt or erase previously learned information. In other words, natural cognitive systems do not, in general, forget catastrophically. Unfortunately, however, this is precisely what occurs under certain circumstances in distributed connectionist networks. It turns out that the very features that give these networks their much-touted abilities to generalize, to function in the presence of degraded input, etc., are the root cause of catastrophic forgetting. The challenge is how to keep the advantages of distributed connectionist networks while avoiding the problem of catastrophic forgetting. In this article, we examine the causes, consequences and numerous solutions to the problem of catastrophic forgetting in neural networks. We consider how the brain might have overcome this problem and explore the consequences of this solution.
Publication Year: 1999
Publication Date: 1999-01-01
Language: en
Type: article
Access and Citation
Cited By Count: 78
AI Researcher Chatbot
Get quick answers to your questions about the article from our AI researcher chatbot