ON THE RELATION BETWEEN CATASTROPHIC INTERFERENCE AND GENERALIZATION IN CONNECTIONIST NETWORKS

Publisher Website
Google Scholar
Abstract
Catastrophic interference refers to the frequent inability of connectionist networks to retain old memories in the presence of new information. Although solutions to the catastrophic interference problem exist, most involve the creation of semi-distributed (only partially overlapping) representations. This has given rise to the new concern that other, desirable properties that derive from the distributed nature of networks might be lost in exchange for reduced interference. For example, it has been feared that networks may lose the ability to generalize if interference is reduced through the creation of semi-distributed representations. This article examines the close conceptual relation between interference and generalization, and reports results from simulations that examined forgetting and generalization in a prototype learning situation. Two classes of solutions to catastrophic interference were investigated: Interference-reduction techniques that rely on rearranging weights into a more robust pattern were found to leave generalizability intact. An alternative technique that reduces interference by reducing the overlap between receptive fields of static hidden units was found to trade off resistance to forgetting against a reduced ability to generalize.

This publication has 0 references indexed in Scilit: