Analysis of Dynamical Recognizers
- 1 July 1997
- journal article
- Published by MIT Press in Neural Computation
- Vol. 9 (5) , 1127-1142
- https://doi.org/10.1162/neco.1997.9.5.1127
Abstract
Pollack (1991) demonstrated that second-order recurrent neural networks can act as dynamical recognizers for formal languages when trained on positive and negative examples, and observed both phase transitions in learning and interacted function system-like fractal state sets. Follow on work focused mainly on the extraction and minimization of a finite state automaton (FSA) from the trained network. However, such networks are capable of inducing languages that are not regular and therefore not equivalent to any FSA. Indeed, it may be simpler for a small network to fit its training data by inducing such a nonregular language. But when is the network's language not regular? In this article, using a low-dimensional network capable of learning all the Tomita data sets, we present an empirical method for testing whether the language induced by the network is regular. We also provide a detailed "-machine analysis of trained networks for both regular and nonregular languages.Keywords
This publication has 14 references indexed in Scilit:
- Extraction of rules from discrete-time recurrent neural networksNeural Networks, 1996
- Recurrent neural networks and prior knowledge for sequence processing: a constrained nondeterministic approachKnowledge-Based Systems, 1995
- Learning the Initial State of a Second-Order Recurrent Neural Network during Regular-Language InferenceNeural Computation, 1995
- First-Order Recurrent Neural Networks and Deterministic Finite State AutomataNeural Computation, 1994
- The calculi of emergence: computation, dynamics and inductionPhysica D: Nonlinear Phenomena, 1994
- Random DFA's can be approximately learned from sparse uniform examplesPublished by Association for Computing Machinery (ACM) ,1992
- Learning and Extracting Finite State Automata with Second-Order Recurrent Neural NetworksNeural Computation, 1992
- The induction of dynamical recognizersMachine Learning, 1991
- Finite State Automata and Simple Recurrent NetworksNeural Computation, 1989
- Learning representations by back-propagating errorsNature, 1986