Learning the Initial State of a Second-Order Recurrent Neural Network during Regular-Language Inference

1 September 1995

journal article
Published by MIT Press in Neural Computation

Vol. 7 (5) , 923-930
https://doi.org/10.1162/neco.1995.7.5.923

Abstract

Recent work has shown that second-order recurrent neural networks (2ORNNs) may be used to infer regular languages. This paper presents a modified version of the real-time recurrent learning (RTRL) algorithm used to train 2ORNNs, that learns the initial state in addition to the weights. The results of this modification, which adds extra flexibility at a negligible cost in time complexity, suggest that it may be used to improve the learning of regular languages when the size of the network is small.

Keywords

NEURAL
FLEXIBILITY
REGULAR LANGUAGES
RTRL
SECOND ORDER
INITIAL STATE
INFER
EXTRA
NEGLIGIBLE

This publication has 1 reference indexed in Scilit:

Induction of Finite-State Languages Using Second-Order Recurrent Networks
Neural Computation, 1992