Online stabilization of block-diagonal recurrent neural networks
- 1 January 1999
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 10 (1) , 167-175
- https://doi.org/10.1109/72.737503
Abstract
Deals with a discrete-time recurrent neural network (DTRNN) with a block-diagonal feedback weight matrix, called the block-diagonal recurrent neural network (BDRNN), that allows a simplified approach to online training and to address network and training stability issues. The structure of the BDRNN is exploited to modify the conventional backpropagation through time (BPTT) algorithm. To reduce its storage requirement by a numerically stable method of recomputing the network state variables. The network and training stability is addressed by exploiting the BDRNN structure to directly monitor and maintain stability during weight updates by developing a functional measure of system stability that augments the cost function being minimized. Simulation results are presented to demonstrate the performance of the BDRNN architecture, its training algorithm, and the stabilization methodKeywords
This publication has 21 references indexed in Scilit:
- Globally asymptotical stability of discrete-time analog neural networksIEEE Transactions on Neural Networks, 1996
- Relating Real-Time Backpropagation and Backpropagation-Through-Time: An Application of Flow Graph InterreciprocityNeural Computation, 1994
- Absolute stability conditions for discrete-time recurrent neural networksIEEE Transactions on Neural Networks, 1994
- Continuous-time temporal back-propagation with adaptable time delaysIEEE Transactions on Neural Networks, 1993
- Progress in supervised neural networksIEEE Signal Processing Magazine, 1993
- An Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network TrajectoriesNeural Computation, 1990
- Identification and control of dynamical systems using neural networksIEEE Transactions on Neural Networks, 1990
- Learning State Space Trajectories in Recurrent Neural NetworksNeural Computation, 1989
- Recurrent Backpropagation and the Dynamical Approach to Adaptive Neural ComputationNeural Computation, 1989
- A parametric approach to the realization of second-order digital filter sectionsIEEE Transactions on Circuits and Systems, 1985