Relating Real-Time Backpropagation and Backpropagation-Through-Time: An Application of Flow Graph Interreciprocity

1 March 1994

journal article
Published by MIT Press in Neural Computation

Vol. 6 (2) , 296-306
https://doi.org/10.1162/neco.1994.6.2.296

Abstract

We show that signal flow graph theory provides a simple way to relate two popular algorithms used for adapting dynamic neural networks, real-time backpropagation and backpropagation-through-time. Starting with the flow graph for real-time backpropagation, we use a simple transposition to produce a second graph. The new graph is shown to be interreciprocal with the original and to correspond to the backpropagation-through-time algorithm. Interreciprocity provides a theoretical argument to verify that both flow graphs implement the same overall weight update.

Keywords

This publication has 4 references indexed in Scilit:

Neural networks for self-learning control systems
IEEE Control Systems Magazine, 1990
Identification and control of dynamical systems using neural networks
IEEE Transactions on Neural Networks, 1990
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Neural Computation, 1989
Inter-reciprocity applied to electrical networks
Applied Scientific Research, Section B, 1957