Stability in contractive nonlinear neural networks

Abstract
We consider models of the form mu chi = -x + p + WF(x) where x = x(t) is a vector whose entries represent the electrical activities in the units of a neural network. W is a matrix of synaptic weights, F is a nonlinear function, and p is a vector (constant or slowly varying over time) of inputs to the units. If the map WF(x) is a contraction, then the system has a unique equilibrium which is globally asymptotically stable; consequently the network acts as a stable encoder in that its steady-state response to an input is independent of the initial state of the network. We consider some relatively mild restrictions on W and F(x), involving the eigenvalues of W and the derivative of F, that are sufficient to ensure that WF(x) is a contraction. We show that in the linear case with spatially-homogeneous synaptic weight, the eigenvalues of W are simply related to the Fourier transform of the connection pattern. This relation makes it possible, given cortical activity patterns as measured by autoradiographic labeling, to construct a pattern of synaptic weights which produces steady state patterns showing similar frequency characteristics. Finally, we consider the relationships, in the spatial and frequency domains, between the equilibrium of the model and that of the linear approximation mu chi = -x + p + Wx; this latter equilibrium can be computed easily from p in the homogeneous case using discrete Fourier transforms.