Asymptotic Properties of Distributed and Communicating Stochastic Approximation Algorithms

Publisher Website
Google Scholar
Abstract
The asymptotic properties of extensions of the type of distributed or decentralized stochastic approximation proposed in [1] are developed. Such algorithms have numerous potential applications in decentralized estimation, detection and adaptive control, or in decentralized Monte Carlo simulation for system optimization (where they can exploit the possibilities of parallel processing). The structure involves several isolated processors (recursive algorithms) that communicate to each other asynchronously and at random intervals. The asymptotic (small gain) properties are derived. The communication intervals need not be strictly bounded, and they and the system noise can depend on the (communicating) system state. State space constraints are also handled. In many applications, the dynamical terms are merely indicator functions, or have other types of discontinuities. The “typical” such case is also treated, as is the case where there is noise in the communication. The linear stochastic differential equation satisfied by the (interpolated) asymptotic normalized error sequence is derived, and issued to compare alternative algorithms and communication strategies. Weak convergence methods provide the basic tools.