Stochastic Approximation Algorithms for System Identification, Estimation, and Decomposition of Mixtures
- 1 January 1969
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems Science and Cybernetics
- Vol. 5 (1) , 8-15
- https://doi.org/10.1109/TSSC.1969.300238
Abstract
A stochastic approximation procedure that minimizes a mean-square-error criterion is proposed in this paper. It is applied first to derive an algorithm for recursive estimation of the mean-square-error approximation of the function which relates the input signals and the responses of a memoryless system. The input signals are assumed to be generated at random with an unknown probability density function, and the response is measured with an error which has zero mean and finite variance. A performance index for evaluating the rate of convergence of the algorithm is defined and then the optimal form of the algorithm is derived. It is shown that the least-square-error fit of the measured output signals of the systems offers a recursive formula which is a special case of the proposed algorithm. A recursive formula for estimation of a priori probabilities of the pattern classes using unclassified samples is then presented. The rate of convergence is computed. A minimum square-error estimate of a continuous probability density function is also obtained by the same algorithm.Keywords
This publication has 3 references indexed in Scilit:
- Learning without a teacherIEEE Transactions on Information Theory, 1966
- Recovery of functions from noisy measurements taken at randomly selected points and its application to pattern classificationProceedings of the IEEE, 1966
- Learning to recognize patterns in a random environmentIEEE Transactions on Information Theory, 1962