A Simple Lemma on Greedy Approximation in Hilbert Space and Convergence Rates for Projection Pursuit Regression and Neural Network Training

A general convergence criterion for certain iterative sequences in Hilbert space is presented. For an important subclass of these sequences, estimates of the rate of convergence are given. Under very mild assumptions these results establish an $O(1/ \sqrt n)$ nonsampling convergence rate for projection pursuit regression and neural network training; where $n$ represents the number of ridge functions, neurons or coefficients in a greedy basis expansion.