Multidimensional density shaping by sigmoids
- 1 September 1996
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 7 (5) , 1291-1298
- https://doi.org/10.1109/72.536322
Abstract
An estimate of the probability density function of a random vector is obtained by maximizing the output entropy of a feedforward network of sigmoidal units with respect to the input weights. Classification problems can be solved by selecting the class associated with the maximal estimated density. Newton's optimization method, applied to the estimated density, yields a recursive estimator for a random variable or a random sequence. A constrained connectivity structure yields a linear estimator, which is particularly suitable for "real time" prediction. A Gaussian nonlinearity yields a closed-form solution for the network's parameters, which may also be used for initializing the optimization algorithm when other nonlinearities are employed. A triangular connectivity between the neurons and the input, which is naturally suggested by the statistical setting, reduces the number of parameters. Applications to classification and forecasting problems are demonstrated.Keywords
This publication has 10 references indexed in Scilit:
- An Information-Maximization Approach to Blind Separation and Blind DeconvolutionNeural Computation, 1995
- Using mutual information for selecting features in supervised neural net learningIEEE Transactions on Neural Networks, 1994
- Mapping neural network derived from the parzen window estimatorNeural Networks, 1992
- Neural Network Classifiers Estimate Bayesian a posteriori ProbabilitiesNeural Computation, 1991
- A neural network approach to statistical pattern classification by 'semiparametric' estimation of probability density functionsIEEE Transactions on Neural Networks, 1991
- Minimum class entropy: A maximum information approach to layered networksNeural Networks, 1989
- Learning in Artificial Neural Networks: A Statistical PerspectiveNeural Computation, 1989
- How to Generate Ordered Maps by Maximizing the Mutual Information between Input and Output SignalsNeural Computation, 1989
- Matrix AnalysisPublished by Cambridge University Press (CUP) ,1985
- Do Robust Estimators Work with Real Data?The Annals of Statistics, 1977