Weight decay and resolution effects in feedforward artificial neural networks

1 January 1991

journal article
Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks

Vol. 2 (1) , 168-170
https://doi.org/10.1109/72.80308

Abstract

Results are presented from a preliminary study on the effects of weight decay and resolution on the performance of typical three-layer, feedforward neural networks. Two types of decay are investigated, unilateral decay toward the most negative weight (unipolar) and bilateral decay toward the median or zero weight value (bipolar), and compared with Gaussian weight perturbations. This analysis is pertinent to the area of VLSI-based network implementations with analog weight storage. The results show that, if weight decay is unavoidable, bipolar decay achieves an order-of-magnitude better performance than unipolar and that the weight resolution required in actual implementations of feedforward, connectionist hardware is higher than predicted by computer simulations of network responses to random or Gaussian weight perturbations.

Keywords

This publication has 9 references indexed in Scilit:

A dynamic CMOS multiplier for analog neural network cells
Published by Institute of Electrical and Electronics Engineers (IEEE) ,2002
Sensitivity of feedforward neural networks to weight errors
IEEE Transactions on Neural Networks, 1990
Programmable analog vector-matrix multipliers
IEEE Journal of Solid-State Circuits, 1990
An electrically trainable artificial neural network (ETANN) with 10240 'floating gate' synapses
Published by Institute of Electrical and Electronics Engineers (IEEE) ,1989
A ±5-V CMOS analog multiplier
IEEE Journal of Solid-State Circuits, 1987
Analogue circuits for variable-synapse electronic neural networks
Electronics Letters, 1987
An artificial neural network integrated circuit based on MNOS/CCD principles
AIP Conference Proceedings, 1986
A 20-V four-quadrant CMOS analog multiplier
IEEE Journal of Solid-State Circuits, 1985
A four-quadrant NMOS analog multiplier
IEEE Journal of Solid-State Circuits, 1982