Statistical mechanics of a multilayered neural network

Statistical mechanics is applied to estimate the maximal information capacity per synapse (${\mathrm{\alpha }}_{\mathit{c}}$) of a multilayered feedforward neural network, functioning as a parity machine. For a large number of hidden units, K, the replica-symmetric solution overestimates dramatically the capacity, ${\mathrm{\alpha }}_{\mathit{c}}$∝${\mathit{K}}^2$. However, a one-step replica-symmetry breaking gives ${\mathrm{\alpha }}_{\mathit{c}}$∼lnK/ln2, which coincides with a theoretical upper bound. It is suggested that this asymptotic behavior is exact. Results for finite K are also discussed.