On Block-Iterative Entropy Maximization

Publisher Website
Google Scholar
Abstract
Entropy maximization under línear equality constraints is consídered and convergence of a special-purpose block-iterative algorithm for its solution is investigated. This algorithm processes in each iterative step a group of constraints. When each block contains a single equation then a well known row-action method, called MART (for “Multiplicative Algebraic Reconstruction Technique”), is obtained. At the other extreme, when all equations are grouped into a single block, a fully simultaneous entropy maximization algorithm is obtained. In the general ease the algorithm permits assignment of weights to the equations within each block to reflect any a priori knowledge, which might be available from the modelling of some real-world process, about their relative importance. The blocks are processed in an almost cyclic order and, in contrast with Bregman’s method, the íterative step does not require inner-loop iterations but is rather represented by a closed form formula. The iterative step of the algorithm resembles that of the algorithm of Darroch and Ratcliff (The Annals of Math. Statistics 43: 1470–1480, 1972). However, additional insight into the nature of the algorithm, offered by a primal-dual approach to its analysis, results here in modifications of both theoretical and practical significance.