Characterization and Computation of Optimal Distributions for Channel Coding
- 27 June 2005
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Theory
- Vol. 51 (7) , 2336-2351
- https://doi.org/10.1109/tit.2005.850108
Abstract
This paper concerns the structure of capacity-achieving input distributions for stochastic channel models, and a renewed look at their computational aspects. The following conclusions are obtained under general assumptions on the channel statistics. i) The capacity-achieving input distribution is binary for low signal-to-noise ratio (SNR). The proof is obtained on comparing the optimization equations that determine channel capacity with a linear program over the space of probability measures. ii) Simple discrete approximations can nearly reach capacity even in cases where the optimal distribution is known to be absolutely continuous with respect to Lebesgue measure. iii) A new class of algorithms is introduced based on the cutting-plane method to iteratively construct discrete distributions, along with upper and lower bounds on channel capacity. It is shown that the bounds converge to the true channel capacity, and that the distributions converge weakly to a capacity-achieving distribution.Keywords
This publication has 28 references indexed in Scilit:
- Capacity-Achieving Probability Measure for Conditionally Gaussian Channels With Bounded InputsIEEE Transactions on Information Theory, 2005
- On Fixed Input Distributions forNoncoherent Communication Over High-SNR Rayleigh-Fading ChannelsIEEE Transactions on Information Theory, 2004
- Geometric Programming Duals of Channel Capacity and Rate DistortionIEEE Transactions on Information Theory, 2004
- Capacity bounds via duality with applications to multiple-antenna systems on flat-fading channelsIEEE Transactions on Information Theory, 2003
- Fading channels: how perfect need "perfect side information" be?IEEE Transactions on Information Theory, 2002
- The capacity of discrete-time memoryless Rayleigh-fading channelsIEEE Transactions on Information Theory, 2001
- The capacity of average and peak-power-limited quadrature Gaussian channelsIEEE Transactions on Information Theory, 1995
- On channel capacity per unit costIEEE Transactions on Information Theory, 1990
- Hypothesis testing and information theoryIEEE Transactions on Information Theory, 1974
- Computation of channel capacity and rate-distortion functionsIEEE Transactions on Information Theory, 1972