Some inequalities for information divergence and related measures of discrimination
Top Cited Papers
- 1 July 2000
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Theory
- Vol. 46 (4) , 1602-1609
- https://doi.org/10.1109/18.850703
Abstract
Inequalities which connect information divergence with other measures of discrimination or distance between probability distributions are used in information theory and its applications to mathematical statistics, ergodic theory, and other scientific fields. We suggest new inequalities of this type, often based on underlying identities. As a consequence, we obtain certain improvements of the well-known Pinsker inequality. Our study depends on two measures of discrimination, called capacitory discrimination and triangular discrimination. The discussion contains references to related research and comparison with other measures of discrimination, e.g., Ali-Silvey-Csiszar (1996, 1966) divergences and, in particular, the Hellinger distance.Keywords
This publication has 21 references indexed in Scilit:
- Two approaches to grouping of data and related disparity statisticsCommunications in Statistics - Theory and Methods, 1998
- Bounding $\bar{d}$-distance by informational divergence: a method to prove measure concentrationThe Annals of Probability, 1996
- Statistical information and discriminationIEEE Transactions on Information Theory, 1993
- Rates of convergence for minimum contrast estimatorsProbability Theory and Related Fields, 1993
- Divergence measures based on the Shannon entropyIEEE Transactions on Information Theory, 1991
- A NEW DIRECTED DIVERGENCE MEASURE AND ITS CHARACTERIZATION∗International Journal of General Systems, 1990
- On the selection of measures of distance between probability distributionsInformation Sciences, 1982
- Sharper lower bounds for discrimination information in terms of variation (Corresp.)IEEE Transactions on Information Theory, 1975
- A Note on Hoeffding's InequalityJournal of the American Statistical Association, 1969
- On Information and SufficiencyThe Annals of Mathematical Statistics, 1951