Correction algorithm for finite sample statistics
- 1 November 2003
- journal article
- research article
- Published by Springer Nature in The European Physical Journal E
- Vol. 12 (4) , 531-541
- https://doi.org/10.1140/epje/e2004-00025-4
Abstract
Assume in a sample of size M one finds M i representatives of species i with $i = 1\dots N^{*}$ . The normalized frequency $p^*_i\equiv M_i/M$ , based on the finite sample, may deviate considerably from the true probabilities p i . We propose a method to infer rank-ordered true probabilities r i from measured frequencies M i . We show that the rank-ordered probabilities provide important informations on the system, e.g., the true number of species, the Shannon- and the Renyi-entropies.
Keywords
This publication has 11 references indexed in Scilit:
- How to decide whether small samples comply with an equidistributionBiosystems, 2003
- Finite-sample frequency distributions originating from an equiprobability distributionPhysical Review E, 2002
- Estimating the Entropy of DNA SequencesJournal of Theoretical Biology, 1997
- A statistical approach to vehicular trafficPhysica A: Statistical Mechanics and its Applications, 1995
- Guessing probability distributions from small samplesJournal of Statistical Physics, 1995
- ENTROPY, TRANSINFORMATION AND WORD DISTRIBUTION OF INFORMATION-CARRYING SEQUENCESInternational Journal of Bifurcation and Chaos, 1995
- Entropy and Long-Range Correlations in Literary EnglishEurophysics Letters, 1994
- Finite sample effects in sequence analysisChaos, Solitons, and Fractals, 1994
- A New Method to Calculate Higher-Order Entropies from Finite SamplesEurophysics Letters, 1993
- Finite sample corrections to entropy and dimension estimatesPhysics Letters A, 1988