Testing the Accuracy, Usefulness, and Significance of Probabilistic Choice Models: An Information-Theoretic Approach

Publisher Website
Google Scholar
Abstract
Disaggregate demand models predict the choice behavior of individual consumers. But while such models predict choice probabilities (0 < p < 1), they must be tested against (0, 1) choice behavior. This paper uses information theory to derive three complementary tests that help analysts select a “best” disaggregate model. “Usefulness” measures the percentage of uncertainty (entropy) explained by the information the model provides. It provides theoretic rigor and intuitive appeal to the commonly used likelihood ratio index and leads to important practical extensions. “Accuracy” is a new two-tailed normal test that determines whether the (0, 1) observations are reasonable under the hypothesis that the model is valid. “Significance” is the standard chi-squared test to determine whether a null model can be rejected. This paper also extends the information test to examine the relationships among successively more powerful null hypotheses. For example, in a logit model one can quantify (1) the contribution due to knowing aggregate market shares, (2) the incremental contribution due to knowing choice set restrictions, and (3) the final incremental contribution due to the explanatory variables. Further extensions provide “explanable uncertainty” measures applicable if choice frequencies are observed. Market research and transportation analysis empirical examples are given.

This publication has 0 references indexed in Scilit: