Evaluation of Prototypical Climate Forecasts: The Sufficiency Relation

Open Access
Publisher Website
Google Scholar
Abstract
The sufficiency relation, originally developed in the context of the comparison of statistical experiments, provides a sound basis for the comparative evaluation of forecasting systems. The importance of this relation resides in the fact that if forecasting system A can be shown to be sufficient for forecasting system B, then all users will find A's forecasts of greater value than B's forecasts regardless of their individual payoff structures. In this paper the sufficiency relation is applied to the problem of comparative evaluation of prototypical climate forecasting systems. The primary objectives here are to assess the basic applicability of the sufficiency relation in this context and to investigate the implications of this approach for the relationships among the performance characteristics of such forecasting systems. The results confirm that forecasting system A is sufficient for forecasting system B when the former uses more extreme probabilities more frequently than the latter. Further, in terms of the relatively simple forecasting systems considered here, it is found that system A may be sufficient for system B even if the former uses extreme forecasts less frequently, provided that A's forecasts are—to a certain degree—more extreme than B's forecasts. Conversely, system A cannot be shown to be sufficient for system B if the former users less extreme forecasts more frequently than the latter. The advantages of the sufficiency relation over traditional performance measures in this context are also demonstrated. Several issues related to the general applicability of the sufficiency relation to the comparative evaluation of climate forecasts are discussed. Possible extensions of this work, as well as some implications of the results for verification procedures and practices in this context, am briefly described.

This publication has 0 references indexed in Scilit: