Does it fit? Is it good? Assessment of scoring systems

Publisher Website
Google Scholar
Abstract
The goal of intensive care is to provide the highest quality care to achieve the best outcomes for patients. Although the random allocation of patients to receive intensive care as part of a randomized controlled trial might in theory be the best method to evaluate its effectiveness, the practical application of such a study design is deemed unethical. The alternative in such a situation is to use observational methods in which the outcome of care patients receive as part of their “natural” treatment is studied. Before drawing inferences from the outcomes of treatment for such groups of patients, the characteristics of the patients admitted to intensive care must be taken into account. The purpose of scoring systems in intensive care, as for many other areas of health care, is to take into account the characteristics of patients that could affect their risk of a particular outcome, irrespective of the effect of the care they receive. Clearly, accounting for such factors, which are outside the control of those providing the care, is essential before any comparison of the outcome of care is possible. For each scoring system, the association between the independent variables (patient characteristics) and the dependent variable (death before discharge from hospital after intensive care) is described in the form of a mathematical model, known as a multiple logistic regression model. To assess any scoring system and model, four steps must be undertaken. First, the goodness-of-fit of the model should be assessed on the development dataset. Second, the goodness-of-fit of the model should be assessed on a subset of the dataset different from the subset used to develop the model. Third, the goodness-of-fit of the model should be assessed on an entirely new dataset. Fourth, the given model should be compared with other existing models on the same dataset. This article briefly describes the concept of scoring systems and their roles, and outlines the requirements for assessment in terms of model validation and goodness-of-fit. It concludes with the outstanding methodological issues in this area.