Abstract
This article analyzes mathematics achievement data from the Second International Mathematics Study (SIMS; Crosswhite, Dossey, Swafford, McKnight, & Cooney, 1985) in which U.S. students are measured at the beginning and end of eighth grade. The aim of the article is to address some substantive analysis questions in the SIMS data and show the potential of multilevel factor analysis methodology. Issues related to between‐ and within‐class decomposition of achievement variance and the change of this decomposition over the course of the eighth grade are studied. As a starting point, random effects ANOVA is considered for each achievement score. Each score contains a large amount of measurement error. The effects of unreliability on variance decomposition are shown with the help of a multilevel factor analysis model. Unreliability has severely distorting effects on this type of ANOVA while multilevel factor analysis gives results corresponding to what would be obtained with perfectly reliable scores.