Construction of a paper-and-pencil Test of Basic Scientific Literacy based on selected literacy goals recommended by the American Association for the Advancement of Science

Publisher Website
Google Scholar
Abstract
We describe the construction of a paper-and-pencil Test of Basic Scientific Literacy (TBSL) specifically designed for high-school leavers entering technikon and university in South Africa. The `true-false-don't know' scientific literacy test-items are based on a pool of 472 items developed previously from selected literacy goals recommended by the AAAS in Science for all Americans. Test-items were pilot-tested on 625 technikon and university students and were included in the 110-item TBSL on the basis of item discrimination, item difficulty and student feedback. The TBSL consists of three subtests based on Jon Miller's three constitutive dimensions of scientific literacy: the nature of science (22 items); science content knowledge (72 items); and the impact of science and technology on society (16 items). About 260 South Africa-based members of various South African professional science and engineering associations participated in setting a performance standard for each of the three dimensions of scientific literacy. These standards were validated using a `contrasting groups' approach. The internal consistency of the individual TBSL subtests, and the reliability of mastery-nonmastery classification decisions based on the performance standard, was found to be about 0.80. The reliability of the overall 110-item TBSL was 0.95.