Controlling the Functional Testing of an Operating System

Abstract
Functional testing of operating systems is in transition from a predominantly imprecise art to an increasingly precise science. The process that controls this testing is maturing correspondingly. The laissez-faire approach is giving way to a disciplined approach characterized by rigorous definition of the test plan, systematic control of the test effort, and objective quantitative measurement of the test coverage. This paper describes just such a disciplined test control process, which is composed of five steps: 1) the survey, which establishes the intended extent of testing; 2) the identification, which creates a list of functional variations eligible for testing; 3) the appraisal, which ranks and subsets the eligible variations so that test resources can be directed at those with the higher payoff; 4) the review, which calculates the test coverage of the test case library; and 5) the monitor, which verifies attainment of the planned test coverage. Throughout the test process, specification testing is distinguished from program testing.

This publication has 16 references indexed in Scilit: