A general procedure for obtaining paired comparisons from multiple rank orders

Abstract
From a theoretical point of view, paired comparisons and the law of comparative judgment provide an excellent approach to the problem of psychological measurement. However, if a reasonably large number of stimuli are to be investigated, paired comparisons become extremely time-consuming and fatiguing to the subjects. A balanced incomplete block design, requiring multiple rank order judgments for each subject, provides an efficient experimental method for obtaining paired comparisons judgments. Features of the analysis proposed for this design are discussed in detail. A program for the analysis is available for the IBM 650 electronic computer.