Unpacking the ‘black box’: the importance of process data to explain outcomes

Publisher Website
Google Scholar
Abstract
Many of those who advance criticisms of randomized controlled trials (RCTs) conclude that RCTs of social interventions are rarely, if ever, appropriate. However, it should be possible to combine the strengths of a randomized approach with the ‘theory-driven’ or ‘realistic’ approach that has been advocated by Pawson and Tilley. The essence of this approach is to base the outcome evaluation on a thorough understanding of the process of the intervention and to complement it with the collection of good process data, in particular using Pawson and Tilley's formula: ‘Outcomes = Mechanism + Context’. In short, an unintelligible intervention, or ‘black box’, has to be unpacked. This is illustrated through two RCTs currently under analysis in Scotland and Tanzania. Following an outline of the projects, this chapter discusses four key intervention factors that can be of critical importance in interpreting outcome evaluations. These factors are: (1) the extent and quality of intervention delivery; (2) the mechanism; (3) the context; and, (4) the response of the target group. Finally, the chapter considers some key problems with process evaluations and how process and outcome data can be integrated.