The What Works Clearinghouse (WWC) is a federally funded program established in 2002 to evaluate educational interventions and provide reliable and trustworthy summary ratings and reports of their effectiveness. Yet, some of their reports directly contradict the conclusions of the research literature, giving positive ratings to a program that scholars have found to be ineffective (Reading Recovery) and failing to give positive ratings to programs the research literature has found to be highly effective (Direct Instruction). This article uses a comparative case study approach to examine how these contradictory conclusions developed. It contrasts the methods used by the scholarly world and the WWC to summarize literature and their conclusions about the two curricula. It then examines errors in the three major steps of the WWC review process: 1) compiling lists of studies to examine, 2) applying WWC criteria to select studies for further analysis, and 3) interpreting and reporting the results of the studies. Extensive problems are documented at each step, systematically favoring RR and not favoring DI. Implications of the results are briefly discussed.
Click here to view all documents.