Content uploaded by Fadhel Kaboub
Author content
All content in this area was uploaded by Fadhel Kaboub on Oct 22, 2018
Content may be subject to copyright.
Book reviews/ The Social Science Journal 41 (2004) 147–161 153
Craig A. Gallet
Department of Economics
California State University at Sacramento
Sacramento, CA 95819-6082, USA
Tel.: +916-278-6223; fax: +916-278-5768
E-mail address: cgallet@csus.edu
doi: 10.1016/j.soscij.2003.10.016
Realistic Evaluation
Ray Pawson and Nick Tilley, Sage, London, 1997, 256 pages
Realistic Evaluation is an original, innovative, and thought-provoking book that I recom-
mend to any scholar interested in program evaluation. Pawson and Tilley provide their readers
with a blueprint for realistic evaluation and display a wealth of detailed examples to illustrate
their approach. The book is an application of critical realism to specific micro contexts. The
authors tried to take critical realism from the metatheory domain to the policy domain by
showing how one can use critical realism in specific policy contexts.
Theauthorsstartwithanintroductorychapterinwhichtheysumupthehistoryofevaluation.
Theythoroughlydelineatefourgenerationsofparadigmsthatprogramevaluationwent through
(i.e., experimentation, pragmatism, constructivism, and comprehensive paradigms). In chapter
two Pawson and Tilley take on the task of comparing and contrasting the “successionist” (or
positivist) and “generative” (or realist) theories of causation. They argue that experimental
evaluation cannot take into account the importance of the generative mechanisms and the
richness of heterogeneous contexts. Their constructive criticism of the successionist approach
inthischapterlaysthegroundworkfor the general framework that the authors invitethe readers
to consider (i.e., realistic evaluation).
Chapter three is where Pawson and Tilley’s critical realist contribution is formulated. The
authors provide a thorough explanation showing how programs work: “causal outcomes follow
from mechanisms acting in contexts.” They even go on to make it so simple as to put it in an
equation format for the readers.
Outcomes =mechanisms +context
Note that the “plus sign” in the above equation is not an additive sign, rather it depicts the
necessary interrelation between contexts and mechanisms that must exist in order for programs
to generate outcomes. This formulation provides the reader with a powerful tool to hypothe-
sise the existence of contexts and mechanisms in the process of explanation and evaluation.
Throughout the book, Pawson and Tilley illustrate their context–mechanism–outcome config-
uration (CMO) with a number of diagrams to help the readers visualize the complexity of the
issues at hand in a simple way. They also provide a wide range of practical interdisciplinary ex-
amples from the fields of sociology, social policy, criminology, health, and education to invite
the readers to consider the method of realistic evaluation in their own field of research. Pawson
and Tilley argue that the goal of the realistic evaluator would be to formulate hypotheses about
154 Book reviews/ The Social Science Journal 41 (2004) 147–161
potential CMO configurations. In order to achieve this goal, the authors declare themselves
to be “whole-heartedly pluralists when it comes to the choice of method.” This leads to the
discovery of what the authors call “the three Ws” or “the realist mantra”; namely: “what works
for whom in what circumstances?”
The authors turn the philosophical notion of critical realism into a methodological notion.
Theydemonstrate howactioniscausedonly if its outcome istriggeredbyamechanism acting in
a context. Pawson and Tilley introduce critical realism as an actual method of experimentation
armedwithcausality, thusbringingcriticalrealismfromanabstractlevelto the levelofresearch
in action (i.e., policy-making).
Pointing to the complexity of the working of programs, Pawson and Tilley recommend
conducting evaluations for subgroups within programs. They do, however, caution the readers
that there might be more than one mechanism at work within each subgroup, thus generating
mixed results.
After outlining the process of designing realistic evaluation in chapter four, Pawson and
Tilley go on to delineate the process of “realistic cumulation” in chapter five. This process,
also described as “theory building,” consists of building a typology of CMO configurations
through continuously and progressively refining the researchers’ understanding of the CMO
patterns by abstracting patterns of contexts, mechanisms, and outcomes during the research
process.
Chaptersixintroduces the readers to therealistdatacollection as well as therealistinterview-
ing methodology; both of which reflect the authors’ philosophy “that knowledge acquisition
is dominated by and organized around the development of realist propositions linking mech-
anisms, contexts and outcomes.” Pawson and Tilley dedicate chapter seven to a rich dialogue
betweenaprogram evaluatorandpolicy-makers regardingasmoke-cessationprogram. In chap-
ter eight, the authors point out the importance of engaging in a teacher-learner cycle between
policy-makers and program evaluators. Pawson and Tilley conclude the book with a summary
chapter for realistic evaluation.
This book is an application of critical realism to specific micro contexts. It is important to
know that, according to critical realism, all micro social contexts are informed and affected
by macro social and philosophical ontology. In my humble opinion, Pawson and Tilley fail to
blend the philosophical ontology of critical realism to its social ontology before moving into
program evaluation. Constructing a consistent and a solid paradigmatic base (philosophical
ontology, epistemology, and social ontology) is a prerequisite for a sound program evaluation.
Needlesstomentionthatin their criticism of positivismforinstance,the authors did not address
any criticism to the positivist social ontology. Having said this, I must confess that Realistic
Evaluation has taken critical realism several steps forward by setting up a blueprint and a
challenging interdisciplinary research agenda for critical realists to pursue. This achievement
I applaud! Fadhel Kaboub
Department of Economics, University of Missouri, Kansas City
5100 Rockhill Road, Kansas City, MO 64110, USA
Tel.: +1-816-235-5648; fax: +1-816-235-2834
E-mail address: kaboubf@umkc.edu
doi: 10.1016/j.soscij.2003.10.017
