Michael Quinn Patton

Utilization-Focused Evaluation · Developmental Evaluation

Utilization-focused evaluation is widely recognized as one of the most influential approaches in the evaluation profession. What is not widely recognized is that the approach is fundamentally applied sociology. Six particular sociological specializations have influenced my evaluation perspective and practice: applied sociology, sociology of knowled...

Our understanding of programs is enhanced when trained, skilled, and observant evaluators go into the field – the real world where programs are conducted – paying attention to what’s going on, systematically documenting what they see, and reporting what they learn. The article opens by presenting and illustrating twelve reasons for building fieldwo...

Empathy is not bias in qualitative inquiry

Anecdotes can be evidence

Top Ten Developments in Qualitatuve Evaluation over the Last Decade

Session Abstract: These series of papers will discuss advances in application of developmental evaluation in Canadian and other international settings. Four examples and four approaches to developmental evaluation will be discussed. The four problems will include the use of video in developing integrated care for the elderly, the use of development...

Theory and practice are integrated in the human brain. Situation recognition and response are key to this integration. Scholars of decision making and expertise have found that people with great expertise are more adept at situational recognition and intentional about their decision-making processes. Several interdisciplinary fields of inquiry prov...

It has become a standard in major high-stakes evaluations to commission an independent review to determine whether the evaluation meets generally accepted standards of quality. This is called a meta-evaluation. Given the historic importance of the Evaluation of the Paris Declaration, the Management Group commissioned a meta-evaluation of the evalua...

Utilization-focused evaluation involves identifying and working with primary intended users to design and interpret an evaluation. This includes the process of working with primary intended users to render judgments about the extent to which the preponderance of evidence supports a meaningful and useful conclusion about degree to which an intervent...

Valuing is context dependent. That is the cross-cutting theme of the chapters in this issue of New Directions for Evaluation. This chapter highlights how each author emphasizes contextual sensitivity and adaptability in the valuing process and concludes with a personal interpretation of each author's evaluator context to add value to the wisdom and...

In the broad field of evaluation, the importance of stakeholders is often acknowledged and different categories of stakeholders are identified. Far less frequent is careful attention to analysis of stakeholders' interests, needs, concerns, power, priorities, and perspectives and subsequent application of that knowledge to the design of evaluations....

Strategic thinking and planning have long been the focus of management training and organizational development, but strategy is a new unit of analysis for evaluation. The authors examine the increasing attention given to being strategic in the private sector, in government, in philanthropy, and in the not-for-profit sectors. To respond and adapt to...

The Gold Standard Debate Evaluability Assessment for Youth Civic Engagement Initiatives Alternative Youth Civic Engagement Theories of Change Simple Versus Complex Models A Youth Civic Engagement Example: Innovations in Civic Participation Complex, Dynamic Systems Maps to Understand Youth Civic Engagement Outcomes Interdependent and Individualized...

Background: Hundreds of evaluators visit the Claremont Colleges in southern California each year to discuss a wide range of topics related to improving the quality of evaluation practice. Debates between thought leaders in the field have been one of the most popular and informative ways to advance understanding about how best to practice evaluation...

Extension and evaluation share some similar challenges, including working with diverse stakeholders, parallel processes for focusing priorities, meeting common standards of excellence, and adapting to globalization, new technologies, and changing times. Evaluations of extension programs have helped clarify how change occurs, especially the relation...

Advocacy and policy change evaluations focus on policy as the unit of analysis rather than the more traditional program or project. Advocacy evaluation, like all evaluation, is guided by the profession’s Principles and Standards. Advocacy evaluation can be, and should be, utilization-focused. That means focusing the evaluation on intended use by in...

Utilization-focused evaluation (UFE) is based on the principle that an evaluation should be judged by its utility. So no matter how technically sound and methodologically elegant, an evaluation is not truly a good evaluation unless the findings are used. UFE is a framework for enhancing the likelihood that evaluation findings will be used and lesso...

Process use is best understood and used as a sensitizing concept. Judging the concept's meaningfulness through the lens of operationalization misconstrues its utility. This closing chapter also examines what other chapters in this volume reveal about process use as a sensitizing concept.

OCIAL SCIENCE HAS PROVEN ESPECIALLY inept in offering solutions for the great problems of our time—hunger, violence, poverty, hatred. There is a pressing need to make headway with these large chal- lenges and push the boundaries of social inno- v a t ion t o m a ke re a l prog re s s . T he ver y possibility articulated in the idea of making a majo...

Qualitative research analyzes data from direct fieldwork observations, in-depth, open-ended interviews, and written documents. Qualitative researchers engage in naturalistic inquiry, studying real-world settings inductively to generate rich narrative descriptions and construct case studies. Inductive analysis across cases yields patterns and themes...

The evaluation profession has been studying ways of increasing the use of evaluations. Here are some of the things we've learned are important for evaluations to be useful: being clear about intended uses by primary intended users; creating and nurturing a results-oriented, reality-testing culture that supports evaluation use; collaboration in deci...

The focus of this teaching case is the Central Valley Partnership initiative of the James Irvine Foundation. This case traces the varying roles of the external evaluator as the program changed and developed.

The three cases in this volume were developed so that each could be taught unto itself. This chapter suggests other uses for the broader context of evaluation teaching and training.

This chapter reviews the benefits of case teaching, examines evaluation issues that cases can be used to surface, and provides guidance for using the case method in evaluation teaching and training.

Sensitivity to and skillful use of language are core evaluation competencies. The language we use, both among ourselves and with stakeholders, necessarily and inherently shapes perceptions, defines “reality,” and affects mutual understanding. Whatever we seek to understand or do, a full analysis will lead us to consider the words and concepts that...

Let a thousand flowers bloom in cross-cultural evaluations, and let no single flower be exalted as the universal exemplar of beauty.

In this chapter, voices from one multicultural evaluation convey a sense of the struggle involved in developing an evaluation that is sensitive to and empowering of multicultural peoples and perspectives.

The challenges ahead are at least as daunting as those already overcome: demonstrating that monitoring and evaluating findings can make a difference in the lives of people.

Incl. bibl.

Appreciative Inquiry evaluation offers an important new option in evaluation's increasingly vast repertoire of approaches. Its ultimate legitimacy and credibility will depend on how it is applied and used.

This case describes the changing roles of an evaluator as the program he was evaluating changed and developed, and as the needs of his client and primary intended users changed over time. The focus in this case is on the relationship between the external evaluator and the staff of a philanthropic foundation as a major initiative unfolded. The focus...

The purposes of this checklist are to guide evaluators in determining when qualitative methods are appropriate for an evaluative inquiry and factors to consider (1) to select qualitative approaches that are particularly appropriate for a given evaluation’s expected uses and answer the evaluation’s questions, (2) to collect high quality and credible...

Feminist evaluation represents an important option in evaluation, but its legitimacy and credibility depend on the criteria used to define high-quality evaluation practice. This commentary examines how feminist evaluation principles and practices look from the perspectives of five different frameworks, each offering different criteria for judging e...

Discusses teaching and training about evaluation with metaphors. Helping those who study evaluation perceive analogies and metaphors related to evaluation helps them understand truths about the rich diversity of evaluation. (SLD)

The publication of the third edition of Qualitative Research and Evaluation Methods offers the author an opportunity to reflect back over two decades of developments in qualitative inquiry. Major developments include: the end of the qualitative-quantitative debate; the flowering of diverse and competing approaches within qualitative inquiry; the in...

Utilization-focused evaluation begins with the premise that evaluations should be judged by their utility and actual use; therefore, evaluators should facilitate the evaluation process and design any evaluation with careful consideration of how everything that is done, from beginning to end, will affect use. This is consistent with standards develo...

Contenido: Parte I.Cuestiones conceptuales en la investigación cualitativa: Naturaleza de la investigación cualitativa; Temas estratégicos en la investigación cualitativa; Diversidad en la investigación cualitativa: orientaciones teóricas; Aplicaciones cualitativas particulares. Parte II. Diseños cualitativos y recolección de datos: Estudios de dis...

This article identifies and discusses the innovative contributions of the Collaborative Evaluation Fellows Project (CEFP) as a model for evaluation training and organizational development. The importance of the CEFP model to the effectiveness and accountability of nonprofit organizations is best understood within the larger context of the historica...

Varying philosophical and theoretical orientations to qualitative inquiry remind us that issues of quality and credibility intersect with audience and intended research purposes. This overview examines ways of enhancing the quality and credibility of qualitative analysis by dealing with three distinct but related inquiry concerns: rigorous techniqu...

Myths offer a source of norms against which contemporary life events can be interpreted in the ancient tradition of meaning making. The author uses his son’s initiation in the Grand Canyon to illustrate the normative use of myth for qualitative inquiry and interpretation. Ancient and cross-cultural coming of age myths, especially the Grail Legend,...

Professor Stanfield’s article (this issue) stimulated several questions that are briefly discussed. How does the lens of race shape and effect our understandings and actions? How do we talk with each other about race and racism? How do we engage each other on these issues authentically at a time dominated by political correctness? What methods and...

Professor Stanfield's article (this issue) stimulated several questions that are briefly discussed. How does the lens of race shape and effect our understandings and actions? How do we talk with each other about race and racism? How do we engage each other on these issues authentically at a time dominated by political correctness? What methods and...

This article explores some of the opportunities open to evaluators as organizational development practitioners and the particular competencies and comparative advantages evaluators can bring to such initiatives. Examples include: (1) making systems connections between program culture and organizational culture (and therefore between program effecti...

As program evaluation has developed over the last twenty years into a viable profession, its central challenge has become to define what it means to be an evaluator. This debate is fueled less by traditional divisions between academic and service-related professionals, or disagreements over methodologies, than by sharply different visions of the ev...

Incl. index, bibliographical references pp. 387-413, biographical note on the author

Fostering self-determination is the defining focus of empowerment evaluation and the heart of its explicit political and social change agenda. However, empowerment evaluation overlaps participatory, collaborative, stakeholder-involving, and utilization-focused approaches to evaluation in its concern for such issues as ownership, relevance, understa...

Patton continues the debate by identifying three arenas of evaluation practice in which the formative/summative dichotomy appears limited: knowledge-generating evaluations aimed at conceptual rather than instrumental use; developmental evaluation; and use of evaluation processes to support interventions or empower participants. In so doing, the ess...

This article describes lessons learned from practice; lessons learned about how generally accepted guiding principles are translated into effective work with poor families. This article is based on the diverse experiences and evaluations of seven effective parent- ing/family stability programs which were part of The McKnight Foundation’s Families i...

Two reviews consider the program-evaluation standards of the Joint Committee on Standards for Educational Evaluations. This edition makes the complexity, risks, and power of evaluation clear through descriptive writing and case illustrations. The organization and content of the standards document are described. (SLD)

Books applicable to the broad field of program evaluation are reviewed. Books may be reviewed singly or in groups to illuminate similarities and differences in intent, philosophy, and usefulness. Persons with suggestions of books to be reviewed or those who wish to submit a review should contact Dr. Byron R. Burnham, Evaluation Specialist (Cooperat...

Provides an introduction to ethnographic (ETH) and qualitative research approaches that have been applied to education and communication disorders. The article describes the functions of ethnography and the themes that form the foundation for qualitative inquiry generally and ethnography specifically. Qualitative designs are naturalistic in that th...

Refutes three evaluation myths--(1) evaluation is findings; (2) outcomes are hard to measure; and (3) evaluation is an add-on--with three principles: (1) evaluation is a learning process; (2) soft data about important issues are better than hard data about unimportant issues; and (3) meaningful evaluation is integrated into teaching and learning. (...

Explores parallels between humanistic psychology (HP) and qualitative research (QR) methods, describing their common principles. These include (1) stressing each person's uniqueness, (2) beginning with the client's perspective, (3) forming an interpersonal connection, (4) using a holistic perspective, and (5) maintaining a flexible approach. Differ...

Promotion of evaluation as a profession in Australia and New Zealand is discussed. The profession is challenged to make a commitment to utility, quality products and processes, and the development of skilled, trained professionals. Ways of overcoming barriers to the establishment of evaluation as a profession are outlined. (TJH)

PART ONE: CONCEPTUAL ISSUES IN THE USE OF QUALITATIVE METHODS The Nature of Qualitative Inquiry Strategic Themes in Qualitative Methods Variety in Qualitative Inquiry Theoretical Orientations Particularly Appropriate Qualitative Applications PART TWO: QUALITATIVE DESIGNS AND DATA COLLECTION Designing Qualitative Studies Fieldwork Strategies and Obs...

This comment begins with the concern that the Chen and Rossi “theory-driven approach to validity” was offered as if it were universally applicable in evaluation. The purpose of this article is to identify the conditions under which the “theory-driven approach to validity” is relevant. Those conditions are when one is undertaking a major summative e...

Editor's Note: At the AEA meeting in New Orleans in October 1988, the contributions of Sara Miller McCune and Sage Publications were recognized with the presentation of a special Lifetime Contributions Award. AEA's recognition was for the extraordinary contributions Sara, with Sage, has made to the field of evaluation. Her acceptance remarks are pr...

The most useful and possibly the cheapest form of evaluation would yield results coinciding with ongoing management procedures. While this kind of evaluation may never be fully realized, it is possible to provide a continuous stream of evaluative data and analysis by introducing evaluation into project management at its inception. This chapter outl...

Cooperative Extension can no longer be understood primarily in terms of technology transfer approaches. The future Cooperative Extension model incorporates educational, developmental, and problem-solving mandates with technology transfer through “issues programming.” This has important political and substantive implications for Extension. The final...

A knowledgeable and experienced evaluator becomes the principal user of a design to make program evaluation and administration mutually reinforcing.