J. Bradley Cousins

• PhD
• Professor Emeritus at University of Ottawa

BACKGROUND Subspecialist training is an important part of developing human resources for health and for some learners, may require taking place in another, higher-resourced country. Despite effective learning of skills and knowledge in a different, more highly resourced context, transfer of these skills and knowledge back to a more poorly resourced...

This practice note provides a review of Dedoose®, a software analysis platform capable of mixed methods analysis, based on a case application. The software has been used infrequently by the evaluation community but has significant potential, particularly for those interested in “quantizing” qualitative data. The evaluation of the World Federation o...

Research on program evaluation in the context of patient/family engagement is highly limited. Logically, collaborative approaches to evaluation (CAE) align well with patient/family-centred care philosophy and hold great promise in leveraging desired outcomes. In this longitudinal case intervention study, we tracked a one-year collaborative initiati...

Research and theory on evaluation capacity building (ECB) and organizational evaluation capacity have been developing at a good pace over the past decade. On the other hand, there is a paucity of research on the nature and consequences of organizational evaluation policy. Evaluation policies are developed and implemented ultimately to inform and sh...

Evaluation policy has been identified as an important means of shaping and influencing organizational evaluation practice, yet, to date, little empirical research has been conducted to deepen our understanding of this relationship. The purpose of this study was to illuminate evaluation policy’s role in leveraging organizational capacity to do and u...

As a deeply relational, dialogic, engaged and political approach, the collaborative research context is fairly unique in the world of research, and as such opens up an entirely new set of ethical considerations that serve to differentiate it from other approaches, repositioning ethics as a fundamental rationale for collaborative inquiry. In this pa...

As a deeply relational, dialogic, engaged and political approach, the collaborative research context is fairly unique in the world of research, and as such opens up an entirely new set of ethical considerations that serve to differentiate it from other approaches, repositioning ethics as a fundamental rationale for collaborative inquiry. In this pa...

Background: Research on the role and effects of evaluation policy is limited. Some research on the policy’s role in enhancing organizational evaluation capacity (EC) is beginning to accrue but to date it has been limited largely to global Western evaluation contexts. Purpose: We employed an ecological conceptual framework arising from our own empir...

Editor J. Bradley Cousins and colleagues meet the needs of evaluators seeking to implement collaborative and participatory approaches to evaluation in Collaborative Approaches to Evaluation: Principles in Use. Using a multi-phase empirical process to develop and validate a set of principles to guide collaborative approaches to evaluation, the book...

Differentiated Instruction (DI) continues to be a framework adopted within K-12 settings as it supports planning for the needs of diverse groups of students. However the research base exploring the implementation of DI is limited. The current study explores the relationships between individual and contextual variables on teachers’ use of DI practic...

This article reports findings from a research program exploring the role of mediation in an "adaptive learning" process through study of developmental evaluation (DE). Our study focuses on how mediators might influence the relationships between components of a social learning system and the implications for adaptive learning. Specifically, we focus...

Social innovations (SIs) frequently bring previously unrelated actors, ideas, and practices together in new configurations with the goal of addressing social needs. However, the dizzying variety of definitions of SI and their dynamic, exploratory character raise dilemmas for evaluators tasked with their evaluations. This article is based on a syste...

J. Bradley Cousins traces some key advice from his mother and the example of his father through serendipitous encounters with mentors and work opportunities as instrumental in helping him develop a practical, utilization and problem-solving orientation to the participatory evaluation approach he has used throughout his informal and professional eva...

Social innovation has gained prominence as a way to address social problems and needs. Evaluators and social innovators are conceptualizing and implementing evaluation approaches for social innovation contexts; however, no systematic effort has yet been made to explore and assess the overlap between evaluation and social innovation based on the emp...

Social innovation has gained prominence as a way to address social problems and needs. Evaluators and social innovators are conceptualizing and implementing evaluation approaches for social innovation contexts; however, no systematic effort has yet been made to explore and assess the overlap between evaluation and social innovation based on the emp...

The purpose of this study was to test the construct validity of the Evaluation Capacity in Organizations Questionnaire (ECOQ). Conceptually, the ECOQ examines the role of evaluation in organizational development and, most notably in organizational learning. In this model, evaluation capacity building (ECB) initiatives are assumed to contribute to t...

Despite efforts for greater patient engagement in health care quality improvement, evaluation practice in this context remains mostly conventional and noncollaborative. Following an explication of this problem we discuss relevant theory and research on patient-centred care (PCC) and patient engagement and then con-sider potential benefits of collab...

Social innovation (SI) is billed as a new way to address complex social problems. Interest in SI has intensified rapidly in the last decade, making it an important area of practice for evaluators, but a difficult one to navigate. Learning from developments in SI and evaluation approaches applied in SI contexts is challenging because of ‘fuzzy’ conc...

Making valid inferences regarding the impact of a program is fundamentally a question of construct validity, the adequacy and appropriateness of causal inferences. A necessary first step in construct validation, the process used to legitimize causal inferences, is to define and represent program theory. However, how do we do that when a program is...

How do evaluators using collaborative approaches to evaluation (CAE) define success? This is the core question being asked in a further analysis of data from our previous work ( Cousins, Whitmore, & Shulha, 2013 ; Shulha et al., 2016) that developed a set of evidence-based principles to guide collaborative evaluation practice. Probing data from 320...

This paper draws on empirical studies on evaluation practices in social innovation contexts published between 2000-2015. We focus on mapping the landscape of the current empirical knowledge base, identifying forces and influences driving evaluation practices, and how these practices affect social innovations. We identify implications for practice a...

Trough a guided discussion, this article explores a fve-year cross-cultural evaluation relationship comprising multiple projects involving an evaluator from Canada and a group of Indian colleagues working on educational reform in India. Te initiative was funded through a multilateral consortium of donors and involved Western evaluation specialists...

Objectives: In this exploratory study, we wanted to know how evaluators differentiate collaborative approaches to evaluation (CAE) perceived to be successful from those perceived to be less-than-successful. Method: In an online questionnaire survey, we obtained 320 responses from evaluators who practice CAE (i.e., evaluations on which program st...

Th rough a guided discussion, this article explores a five-year cross-cultural evaluation relationship comprising multiple projects involving an evaluator from Canada and a group of Indian colleagues working on educational reform in India. Th e initiative was funded through a multilateral consortium of donors and involved Western evaluation special...

This article introduces a set of evidence-based principles to guide evaluation practice in contexts where evaluation knowledge is collaboratively produced by evaluators and stakeholders. The data from this study evolved in four phases: two pilot phases exploring the desirability of developing a set of principles; an online questionnaire survey that...

Calls for more and better research on evaluation (RoE) have been sounded for some 20 years now and the recent appearance of several significant reviews of empirical research suggests that interest in RoE is on the rise. Although many empirical studies on evaluation and syntheses of such studies result in implications for evaluation policy and pract...

This article is a review and integration of evaluation utilization literature with a new focus on the use of technology to increase evaluation utility. Scholarship on evaluation utilization embodies one of the major and ongoing quandaries in the evaluation profession: What constitutes usefulness and relevance to stakeholders? We think that a constr...

This article is a review and integration of evaluation utilization literature with a new focus on the use of technology to increase evaluation utility. Scholarship on evaluation utilization embodies one of the major and ongoing quandaries in the evaluation profession: What constitutes usefulness and relevance to stakeholders? We think that a constr...

It can be argued that participatory evaluation and social pedagogy share several interconnections given their respective developments and evolutions. In this paper we explore affinities between these two domains of inquiry in order to understand points of intersection and departure at deeper levels that has occurred to date. Following a recent revi...

It can be argued that participatory evaluation and social pedagogy share several interconnections given their respective developments and evolutions. In this paper we explore affinities between these two domains of inquiry in order to understand points of intersection and departure at deeper levels that has occurred to date. Following a recent revi...

The construct of organizational evaluation capacity is a concept that is receiving increasing attention in theoretical and research-based literature. It is situated within a stream of inquiry that has come to be known as evaluation capacity building (ECB). This chapter reviews evolving conceptions of ECB and recent research and theory in the area....

Research on organizational evaluation capacity building (ECB) has focused very much on the capacity to do evaluation, neglecting organizational demand for evaluation and the capacity to use it. This qualitative multiple case study comprises a systematic examination of organizational capacity within eight distinct organizations guided by a common co...

Chapter 2 of this volume provided a close look at the evaluation capacity of eight distinct organizations. The organizations were selected on the basis of their interest in and commitment to evaluation as leverage for program and organizational change. Having examined each of the organizations in terms of context, evaluation drivers, evaluation cap...

We provide some concluding reflections about the sustainability of organizational capacity for evaluation arising from informal communications with some of our case organizations.

In this paper, we focus on participatory evaluation in the context of international development and specifically on the emerging empirical knowledge base. In a prior review and critique of research on participatory evaluation (Cousins and Chouinard 2012), we examined 121 studies, with only 21 (17 %) situated in development contexts. However, the ci...

Research on evaluation capacity is limited although a recent survey article on integrating evaluation into the organizational culture (Cousins, Goh, Clark, & Lee, 2004) revealed that interest in the topic is increasing. While knowledge about building the capacity to do evaluation has developed considerably, less is understood about building the org...

Program evaluation is becoming increasingly important in pediatric rehabilitation settings that adhere to the family-centered service (FCS) philosophy. However, researchers know little about the specific evaluation activities occurring in these settings or the extent to which evaluators/service providers uphold FCS in their program evaluation activ...

Organizational evaluation capacity building has been a topic of increasing interest in recent years. However, the actual dimensions of evaluation capacity have not been clearly articulated through empirical research. This study sought to address this gap by identifying the key dimensions of evaluation capacity in Canadian federal government organiz...

Purpose: This study explored ways in which program evaluation activities in pediatric rehabilitation settings can become congruent with family-centered service (FCS) philosophy. Methods: Two Canadian pediatric rehabilitation centers participated in this study, which included focus groups with staff members and interviews with parents. Results:...

In this article, we critique two recent theoretical developments about collaborative inquiry in evaluation—using logic models as a means to understand theory, and efforts to compartmentalize versions of collaborative inquiry into discrete genres—as a basis for considering future direction for the field. We argue that collaborative inquiry in evalua...

In this short paper I reflect on the application of a program logic modeling approach to visualizing evaluation theory as it relates to practical participatory evaluation (PPE). The work of Hansen, Alkin and associates presented in this volume is well-thought out, carefully done, rigorous, and important. I found that their application of the approa...

Explanation seeking in academic settings has been characterized in recent research as an achievement related strategy. The impact of explanation seeking on achievement and attitudes was examined in three studies (N = 65 Grade 7 and 8 students and 96 Grade 9 and 10 students) in which the participants learned how to solve correlational reasoning prob...

As a fairly new and emergent construct, there remain many gaps in our knowledge about how to integrate notions of culture and cultural context into evaluation theory and practice, as well as gaps in our knowledge about how to conduct and implement evaluations in immigrant and indigenous communities. In this article, the authors provide a comprehens...

This paper provides a reflective account of a consultation process on professional designations for evaluators initiated and coordinated by the Canadian Evaluation Society (CES). Described are (1) the forces leading CES to generate discussion and debate about professional designations for Canadian evaluators, (2) the process of developing and imple...

This article provides commentary for the thematic segment titled "Applying a variety of methods to the evaluation of various efforts aimed at transferring knowledge generated from research." The authors revisit arguments supporting inquiry that takes up the challenge of connecting cognate fields of evaluation utilization and the broader domain of k...

This article describes the planning of a multi-agency and multi-stakeholder evaluation of the implementation of supportive housing. The Ottawa HousingPlus Collaborative Communities project is a collaboration between university-based evaluators and members of a local network of supportive housing providers. The objectives of the initiative are (1) t...

According to the literature published on the topic, the development of an organization's capacity to do and use evaluation typically follows four stages: traditional evaluation, characterized by externally mandated evaluation activities; awareness and experimentation, during which organizational members learn about evaluation and its benefits by pa...

Despite increasing interest in the integration of evaluative inquiry into organizational functions and culture, the availability of empirical research addressing organizational capacity building to do and use evaluation is limited. This exploratory descriptive survey of internal evaluators in Canada asked about evaluation capacity building in the c...

The special issue is devoted to the examination of organizational capacity for evaluation and evaluation capacity building (ECB) through empirical inquiry. The compilation consists of two quantitative surveys of evaluators and seven single or multiple case studies across a broad array of organizations in a diverse contexts (e.g., east-central Ontar...

An earlier study by Chakrabarty and Rogé evaluated the dimensionality of the Organizational Learning Survey for assessing the learning capability of organizations developed by Goh and Richards. In this paper, the survey was re-evaluated for unidimensionality using both exploratory and confirmatory factor analysis. Retaining 18 of the original 21 it...

The authors draw on empirical research on evaluation to consider how process use has been operationalized to date and what this may mean for ongoing research in this area.

Much of program evaluation is concerned with understanding and improving social programs so that they are ultimately more responsive and more reflective of program participant needs. At the same time, these programs exist and are embedded within specific social, cultural and historical contexts which impact program development, implementation, and...

In this commentary, a longtime admirer of Cousins and Whitmore discusses why their 1998 article on participatory evaluation made an important contribution to the field. This chapter posits two principal streams of participatory evaluation, practical participatory evaluation and transformative participatory evaluation, and compares them on a set of...

The problem of bullying has gained attention in recent years. Schools are investing significant resources into antibullying programs, despite scant evidence of program effectiveness. Three hundred ninety-five schools responded to a province-wide survey, whose purpose was (a) to describe the evaluation activities of schools relative to their antibul...

In recent review of the literature on integrating evaluative inquiry into organizational culture, Cousins, Goh, Clark and Lee [Cousins, J.B., Goh, S., Clark, S. & Lee, L. (2004). Canadian Journal of Program Evaluation 19(2), 99–144] suggest that there is a link between evaluative inquiry and organizational learning in schools. However, there have b...

When I was initially approached by Michael Scriven to review Collaborative Evaluations I was entirely enthusiastic about the invitation. It was only after having received the book that I took note of subtitle ‘A step-by-step model for the evaluator.’ Hmmm, I thought. Now this looks like an interesting twist. To be completely honest, my enthusiasm t...

Despite growing interest in evaluative inquiry as a trigger for development of organizational learning capacity, there exists a paucity of empirical research in this area. This is particularly the case in the context of schools, where systematic inquiry to support decision making and problem solving is hardly a mainstream activity. The purpose of t...

Evaluative inquiry--systematically gathering and analyzing data for the purposes of judging program or innovation merit, worth or significance and/or supporting ongoing school-based decision making and problem solving--is an option available to educators that is not often used; this despite recent interest in action research and school-based evalua...

Because bullying is a serious problem in Canadian schools, antibullying programs have been widely implemented to redress the problem. School principals in Ontario (N=395) completed a questionnaire to document the severity of bullying, the amount of anti-bullying resources, and the variety of antibullying activities in their schools. Results reveal...

Fidelity measurement is an evolving field in mental health case management program evaluation. This article presents an exploratory study in which two separate fidelity measures, the Dartmouth Assertive Community Treatment Scale (DACTS) and the Key Component Profiles (KCP), were used to assess structure and process elements of three mental health c...

This chapter posits two principal streams of participatory evaluation, practical participatory evaluation and transformative participatory evaluation, and compares them on a set of dimensions relating to control, level, and range of participation. The authors then situate them among other forms of collaborative evaluations.

This paper questions the adequacy of the process dimensions of the earlier version of the participatory framework. Their ongoing analysis of process dimensions reveals that the stakeholder selection dimension is problematic and needs reconsideration. This paper represents the framework and describes enhancements to the process dimension component....

The purpose of this article is to explore, through an extensive review and integration of recent scholarly literature, the conceptual interconnections and linkages among developments in the domains of evaluation utilization, evaluation capacity building, and organizational learning. Our goal is to describe and critique the current state of the know...

Despite increasing interest in process evaluation and the development of a variety of approaches to assess program implementation, several methodological challenges remain. Of particular concern is the development of ways not only to adequately determine the extent to which programs are implemented as intended but the provision of effective means t...

Although stakeholder-based evaluation has been fairly well developed in the general program evaluation literature, it remains barely recognized in training evaluation practice. This article aims to contribute to our understanding of multiple stakeholder perceptions about training evaluation in an organizational context. Extending prior empirical wo...

Interest in collaborative and participatory forms of evaluation — evaluation that involves evaluators working directly with nonevaluator program practitioners or stakeholders — has increased substantially in recent years. Yet research-based knowledge about such approaches remains limited. Moreover, empirical studies have focused almost exclusively...

The purpose of this study was to examine the impact of applying adult education principles to training in advanced-technology companies. First, we wanted to identify strengths and weaknesses of the training program’s content and delivery using a framework of adult education principles, in an effort to improve program design, curriculum development,...

Concept mapping and pattern matching techniques were used in exploratory research to investigate differences in stakeholder perceptions of training results and evaluation in a major division of a multinational network-design and engineering company. Referencing a research framework informed by multiple constituency views of organizational effective...

This exploratory survey study of 310 educators was conducted to investigate what variables best predict educators’ attitudes toward systematic inquiry in schools. Eight variables were selected as potential predictors of educators’ self-reported views about applied research utility and relevance, their personal ability to do research, the need for t...

Previous studies have treated teacher efficacy as a unitary trait without considering how teachers’expectations of their ability to produce student learning varies within teaching assignments. In this study, teachers in nine restructuring secondary schools in one district estimated their ability to perform common teaching tasks in four of the cours...

Few studies exist on the impact of practical participatory evaluation when the evaluator is not only a member of the organization, but also has program knowledge and expertise. This paper addresses this gap in the literature by reporting on a 2.5-year longitudinal single case study of practical participatory evaluation of a national, publicly funde...

This paper examines how teachers' expectations of their ability to produce student learning varies within teaching assignments. In the study described here, 359 teachers in 9 restructuring secondary schools in 1 district estimated their ability to perform common teaching tasks in 4 of the courses they expected to teach in the coming school year. Re...