ArticlePDF Available

Training The Success Case Method: A Strategic Evaluation Approach to Increasing the Value and Effect of

Authors:

Abstract

The problem and the solution. Despite the fact that effective human resource development (HRD) operations are vital to overall organization success, most organizations fail to evaluate the impact and return on training investments that they could and should. Traditional evaluation models and methods, with their focus on simply assessing the scope of training’s effect, do little to help reap greater performance and organizational impact from HRD and, in fact, can even undermine this purpose. This article argues that it is performance, not HRD, that achieves (or does not achieve) results, and thus impact evaluation must inquire more broadly into the performance management context. Consequently, the Success Case Method (SCM) is presented and discussed. The final portion of the article presents a case study derived from a recent SCM evaluation project for a major business client that demonstrates and illustrates the working of the method.
A preview of the PDF is not available
... This study adopted the modified Success Case Method [32,33] ...
... We adopted modified Success Case Method (SCM) with a case study design for this qualitative evaluation study. Traditionally, SCM analyzes the most and least successful groups to investigate how well an organizational initiative (e.g., a training program) is working along with the causal link from program activities to impact [32,33]. SCM achieves efficiencies by focusing the bulk of inquiries on relatively few subjects. ...
... We used a modification of Brinkerhoff 's success case method (SCM) study design [32,33] which helped us identify positive deviant samples fitting into the success case definition. Selection of Fellows by extreme (or deviant) case sampling method was well aligned with the nature of research design and study aims. ...
Article
Full-text available
Background Faculty development programs (FDPs) in health professions education (HPE) are instrumental in supporting, promoting, and improving HPE curricula and activities that target individual- and system-level capacity strengthening. FAIMER, a division of Intealth, has been offering FDPs for global health professions educators in HPE and leadership for over two decades through the International FAIMER Institute in the USA and FAIMER Regional Institutes located globally. This paper explores perceptions of former program participants (Fellows) of FAIMER global FDPs by eliciting their success stories. Methods The study utilized a modified Success Case Method approach. Study participants recruited were 14 FAIMER Fellows nominated as exemplars of success cases by program directors and faculty members from seven global sites, using extreme case sampling technique. Of the 14 Fellows, two were from Africa, nine from Asia and three from Latin America. In-depth interviews were conducted with each Fellow to understand their self-perceived success, and factors from both their local context and FAIMER’s FDPs that have contributed to their success. Theoretical thematic analysis method was used to analyze the data. Results The top three areas of success perceived by Fellows were career advancement & recognition, professional development, and advancement of HPE. Fellows identified both institutional-level factors and personal attributes as contributing to their success. They also reported that FAIMER global FDPs have substantially contributed to their success by expanding professional networks, fostering academic achievement, and enhancing interpersonal development and leadership skills. Conclusions The findings show that Fellows perceive that there are multiple pathways for them to succeed in advancing the field of HPE. The Fellows’ stories highlight the continuing value of FAIMER global FDPs for HPE educators worldwide in professional development and leadership. The study also proposes recommendations for enhancing faculty development activities in HPE.
... For example, only one-third of transformations are self-reported as being successfully sustained (Bucy et al., 2021). Brinkerhoff (2005) finds that historically, only 25% of training participants actively apply learning to generate the desired business results. To address these and other shortcomings with Kirkpatrick's (1976) and Phillips' (2012) training evaluation methods, Brinkerhoff (2003) developed the success case method (SCM) as a mixed-method case study to address the connection with business outcomes. ...
Thesis
Full-text available
This study examines the factors that contribute to the long-term sustainability of a small manufacturer from a manufacturing extension practitioner's perspective. The longevity subject used in this study was XYZ Company, a 90-year-old, family-owned manufacturing organization. The findings are explored in an organizational learning theory context to provide change practitioners with a roadmap of recommended action based on an exemplary case study. Using Brinkerhoff's success case methodology, the key factors for long-term success and the impact of change events were evaluated using a mixed-method approach. The initial quantitative survey determined which factors and events had the most extreme positive and negative cases. Individual participants were then interviewed from these success and nonsuccess cases to gain a deeper understanding of the individuals' experiences. The study found that all the key factors previously identified for century club corporate longevity were extremely important to XYZ Company. An analysis using an artificial intelligence platform identified the following overarching themes: mission, continuous improvement, relationships, and learning. Furthermore, the study found that 13 of 14 of the change events had concrete positive impacts, with the top six most impactful events being lean principles, 5S training, lean certificate series, Kolbe workshop, and quality management system. These results are organized into a roadmap of events, which includes the related markers of progress toward a learning organization. This offers organizations and practitioners a guide to the key factors and the change events that have been found at a small manufacturing company approaching a century of successfully sustained operations.
... Menetapkan mekanisme pemantauan yang baik memungkinkan perguruan tinggi untuk menilai dampak dari langkah-langkah yang diambil. Menurut Brinkerhoff (2006), evaluasi yang berkelanjutan dapat memberikan wawasan berharga tentang efektivitas program. Pendekatan berbasis data sangat penting dalam evaluasi kinerja dosen. ...
Book
Full-text available
Secara sadar atau tidak, setiap orang pasti berhubungan dengan organisasi, baik sebagai bagian dari organisasi maupun bukan. Oleh karena itu, adalah wajar apabila istilah organisasi bukan suatu hal yang baru bagi setiap orang dengan berbagai bentuk namanya. Organisasi adalah suatu tempat berkumpulnya sejumlah orang yang melakukan aktivitas bersama untuk mencapai tujuan. Dalam pelaksanaannya, organisasi dapat menjadi alat yang efektif untuk mencapai tujuan, sekaligus juga dapat menjadi bumerang manakala orang-orang yang terlibat di dalamnya tidak sadar akan tugas, wewenang dan tanggung jawab yang harus dilakukannya. Organisasi yang baik adalah organisasi yang mempunyai tujuan yang jelas, lingkungan untuk pengembangan yang jelas dan mempunyai prediksi keuntungan dan kerugian yang akan terjadi dengan berlandaskan pada data informasi yang ada di lingkungannya, baik internal maupun eksternal. Organisasi sebagai suatu sistem perilaku mencakup input, proses dan output. Sekumpulan orang, sarana dan prasarana dapat dijadikan sebagai masukan untuk proses selanjutnya. Wujud dan proses tersebut dapat berupa interaksi orang-orang yang terlibat dalam organisasi tersebut. Adapun wujud keluaran berupa hasil kerjasama dalam melaksanakan sesuatu untuk mencapai tujuan yang sama. Secara operasional tujuan umum belum dapat berfungsi bila tidak dijabarkan ke dalam tujuan-tujuan khusus. Dalam hal ini, tujuan memiliki sebagai pedoman kegiatan, sumber legitimasi, standar pelaksanaan, sumber motivasi, dasar rasional pengorganisasian. Pencapaian tujuan tidak mudah diukur bagi organisasi yang tidak memproduksi keluaran nyata yang bergerak di bidang usaha jasa pendidikan. Dalam organisasi seperti ini, dosen merupakan faktor input yang melakukan suatu proses agar dapat menciptakan output yang baik. Salah satu masalah yang masih ada dihadapi oleh kampus hingga saat ini adalah “Mengapa beberapa dosen bekerja lebih baik dari pada dosen lainnya”.
... There are several evaluation models for training or workshop programs that can help to assess the effectiveness of any training and workshop programs (Table 1). For example, Kirkpatrick's Four-Level Model (Kirkpatrick & Kirkpatrick, 2007), Phillips' ROI Model (Philips, 2003), Brinkerhoff's Success Case Method (Brinkerhoff, 2005) and Kaufman's Five Levels of Evaluation (Kaufman & Keller, 1994). However, there is no one-size-fits-all answer to which evaluation model is the best for evaluating training and workshops, as each model has its strengths and weaknesses. ...
Article
Full-text available
The integration of makerspace principles presents a promising avenue for educators to implement impactful STEM project-based learning initiatives. To this end, a three-day workshop entitled "Introductory STEM Makerspace Workshop – Learning Through Making" was conducted for lower secondary science teachers, with a particular focus on electric circuits. Throughout this workshop, educators were furnished with the basic knowledge and skills to facilitate maker-centered project-based learning experiences in science education. Furthermore, participants were challenged to conceptualise and develop lessons incorporating maker-centered learning in both classroom settings and extracurricular activities. This study evaluated the effectiveness of this workshop for 18 teachers employing three of the four levels of Kirkpatrick’s Evaluation Model. At the first level, encompassing participants' reactions, data were collected through an online evaluation form and direct observation during the workshop sessions, revealing a predominantly positive response from the participants.. Subsequently, at the second level, which pertains to learning outcomes, participants' acquisition of skills was evaluated via examination of photographs and reflective submissions on Google Drive subsequent to each workshop session. Findings indicated that educators attained proficiency in fundamental tasks such as soldering, 3D printing design, and fabrication of basic electrical devices, exemplified through their application of parallel circuit concepts. As for the third level, concerning behavioural change, was assessed through voluntary feedback solicited from participants. Notably, four respondents reported implementing acquired knowledge and skills within their educational contexts at the time of data collection. Thus, this evaluation determines whether and to what extent the workshop's effectiveness is for the participants. Additionally, it aids in identifying strengths and shortcomings and serves as a decision-making tool for future improvements. Keywords: STEM makerspace, teachers’ training, evaluation, Kirkpatrick’s Evaluation Model
... As many researchers and practitioners confirm the Kirkpatrick model's ongoing utility, usefulness and effectiveness, the model continues to be widely used even in the face of criticism [36,40]. Nonetheless, a number of models like the CIRO model by Warr, Bird [41], Philips's ROI model by Phillips [40], Brinkerhooffs success case method by Brinkerhoff [42], CIPP model by Stufflebeam [43] were put out to assess post-training. As Tamkin, Yarnall [44] and Bove and Little [45] noted, several assessment models are essentially Kirkpatrick's model extensions. ...
Article
Full-text available
This study aims to enhance the post-training evaluation of the annual performance agreement (APA) training organized by the Bangladesh Public Administration Training Centre (BPATC), the apex training institute for civil servants. Utilizing fuzzy-set qualitative comparative analysis (fsQCA) and artificial neural network (ANN) techniques within Kirkpatrick’s four-stage model framework, data were collected from a self-administered questionnaire survey of 71 in-service civil servants who participated in the APA training program. This study employs an asymmetric, non-linear model analyzed through a configurational approach and ANN to explore interrelationships among the four Kirkpatrick levels namely, reaction, learning, behavior, and results. Findings indicate that trainees were satisfied across all levels, identifying a non-linear relationship among these levels in post-training evaluation process. The research highlights that "learning skills" are most significant in the APA post-training evaluation, followed by behavior, results, and reaction. Theoretically, this research advances Kirkpatrick’s model and adds to the literature on public service post-training evaluation. Practically, it recommends prioritizing strategies that address cognitive barriers to enhance training effectiveness. This study’s innovative approach lies in its concurrent use of fsQCA and ANN methods to analyze the success or failure of APA-related trainees, offering alternative pathways to desired outcomes and contrasting traditional quantitative methods that provide a single solution. The findings have practical implications for public service training institutions and bureaucratic policymakers involved in capacity development, guiding the creation of more effective in-service training courses for public officials. The methodology and analysis can be applied in other contexts, allowing bureaucratic policymakers to replicate these findings in their learning institutes to identify unique configurations that lead to successful or unsuccessful training outcomes, adopt effective strategies, and avoid detrimental ones.
Article
Local governments were instrumental in managing the COVID‐19 crisis in countries worldwide. This study examines the methods that local governments in Israel used to successfully manage the COVID‐19 crisis. We explored the structural characteristics of the localities that excelled at managing the pandemic. Furthermore, we used measures of the spread of the virus and vaccination rates to identify a group of localities that managed the crisis with relative success. We conducted interviews with officials from these localities to determine the methods, policies, and conditions that led to success. We found that ethnicity, distance from urban centers, and socioeconomic status were associated with effective COVID‐19 management. We also identified several intraorganizational and interorganizational policies and practices that were successful: crisis‐adjusted management, a unified organizational atmosphere, digitalization, information management, autonomous decision making, and the fostering of collaborative relationships. Implications of the findings for the All Hazards Approach to disaster management are discussed.
Article
Full-text available
Transfer of training is of paramount concern for training researchers and practitioners. Despite research efforts, there is a growing concern over the "transfer problem." The purpose of this paper is to provide a critique of the existing transfer research and to suggest directions for future research investigations. The conditions of transfer include both the generalization of learned material to the job and the maintenance of trained skills over a period of time on the job. The existing research examining the effects of training design, trainee, and work-environment factors on conditions of transfer is reviewed and critiqued. Research gaps identified from the review include the need to (1) test various operationalizations of training design and work-environment factors that have been posited as having an impact on transfer and (2) develop a framework for conducting research on the effects of trainee characteristics on transfer. Needed advancements in the conceptualization and operationalization of the criterion of transfer are also discussed. ABSTRACT FROM AUTHOR Copyright of Personnel Psychology is the property of Blackwell Publishing Limited and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts)
Article
Context is a pervasive and potent force in any learning event. Yet instructional design models contain little guidance about how to accommodate contextual elements to improve learning and transfer. This paper defines context, outlines its levels and types, specifies some pertinent contextual factors within these types, suggests methods for conducting a contextual analysis and utilizing its results for instructional design, and outlines future issues for context-based instructional design. The incorporation of a contextual approach to instruction will make our design models systemic as well as systematic.
Article
Presents a descriptive review regarding current training theory and research. Prescriptions regarding future research; Ways to identify research gaps; Highlighted issues which require a different research orientation; Organizational analysis; Discussion.
The power of appreciative inquiry: A practical guide to positive change
  • D Whitney
  • A Trosten-Bloom
  • D Cooperrider
Whitney, D., Trosten-Bloom, A., & Cooperrider, D. (2003). The power of appreciative inquiry: A practical guide to positive change. San Francisco: Berrett-Koehler.