ArticlePublisher preview available

Effectiveness of Training in Organizations: A Meta-Analysis of Design and Evaluation Features

To read the full-text of this research, you can request a copy directly from the authors.


The authors used meta-analytic procedures to examine the relationship between specified training design and evaluation features and the effectiveness of training in organizations. Results of the meta-analysis revealed training effectiveness sample-weighted mean ds of 0.60 (k = 15, N = 936) for reaction criteria, 0.63 (k = 234, N = 15,014) for learning criteria, 0.62 (k = 122, N = 15,627) for behavioral criteria, and 0.62 (k = 26, N = 1,748) for results criteria. These results suggest a medium to large effect size for organizational training. In addition, the training method used, the skill or task characteristic trained, and the choice of evaluation criteria were related to the effectiveness of training programs. Limitations of the study along with suggestions for future research are discussed.
Effectiveness of Training in Organizations:
A Meta-Analysis of Design and Evaluation Features
Winfred Arthur Jr.
Texas A&M University Winston Bennett Jr.
Air Force Research Laboratory
Pamela S. Edens and Suzanne T. Bell
Texas A&M University
The authors used meta-analytic procedures to examine the relationship between specified training design
and evaluation features and the effectiveness of training in organizations. Results of the meta-analysis
revealed training effectiveness sample-weighted mean ds of 0.60 (k15, N936) for reaction criteria,
0.63 (k234, N15,014) for learning criteria, 0.62 (k122, N15,627) for behavioral criteria, and
0.62 (k26, N1,748) for results criteria. These results suggest a medium to large effect size for
organizational training. In addition, the training method used, the skill or task characteristic trained, and
the choice of evaluation criteria were related to the effectiveness of training programs. Limitations of the
study along with suggestions for future research are discussed.
The continued need for individual and organizational develop-
ment can be traced to numerous demands, including maintaining
superiority in the marketplace, enhancing employee skills and
knowledge, and increasing productivity. Training is one of the
most pervasive methods for enhancing the productivity of individ-
uals and communicating organizational goals to new personnel. In
2000, U.S. organizations with 100 or more employees budgeted to
spend $54 billion on formal training (“Industry Report,” 2000).
Given the importance and potential impact of training on organi-
zations and the costs associated with the development and imple-
mentation of training, it is important that both researchers and
practitioners have a better understanding of the relationship be-
tween design and evaluation features and the effectiveness of
training and development efforts.
Meta-analysis quantitatively aggregates the results of primary
studies to arrive at an overall conclusion or summary across these
studies. In addition, meta-analysis makes it possible to assess
relationships not investigated in the original primary studies.
These, among others (see Arthur, Bennett, & Huffcutt, 2001), are
some of the advantages of meta-analysis over narrative reviews.
Although there have been a multitude of meta-analyses in other
domains of industrial/organizational psychology (e.g., cognitive
ability, employment interviews, assessment centers, and
employment-related personality testing) that now allow research-
ers to make broad summary statements about observable effects
and relationships in these domains, summaries of the training
effectiveness literature appear to be limited to the periodic narra-
tive Annual Reviews. A notable exception is Burke and Day
(1986), who, however, limited their meta-analysis to the effective-
ness of only managerial training.
Consequently, the goal of the present article is to address this
gap in the training effectiveness literature by conducting a meta-
analysis of the relationship between specified design and evalua-
tion features and the effectiveness of training in organizations. We
accomplish this goal by first identifying design and evaluation
features related to the effectiveness of organizational training
programs and interventions, focusing specifically on those features
over which practitioners and researchers have a reasonable degree
of control. We then discuss our use of meta-analytic procedures to
quantify the effect of each feature and conclude with a discussion
of the implications of our findings for both practitioners and
Overview of Design and Evaluation Features Related to
the Effectiveness of Training
Over the past 30 years, there have been six cumulative reviews
of the training and development literature (Campbell, 1971; Gold-
stein, 1980; Latham, 1988; Salas & Cannon-Bowers, 2001; Tan-
nenbaum & Yukl, 1992; Wexley, 1984). On the basis of these and
other pertinent literature, we identified several design and evalu-
ation features that are related to the effectiveness of training and
development programs. However, the scope of the present article
is limited to those features over which trainers and researchers
have a reasonable degree of control. Specifically, we focus on (a)
the type of evaluation criteria, (b) the implementation of training
needs assessment, (c) the skill or task characteristics trained, and
Winfred Arthur Jr., Pamela S. Edens, and Suzanne T. Bell, Department
of Psychology, Texas A&M University; Winston Bennett Jr., Air Force
Research Laboratory, Warfighter Training Research Division, Mesa,
This research is based in part on Winston Bennett Jr.’s doctoral disser-
tation, completed in 1995 at Texas A&M University and directed by
Winfred Arthur Jr.
Correspondence concerning this article should be addressed to Winfred
Arthur Jr., Department of Psychology, Texas A&M University, College
Station, Texas 77843-4235. E-mail:
Journal of Applied Psychology Copyright 2003 by the American Psychological Association, Inc.
2003, Vol. 88, No. 2, 234–245 0021-9010/03/$12.00 DOI: 10.1037/0021-9010.88.2.234
... Although scholars and practitioners have long been skeptical of the effectiveness of training for value change, there is evidence that benevolent values can be enhanced through specific interventions (Arieli et al., 2014). In addition, it has been argued that the right type of training, not training per se, is capable of promoting change within organizations (Arthur et al., 2003;Garofalo, 2003). Moreover, recent research in Public Service Motivation has shown that public values are dependent on context (Andrews, 2016). ...
The purpose of this paper is to investigate how selflessness is to be measured and how it is to be explained. Adopting an innovative approach to measure selflessness, we assess its prevalence in different regions across the world. We also investigate the factors that explain its emergence and how they interact in predicting it. The level of selflessness differs significantly across the world. Considering the factors predicting it, we find that the likelihood of an individual exhibiting selflessness largely depends on the region they live in. Workplace also plays a crucial role – in OECD countries, working in the public sector increases the propensity for selflessness, but it has the opposite effect in Africa.The research design involves a multivariate analysis of data from the World Values Survey using both straight-forward regression analyses and binary logistic regression.The main findings show that the impact of factors traditionally considered important, such as religious affiliation and employment in the public, private, or not-for-profit sectors, varies across regions. These region-specific interaction effects are seen, for instance, in the influence of one’s workplace on their propensity for selflessness. Public sector employees are significantly more likely to be selfless than their private sector counterparts. However, while working in the public sector increases the likelihood of being selfless in OECD countries, this relationship is reversed in Africa.In previous academic studies, little attention was paid to such interaction effects and the impacts of religious affiliation, public sector employment, gender, etc. were assumed to be stable all over the world. This study shows otherwise.As per practical implications, the results of our analyses suggest that research on values needs to be contextualized. This is particularly important when research aims at offering advice to practitioners. Our investigation has shown that the same factors that enhance selflessness in one part of the world may decrease its presence in other regions. A one-size-fits-all approach is therefore not adequate.
... One thing we do not know is why The Think Like an Entrepreneur Summer Academy succeeded. We gathered data on what students liked and disliked about the program, but reactions are generally considered to constitute poor evaluative criteria because they do not measure learning, behaviors, or outcomes (Arthur et al., 2003). ...
Full-text available
This paper overviews a pre-college entrepreneurial education program and examines its impact on entrepreneurial intention. The program, entitled the Think Like an Entrepreneur Summer Academy, aims at disadvantaged and underrepresented high school students and is developed by a public research university in the northeast USA. We analyzed the data from 154 graduates of the program and found a significant increase in entrepreneurial intention in the posttest compared to the pretest. The study adds to the literature on exploring and identifying the characteristics of pre-college entrepreneurial education interventions that help increase entrepreneurial intent and provides suggestions for future research, including comparing dropouts to graduates and exploring the interplay between entrepreneurial intention and entrepreneurial mindset.
... It is thus not surprising that numerous studies emphasize the importance of identifying the determinants of job satisfaction, exploring both work and non-work related factors. Earnings, working hours, working environment, workplace socialization, autonomy, organizational control, and participation in training schemes are all work-related factors shown to be affecting job satisfaction (Agho, Mueller, & Price, 1993;Arthur, Edens, Bell, & Bennett, 2003;Georgellis & Lange, 2007). Studies exploring non-work related factors affecting well-being at work focus mainly on personality, disposition, and work-life conflict arguments (Dormann & Zapf, 2001;Eby, Casper, Lockwood, Bordeaux, & Brinley, 2005;Eby, Maher, & Butts, 2010). ...
... Most organizations do use post-delivery feedback (commonly called "happy sheets") to understand whether the goals of the training programme have been achieved and whether participants have enjoyed the experience. However, this type of evaluation fails to give account of learning taking place in the workplace following delivery or behaviour changes that can occur whether participants have enjoyed the programme or not (Arthur, Bennett, Edens, & Bell, 2003). This is illustrated by the quotation below from Arthur and colleagues. ...
Following Investors in People (IiP) assessment a membership organization wanted to understand how evaluation of learning and development (L&D) activities for 800 UK staff could be improved and aligned with business objectives. We carried out focus groups, interviews with key stakeholders and a series of trio interviews with people who had recently undertaken L&D along with their manager and a peer. Analyses confirmed the need for knowledge sharing and quantification of behavioural and cognitive changes linked to current systems. These themes were explored by using the TOTADO model of evaluation (Birdi, 2006) to ensure future evaluation would be a marriage of theory and practice. We created an evaluation model that interacted with human resource systems as well as shaping future L&D activities. This was supported by a set of tools utilising the organization‟s IT platform. Use of the model encourages role innovation and reflection in the workplace based on knowledge sharing. We delivered the model earlier this year and advised the roll-out across the UK. Immediate results were IiP accreditation along with increased conversations about performance and the use of innovative approaches to learning. It is too early to measure financial or membership benefits for the organization but these are expected to occur after full implementation.
... Furthermore, Belling, James and Ladkin (2004) reported that pressurised work environments requiring employees to meet work demands under strict deadlines, can hinder transfer. Characteristics of the learning programme include the training system (Arthur, Bennett, Edens, & Bell, 2003) and the relevance of the programme content (Axtell, Maitlis & Yearta, 1997). Metaanalysis of 89 empirical studies exploring the influence of trainee characteristics, work environment and training interventions on the transfer of training to different contexts, carried out by Blume and colleagues (Blume et al, 2010), confirmed significant relationships existing between transfer and predictor variables such as work support and personal motivation, especially when the training was related to leadership development. ...
This is an evaluation of a Leadership Development Programme (LDP) consisting of a number of development approaches such as leadership coaching and action learning; carried out for leaders in a London Local Authority. Using semi-structured interviews and a focus group discussion the Taxonomy of Training and Development Outcomes (TOTADO) framework is applied to evaluate the influence of the LDP on individual, team and organizational level outcomes. Characteristics of coaching and action learning were examined; along with factors that may influence learning and transfer of learning to the workplace. Findings suggest that the LDP led to increased confidence and awareness of leadership behaviours and improved teamwork amongst leaders. Coaching content, the coach’s experience and experience sharing in action learning were found to promote leadership development. Manager and peer support, opportunity and work demands were found to influence transfer of learning to the workplace.
Purpose Motivated by the organizational challenge coined the great discontent, employees are dissatisfied with their jobs, see minimal opportunities for growth and are actively searching for new roles. This research aims to take a novel approach to internal communication strategy by introducing employability culture and leadership empowerment as mechanisms for supporting employees' career growth and additional positive workplace outcomes. Design/methodology/approach An online survey was designed and administered in the United States. The final sample size includes 425 full-time employees working in a variety of roles, industries and work arrangements. Findings Findings point to the inherent need for revised internal communication strategy that goes beyond managing and disseminating information. Organizations must develop cultures and their leaders in ways that empower employees and help them understand the meaning of their work. Employability culture, or an organization's support for developing employees' adaptive skills as work roles change, positively predicted employees' perceptions of their career growth opportunities at their current place of employment, employee loyalty and engagement, and job satisfaction. Leadership empowerment behaviors also positively predicted all previously listed workplace variables. These perceptions as influenced by work arrangement (onsite, hybrid, fully remote) and younger versus older generations were also analyzed. Originality/value Research findings offer new strategies for internal communications. Internal communication teams can partner alongside executive leadership to develop a culture that helps employees envision how their skills and expertise translates to different areas of the organization, empowering them to find meaning in their work, and be driven to support organizational growth.
Full-text available
Leadership integrity is crucial in shaping ethical leadership or promoting a rigorous adherence to moral principles and standards. This study explores the intricate relationship between leader integrity, moral identification, self-regulation, and ethical leadership behavior, providing practical insights for healthcare leaders seeking to enhance ethical practices. It addresses a critical gap in the research landscape by focusing on individual-level ethical leadership within the healthcare sector, where prior investigations have been limited. This study examines the mediating role of moral identification among leader integrity and ethical leadership behavior. We surveyed 181 health sector workers and employed SmartPLS to assess the conceptualized relationships. The analyses reveal a significant indirect influence of leader integrity on ethical leadership behavior, whereas moral identification mediates the relationship. Our findings further indicate an intriguing moderation effect of self-regulation on the relationship between moral identification and ethical leadership behavior. This divergence from previous research underscores the significance of contextual and methodological factors in studying leadership integrity and ethical behavior. Our study contributes to the literature on the relationship through planned behavior theory by demonstrating that moral identification mediates the relationship between leader integrity and ethical leadership behavior in the context of the theory of planned behavior. Our findings underscore the significance of fostering leader integrity within organizations to indirectly promote ethical leadership behaviors through moral identification. Organizations should prioritize initiatives that cultivate moral identification among their members to enhance ethical cultures.
The study aims to examine the impact of training and development on organizational performance, with a mediating role of employee performance. The importance of Human Resource Management (HRM) practices in achieving organizational success has gained recognition, as organizations realize that it is people who drive their performance. However, many companies have not fully embraced HRM practices, including training and development, due to affordability, fear, or a lack of understanding of their value and contribution. The primary objective of HRM is to ensure organizational success through a competitive, competent, and self-motivated workforce. Rigorous HRM practices, particularly training and development, play a vital role in achieving this objective. Extensive research has been conducted to establish the connection between HRM practices and organizational performance, and numerous studies have found positive relationships between them. This study will be investigating the impact of training and development on an organizational performance. Additionally, the study aims to examine the mediating role of employee performance between HRM practices and organizational perceived performance. The findings of this study will be contributing to the existing body of knowledge on HRM practices and their impact on organizational performance. It will provide insights into the significance of training and development in enhancing employee performance and subsequently driving organizational success. The study will also shed light on the mediating role of employee performance in the relationship between HRM practices and organizational perceived performance. By understanding the linkages between training and development, employee performance, and organizational performance, organizations can make informed decisions about the implementation and improvement of HRM practices. Ultimately, this study aims to highlight the importance of effective HRM practices, particularly training, and development, in achieving organizational success in today's competitive business environment.
Purpose This paper aims to establish a service efficiency-oriented framework for training design and evaluation as a pivotal service procedure in the workplace to fill the gap between training and organizational performance in a service context. Design/methodology/approach A semi-structured interview was first employed to confirm the primary indicator for training programs and criteria design as the pivotal factor for operational efficiency. An observation experiment was subsequently conducted to reveal that the training program can be redesigned according to the concrete operation effects and influencing factors for operational efficiency in the workplace. Findings The proposed service efficiency-oriented training model is suggested to underline and guide the activities for training requirements, training methods, training criteria and training evaluation for the service sector. Training auditing, analyzing and redesigning based on service efficiency could help to integrate service efficiency so that service organizations can readjust their specific training needs and concise the training program in the human resource management practice. Research limitations/implications This study only conducted an on-site observational experiment on one of the casinos in Macau. An observational method assessed the conceptual model in the context of table game operations. More quantitative approaches like AI-assisted systems may be employed in the future. The representativeness of the sample is somewhat limited. In addition, the service efficiency-oriented training concept model is an open system that any organization could extend by incorporating more elements in each part that can be developed to meet their human resource management needs. Finally, other service-oriented organizations like airlines and banks can learn from the theoretical model proposed in this article. It is suggested that non-profit organizations would be a better research area. Practical implications The finding can provide organizations and practitioners with insights and tools on how to provide and evaluate service efficiency and assess employee performance. Social implications The proposed service efficiency-oriented training model provides a theoretical foundation for training and organizational performance for service organizations. Originality/value This study is the first to develop a service efficiency-oriented training framework with training needs, methods, criteria and evaluation. A service industry sample was used to verify the framework in the context of casino game pace and dealer training for table games. Suggestions for a combination of management are provided for casino operators to redesign and evaluate the dealer training program for service improvement.
Full-text available
Although training evaluation is recognized as an important component of the instructional design model, there are no theoretically based models of training evaluation. This article attempts to move toward such a model by developing a classification scheme for evaluating learning outcomes. Learning constructs are derived from a variety of research domains, such as cognitive, social, and instructional psychology and human factors. Drawing from this research, we propose cognitive, skill-based, and affective learning outcomes (relevant to training) and recommend potential evaluation measures. The learning outcomes and associated evaluation measures are organized into a classification scheme. Requirements for providing construct-oriented evidence of validity for the scheme are also discussed.
Full-text available
Tests for experiments with matched groups or repeated measures designs use error terms that involve the correlation between the measures as well as the variance of the data. The larger the correlation between the measures, the smaller the error and the larger the test statistic. If an effect size is computed from the test statistic without taking the correlation between the measures into account, effect size will be overestimated. Procedures for computing effect size appropriately from matched groups or repeated measures designs are discussed.
Die Reduktion der Folsäurederivate (Ia) und (Ib) mit NaBD4 bzw. NaBH4 (in DMSO/Pyridin) verläuft stereoselektiv unter Bildung der Diastereomeren (II) bzw. (III) (NMR-spektrometrische Untersuchung).
This study used a pre/post design to assess student learning for the purposes of examining relationships among student grades, student learning, and student evaluations of teaching. These relationships were reframed in terms of reaction (Level I) and learning (Level II) evaluation criteria. Participants were 652 undergraduate students enrolled in seven sections of an introductory psychology course. Our results indicated a medium relationship between student grades and the pre/post learning measure. In addition, a small relationship was observed between student ratings of teaching effectiveness and a pre/post measure of learning. We conclude that student ratings and learning measures assess different aspects of teaching effectiveness and should not be used interchangeably. The most appropriate criterion for assessing teaching effectiveness is a function of the goal of evaluation. However, reaction and learning measures may be used in conjunction to obtain a more complete picture of instructor effectiveness.