ArticlePublisher preview available

Measuring the organisational impact of training: The need for greater methodological rigor

Authors:
Thomas N. Garavan at University College Cork
Thomas N. Garavan
Alma Mccarthy at University of Galway
Alma Mccarthy
Maura Sheehan at Edinburgh Napier University
Maura Sheehan
Yanqing Lai at Manchester Metropolitan University
Yanqing Lai
Show all 8 authorsHide
Read publisher preview
Download citation
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

We review the methodological rigor of empirical quantitative studies that have investigated the training and organisational performance relationship. Through a content analysis of 219 studies published in quality journals, we reveal significant validity threats (internal, external construct and statistical conclusion validity) that raise questions about the methodological rigor of the field. Our findings suggest that the time is appropriate for a renewed methodological endeavour to understanding the relationship between training and organisational performance. We make specific recommendations to enhance methodological rigor and generate research finding will enhance operationalisation of theory, help researchers to make inferences about causality and inform the decision making of HRD practitioners.
A preview of this full-text is provided by Wiley.
Learn more
Content available from Human Resource Development Quarterly
This content is subject to copyright. Terms and conditions apply.
LITERATURE REVIEW
Measuring the organizational impact of training:
The need for greater methodological rigor
Thomas Garavan
1
| Alma McCarthy
2
| Maura Sheehan
1
|
Yanqing Lai
1
| Mark N. K. Saunders
3
| Nicholas Clarke
4
|
Ronan Carbery
5
| Valerie Shanahan
6
1
Department of International Business,
Edinburgh Napier Business School, Edinburgh,
Scotland, UK
2
Discipline of Management, Cairnes School of
Business and Economics, National University
of Ireland Galway, Galway, Ireland
3
Department of Management, Birmingham
Business School, University of Birmingham
4
Department of Strategy, Leadership and
People, EADA Business School, Barcelona,
Spain
5
Department of Management and Marketing,
University College Cork, Cork, Ireland
6
Global Senior Learning Partner, Squarespace,
Dublin, Ireland
Correspondence
Thomas Garavan, Edinburgh Napier Business
School, Edinburgh, Scotland, UK.
Email: T.Garavan@napier.ac.uk
All authors contributed equally to this article.
We review the methodological rigor of empirical quantitative studies
that have investigated the training and organizational performance
relationship. Through a content analysis of 217 studies published in
quality journals, we demonstrate significant validity threats (internal,
external construct, and statistical conclusion validity) that raise ques-
tions about the methodological rigor of the field. Our findings suggest
that the time is appropriate for a renewed methodological endeavor
to understanding the relationship between training and organizational
performance. We make specific recommendations to enhance meth-
odological rigor and generate research findings that will enhance oper-
ationalization of theory, help researchers to make inferences about
causality, and inform the decision-making of Human Resource Devel-
opment (HRD) practitioners.
KEYWORDS
methodological rigor, training and organizational performance,
validity
1|INTRODUCTION
In this paper, we review 40 years of quantitative empirical studies that have investigated the training–organizational
performance relationship to identify the methodological features of these studies and the extent to which they are sub-
ject to validity threats. Training is an important construct in the HRD and learning and development (L&D) disciplines
(Bell, Tannenbaum, Ford, Noe, & Kraiger, 2017), and numerous industry-based reports document the considerable
investment made by organizations in employee training and development (e.g., Bersin by Deloitte, 2016). In addition,
scholars have argued that training enhances organizational performance, including productivity, innovation, customer
service quality, and financial performance (Aguinis & Kraiger, 2009; Kim & Ployhart, 2014; Noe, Clarke, & Klein, 2014),
yet the evidence base to make these claims is based on a preponderance of cross-sectional research designs that shed
DOI: 10.1002/hrdq.21345
© 2019 Wiley Periodicals, Inc.
Human Resource Development Quarterly. 2019;30:291–309. wileyonlinelibrary.com/journal/hrdq 291
... There exists a positive correlation between investment in training and enhanced organisational performance, as evidenced by studies conducted by Garavan et al., (2021a), Muhammed et al., (2023), Rahman andSahu (2022), andYe et al., (2023). According to Garavan et al., (2021b), a considerable body of research has indicated that training has a positive impact on various aspects of organisational performance, including innovation, customer service quality, productivity, and financial performance. According to Muhammed et al. (2023), the literature indicates that training and development initiatives have the potential to facilitate employee self-development and satisfaction, enhance employee skills and knowledge, increase performance, align with organisational goals, and ultimately lead to the attainment of a competitive advantage. ...
... Furthermore, the majority of research on the influence of training on organisational performance has been carried out in an Anglo-American setting, specifically in the United Kingdom, United States, Australia, Canada, and New Zealand. As a result, the existing understanding of the connection may not be entirely precise when applied to the economies of Asia-Pacific, Africa, and the Middle East (Garavan, 2021b). Therefore, the objective of this study is to comprehend the obstacles that hinder the assessment of return on investment (ROI) for training and development initiatives offered to academics in the higher education setting of South Africa. ...
... The challenge when it comes to measuring ROI is when the required information cannot be located (Phillips & Phillips, 2016). Provided that the relationship between training and associated organizational performance has been studied widely and is essential to the arguments that learning, and development practitioners make to rationalize investment in training, a major concern pertains to the quality of the evidence accessible (Garavan, 2021b). ...
Exploring barriers to measuring ROI for training and development interventions: A case study on academic staff in South Africa
Article
Full-text available
  • Apr 2024
The objective of this article is to investigate obstacles that hinder the assessment of return on investment (ROI) for academic staff development initiatives in public universities in South Africa. The primary objective of this study is to address the following research inquiry: What are the obstacles hindering the assessment of return on investment (ROI) for training and development interventions offered to academics within the higher education landscape of South Africa? The study employed a qualitative research methodology, which was based on an interpretive philosophical standpoint. The researchers employed purposive sampling to carefully select a total of fourteen participants who were selected to represent a range of cases or institutions. These included six universities and government institutions, namely the Department of Higher Education and Training (DHET), the Department of Science and Innovation (DSI), and the Education, Training, and Development Practices Sector Education and Training Authority (ETDPSETA). The interview data was subjected to thematic analysis using ATLAS.ti. The factors that hinder the measurement of ROI include the intricacy of the process, insufficient capacity, time lag between training and improved performance, lack of motivation, financial limitations, absence of processes, fragmentation of training and development interventions, interventions producing non-financial advantages, and insufficient data. The study enhanced the existing information on human resource development and practice by identifying the obstacles to measuring return on investment (ROI). Additionally, it offered practical advice and suggestions for future research.
View
... Given that the fervent efficacy claims, pedagogical decisions, and national and international university initiatives surrounding active learning adoption have been rooted in active learning research, particularly Freeman et al. (2014), it is important to examine the methodological strengths and weaknesses of the literature base to ensure these research-based recommendations are rooted in rigorous research. There have been many approaches taken to assess the methodological rigor in a number of disciplines (e.g., Bratt & Moons, 2015;Garavan et al., 2019;Pennington et al., 2021;Ramirez et al., 2017;Yang et al., 2012). Methodological rigor refers to the "thoroughness and accuracy with which research is conducted and it therefore involves elements such as empirical validity, technical quality, statistical significance, and the generalizability of results" (Flickinger et al., 2014, p. 105). ...
... Researchers typically choose methodological controls within categories of validity from which to assess the rigor of their respective literature. For example, Garavan et al. (2019) investigated four categories of validity-each with several dimensions-to determine the methodological rigor of empirical studies on the training and organizational performance relationship. These categories included internal validity, external validity, construct validity, and statistical conclusion validity. ...
How Rigorous is Active Learning Research in STEM Education? An Examination of Key Internal Validity Controls in Intervention Studies
Article
Full-text available
  • Nov 2023
  • EDUC PSYCHOL REV
Active learning is a popular approach to teaching and learning that has gained traction through research on STEM educational improvement. There have been numerous university- and national/international-level efforts focused on transitioning courses from the lecture method to active learning. However, despite these large-scale changes, the active learning literature has not been assessed on its methodological rigor to ensure instructional recommendations are rooted in rigorous research studies. The purpose of the present review was to determine areas of strengths and areas in need of improvement and to provide specific recommendations on how to continue or improve active learning research to strengthen the respective literature base and increase confidence in results. We assessed the articles included in the Freeman et al. (PNAS, 111:8410–8415, 2014) meta-analysis as well as a random sample of more recent active learning articles (2015–2022) on 12 internal validity controls (i.e., control procedure used to prevent a threat to the internal validity of a study). Results indicated that there were high percentages of articles that did not meet each internal validity control. In fact, no articles from the Freeman et al. meta- analysis and no sampled 2015–2022 articles met each of the 12 internal validity controls. Therefore, the active learning literature contains numerous internal validity control issues that need to be addressed if we are to determine the extent to which active learning interventions are effective and if there are any boundary conditions for when particular active learning interventions are or are not effective.
View
... Furthermore, training needs analysis (TNA) is the process through which institutions gather data to support decisions, particularly those about whether or not training may improve performance, who should receive training, and what should be covered in that training. Presenting a framework to assist practitioners in recognizing when and how organizational politics may affect such kind of assessment, with attempts to discover the specific political dimensions that impact on training decisions, may help other people to guide future research in this area (Clarke, 2003;Garavan et al., 2019). ...
SDCA Asia-Pacific Multidisciplinary Research Journal (Volume 5, Issue 2, December 2023)
Article
Full-text available
  • Dec 2023
All articles in the SDCA Asia-Pacific Multidisciplinary Research Journal (Volume 5, Issue 2, December 2023) were authored/co-authored by 31 authors.
View
... Um dos maiores desafios da área de gestão de pessoas é evidenciar o retorno de investimento em treinamento (Garavan et al., 2019), especialmente quando o objetivo é desenvolver competências individuais com alto nível de abstração em profissionais, os quais precisam honrar expectativas organizacionais em troca dos recursos investidos apresentando resultados com perspectivas de longo prazo. ...
View
View
The relationship between training effectiveness and financial performance in cooperative organizations
Article
  • Apr 2024
Purpose The purpose of this study was to identify the relationship between training and financial performance in cooperative organizations. Design/methodology/approach To achieve this goal, the fixed-effect panel regression technique was used, from a single database containing hours and amounts invested in training by 35 large Brazilian agribusiness cooperatives over 10 years as the main independent variable of the econometric model. Financial performance was operationalized by the Net Margin and ROE. Findings It was possible to identify a positive relationship between expenditure on training and the future rate of return and profitability of the organizations in question. The results also indicate that this relationship grows stronger over the first three years after the investments are made and ceases to exist after this period. The findings are robust with regard to a series of alternative explanations and contribute to understanding the relationship between training and organizational performance in financial terms, considering the extent and duration of training. Originality/value The originality this study is justified by the pioneering spirit of presenting direct evidence linking investment in training and financial performance and the duration of this relationship. Thus, the study makes a significant contribution to the construction of knowledge on the subject.
View
Learning And Development Practitioners in Organisations: Organising, Belonging, Learning And Performing Tensions
Article
Full-text available
  • Mar 2024
This paper provides a review of the literature on the learning and development (L&D) role in organisations using the lens of paradox theory. I evaluate prior literature and organise it around four paradoxes: organising, belonging, learning and performing. The L&D organisational role is fraught with paradox due to tensions around organising, belonging, learning and performance. I evaluate the different ways in which L&D practitioners respond to these tensions, I highlight future research avenues and suggest practice implications for L&D practitioners.
View
Determinants of Organisations’ Decisions Regarding Investments in Human Resource Development
Chapter
  • Feb 2024
The study analyses the determinants influencing the decisions of organisations to invest in human resource development. The researchers in the field have been particularly interested in investments in human resources, having analysed various aspects of such investments. At the same time, it should be noted that most research in this field investigated the impact of human resource development investments on organisational performance as it could influence future decisions on the amount of investments. We also found that there is insufficient research on the perception of specific determinants of managerial decisions on human resource development investments. In this context, we have identified several determinants that, depending on the way they are perceived and valued by organisations, could influence managerial decisions regarding investments in human resource development. We also believe that the existence of a human resource development policy at the organisational level can have an impact on the assessment of the determinants that influence the level of investment for employee training. Based on the aforementioned, we formulated the following research question: Could an assessment of determinants influence the organisations’ decisions regarding investment in human resource development? For this purpose, we have developed a survey comprising several determinants that can more or less influence investments in human resource development. The Likert scale from “1” to “5” was used to evaluate the determinants. The study results show that the determinants with a higher score give greater responsiveness to organisations in terms of human resource development investments.
View
Human Resource Development and Organizational Practice linkages through Job Satisfaction: Evidence from the Indian Banking Sector
Article
Full-text available
  • Jan 2023
Every organization has its own ways to manage and develop human resources in their organizations. Human resource development practices prevailing in an organization can express the effectiveness of the organization and its impact on overall organizational performance. The present research tried to identify the principal practices of human resource development in the Indian banking sector and further to see that how these practices are impacting the productivity of the organization as well as affecting the satisfaction level of the employees with the help of job satisfaction. To conduct the statistical analysis a questionnaire was circulated amongst 500+ professionals, they are target respondents as per the objectives of the research. Finally, 400 responses of target employees were taken from the different public and private sector banks (Delhi/NCR) for data analysis. Multiple regression analysis was used to determine the contribution of eight human resource development Mechanisms toward influencing job satisfaction. The study highlights that the human resource development mechanism and organizational practices have a positive impact on the job satisfaction and performance of the employees. Our results are exhibiting that human resource development practices have a positive impact on organizational performance and job satisfaction. The study provides useful implications for managers, banking institutions, and policymakers to enhance the productivity of the workforce and the profitability of the organization.
View
Factors affecting transfer of online training: A systematic literature review and proposed taxonomy
Article
Full-text available
  • Nov 2023
  • Hum Resource Dev Q
Training employees in new competencies is crucial for ensuring organizational sustainability and success. Consequently, organizations allocate significant resources to training initiatives. As training constitutes a substantial investment, it becomes imperative to identify and categorize factors that enhance the transferability of training. This necessity becomes particularly pronounced in the context of online training, which is being increasingly adopted by organizations for various reasons, including cost efficiency and flexibility. While the adoption of online training is on the rise, research on factors that enhance the effectiveness of such training remains limited. This study aims to address this gap by introducing a taxonomy of factors that influence the transfer of online training. These factors are categorized into four broad themes: computer (digital) literacy and attitudes, learner characteristics, training design and planning, and the learning environment along with organizational factors. Additionally, the study maps these factors to the intended outcomes of training (knowledge, skills, and attitude) and the participants engaged in the training sessions, providing a comprehensive overview of ongoing research and identifying areas for future investigation. Drawing insights from an evaluation of existing literature, the research not only points toward potential research directions but also offers practical implications for designing more effective online training sessions.
View
View
View
View
View
View
View
View
Revisiting the rigor-relevance relationship: An institutional logics perspective
Article
  • Mar 2018
  • HUM RESOUR MANAGE-US
The question of whether academic research should emphasize scientific rigor, practical relevance, or both simultaneously has been hotly debated in HRM research and other related disciplines for much of the past century. That said, empirical investigations of whether these values are mutually exclusive or compatible are surprisingly rare. Moreover, the perspective of the end consumers of research—practitioners—as to what research is relevant has been almost completely ignored. In the present study, we adopt an institutional logics perspective to assess the perceived relevance of common management research findings with two samples of 298 and 143 practicing managers, respectively. Further, we examine whether objective indicators of rigor and manager‐rated relevance impact academic legitimacy. The results indicate a positive relationship between rigor and relevance. Interestingly, practitioner ratings of relevance were stronger predictors of academic legitimacy than methodological rigor. Finally, research findings that deal with leadership topics are rated as most relevant to practitioners, whereas job characteristics, demographics, and human resource practices are seen as the least relevant to managers. The contributions of this study and implications for future research conclude the article.
View
Beyond bridging rigor and relevance: The three-body problem in entrepreneurship
Article
  • Mar 2018
The purpose of this special issue is to outline a distinct third body of knowledge in the form of pragmatically oriented entrepreneurial design principles, to discuss whether it deserves a position on par with theory and practice, and to explore its interfaces with both the causal mechanisms of entrepreneurship theory and the complex realities of entrepreneurial practice (c.f. Romme and Endenburg, 2006; Van Burg et al., 2008). By design principles we mean context specific and pragmatic heuristics that prescribe actions often with the following syntax: ’to achieve X in situation Y, something like Z will help’ (Van Aken, 2004: 227). By highlighting design as a valuable third body of knowledge, in this virtual special issue we depart from the commonly proposed way to bridge the rigor-relevance gap that simply encourages closer collaboration and more intimate involvement of practitioners in the research process (e.g. Shapiro et al., 2007; Starkey and Madan, 2001). While such closeness may very well be valuable, we submit that interlocking theory and practice may not be the best option to produce a stable system. Instead, we follow Simon (1996) who argued for a science of design whose purpose is not to produce descriptive theories of the world as it is, but rather to develop pragmatic tools "in the service of action" (Romme, 2003, p. 562).
View
View