Available via license: CC BY 4.0
Content may be subject to copyright.
ARTICLE
1. Introduction
The business world is constantly changing, requiring organisations to create new production
processes, products and services at a rapid pace to improve quality and customer service (Rial,
Applied Business and Economics Journal
e-ISSN: 2956-7432 2024 Vol.2 No 1 pp. 76–99 DOI: 10.61089/abej.2024.2.86
Using the analytic hierarchical process to investigate the impact of training on
KPIs in an SME: A case study in the printing industry
Galleguillos-Pozo, Rosa Elizabeth 1 Rial-González, Pablo 2
1 2
Euncet Business School
Recived: 13
February 2024
Accepted:
20 April
2024
Published: 30
September 2024
Abstract:
This paper proposes an AHP and BSC based approach for evaluating the
impact of training investments on KPIs in a manufacturing SME in Chile.
The analytic hierarchy is structured in three steps: initial analysis
(economic factors), organisational analysis (observation) and the impact of
training investment on the economic factors in the initial analysis, in terms
of the key performance indicators in each step. The aim of this study is to
serve as a first stage in the development of an integrated approach linking
key performance indicators (KPIs) to training plans, in order to obtain a
working process that can be used t
o determine the weighted impact of
training on KPIs, with a view to calculating the return on training
investment in a future second stage of research. This paper integrates a
theoretical approach with analytic hierarchical process (AHP)
methodology. This new approach has been called hierarchical participatory
action analysis (APAJ) and is carried out in five steps. APAJ was found to
be a more realistic tool for assessing the cost-effectiveness of training in an
organisation. A case study focusing on an SME
operating within the
printing sector is provided to demonstrate the practical application of the
proposed methodology.
Keywords:
analytical kierarchical process, key performance indicators, small and
medium-sized enterprises, skills, training, return on investment
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 2
Applied Business and Economics Journal
2024 vol. 2
2016). In order for a company to maintain its competitiveness and success, each decision must
be properly evaluated and selected to increase its competitive advantage (Galleguillos et al.,
2017).
The balanced scorecard (BSC) is a tool that can be used to improve competitiveness, but it has
some shortcomings in terms of implementation on a quantitative basis (Mardani et al., 2015;
Quezada et al., 2014). This has led to the integration of methodologies such as the analytic
hierarchical process (AHP), which prioritises and weights the perspectives and indicators of the
BSC (Álvarez et al., 2017). Many studies focus on financial and customer indicators while
neglecting the training perspective. Others show that strengthening the training perspective
could help to build trust among investors and stakeholders and improve future performance
(Massingham et al., 2018).
Today’s companies are increasingly investing in the training of their employees because of the
positive economic impact it can have on their success (Tonhäuser & Seeber, 2014). On-the-job
training is strengthened by the creation of models and proposals that link the education-
employment binomial. The competence-based approach to training involves a reformulation of
the concept, linking it to the demands of economic and social development and the needs of
each company. This implies important pedagogical changes and innovations in the definition
and organisation of learning processes and their subsequent assessment (Clares et al., 2019).
Urbancová et al., 2021, point out that to evaluate the organisational-level benefits of
implemented training and development programs, they must align with the organisation's needs.
Management should identify areas where monitoring effectiveness is essential. Effectiveness
needs observation in all training activities without exceptions. The simplest method is to assess
financial costs, but it's also possible to consider the nature of the training activity or the time
invested. Additionally, it's crucial for the organisation's management to appropriately determine
the variables by which training benefits will be evaluated. These variables should be monitored
even before the actual training implementation. All involved parties should be familiar with the
objectives and methodology of evaluating training effectiveness, emphasizing that the
effectiveness of evaluation is inseparable from the identification of training needs and planning
Obtaining feedback on the effectiveness of training not only helps managers and decision
makers to identify areas for improvement, but also provides employees with the knowledge
they need to progress and develop their careers (Karim et al., 2019). Ultimately, having a
competent and flexible workforce is a must for any competitive business. Training activities
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 3
Applied Business and Economics Journal
2024 vol. 2
lead to financial benefits such as revenue, cost reduction, savings and profitability. Therefore,
measuring the return on investment (ROI) in training is essential for business continuity and
success (Salimian, 2021).
There is great interest in measuring the ROI in training using different methodologies and/or
tools to improve companies’ competitiveness. For example, a method has been defined and
tested to empirically measure ROI in architectural firms (Lechhab et al., 2022).
The literature has not yet identified a satisfactory method for evaluating the economic impact
of training investment. To fill this gap, the present study aims to serve as a first stage in the
creation and application of an integrated approach linking organisational factors (KPIs) to
training plans. The ultimate aim is to develop a working process for determining the impact of
training on KPIs (first stage) and, as a continuation of this study (second stage), a standardised
procedure for calculating the return on training investment. In light of the above, our main
research focus is on establishing an approach that incorporates a decision model of key
performance indicators (including the impact of training) and prioritises these indicators by
taking into account all relationships.
This research aims to answer the main question: How can we develop an approach that
integrates a decision model? This model should incorporate key performance indicators (KPIs)
that capture the impact of training and prioritize them, taking into account all the
interrelationships between them.
The rest of the paper is structured as follows: Section 2 presents the literature review that was
used as a starting point for this research. Section 3 describes the methodology of the study.
Section 4 illustrates a case study using the proposed approach and analyses the results. Finally,
Section 5 outlines some conclusions and future lines of research.
2. Literature preview
Work-based training approaches have been consolidated over the last decade, generating new
models and proposals to improve the effectiveness and profitability of in-company training.
Castillo and Villalpando (2019) support the competency-based training approach by
proposing a reformulation of the concept of training, linking it to the demands of economic
and social development and, more specifically, to the needs of the productive world. This
implies important pedagogical changes and innovations in company training, especially with
regard to the assessment of learning and the definition and organisation of learning processes.
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 4
Applied Business and Economics Journal
2024 vol. 2
Soft skills are becoming as important as hard skills in the context of job performance and
employee competence building. These skills are necessary not only to secure a job, but also
to grow within a company and reach one’s maximum potential to contribute to its strategic
goals (Elmoutanna & Motii, 2022). According to Matia (2016), the competency-based model
of vocational education and training (VET) has the following features: it addresses
professional qualification processes; it defines learning processes from a global, flexible and
dynamic perspective; it promotes the use of innovative, active and problem-solving
methodologies; it manages the mobilisation of cognitive, motor, affective and psychosocial
resources to find solutions to different challenges; and it facilitates responsibility,
cooperation, initiative, creativity and autonomy in the learning process.
There are also other models, such as Kilpatrick’s project-based model (Domènech-Casal,
2019), developed by Van der Klink et al. (2007), which focuses on collaborative learning
methods that encourage peer learning, project-based learning and a sense of belonging and
cohesion in the group. Building on the above models, newer models have also been
developed. These include problem-based learning (PBL), which is designed to develop
competencies through involvement in projects, while also encouraging self-learning and
collaborative learning. PBL does not aim to have people memorise concepts or solve
problems individually, but rather to develop competencies that allow for individual
knowledge development with the common goal of solving a problem as a team (Tan, 2021).
Another model is work-based learning (WBL), a variant of the PBL methodology, which
aims to integrate knowledge with the real needs of each professional sector. WBL is used in
continuous in-company training, is based on practice (experience) and learning by doing, and
works with a competence development plan for each employee’s career. WBL provides a
tailor-made training programme that meets the training needs of the company and the
individual ambitions of its employees. The aim of WBL is to ensure that the knowledge
acquired in different training contexts is applied jointly by several people in order to solve
problems as a team (Murtazin et al., 2020; Perusso & Wagenaar, 2022).
Methodologies also exist in the area of workplace learning analysis (WLA). WLA provides
structured and holistic assessments to communicate learning outcomes to managers in a
summarised and simplified way. This conceptual approach has led to the development of a
framework for measuring the impact of workplace learning interventions and has
demonstrated the positive impact of social learning in organisations (Kopp & Kinkel, 2020).
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 5
Applied Business and Economics Journal
2024 vol. 2
In this context, Van Rooij and Merkebu’s (2015) study used multiple measures, including
win rate, to provide managers with greater insight into how investment in learning
contributes to overall organisational performance. It should be noted, however, that the
decision makers who participated in this study did not provide information on the relative
weight given to each measure.
Within the diverse landscape of training methodologies, each approach possesses unique
strengths and weaknesses that warrant careful consideration to ensure optimal effectiveness
and tailoring to specific training needs. Kilpatrick's PBL shines in its ability to ignite learner
engagement through active participation in projects. By tackling real-world challenges,
students cultivate strong motivation, curiosity, and collaboration skills. However, challenges
arise concerning resource intensiveness, requiring significant planning and expertise to
design engaging projects (Evensen et al., 2000). Additionally, assessing individual
contributions within collaborative projects can be challenging, and the open-ended nature
might not suit all learning styles (Pereira Pessoa, 2023).
Traditional PBL excels in nurturing problem-solving abilities. Learners actively research and
apply knowledge to tackle fictitious scenarios, promoting self-directed learning and critical
thinking. In medical education, PBL scenarios simulating complex patient diagnoses hone
students' research skills and teamwork in presenting treatment options (Barrows & Tamblyn,
1980). Nonetheless, its suitability for diverse subjects and learning objectives remains
uncertain (Savery, 2006). Moreover, the time-consuming nature of problem analysis and
research demands strong facilitation skills from instructors to ensure productive team
collaboration (Dolmans et al., 2005).
WBL stands out for its emphasis on practical skill application in real-world workplaces.
Internships like IT network maintenance projects provide invaluable experiential learning
and career readiness. Studies highlight its positive impact on graduate employability and skill
development (Abelha et al., 2020). However, access to high-quality WBL opportunities can
be unequal, and ensuring qualified mentors and structured learning experiences within
diverse organizational settings presents challenges (Bahl & Dietzen, 2019). Furthermore,
potential safety concerns require careful planning and oversight, particularly in high-risk
environments.
Each model possesses distinct strengths and weaknesses. Kilpatrick's PBL fosters
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 6
Applied Business and Economics Journal
2024 vol. 2
engagement and authentic learning, traditional PBL cultivates problem-solving and self-
directedness, while WBL prioritizes practical application and career readiness. Recognizing
these nuances allows for informed selection and adaptation of these models to suit diverse
learning needs and contexts. In today's rapidly evolving work environment, combining
elements from different training models holds immense potential to create a robust training
approach that enhances engagement, develops critical skills, and prepares individuals for
success in the real world.
The literature offers several training evaluation models, but the most common methods
follow Kirkpatrick’s evaluation methodology, which has specific evaluation levels for short
and long training programmes: reaction, learning, behaviour and outcomes (Fregonese et al.,
2018; Grohmann & Kauffeld, 2013). Phillips and Phillips’ (2016) method has five levels of
evaluation and focuses on the returns associated with investment in training, expressed as a
percentage or proportion. There are many ways to isolate the impact of training, including
performance monitoring (Brahmana et al., 2018), surveys (Asadullah et al., 2015; Chochard
& Davoine, 2011; Kucherov & Manokhina, 2017; Fregonese et al., 2018; Grohmann &
Kauffeld, 2013), management estimates and information reported directly by programme
participants (Subramanian et al., 2012).
Curado and Bernardino’s (2018) study, based on the Kirkpatrick and Phillips models, used
fuzzy set qualitative comparative analysis (fsQCA) to provide a deeper analysis of the factors
associated with higher or lower ROI. The study highlights the importance of organising
training programmes in small groups. However, its results cannot be generalised due to the
use of national data. Another limitation is the lack of some training-related data, such as the
gender of programme trainers, the predominant gender of trainees for each programme, and
the evaluation scores of trainees and trainers.
In a pioneering move within the insurance field, a recent study introduced a framework that
integrates Balanced Scorecard (BSC) models with the best-worst method (BWM) to assess
performance across two distinct time periods. The amalgamated BSC-BWM model serves
as a valuable tool for managers and decision-makers, enabling them to discern and interpret
the competitive strengths of a company. In summary, this model considers both past and
future parameters with two different time periods. This allows for a better analysis and study
of the organisation's operational business management data. This, in turn, facilitates swift
and effective decision-making processes. It's important to note that while this integrated
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 7
Applied Business and Economics Journal
2024 vol. 2
model is currently tailored for a specific business category, there is ample potential for its
application across various industries (Dwivedi et al., 2021).
Salimian (2021) used the general Phillips model and the AHP approach to calculate the ROI
of two training modules on electrical protection and uninterruptible power supply (UPS) in
a gas company. Although the results are reasonable, they are specific to these two training
modules. In addition, they do not evaluate the relationship between multiple KPIs, such as
financial, customer, internal process and training KPIs, at a general level.
Several studies in the literature have developed models to measure the effectiveness of
training in companies (Grohmann & Kauffeld, 2013; Phillips & Phillips, 2016; Fregonese et
al., 2018). However, no model or approach has been found that both uses AHP decision
making tools and also integrates an organisation’s KPIs and evaluates the impact of training
investment on them. Our aim is to propose an approach for measuring the impact of training
investment indicators in conjunction with the AHP methodology, in order to reduce the
qualitative factor in the evaluation of training.
3. Methodology
This paper is based on the integration of a theoretical approach for evaluating the impact of
training on KPIs with the AHP methodology. This new approach has been named
hierarchical participatory action analysis (APAJ) and is carried out in the five steps described
below.
Step 1. Creation of an integrated approach to training profitability: this approach was defined
and developed by an in-company training expert with the aim of reliably measuring the
profitability of training investments and improving efficiency. To this end, observations were
first carried out in six companies where the expert works as a consultant. Key questions were
then identified in relation to what was observed (initial analysis and organisational analysis),
followed by the relevant actions. A literature review was subsequently conducted to
corroborate the selected KPIs.
The proposed KPIs offer a wide range of measures covering both financial and non-financial
aspects of in-company training. These KPIs play a key role in justifying investment in
training. The collection of data combines internal sources with insights from staff surveys
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 8
Applied Business and Economics Journal
2024 vol. 2
and questionnaires. This integrated approach ensured that our data collection captures both
quantitative metrics and qualitative insights essential for our new methodology.
Training costs (KPI-1 and KPI-2): These indicators enable the calculation of both direct and
indirect costs associated with training programmes. An accurate assessment of these costs
provides a complete picture of training-related expenditure and is essential for calculating
ROI (Devarakonda, 2019).
Competencies (KPI-3): Competency levels should be measured before and after the training
intervention to assess the impact of development programmes on employee skills and
knowledge. This KPI makes it possible to assess whether employees acquire the necessary
competencies to achieve the company’s objectives (Gómez, 2022).
The economic variables (KPI-4 to KPI-8) quantify the financial impact of training.
Productivity, turnover, efficiency, quality and lead time can increase as a result of training.
These KPIs provide concrete data on how training contributes to financial results (Aragón-
Sánchez et al., 2003).
Organisational factors (KPI-9 and KPI-10) are taken into account to measure the work
climate and competitiveness. García et al., (2012) conducted a study on the impact of training
on company culture and competitive position. A favourable work climate can lead to higher
employee retention and better performance, while improved competitiveness can generate
strategic advantages.
Social factors (KPI-11 to KPI-13): These KPIs assess the impact of training on the social
dynamics of the organisation. Internal promotion, attrition and absenteeism are indicators of
how training affects internal mobility and talent retention. Continuous training can be used
as a tool to motivate employees (Rial, 2016).
Training factors (KPI-14 to KPI-16): These indicators evaluate the effectiveness of training
programmes in terms of learning transfer, thus assessing the new level of competence as well
as the development of transferable skills, such as the previously mentioned soft skills
(Sreehari, 2021).
This analysis provided the final approach to be used in the study. Information on the proposed
KPIs (analysed in the first three steps) is collected and combined with the AHP methodology
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 9
Applied Business and Economics Journal
2024 vol. 2
to determine the impact of training on them. These KPIs are shown in Figure 1.
Step 2. Establishment of hierarchical relationships: the expert team established the hierarchy
and relative importance of each step (criteria) in relation to each KPI (alternatives). The
Super Decisions software was used to extract the weights of the established hierarchy and to
perform the subsequent analysis of the AHP methodology.
Step 3. A comparison matrix (j) was created using a pairwise comparison index (jij). Each
decision maker was asked to express the relative importance of two decision items at the
same level (e.g. two criteria) using a nine-point scale. The pairwise comparison scores were
compiled and the pairwise comparison matrices were constructed for each decision maker.
The pairwise comparison was performed using the Saaty (1980) scale of relative importance
scores, which ranges from 1 to 9, where 1 indicates equal importance of the two items and 9
indicates high importance of one item (matrix row component) compared to the other item
(matrix column component) (Meade & Sarkis, 1999). For n number of comparison items, the
comparison matrix is defined in Eq 1:
=
(1)
For diagonal entries, i.e. i=j, jij=1, the upper right triangular comparison elements jij must be
defined by the decision maker, while the lower left triangular entries are obtained by taking
reciprocals, i.e. jji=1/jij.
Step 4. The consistency of the pairwise comparisons of Eq 1 had to be checked as it depends
on people’s preferences (Zahedi, 1986). The consistency index (CI) is estimated with the
eigenvalue λ and the eigenvector W by solving the formulation defined in Eq 2.
()= 0 (2)
The maximum eigenvalue is therefore given by λmax=max(λ) (Saaty, 1980). The CI indicates
whether a decision maker provides consistent values, where n is the dimension of the
comparison matrix. The CI is defined in Eq 3.
=() (1) (3)
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 10
Applied Business and Economics Journal
2024 vol. 2
where n is the number of elements to be compared in the matrix. The final inconsistency is
obtained using the consistency ratio (CR) defined in Eq 4.
=(
)100 (4)
Where RI is the random index obtained by averaging the CI of a randomly generated
reciprocal matrix (Saaty, 1980). The maximum accepted value of CR is 10% (Saaty, 1980).
If the consistency test is not passed, the decision maker must revise the original scores in the
pairwise comparison matrix. This would mean consulting the experts again.
Step 5. Calculate the weights: the result of the priority vectors was determined using Eq 5:
= (5)
Where,
j is the pairwise comparison matrix;
W is the vector of priorities; and
λmax is the maximum eigenvalue of the matrix A.
4. Results
The case study is a Chilean printing SME with a market segment of small businesses in the
city of Rancagua, Chile. An analysis of Porter’s 5 Forces showed that the sector is highly
competitive. Because of this, the company is always looking to increase its competitiveness
by training its workers, with a focus on measuring the return on investment of this training.
To illustrate this, the APAJ approach is described below, step by step.
Step 1: In this case, the owners of the SME met with the expert trainer who created the
approach to review and define the KPIs to be measured in the company. Figure 1 shows the
steps of the process and the KPIs included in each one.
Figure 1. Steps of the APAJ approach
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 11
Applied Business and Economics Journal
2024 vol. 2
Source: Author’s own elaboration
Step 2: The hierarchical structure was constructed with decision elements (e.g. criteria and
detailed criteria). The KPI levels and names are shown in Table 1. Figure 2 then presents the
actual hierarchy.
Table 1. KPI levels and names
Training cost: Organisational factors: Economic variables:
KPI-1 Direct cost
KPI-2 Indirect cost
KPI-9 Work climate
KPI-10 Competitivity
KPI-4.e Productivity
KPI-5.e Turnover
KPI-6.e Efficiency
KPI-7.e Quality
KPI-8.e Time
Competencies: Social factors:
KPI-3 Competency level
KPI-11 Internal promotion
KPI-12 Attrition rate
KPI-13 Work absenteeism
Economic variables: Training factors:
KPI-4 Productivity
KPI-5 Turnover
KPI-6 Efficiency
KPI-14 Learning transfer
KPI-15 New competency level
KPI-16 New transferable skills
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 12
Applied Business and Economics Journal
2024 vol. 2
KPI-7 Quality
KPI-8 Time
Source: Author’s own elaboration
Figure 2. Hierarchical levels of the case study using the APAJ approach
Source: Author’s own elaboration
Step 3: The pairwise comparison matrix was then developed. For the case study, each KPI
was compared for each hierarchical level. The group of experts that defined the weight of
each KPI in relation to the others was composed of owners, shareholders of the SME, the
expert/creator of the approach and company employees such as its accountant. As mentioned
above, the relative weights were determined using a scale of 1 to 9, where 1 indicates equal
importance of the two elements (KPIs) and 9 indicates greater importance of one of the two
elements (KPIs). As an example of all the comparisons, Figure 3 shows the pairwise
comparison of the KPIs at the “Impact analysis: Economic variables” level.
Figure 3. Pairwise comparison of the “Impact analysis” level
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 13
Applied Business and Economics Journal
2024 vol. 2
Source: Author’s own elaboration
Step 4. For the case study, the inconsistency of each of the scores was checked. For all the
scores, the CR was less than 10%. For example, as shown in Figure 4, the CR for KPI-16
(New transferable skills) was 0.08363, which is acceptable since the maximum is defined.
Figure 4. Inconsistency index of KPI-16: New transferable skills
Source: Author’s own elaboration
Step 5. In this step, the weight of each KPI was calculated using the Super Decision software.
Table 2 shows the final weight of each KPI. According to the AHP methodology, the KPI
pairs should be compared from the first level (“Initial analysis”) to the following levels
(“Organisational analysis: Observation” and “Impact analysis: Economic factors”). Although
the methodology establishes these weights hierarchically for each level, i.e. for each node
(KPI) of each cluster (hierarchy level), the aim of this study is to determine the importance
of the KPIs at the “Impact analysis” level compared to the next highest level. Therefore,
these last two levels are normalised in Table 2.
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 14
Applied Business and Economics Journal
2024 vol. 2
Finally, the indicators with the greatest importance were found to be KPI-5.e and KPI-7.e,
i.e. impact on turnover (14.1%) and impact on quality (14.8%), which indicates that SME
decision makers should emphasise training related to these two KPIs, as this would improve
the company’s performance by almost 30%. This in turn would affect the overall
performance of the organisation. Another important KPI is impact on time (11.7%), which,
together with the above two indicators, would help to improve SME performance by up to
41%.
Table 2. Priorities – Overall weights of the KPIs
KPI name Name Percentage
Organisational
analysis: Observation
Competitivity KPI-10 10.0%
New competency level KPI-15 8.4%
New transferable skills KPI-16 7.3%
Work absenteeism KPI-13 6.9%
Internal promotion KPI-11 6.6%
Learning transfer KPI-14 6.2%
Work climate KPI-9 2.2%
Attrition rate KPI-12 1.5%
Impact analysis:
Economic factors
Impact on quality KIP-7.e 14.8%
Impact on turnover KPI-5.e 14.1%
Impact on time KPI-8.e 11.7%
Impact on productivity KPI-4.e 5.4%
Impact on efficiency KPI-6.e 5.0%
Total 100.0%
Source: Author’s own elaboration
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 15
Applied Business and Economics Journal
2024 vol. 2
5. Discussion
The research investigated the impact of training investments on key performance indicators
(KPIs) in a Chilean manufacturing SME, employing an AHP-based approach. The analysis
identified relevant KPIs across three stages: initial economic factors, organisational
observation, and the impact of training on the initial factors. By focusing on employee
retention (KPI-5.e), product quality (KPI-7.e), and production time reduction (KPI-8.e), the
research revealed the potential for training to significantly improve overall organisational
performance.
The emphasis on employee retention (KPI-5.e) aligns with the work of Bramhanti et al.
(2018) and Asadullah et al. (2015), who demonstrated the positive influence of training on
reducing turnover. This research builds upon those findings by pinpointing specific areas
like retention-focused programs, offering a more targeted approach to maximizing this
impact and empowering SMEs to optimize their training investments.
Similarly, the focus on quality (KPI-7.e) echoes the insights of Phillips and Phillips (2016),
who highlighted the role of training in improving manufacturing quality. The present
research goes beyond this by offering a more granular perspective. Identifying skill
development and process optimization training as key contributors, it provides actionable
advice for SMEs to refine their training programs for targeted quality improvement.
Furthermore, the importance placed on reducing production time (KPI-8.e) aligns with the
research of Curado and Bernardino (2018) and Salimian (2021), who established a
connection between training and increased efficiency. This research adds specificity by
suggesting that process optimization training directly impacts production time, contributing
to overall efficiency gains. This targeted approach empowers SMEs to prioritize specific
training areas that directly address their productivity needs.
While the results are promising, the single case study design necessitates further research
with larger, diverse samples. Expanding the scope across industries and contexts would
enable validation and generalization of the findings, ensuring their broader applicability.
Additionally, while the AHP-based approach provided a structured framework, future
studies could explore alternative methods for enriched understanding. The Analytic Network
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 16
Applied Business and Economics Journal
2024 vol. 2
Process (ANP) might offer valuable insights by considering interdependencies between
KPIs, while fuzzy logic could address inherent human judgment uncertainties, adding
nuance to the analysis.
Finally, exploring the influence of cultural factors on training effectiveness and ROI holds
immense potential. Tailoring training design and evaluation methods to specific cultural
contexts could yield valuable insights, ensuring the effectiveness of training initiatives across
diverse settings. This opens exciting avenues for future research, leading to a more nuanced
and culturally informed understanding of training's impact on SMEs.
By comparing our findings with existing research, this paper highlights the promising role
of targeted training in impacting key performance indicators like employee retention,
quality, and production time in SMEs. However, acknowledging limitations and considering
future research directions using larger datasets, diverse samples, and alternative evaluation
methods is crucial for enriching our understanding of training ROI and its broader impact on
organisational performance across various contexts and cultures.
6. Conclusions
Although there is broad consensus on the need to assess the return on investment in
training within companies, often the evaluation is limited to measuring the satisfaction of
the students and the individual assessment of their knowledge. This is mainly because,
when designing training, there is a lack of educational planning followed by evaluating
through cost analysis, performance, and the potential transfer of knowledge to the
workplace.
There are many technical difficulties in calculating the return on investment in training, with
the most significant obstacle in the majority of studies conducted in this field.
This paper proposes an AHP and BSC based approach for evaluating the impact of training
investments on KPIs in a manufacturing SME in Chile. The analytic hierarchy is structured
in three steps: initial analysis (economic factors), organisational analysis (observation) and
the impact of training investment on the economic factors in the initial analysis, in terms of
the key performance indicators in each step. Since the human decision-making process
usually involves confusion and vagueness, a well-organised AHP information system was
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 17
Applied Business and Economics Journal
2024 vol. 2
constructed to make it easier. This is a first stage of research (determining the impact of
training investment on KPIs), the ultimate aim of which is to determine the return on
investment (second stage), which will be developed in subsequent studies based on this
research.
Finally, the KPIs with the greatest impact on overall organisational performance were found
to be impact on quality and impact on turnover. Therefore, SME owners should focus
training initiatives on these two KPIs in order to achieve overall improvement in their
organisations. This research opens up new avenues for research. For instance, other methods
such as the analytic network process (ANP) and fuzzy logic should be evaluated, and the
proposed approach should be applied to companies in other sectors.
Some comments and conclusions from the studied company include the following: they have
pointed out that the method is difficult to understand, but they recognize the need for the
staff to internalize the methodology. They have also grasped the utility of the method.
Ultimately, the owner and director of the company believe that the obtained methodology
appropriately represents and supports the business strategy.
Regarding the study's limitations, it's essential to note that it was confined to a single SME
in the province of Rancagua, Chile, using a small sample. The findings may not be
generalizable to other organisations in different provinces or sectors. A larger sample and
the inclusion of participants from various organisations could have enhanced the scope of
this study and the transferability of the findings.
As future research directions, exploring alternatives such as other methods like ANP or fuzzy
logic is suggested. In the first case, the goal is not only to establish a hierarchical structure
among indicators but to treat them as a network. In the second case, it is proposed to evaluate
the decisions used in this study not only as binary outcomes but to consider that decisions
may vary between these two options, establishing a range in decision-making.
Furthermore, applying this study to SMEs in different areas and other types of companies,
whether medium or large, is recommended to assess the method obtained in the research.
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 18
Applied Business and Economics Journal
2024 vol. 2
References
Abelha, M., Fernandes, S., Mesquita, D., Seabra, F., & Ferreira-Oliveira, A. T. (2020).
Graduate Employability and Competence Development in Higher Education—A
Systematic Literature Review Using PRISMA. Sustainability, 12(15), 5900.
https://doi.org/10.3390/su12155900
Aragón-Sánchez, A., Barba-Aragón, I., & Sanz-Valle, R. (2003). Effects of training on business
results1. The International Journal of Human Resource Management, 14(6), 956-980.
https://doi.org/10.1080/0958519032000106164
Álvarez Pérez, C., Rodríguez Montequín, V., Ortega Fernández, F., & Villanueva Balsera, J.
(2017). Integrating analytic hierarchy process (AHP) and balanced scorecard (BSC)
framework for sustainable business in a software factory in the financial sector.
Sustainability, 9(4), 486. https://doi.org/10.3390/su9040486
Asadullah, M. A., Peretti, J. M., Ghulam Ali, A., & Bourgain, M. (2015). Firm size, ownership,
training duration and training evaluation practices. European Journal of Training and
Development, 39(5), 429-455. https://doi.org/10.1108/EJTD-10-2014-0072
Bahl, A., & Dietzen, A. (Eds.). (2019). Work-based Learning as a Pathway to Competence-
based Education. A UNEVOC Network Contribution.
https://doi.org/10.3390/su12155900
Barrows, H. S., & Tamblyn, R. M. (1980). Problem-based learning: An approach to medical
education (Vol. 1). Springer Publishing Company.
Brahmana, R. K., Brahmana, R. K., & Fei Ho, T. C. (2018). Training and development policy,
corporate governance, and firm performance. Gadjah Mada International Journal of
Business, 20(1), 59-87.
https://search.informit.org/doi/10.3316/informit.569434992657313
Castillo Sarabia, J. C., & Villalpando Cadena, P. (2019). El papel de las competencias laborales
en el ámbito educativo: una perspectiva de reflexión e importancia. Revista Daena
(International Journal of Good Conscience), 14(1). http://www.spentamexico.org/v14-
n1/A3.14(1)30-51.pdf
Chochard, Y., & Davoine, E. (2011). Variables influencing the return on investment in
management training programs: a utility analysis of 10 Swiss cases. International Journal
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 19
Applied Business and Economics Journal
2024 vol. 2
of Training and Development, 15(3), 225-243. https://doi.org/10.1111/j.1468-
2419.2011.00379.x
Clares, P. M., Lorente, C. G., & Quintela, N. R. (2019). Competencias para la empleabilidad:
un modelo de ecuaciones estructurales en la Facultad de Educación. Revista de
Investigación Educativa, 37(1), 57-73. https://doi.org/10.6018/rie.37.1.343891
Curado, C., & Bernardino, G. (2018). Training programs’ return on investment in the
Portuguese railway company: A fuzzy‐set qualitative comparative analysis. International
Journal of Training and Development, 22(4), 239-255. https://doi.org/10.1111/ijtd.12136
Devarakonda, S. (2019). Calculating the Economic Viability of Corporate Trainings
(Traditional & eLearning) using Benefit-Cost Ratio (BCR) and Return On Investment
(ROI). International Journal of Advanced Corporate Learning (iJAC), 12(1), 41-57.
https://doi.org/10.3991/ijac.v12i1.9735
Dolmans, D.H., De Grave, W., Wolfhagen, I. H., & van der Vleuten, C. P. (2005). Problem-
based learning: future challenges for educational practice and research. Medical
education, 39(7), 732-41. https://doi.org/10.1111/j.1365-2929.2005.02205.x
Domènech-Casal, J. (2019). Contexto y modelo en el aprendizaje basado en proyectos.
Alambique Didáctica de las Ciencias Experimentales, núm. 98, 71-76.
Dwivedi, R., Prasad, K., Mandal, N., Singh, S., Vardhan, M., & Pamucar, D. (2021).
Performance evaluation of an insurance company using an integrated Balanced Scorecard
(BSC) and Best-Worst Method (BWM). Decision Making: Applications in Management
and Engineering, 4(1), 33-50.
Elmoutanna, N., & Motii, N. (2022). Soft skills from university to workplace: A literature
review. International Journal of Accounting, Finance, Auditing, Management and
Economics, 3(5-1), 187-198.
https://www.ijafame.org/index.php/ijafame/article/view/734
Evensen, D. H., Hmelo, C. E., & Hmelo-Silver, C. E. (2000). Problem-based learning: A
research perspective on learning interactions. Routledge.
Fregonese, C., Caputo, A., & Langher, V. (2018). Italian translation of the questionnaire for
professional training evaluation. International Journal of Training and Development,
22(1), 34-50. https://doi.org/10.1111/ijtd.12117
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 20
Applied Business and Economics Journal
2024 vol. 2
Galleguillos, R., Altamirano, S., García, M. V., Pérez, F., & Marcos, M. (2017, 12-17
September). Low cost CPPs for industrial control under FAHP algorithm. 22nd IEEE
International Conference on Emerging Technologies and Factory Automation (ETFA).
Limassol, Cyprus. https://doi.org/10.1109/ETFA.2017.8247775
García Govea, E. M, Escalate Domínguez, M., & San Agustín, Y. Q. (2012). Importancia del
clima laboral en los resultados de una empresa y la competitividad. Contribuciones a la
Economía, (2012-11). https://www.eumed.net/ce/2012/clima-laboral-empresa-
competitividad.html
Gómez Lega, J. L. (2022). Modelo de cálculo del ROI de la formación en el área de desarrollo
de competencias y su repercusión en la organización empresarial. [Tesis doctoral,
Universidad a Distancia de Madrid]. Repositorio Institucional de la Universidad a
Distancia de Madrid (UDIMA). http://hdl.handle.net/20.500.12226/1164
Grohmann, A., & Kauffeld, S. (2013). Evaluating training programs: development and
correlates of the Questionnaire for Professional Training Evaluation. International
Journal of Training and Development, 17(2), 135-155. https://doi.org/10.1111/ijtd.12005
Karim, M. M., Choudhury, M. M., & Latif, W. B. (2019). The impact of training and
development on employees’ performance: an analysis of quantitative data. Noble
International Journal of Business and Management Research, 3(2), 25-33.
Kopp, T., Kinkel, S., Schäfer, T., Kieslinger, B., & Brown, A. J. (2020). Measuring the impact
of learning at the workplace on organizational performance. International Journal of
Productivity and Performance Management.
https://www.emerald.com/insight/content/doi/10.1108/IJPPM-12-2018-0443/full/html
Kucherov, D., & Manokhina, D. (2017). Evaluation of training programs in Russian
manufacturing companies. European Journal of Training and Development, 41(2), 119-
143. https://doi.org/10.1108/EJTD-10-2015-0084
Lechhab, N., Iordanova, I., & Forgues, D. (2022, May). Evaluation of the return on investment
of BIM—the case of an architectural firm. In S. Walbridge, M. Nik-Bakht, K. Tsun Wai
Ng, M. Shome, M. S. Alam, A. el Damatty, & G. Lovegrove (Eds.), Proceedings of the
Canadian Society of Civil Engineering Annual Conference 2021, (pp. 43-443). Springer
Nature Singapore. https://doi.org/10.1007/978-981-19-1029-6_33
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 21
Applied Business and Economics Journal
2024 vol. 2
Lorente, Bilbao, J.I., Doblas Pozo, P., & Zaldumbide Amezaga, I. (2011). La competencia en
comunicación en el aprendizaje orientado a proyectos (PBL). DUGiDocs. https://dugi-
doc.udg.edu/bitstream/handle/10256/3716/296.pdf?sequence=1&isAllowed=y
Mardani, A., Jusoh, A., & Zavadskas, E. K. (2015). Fuzzy multiple criteria decision-making
techniques and applications – Two decades review from 1994 to 2014’, Expert Systems
with Applications. Elsevier Ltd, 42(8), pp. 4126–4148.
https://doi.org/10.1016/j.eswa.2015.01.003
Massingham, R., Massingham, P. R., & Dumay, J. (2018). Improving integrated reporting: a
new learning and growth perspective for the balanced scorecard. Journal of Intellectual
Capital, 20(1), 60-82. https://doi.org/10.1108/JIC-06-2018-0095
Matia Portilla, V. (2016). Formación profesional y formación para el empleo en España: del
aprendizaje de tareas al desarrollo de competencias. [Tesis doctoral, Universidad de
Valladolid]. Repositorio documental de la Universidad de Valladolid.
https://doi.org/10.35376/10324/22300
Meade, L. M., & Sarkis, J. (1999). Analyzing organizational project alternatives for agile
manufacturing processes: an analytical network approach. International Journal of
Production Research, 37(2), 241-261. https://doi.org/10.1080/002075499191751
Murtazin, K., Shvets, O., & Piho, G. (2020, 21-24 October). Literature review on work-based
learning. 2020 IEEE Frontiers in Education conference (FIE), Uppsala, Sweden.
https://doi.org/10.1109/FIE44824.2020.9274264
Pereira Pessoa, M. V. (2023). Guidelines for teaching with ill-structured real-world engineering
problems: insights from a redesigned engineering project management course. European
Journal of Engineering Education, 48(4), 1-18.
https://doi.org/10.1080/03043797.2023.2194850
Perusso, A., & Wagenaar, R. (2022). The state of work-based learning development in EU
higher education: learnings from the WEXHE project. Studies in Higher Education,
47(7), 1423-1439. https://doi.org/10.1080/03075079.2021.1904233.
Phillips, J. J., & Phillips, P. P. (2016). Handbook of training evaluation and measurement
methods. Routledge. https://doi.org/10.4324/9781315757230
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 22
Applied Business and Economics Journal
2024 vol. 2
Quezada, L., Palominos, P., Galleguillos, R., & Olmedo, A. (2014) A method for generating
strategy maps using ANP. Journal of Manufacturing Technology Management, 25(8),
1090-1104. https://doi.org/10.1108/JMTM-06-2014-0081.
Rial, P. G. (2016). El marketing aplicado a la formación en la empresa: propuesta para la
promoción de la formación entre los empleados a través de "lifelong learning”. [Tesis
doctoral, Universidad de Santiago de Compostela]. Minerva – Repositorio Institucional
de la USC. https://minerva.usc.es/xmlui/handle/10347/14634
Saaty, T. L. (1980). The analytic hierarchy process. McGrew-Hill.
Salimian, F. (2021). Measuring the return on investment of training modules of electrical
protection and uninterruptible power supply (UPS) using the corrective and AHP
approaches. Mathematical Problems in Engineering, 2021, (Article ID 2635761), 1-10.
https://doi.org/10.1155/2021/2635761
Sreehari, P. (2021). Essential Soft Skills for Workplace Success. Revista Gestão Inovação e
Tecnologias, 11(4), 2648-2654. http://revistageintec.net/old/wp-
content/uploads/2022/03/2306.pdf
Subramanian, K. S., Sinha, V., & Gupta, P. D. (2012). A study on return on investment of
training programme in a government enterprise in India. Vikalpa: The Journal for
Decision Makers 37(1), 31-48. https://doi.org/10.1177/0256090920120103
Savery, J. R. (2006). Overview of Problem-based Learning: Definitions and Distinctions.
Interdisciplinary Journal of Problem-Based Learning, 1(1), 9-20.
https://doi.org/10.7771/1541-5015.1002
Tan, O. S. (2021). Problem-based learning innovation: Using problems to power learning in
the 21st century. Gale Cengage Learning.
Tonhäuser, C., & Seeber, S. (2014). Assessing the Return on Investments in Human Resource
Development: Theory and Practice. In Andresen, M., Nowak, C. (Eds). Human Resource
Management Practices: Assessing Added Value (pp. 69-87). Springer, Cham.
https://doi.org/10.1007/978-3-319-08186-1_5
Urbancová, H., Vrabcová, P., Hudáková, M., & Petrů, G. J. (2021). Effective training
evaluation: The role of factors influencing the evaluation of effectiveness of employee
training and development. Sustainability, 13(5), 1-14.
https://doi.org/10.3390/su13052721
Using the analytic hierarchical process to investigate the impact of training on KPIs in an
SME: A case study in the printing industry…. 23
Applied Business and Economics Journal
2024 vol. 2
Van der Klink, M., Schlusmans, K., & Boon, J. (2007). Designing and implementing views on
competencies. In M. Sicilia (Ed.), Competencies in organizational e-learning: Concepts
and tools (pp. 221-233). IGI Global. https://doi.org/10.4018/978-1-59904-343-2.ch010
Van Rooij, S. W., & Merkebu, J. (2015). Measuring the business impact of employee learning:
A view from the professional services sector. Human Resource Development Quarterly,
26(3), 275-297. https://doi.org/10.1002/hrdq.21211
Zahedi, F. (1986). The analytic hierarchy process—a survey of the method and its applications.
Interfaces, 16(4), 96-1. https://www.jstor.org/stable/25060854