Content uploaded by Muhammad Ashraf
Author content
All content in this area was uploaded by Muhammad Ashraf
Content may be subject to copyright.
African Journal of Business Management Vol. 6(1), pp. 274-279, 11 January, 2012
Available online at http://www.academicjournals.org/AJBM
DOI: 10.5897/AJBM11.2087
ISSN 1993-8233 ©2012 Academic Journals
Full Length Research Paper
Exploring the link between Kirkpatrick (KP) and
context, input, process and product (CIPP) training
evaluation models, and its effect on training evaluation
in public organizations of Pakistan
Muhammad Maqsood Khalid
1
, Chaudhry Abdul Rehman
2
and Muhammad Ashraf
3
*
1
HRM Training and Research Directorate, FBR Islamabad, Pakistan.
2
Superior University, Raiwind Road, Lahore, Pakistan.
3
Azra Naheed Center for Research and Development, Superior University/College, Lahore, Pakistan.
Accepted 1 August, 2011
In the awake of performance gap detected in public organizations of Pakistan, this paper proposes a
training evaluation framework to ensure training transfer. This study is based on theoretical perspective
of literature review and finds a link between Kirkpatrick (KP) and context, input, process and product
(CIPP) training evaluation models to enhance efficacy of training in Pakistan. This study reiterates that
pre-training context is imperative to finding viable training criterion to make training successful.
Evaluation lacks criteria setting in Pakistan training institutions. This study is attempting to propose a
training evaluation framework using KP and CIPP models to ensure financial viability as well as,
alleviating and declining performance of public organizations. The framework proposes a strategy to
ensure training transfer. Context is seen as a primary requirement in this framework, to help framing a
viable training design aimed at training transfer. It thus, presents useful information for organizations
with limited resources, human resource research fellows and research students as well. This paper
proposes a new framework of training evaluation based on vision of Kirkpatrick and Stufflebeam (CIPP)
models. It also emphasizes on understanding the entire situation of the organization from the
beginning, aids in tracking organization needs, its operational objectives, training designs,
implementation and monitoring. These items will pave way for subsequent evaluation. The researcher
has shared his experience as a trainer with body of knowledge on training evaluation.
Key words: Pakistan, training evaluation framework, transfer of training, Kirkpatrick, CIPP models.
INTRODUCTION
Pakistan is a fertile land of four seasons, which has
grown amidst crisis of governance, poor performance of
public institutions in terms of lack of accountability,
inefficient management and external debts. Public sector
has a keynote role in socioeconomic development of the
country, makes use of a significant part of the country’s
resources, creates wealth competencies and better
innovative services (Jia and Fan, 2008).
*Corresponding author. E-mail: Ashraf.scholar@gmail.com.
Efficiency of the public service largely depends upon the
quality of its people that constitute the public service
(Garavan and McGuire, 2001). Its performance and
productivity boost the progress and efficiency of other
sectors. During the last decades, there has been a
question mark about the declining performance standards
of public sector employees. To fill the performance gap,
training intervention has been an effective tool, in
developing the workforce and ensuring transfer of training
to the workplace.
The purpose of this study is to check whether training
investment has been effective, enabled training transfer
and contributed to improved performance on job.
IMPORTANCE OF TRAINING EVALUATION
Training has been accepted as a viable human resource
practice to get workplace transformation (Pidd, 2004).
Training has a structured plan and format, targets
employees to improve efficiency and enhance perfor-
mance on job (Baldwin and Ford, 1988). But training is
not like a garage where people problems can be solved
in a short period of time (Holton, 1996). Training is not
always an affair of skill acquirement (Saks and Belcourt,
2006). It aims to contribute to improved job performance
on workplace and augments training efficacy (Pidd,
2004). Despite, the gross importance of training, training
evaluation is not carried out in majority of organizations.
Almost 99% of organizations do not like to evaluate
impact of training on performance (Alvarez et al., 2004).
This is due to scantiness of feasible training evaluation
methods (Aghazadeh, 2007). Little work has been done
in developing countries to introduce fresh approaches to
evaluate training (Kontoghiorghes, 2004). Evaluation
means overall social and financial value of a training
system in delivering worth (Pineda, 2003). Training loses
its efficacy when it is evaluated loosely (Burke and
Hutchins, 2008). Evaluation should be a part of planning
process (Meignant, 1997). Information generated by
evaluation becomes vital, for planning, decision making
and training of an investment and not expense.
Brinkerhoff (2006) regards training as an investment
and according to him, every investment bears an return
on investment (ROI) value. He uses $ value at level 3 of
his training evaluation model to reduce margin of errors.
The benefits associated with investment in training are
enormous but hard to measure (Kirkpatrick, 1994).
Training affects attitude of employees, and if they show
positive change in behavior on job, expense on training
becomes investment (Brinkerhoff, 2006). Training can
therefore be planned, to assess any change in
performance, and behavior on job to know training effec-
tiveness. If a trainee accepts learning during training, he
will definitely transfer it to the workplace (Bartlett, 2001).
Impact of training on training transfer
Training transfer refers to an application of knowledge,
skill and attitude (KSA), learnt during training at the
workplace (Baldwin and Ford, 1988). Transfer of training
is not so high in developing countries, as a result, skills
learnt are faded not applied on job. Investment made on
training wastes away and training loses its efficacy
(Pineda, 2008). There is the need is to know what
trainees learn during training so as to reproduce it on job,
in form of better performance. A study with samples of
Khalid et al. 275
150 organizations reported that, within 6 months to 1 year
after training, less than 50% of staffs on average were
able to transfer the training to the job (Saks and Belcourt,
2006). This gives a disheartening outcome of training.
Another study, showed dismal results when it was found
that less than 15% trainees are able to learn and transfer
to workplace in form of improved job performance (Velda
et al., 2007). This scenario warrants creating a model to
ensure training transfer to workplace within prescribed
time. Conditions look bleaker when it was researched
that 10 to 15% of learning in a training program is applied
on job (Brinkerhoff, 2006). Various other studies found
transfer rates between 10 to 40% (Baldwin and Ford,
1988; Burke and Hutchins, 2007). Transfer of training to
the workplace is imperative in raising the efficacy of
training and its effectiveness (Barlow, 2006).
To ensure transfer of training, training program must be
evaluated at all levels (Bartlett, 2001). Any study
identified as faltering factors inhibit transfer of training.
Pidd (2004) conducted a study and identified two factors,
that is, personal characteristics of trainees and social
support at workplace; the key factors that supplement
transfer rate to the workplace. Characteristics of trainees
such as, ability, aptitude, personality, self efficacy, desire
for success, willingness to attend training, value beliefs
about training, and prior experience of trainees, enables
training transfer (Swanson, 1996). Workplace
characteristics such as, management support and job
atmosphere were found important predictors of training
effectiveness (Baldwin and Ford, 1988; Maqsood et al.,
(2011). Situational factors like, line management
commitment and organization support in form of equitable
reward system, also contributes to transfer of training to
the work place (Kontoghiorghes, 2004). Trainees’ quest
for learning is a prerequisite in learning something new to
apply on job (Lim and Johnson, 2002; Alvarez et al.,
2004) and adds good training design to ensure training
impact. Appropriate design and delivery of a training
program is a stepping stone to transfer of learning (Axtell
et al., 1997; Maqsood et al., 2011). According to the
findings of Noe (2004) salient features of the learning
environment helps in building an effective training design.
Another study found that personal characteristics of the
trainees and the social support given at the workplace
prove supportive in implementing training (Pidd, 2004).
When we look at trainees’ characteristics and social
support at workplace in Pakistani scenario, these two
areas were found to be weak (Khilji, 1999). Transfer of
training takes place when trainees effectively apply KSA
learnt during training (Saks and Belcourt, 2006). In this
context, a training evaluation mechanism is direly needed
to verify training transfer to workplace in developing
countries environment. This seems an effective way to
address performance gaps in public organizations. To
search such framework, four models were taken into
account and they are presented thus.
276 Afr. J. Bus. Manage.
Table 1. Techniques for evaluating training programs.
Level of measurement Application Critics
Level 1
Participant’s degree of
satisfaction is taken during or
after the training session
participants’ reaction towards training and its
effectiveness is captured through a questionnaire.
Verbal feed, body language and observation of
participants are a part of reaction.
it is silent about measuring ability of
participants to ensure transfer of training
to the workplace.
Level 2 Learning
What did they learn during
training?
Pre-test before the start of training and post test at the
end of the training gives a clue of learning.
It does not guide about the future ability
of trainees to carry their learning from
training environment to the job
environment
(Bushnell, D.S 1990)
Level 3 Behavior
Is there visible change or
variation occurred in behavior
required to apply on the job
Pre and Post training tests are conducted to determine
change in behavior. Later a 360° assessment is
conducted through survey and interview using control
group technique.
This does not reflect any change in
behavior and it is not able to guarantee
positive impact on the organization.
Level 4 Results
Change in Performance
outcomes after the training
program
Results obtained are compared with training goals to
check transfer of training to the workplace
It does not identify cost versus
achievement (Brinkerhoff R.O. 2006)
Source: Kirkpatrick (1959).
Swanson model of training evaluation
Swanson (1996) presented a training evaluation model
based on performance, learning and satisfaction (PLS)
factors. To evaluate training results in terms of training
transfer to the workplace, Swanson guides to compare
training goals with results achieved after training.
Holton evaluation model
Holton evaluation model postulates that, evaluation takes
two forms: Generic and specific. Generic evaluation
covers the aspects such as, level of satisfaction of
trainees and transfer of training to the workplace.
Evaluation is conducted through a questionnaire. Specific
evaluation focuses on learning, educational, transfer
capability of trainees and impact of training in post
training scenario (Holton, 2005). Holton training
evaluation model stresses upon the following dimensions
to evaluate transfer of training to the workplace:
Dimension 1 - job profile of employees: Job profile of
an employee includes his personal characteristics (age,
gender) and workplace features like location, office type.
Dimension 2 - learning acquired: Learning acquired
can be sought through a well knitted set of questions;
what type of learning did you acquire? To what extent
learning was achieved? To what degree training meets
the needs of the trainees?
Dimension 3 - implementation of training: What type and
extent of improvement occurred in performance after
training? Whether improvement directly relates with the
skills acquired during training? Whether skills learned
during training relates to need areas?
To ensure implementation of training, any other valid
factor can be incorporated.
Dimension 4 - factors affecting implementation; This
dimension relates to individual motivation and organi-
zational back up that affects transfer of training. Personal
characteristics include motivation and expectations while
organizational characteristics include workplace
conditions, and available resources (Holton, 2005). Every
organization can develop an instrument as per its needs
and culture.
Kirkpatrick four level learning model
Kirkpatrick offered training evaluation framework based
on four concise levels; Level 1 (reaction), Level 2
(learning) Level 3 (behavior) Level 4 (results). Despite
criticisms, this model bears credible reputation and is
recognized as an age old training evaluation model since
1959 (Holton, 2005) (Table1).
Use of Kirkpatrick each level of evaluation: Table 2
shows the percentage age of use of each level of
Kirkpatrick in the organizations. Reaction based Level 1
has greater use at 95%. Effectiveness at level 1 could be
measured in the awake of scoring 4 out of 5 in each area
Khalid et al. 277
Table 2. Percentage age of use of KP.
KP four levels Percentage age of use Use of each level in the organizations (%)
Level 1 95 86-100
Level 2 37 71-90
Level 3 13 43-83
Level 4 3 21-49
Source: McMurrer et al. (2000) and Twitchell et al. (2000).
being rated. If all the trainees rate the area 4 out of 5,
level 1 would be showing 100% effectiveness. If 50%
shows response at 4, training will be considered 50%
effective (McMurrer et al., 2000; Twitchell et al.,
2000).Kirkpatrick (1959) confirms that Level 1 evaluation
can be used for all type of courses. At Reaction Level 1,
Learners will be able to illustrate their learning
experiences. At learning level 2, learning experience will
reveal change in knowledge, skill and attitudes before
and after the knowledge occurrence. As this knowledge
learnt, will be applied on workplace, behavior evaluation
occurs and this is level 3 behavior. Results obtained at
level 4, raise organizational value.
CIPP model (1987)
The CIPP model (context, input, process and product)
was proposed by Stufflebeam (1987). He presented a
systematic methodology through a series of questions to
probe into the curriculum development process:
1. Context: Obtaining situational data to determine
program objectives and learning linked with it
2. Input: Strategies are devised to achieve the desired
results
3. Process: It involves program implementation
4. Product: To evaluate the outcome in terms of program
worth and effectiveness.
Commentary
Swanson and Holton talked about performance, learning
and satisfaction. In Pakistan, there is less stress on these
factors. Trainings are conducted to comply with
mandatory regulations. There is no real learning or deep
satisfaction or training transfer involved in this
phenomenon. Training evaluation is loosely conducted in
a bureaucratic environment. Critical analysis reveals that,
Kirkpatrick model is silent about measuring ability of
participants to ensure transfer of training to the work-
place. Moreover, it does not guide about the future ability
of trainees to carry their learning from training to the job
environment to bring change in behavior that will
guarantee positive impact on the organization (Bushnell,
1990). Brinkerhoff (2006) finds out that, Kirkpatrick model
does not identify cost against achievement.
To enhance efficacy of training and improvement in job
performance, context (training criteria) needs to be set,
while planning a training course. For developing
countries, CIPP model has more attraction with its
context approach and KP model can be trusted onward.
Context approach helps to build training criteria with a
pre-defined outcome. In this backdrop, KP and CIPP
models have practical validity in Pakistani environment.
This study aims to integrate these two models and look
for a workable training evaluation framework. The
proposed diagrammatic flow of model is presented in
Figure 1.
This framework covers the key dimensions of training
evaluation to ensure training effectiveness through
achieving objectives at desired situations. At stage 1,
evaluation begins when the proposed model focuses on
the pre-training context as fundamental requirement. This
framework is expected to overcome diverse evaluation
practices of public sector organizations and offer a
methodical strategy. The proposed framework is standing
on five stages; context, reaction, learning, behavior and
results. These stages deal with evaluation activity
effectively. It is an interlinked and integrated chain of
processes that covers the entire training evaluation right
from planning to training outcome.
DISCUSSION
In Pakistan, heavy investment is made on training of civil
servants. To maintain check and balance on the public
investment, a viable training evaluation framework is
direly needed. The proposed framework offers step by
step evaluation of a training activity right from the
inception (criteria setting) to execution (results). Literature
review suggests that, all the evaluation methods offer
different analytical tools to evaluate training activity in
developed countries but their application in under
developed countries, like Pakistan, is debatable where
mind set and educational levels are stunningly low.
Experts have agreed that employees have diverse
evaluation needs (Nijman et al., 2006). Evaluation needs
278 Afr. J. Bus. Manage.
Stage 1
Context
(Training Criteria)
Stage 3
Learning
Secondary Stage Evaluation
Stage 4
Behavior
Tertiary Stage
Evaluation
Stage 2
Reaction
Primary
Evaluation
Stage 5
Results
Later Stage
Evaluation
Figure 1. Diagrammatical flow of proposed training evaluation framework (source: Author); dotted
line indicates continuous and unbroken link between stage 5 and stage 1. context is seen as a primary
requirement to help framing a viable training design aiming training transfer; arrow shows flow of
events and their direction of impact.
of employees of developed and developing countries are
distinct (Khilji, 1999). Kirkpatrick training evaluation
model, takes its initiative from participants’ reactions.
CIPP model starts training evaluation from context which
focuses on development of criterion of training. Context is
important in Pakistani environment and training plan
needs to be designed on real time indicators to produce
positive results. Should in case, this stage is ignored and
training plan is designed carelessly, trainees’ on board
will reflect negative reaction. Trainees’ reactions cannot
provide substantial base to get reliable results. Reaction
may undergo change at any of the three stages of
training, that is, before, during, and after training. Litera-
ture review suggests that training criteria, management
support and social support will serve as input to
implement training afterwards.
The framework re-engineered on the basis of KP and
CIPP training evaluation models will run its evaluation at
all levels and is proposed for application in developing
countries because, situation prevailing in developing
countries cannot be matched with developed countries.
Low educational standards, static mentality, and
unwillingness to learn are frequently found in developing
countries and people’s perception are hard to evaluate in
this environment.
PESTLE model has already identified that the political,
economic, sociological, technological, legal and
environmental factors prevailing in a country cannot be
ignored. These factors are beyond the control of
business. PESTLE model scans the political conditions
as (peace or turmoil), economic graph (going upward or
downward), sociological (social norms and cultures),
technological (modern technologies or old methods),
legal situation (litigations) and finally environment
(polluted or pollution free). This study gets support from
the PESTLE model.
CONCLUSIONS
There is no second opinion about the importance of
training and to make the investment in training fruitful is
the priority of every nation so as, to lay stringent check on
public investments. Training evaluation system is
considered vital for public or private organizations and
inorder to get desired efficacy of training, evaluating
training is a fruitful activity. Currently, there is a gap
between desired and actual performance in Pakistani
public organizations. This situation is alarming. Every
country needs a viable training evaluation framework to
get training outcome. Literature review identifies that,
trainings are conducted without predefined criteria in
Pakistan. This gap needs to be filled with due planning to
ensure training transfer to the workplace. This study
reiterates that pre-training context is imperative to finding
training criteria successful. This study offers new strategy
in the area of training evaluation and various studies
have already worked in the direction of this study.
Literature review, training evaluation models and experts
opinion leads to a conclusion that training conditions in a
particular sector, office, or area needs to be explored
carefully to design an effective training program. This is
taken as context (training criteria) in the proposed
framework. As a result of the exploration of training
conditions, factors like training criteria, management
support and social support will serve as input to
implement training adequately and these factors can be
used as input while developing training design and
delivery. Conceptual framework presented in this study
came into being as result of integration of CIPP and KP
training evaluation models. These model provided a base
to help in evaluating training activity. The proposed
framework is expected to enhance efficacy of training.
The subject framework emphasizes on the under-
standing of the entire situation of the organization from
the beginning and aids in tracking organization needs,
operational objectives, training design, realization and
monitoring. These items will pave way for subsequent
evaluation. The framework proposes a strategy to ensure
training transfer. Context is seen as a primary
requirement in this framework, to help in framing a viable
training design aimed at training transfer. This ambition
will remain unfulfilled until other characters such as
training management, trainer and trainees take active
part to make the training event a success story.
REFERENCES
Aghazadeh SM (2007). Re-examining the training side of productivity
improvement: evidence from service sector. Int. J. Productivity
Perform. Manage., 56(8).
Alvarez K, Salas E, Garofano CM (2004). An integrated model of
training evaluation and effectiveness. Hum. Resour. Dev. Rev., 3(4),
386-416.
Axtell CM, Maitlis S, Yearta SK (1997). Predicting immediate and
longer-term transfer of training. Pers. Rev., 26(3).
Khalid et al. 279
Baldwin TT, Fold JK. (1988). Transfer of training: a review and
directions for future research. Personnel Psychol., 41(1).
Barlow L (2006). Talent development: the new imperative? Dev. Learn.
Organ., 21(3).
Bartlett KR (2001). The Relationship between training and
organizational Commitment: A Study in the health care field. Hum.
Resour. Dev. Quart., 12.
Bersin J (2006). Companies still struggle to tie training to business
goals. Training, 43(10).
Birdi K, Allan C, Warr P (1997). Correlates of perceived outcomes of
four types of employee development activity. J. Applied Psychol.,
82(6).
Brinkerhoff RO (2006). Increasing impact of training investment: an
evaluation strategy for building organizational learning capability. Ind.
Commer. Training, 38(6).
Brinkerhoff RO, Montesino MU (1995), “Partnerships for training
transfer: lessons from a corporate study”, Hum. Resour. Dev. Quart.,
6(3):263-74.
Burke LA, Hutchins HM (2007). Training transfer: an integrative
literature review. Hum. Resour. Dev. Rev., 6(3).
Garavan T, McGuire D (2001). Competencies and workplace learning:
some reflections on the rhetoric and the reality. J. Workplace
Learning, 13(4).
Holton EF (1996). The flawed four-level evaluation model. Human
Resour. Dev. Quart., Vol.7.
Holton EFI (2005). Holton’s evaluation model: new evidence and
construct elaboration. Adv. Dev. Hum. Resour., 7(37).
Jia J, Fan L (2008). Research framework of human resource
development based on competency about knowledge enterprise. Int.
J. Bus. Manage., Vol. 8(Nov.).
Khilji SE (1999). Management in Pakistan. In M. Warner (Ed.)
International Encyclopedia for Business and Management. London:
International Thompson Press.
Kirkpatrick DL (1994). Evaluating training programs: the four levels. CA:
San Francisco: Berrette-Koehler.
Kirkpatrick, DL (1998). Evaluating Training Programs. 2nd ed., Berrett
Koehler, San Francisco.
Kontoghiorghes C (2004). Reconceptualizing the learning transfer
conceptual framework: empirical validation of a new systemic model.
Int. J. Training Dev., Vol. 8.
Lim DH, Johson SD (2002). Trainee perceptions of factors that
influence learning transfer. Int.l J. Training Develop., 1.
Maqsood KUY, Muhammad A, Chaudhry AR (2011). Assessing Impact
of Management Support on Perceived Managerial Training
Effectiveness in Public Organizations of Pakistan. Euro. J. Soc. Sci.,
22(1): 106-125.
Noe K (2004). Trainees’ attributes and attitudes: neglected influences
on training effectiveness. Acad. Manage. Rev., 11(4).
Pidd K (2004). The impact of workplace support and identity on training
transfer. Int. J. Train. Dev., 8(4).
Pineda R (2007). The continuing training in Spain. Balanace and future
challenges. Relief, Vol. 3.
Saks AM, Belcourt M (2006). An investigation of training activities and
transfer of training in organizations. Hum. Resour. Manage., 45(4).
Swanson RA (1996). Evaluation systems in HRD, PLS evaluation
system; Sales Communication case study, Symposium Evaluation
Systems in HRD. Minneapolis.
