Content uploaded by Frank Nguyen
Author content
All content in this area was uploaded by Frank Nguyen on Sep 07, 2022
Content may be subject to copyright.
The voice on the other end of
the phone line says, “I believe
my customer’s problem could
benefit from a performance
support system. Unfortunately, I’m not
sure where to begin. Can you help?”
As organizations seek to reduce the time
and cost associated with training, this
type of phone call has become increas-
ingly common. Organizations are looking
more toward noninstructional interven-
tions, like electronic performance
support systems (EPSS), to provide
employees with the information they
need and to do so in increasingly faster
ways. Unfortunately, despite the poten-
tial for EPSS to mitigate performance
problems, many practitioners are not
sure which systems would be best suited
to solving their current issue or some-
times even where to start.
The general human performance technology
(HPT) model available on the ISPI website
(http://www.ispi.org/services/whatshpt-
model.pdf) provides practitioners with a
process to analyze performance problems
and select the appropriate intervention.
However, once an intervention is
selected, the HPT model typically relies
on the processes, technologies, and best
practices commonly used in each particu-
lar domain to guide the design and
development of the selected intervention.
For instance, when training is selected as
part of the intervention portfolio, a practi-
tioner may rely on the ADDIE model or
on other competing models from the
instructional design domain.
When it comes to electronic performance
support, no clear and concise model to
guide the practitioner currently exists.
Models from other domains could cer
tainly be applied to EPSS. For instance,
Witt & Wager (1994) compared the EPSS
design process to the general ADDIE
model used in instructional design.
Raybould (2000) introduced the
Performance Support Mapping method-
ology. Although this approach is quite
robust and heavily emphasizes analysis,
mapping, and carefully designed perfor-
mance support interfaces, it is
unfortunately not yet widely known or
adopted by practitioners. As Fischer and
Horn (1997) noted, “without tools that
are primarily EPSS tools and with no
clear methodology for building them . . .
EPSS will be limited to an approach.”
This article addresses this perceived gap.
Figure 1 illustrates a model that the
authors have developed, applied, and
refined over the course of working with
various customers and performance
problems at a Fortune 100 company. The
goal of this model is not to be compre-
hensive and all encompassing. Instead,
the intent is to provide a model that
human performance technologists can
readily apply in the design and develop-
ment of performance support systems.
Where possible, actual instruments and
templates (identified in Figure 1 with
shaded boxes) will be provided so that
practitioners can use and adapt these
tools to their particular organization’s
needs and performance problems.
A Practitioner’s Guide for Designing
Performance Support Systems
by Frank Nguyen and Craig Woll
37
Performance Improvement, vol. 45, no. 9, October 2006
© 2006 International Society for Performance Improvement
Published online in Wiley InterScience (www.interscience.wiley.com) •DOI:10.1002/pfi.001
Phase 1: Performance Analysis
Conduct HPT Analysis
In the first phase of solving any human performance prob-
lem, a performance analysis should be completed. Some
common tools for analyzing performance include organiza-
tional, environmental, workflow, task, and gap analyses.
The appropriate combination of these analyses will paint a
clear picture of the performance problem to be addressed.
Because this article is focused on performance support sys-
tems, we will not attempt to address the general topic of
performance analysis in detail. More information on this
topic can be obtained from the ISPI HPT Institute (see
http://www.ispi.org/hpt_institute) or publications such as
First Things Fast (Rossett, 1998).
Select EPSS Intervention
According to Gilbert’s (1978) Behavior Engineering Model
(BEM) the root cause of a performance problem falls into
one or a combination of six categories: data, resources,
incentives, knowledge, capacity, and motives. Chevalier
(2003) updated the language of the Gilbert model to better
reflect the terminology used in today’s performance analy-
sis, as is shown in Table 1. Placing the root cause(s) into
these categories simplifies the intervention selection pro-
cess by clearly articulating the type of problem that has been
encountered. An EPSS intervention is most commonly asso-
ciated with solving problems with a root in the information
category but can also be used to supplement interventions
in any of the other five categories.
Carr (1992) mentions five guidelines for the selection of an
EPSS as an intervention: (1) “Skilled performers spend sig-
nificant amounts of time helping and correcting unskilled
performers”; (2) “A lot of
documentation exists, so
that employees must do
extensive searches in order
to find the right informa-
tion”; (3)”Neophyte workers
must begin to perform effec-
tively and training is either
impractical or unavailable”;
(4) “Individual technicians,
specialists, managers, or
other performers need to be
guided through complex
processes”; (5) “Teams of
technicians, specialists,
managers, or other perform-
ers need to be guided
through a complex process”
(p. 37). Ladd (1993) devel-
oped a checklist like the one
shown in Table 2 to assist in
determining if EPSS was right for the performance problem
identified in the analysis.
Check the items that apply to your organization. Your orga-
nization is a candidate for the use of electronic performance
support if you check three or more items in the first section
or four or more items in the second section.
EPSS are best suited for tasks that are not done frequently.
They are most effective when they are implemented in the
context of the work itself. Research shows that adult learners
retain information better when it is related to their personal
experience and is applied to a context with which they are
familiar (Knowles, 1984; Caffarella, 1994; MacKeracher,
1996; Daley, 1998). EPSS are also a better fit with noncritical
tasks and objectives and can also be associated with tasks or
processes that are not frequently performed. The selection of
an EPSS as an intervention to solve a problem or need iden-
tified in a performance analysis is more likely to succeed
when the human performance technologist adheres to the
simple principles previously stated.
Measure Performance Problem
The time and cost associated with some interventions, like
EPSS, often require strong management and organizational
support. One of the best ways to ensure that the project is
properly funded and supported is to identify key perfor-
mance measures against which the success of the
intervention can be measured. These measures can be financial
(improved sales, reduced operating costs), organizational
performance (employee retention or satisfaction), or indi-
vidual performance (reduced errors, decrease in workflow
cycle time). Ideally, the measures you choose should be
ones that are directly related to the performance problem
38 www.ispi.org •DOI:10.1002/pfi •OCTOBER 2006
Figure 1. A Practitioner’s Model for Designing EPSS.
you are trying to solve, that affect the organization in a
meaningful way, and that are valued by the customer. A
baseline of these measures should be taken during Phase 1
to allow you to calculate the impact that the EPSS interven-
tion has made after implementation. Start by identifying the
key indicators and then measure them longitudinally
throughout the process, beginning prior to the implementa-
tion of the intervention.
Webb (1999) identified four key factors in calculating the
ROI of a system: (1) isolating your data from other variables,
(2) limiting the number of key metrics, to avoid saturation,
(3) converting the data collected into money; and (4) mea-
suring the same indicators before and after the project. The
metrics selected can measure value in a number of ways, but
the most useful is generally a financial indicator shown in
dollars. Some examples of indicators are displayed in Table
39
*Performance Improvement •Volume 45 •Number 9 •DOI:10.1002/pfi
Table 1. Updated Behavior Engineering Model. Source: Adapted from Gilbert, 1978, p. 88, by Chevalier, 2003, p. 9.
Table 2. Is Performance Support Right for Your Organization? Source: Adapted from Ladd, 1993, p. 24.
3. The final metrics selected should be understood and rat-
ified by the key stakeholders of the project.
Phase 2: EPSS Analysis
Once EPSS has been selected as an intervention it is impor-
tant to conduct a more specific needs assessment directly
related to the EPSS. This assessment will provide critical
information that is a key to the high-level design carried out
in Phase 3. This assessment information is also important in
pinpointing the exact need of the customers to guarantee
that the EPSS is properly outfitted to mitigate their perfor-
mance problem.
The data for the EPSS analysis can be gathered through sur-
veys, observations, collection of the company’s operational
data, or any of a number of data gathering methods. This
article will make one recommendation of how the analysis
can be administered. Customization of this method is rec-
ommended to match unique situations. Table 4 provides an
example of a simplified instrument for gathering the assess-
ment data. A precursor survey may be administered prior to
using this instrument in order to understand the general
audience. Once again timeliness of the assessment rollup
and analysis completion is important in maintaining a qual-
ity relationship with your customer.
Conduct Quantitative Assessment
The sample size for the assessment should be sufficient to
reveal the need clearly but not so large that the data rollup
becomes overbearing. We recommend limiting the sample to
no fewer than 10 and no more than 30 performers. This
means that for large populations the sample should be care-
fully selected to be random yet representative of the target
audience. The job roles of the individuals selected in the
sample may include end users, managers, peer trainers, and
others, depending on the intended use of the system.
40 www.ispi.org •DOI:10.1002/pfi •OCTOBER 2006
Table 3. Examples of EPSS Indicators. Source: Adapted from Hawkins, Gustafson, & Nielson, 1998, pp. 19-20.
Conduct Qualitative Assessment
Data collected as part of the quantitative needs assessment
can often reveal worker preferences, environmental condi-
tions, and trends. However, data do not typically explain
why these conditions may exist. To better understand these
results, a performance technologist can use qualitative
research methods. This research can be easily conducted
using interviews, focus groups, job observations, or other
methods. This follow-up is a deterrent to costly defects that
can occur in the development phase if the design does not
match the true user need.
When the final assessment is complete and the data have
been placed in a meaningful summary document, that
assessment should be validated by the key stakeholders
prior to moving into the succeeding phases of the project.
The metrics collected during Phase 1 should also be
reviewed as a reminder of what the project outcomes will be
measured against.
Phase 3: EPSS Design
After collecting the requisite analysis information, the next
phase in the performance support design process is to use
these data to inform the design of the EPSS. As shown in
Figure 1, the design phase includes four steps: selecting the
type(s) of EPSS to deploy, designing a high-level architec-
ture, developing a detailed design, and validating these
design products with the customer.
Select EPSS Type
Arguably the most important yet most often neglected step
in the EPSS design process is the careful and deliberate
selection of the appropriate type of performance support
system. It is also important to note that in certain circum-
stances, human performance technologists may choose to
implement more than one type of system. Although more
research needs to be conducted, some guidelines and empir-
ical studies currently exist to guide practitioners in this
selection process.
Table 5 defines the three categories of performance support
systems proposed by Gloria Gery (1995). These three types
of EPSS differ in the level of integration between the sup-
port system and the users’ work interface. For instance,
external systems have minimal integration and therefore
require the learner to stop what he or she is doing, find
information in the EPSS, learn it, and then return to the task
at hand. In contrast, intrinsic systems are so integrated into
the work interface itself that users do not have to interrupt
41
Performance Improvement •Volume 45 •Number 9 •DOI:10.1002/pfi
Table 4. Simplified EPSS Needs Assessment Template.
their workflow to learn. “They simply feel that they are just
doing the work” (Gery, 1995, p. 51).
Gery implicitly suggests that the more integrated a perfor-
mance support system is to the work interface, the better it is.
In fact, she provides practitioners with the goal of integrating
“as much as 80% of the required performance support as
intrinsic,” with the remaining percentage equally allocated
between extrinsic and external solutions (Gery,
1995, p. 53). Raybould (2000) confirms this
notion by arguing that “[a]s support moves fur-
ther from the tool and requires more time off
the job it becomes less powerful” (p. 35). These
assertions have been partially validated by
more recent empirical research by Nguyen,
Klein, and Sullivan (2005). They found that
users provided with intrinsic and extrinsic per-
formance support performed significantly
better on a software procedure than a control
group that was not provided with an EPSS. In
addition, the intrinsic users accessed their
EPSS 1.5 times more than the control group
did, whereas the extrinsic users accessed their
EPSS 3 times more often.
However, integrated performance support solutions like
intrinsic and extrinsic systems may not be possible in all
circumstances or may prove too costly to build. For exam-
ple, the root cause of your customer’s performance problem
may be related to an off-the-shelf software application.
Because the software was purchased from a vendor, the
interface may be compiled in such a way as to prevent
modifications required by an integrated EPSS. In addition,
certain workers, such as those
in factories, warehouses, or
repair centers, perform tasks
that are not primarily computer
based. In fact, they may not
have immediate access to com-
puters, making it inconvenient
or impossible for them to use
electronic forms of perfor-
mance support. As noted by
Raybould (2000), “when a par-
ticular [EPSS] proves
infeasible,” practitioners may
need to look at less-embedded
support systems (p. 35).
Computer technologies and
HPT interventions have
advanced tremendously since
Gery first introduced her three
EPSS categories. Figure 2 pro-
vides an updated taxonomy that
illustrates just some of the per-
formance support systems that
practitioners have available to
them today. Again, this taxon-
omy is not meant to be an
exhaustive list but a framework
that illustrates some technolo-
gies that can be leveraged for
performance support purposes.
42 www.ispi.org •DOI:10.1002/pfi •OCTOBER 2006
Table 5. Types of Electronic Performance Support Systems. Source: Adapted from Gery,
1995, p. 51.
Figure 2. Taxonomy of Performance Support Systems.
As you can see, performance technologists now enjoy more
EPSS options for mitigating a performance problem than
they did fifteen years ago. To make sense of these choices, a
needs assessment was recently conducted in conjunction
with ASTD Benchmarking Forum companies. This study
provides insight into the particular types of EPSS that corpo-
rate employees perceive as useful and potentially effective
(Nguyen, 2005). As noted earlier, though, selecting the most
effective EPSS for a given performance problem may be dif-
ficult due to the current lack of more substantive research.
Develop High-Level Architecture
Once one or more types of performance support systems
have been selected, the next step in the process is to develop
a high-level architecture. Figure 3 shows the basic compo-
nents that need to be considered when designing an EPSS:
the performance problem, the end user, the type of elec-
tronic device (if any) the
individual can use to access
the performance support sys-
tem while on the job, the
EPSS or work interface that
he or she will use to access
the support content, and the
databases or content reposi-
tories he or she may have
access to from the EPSS.
This architecture template
may be downloaded, and
then adapted for your use,
from http://www.frankn.net/
epss/architecture.zip.
The information from the
analysis phase can be com-
bined with the EPSS selection
to complete this architecture
diagram. In particular the
quantitative data should inform
this step. You will notice that
the items from the needs
assessment instrument in
Figure 3 map directly to the
components of the archi-
tecture diagram. Figure 4
presents an example of an
EPSS architecture. This
design addresses an equip-
ment repair problem where
technicians had access
only to computer kiosks. To
improve the technicians’
ability to access content
while working on equip-
ment, the architecture calls
for the adoption of wireless tablet computers. The needs
assessment also revealed that the technicians wanted access
to manuals and training materials as well as the ability to
share their knowledge among peers. As a result the high-
level design calls for an EPSS that can access a learning
content management system (LCMS) and knowledge man-
agement (KM) system.
Upon completing an initial version of the architecture, it is
not unusual to change the selected performance support
systems. For example, the needs assessment data may sug-
gest that users require access to more content repositories
than the work interface can realistically support. To
accommodate this, a practitioner may choose to integrate
certain content intrinsically or extrinsically in the work
interface. The remaining content may be accessed exter-
nally through a list of frequently asked questions (FAQs)
or a search engine.
43
Performance Improvement •Volume 45 •Number 9 •DOI:10.1002/pfi
Figure 3. Simplified EPSS Architecture Template.
Figure 4. EPSS Architecture Example.
Technician
encounters a
repair error
code
Computer
kiosk
External
EPSS:
Search
engine for
error codes
displayed by
equipment
Wireless
tablet PC
Vendor
Equipment
Manuals
Training
LCMS
Knowledge
Mgmt.
System
Develop Low-Level Design
The next step in the process is to define in more detail the
design of the performance support system. Software devel-
opers often refer to this step as low-level or detailed design.
Because low-level design tends to be specific to a work inter-
face and EPSS type, it is not possible in this article to
provide a generic framework or template for this activity.
Nevertheless the goal of this activity remains the same in all
situations: develop a design of sufficient detail that a perfor-
mance support development team can build or purchase a
system that will address the identified performance problem.
Validate with Customer
As with all the phases in the EPSS design model, the perfor-
mance technologist should validate the EPSS selection and
the high-level and low-level design products with the cus-
tomer before moving to the next phase.
Phase 4: EPSS Development
Once the design for the EPSS has been completed, the next
phase in the performance support design model is to use the
high-level architecture and low-level design to build an
EPSS system, purchase an off-the-shelf EPSS tool that meets
the design requirements, or adapt an existing EPSS system
to the current performance problem. As shown in Figure 1,
the development phase includes at least four steps: develop-
ing or purchasing the EPSS, developing support content to
be accessed from the EPSS, integrating the EPSS into the
users’ work interface, and of course validating the devel-
oped performance support system with the customer.
Develop or Purchase EPSS
Where possible, it may be more cost effective to purchase ven-
dor-developed software packages to support your performance
support system design. A relatively comprehensive list of
EPSS tools is available from EPSScentral (at http://www.epss-
central.info/knowledgebase/desdev). However, it is quite
common that performance support designs call for an EPSS
that does not currently exist or that cannot be met by off-the-
shelf software. This circumstance is particularly likely for
highly integrated performance support, as found in intrinsic
systems. In these instances, organizations are forced to pursue
a custom-developed performance support system.
Develop Content
Because extrinsic and external performance support sys-
tems rely on content stored outside the work interface, it
may be necessary to develop new content to support users
on the job, and store it in a database or repository. To opti-
mize project timing and resources, this step could be done
in parallel with the preceding EPSS development step.
In certain situations it may also be possible to leverage con-
tent stored in other systems in the environment. For
instance, if the performance problem being addressed is
caused by lack of knowledge or discipline in following busi-
ness processes, one could look to workflow diagrams or
other business process engineering documents. If the perfor-
mance problem is related to software procedures, one could
look to vendor-developed manuals, job aids, or simulations.
Furthermore, a growing trend in the training world is the
adoption of an LCMS. These systems allow training devel-
opers to chunk e-learning and instructor-led content into
reusable learning objects (RLOs). Although the original
intent of these objects is to be combined and reused for
training offerings, they can also be leveraged as content for
performance support systems. In this way performance sup-
port content can be introduced and updated as training
content is created, thereby reducing overall development
and maintenance costs.
Integrate into Work Interface
In intrinsic and extrinsic EPSS designs, it is necessary to
integrate support content directly into the users’ work inter-
face. It is sometimes helpful to consult with a usability
expert, human factors engineer, or industrial engineer. By
doing so the human performance technologist can determine
potential problem areas, opportunities for interface redesign,
and other strategic locations in the work interface to inte-
grate performance support content. Even in external
performance support designs, it may be helpful to provide a
link in the users’ work interface to the EPSS. This is often
seen in software applications that provide a Help button that
launches an external search engine, FAQ page, or help index.
Validate with Customer
As in the analysis and design phases the performance tech-
nologist should validate the developed or purchased
performance support system with the customer. This step
may involve a number of tests common in the software
development world, such as functional testing, performance
testing, and user acceptance testing. Some of these tests may
occur during the EPSS development phase, and others may
occur at the conclusion.
Phase 5: EPSS Implementation, Evaluation
The implementation and evaluation of the EPSS occurs fol-
lowing the development. In reality the foundation for both
implementation and evaluation are laid long before the end
of the project. The successful adoption and validation of the
usefulness of the system is contingent on tying the previous
phases together in this phase. In Phase 1 the metrics that
will be referenced in the evaluation were created and vali-
dated by the stakeholders. In Phase 2 the population that
would use the EPSS as an intervention was identified and
44 www.ispi.org •DOI:10.1002/pfi •OCTOBER 2006
an assessment of its needs was performed. A sample from
that audience was carefully selected to represent the popu-
lation. This same sample, or a small audience of choice, was
likely used in the functional testing of Phase 4. During the
implementation portion of phase 5, emphasis must be
placed on communication, training, change management,
support, and marketing of the system. Each component
plays an important role in the adoption of the performance
support system by employees.
Although it is possible and potentially valuable to evaluate
the EPSS intervention using Kirkpatrick’s Level 1-4 frame-
work, we believe it is particularly important to determine
the direct impact of the EPSS intervention on the perfor-
mance problem. During Phase 1, baseline data about key
performance metrics were gathered. During the evaluation
portion of Phase 5, these baseline data should be compared
with data collected during the implementation phase and at
predefined intervals for some time after implementation.
The system’s success or failure in meeting the predeter-
mined success criteria should be reported out to the key
stakeholders in the form of a periodic report or a dashboard.
This will not only be useful in performance management
and continuous improvement of the system but will also act
as a data point for future performance analysis to determine
if EPSS is the right intervention.
Conclusion
In the coming years new technologies will continue to pro-
vide burgeoning opportunities to support the performance
of users in innovative ways. Our understanding of the fac-
tors that influence the effectiveness of performance support
systems will grow, and the field of EPSS will continue to
mature. The design model introduced in this article pro-
vides a simple framework that human performance
technologists can readily apply, refine, and adapt as elec-
tronic performance support systems continue to evolve.
References
Caffarella, R. S. (1994). Planning programs for adult learn-
ers: A practical guide for educators, trainers, and staff
developers. San Francisco: Jossey-Bass.
Carr, C. (1992). PSS! Help when you need it. Training &
Development, 46(6), 31-37.
Chevalier, R. (2003). Updating the behavior engineering
model. Performance Improvement, 42(5), 8-14.
Daley, B. J. (1998). Novice to expert: How do professionals
learn? In 39th Annual Adult Education Research
Conference Proceedings. San Antonio, TX: University of
Incarnate Word and Texas A&M University.
Fischer, O., & Horn, R. (1997). Electronic performance sup-
port systems. Communications of the ACM, 40(7), 31-32.
Gery, G. (1995). Attributes and behaviors of performance-
centered systems. Performance Improvement Quarterly,
8(1), 47-93.
Gilbert, T. F. (1978). Human competence: Engineering wor-
thy performance. New York: McGraw-Hill.
Hawkins, C. H., Jr., Gustafson, K. L., & Nielson, T. (1998).
Return on investment (ROI) for electronic performance
support systems: A web-based system. Educational
Technology, 38, 15-22.
Knowles, M. (1984). The adult learner: A neglected species
(3rd ed.). Houston, TX: Gulf.
Ladd, C. (1993). Should performance support be in your
computer? Training & Development, 47(8), 22-26.
MacKeracher, D. (1996). Making sense of adult learning.
Toronto: Culture Concepts.
Nguyen, F. (2005). Oops, I forgot how to do that: A needs
assessment of electronic performance support systems.
Performance Improvement, 44(9), 33-39.
Nguyen, F., Klein, J. D., & Sullivan, H. (2005). A compara-
tive study of electronic performance support systems.
Performance Improvement Quarterly, 18(4), 71-86.
Raybould, B. (2000). Building performance-centered web-
based systems, information systems, and knowledge
management systems in the 21st century. Performance
Improvement, 39(6), 69-79.
Rossett, A. (1998). First things fast: A handbook of perfor-
mance analysis. San Francisco: Jossey-Bass/Pfeiffer.
Webb, W. (1999). Show me the return. Inside Technology
Training. Retrieved February 3, 2006, from
http://www.trainingsupersite.com
Witt, C. L., & Wager, W. (1994). A comparison of instruc-
tional systems design and electronic performance support
systems design. Educational Technology, 34(6), 20-24.
Frank Nguyen is a doctoral candidate focusing on performance support sys-
tems. He has managed the development and deployment of enterprise
e-learning and performance support systems at Intel Corporation for the last
six years. He is coauthor of Efficiency in Learning (with Ruth Colvin Clark and
John Sweller,Jossey-Bass, 2006) and holds a master’s degree in Educational
Technology from Arizona State University. He may be reached at
frank.nguyen@asu.edu.
Craig A. Woll,PhD, is a Senior Learning Technologist for Intel Corporation.
He received his PhD degree in Instructional Technology from Utah State
University. He works with internal and external customers to develop strategic
learning packages to meet Intel’s strategic objectives. Craig may be reached
at craig.a.woll@intel.com.
45
Performance Improvement •Volume 45 •Number 9 •DOI:10.1002/pfi
