E-Learning in Organizations

Abstract

E-learning, an instructional strategy for imparting needed knowledge, skills, and attitudes in organizations, is here to stay. Its viability, effectiveness, and potential to return tangible benefits to organizations depend largely on how it is designed, delivered, and evaluated. This article provides a comprehensive review of the state of the art of e-learning methods in organizations. The authors also critically examine e-learning’s effectiveness by reviewing the current literature on the outcomes of e-learning. Finally, they offer a research agenda designed to bridge the gap between the practice and science of e-learning.

Full text

http://jom.sagepub.com/
Journal of Management
http://jom.sagepub.com/content/31/6/920
The online version of this article can be found at:
DOI: 10.1177/0149206305279815
2005 31: 920Journal of Management
Renée E. Derouin, Barbara A. Fritzsche and Eduardo Salas
E-Learning in Organizations
Published by:
http://www.sagepublications.com
On behalf of:
Southern Management Association
can be found at:Journal of ManagementAdditional services and information for
http://jom.sagepub.com/cgi/alertsEmail Alerts:
http://jom.sagepub.com/subscriptionsSubscriptions:
http://www.sagepub.com/journalsReprints.navReprints:
http://www.sagepub.com/journalsPermissions.navPermissions:
http://jom.sagepub.com/content/31/6/920.refs.htmlCitations:
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

10.1177/0149206305279815ARTICLEJournal of Management / December 2005DeRouin et al. / E-Learning in Organizations
E-Learning in Organizations
Renée E. DeRouin
Barbara A. Fritzsche
Eduardo Salas*
Department of Psychology, University of Central Florida,
P.O. Box 161390, Orlando, FL 32816
E-learning, an instructional strategy for imparting needed knowledge, skills, and attitudes in
organizations, is here to stay. Its viability, effectiveness, and potential to return tangible benefits to
organizations depend largely on how it is designed, delivered, and evaluated. This article provides
a comprehensive review of the state of the art of e-learning methods in organizations. The authors
also critically examine e-learning’s effectiveness by reviewing the current literature on the out-
comes of e-learning. Finally, they offer a research agenda designed to bridge the gap between the
practice and science of e-learning.
Keywords:
e-learning; distance learning; distance training; computer-based instruction;
Internet in training
E-Learning in Organizations
Today, technology is the driving force of workplace training. In fact, 95% of respondents to
a 2003 survey by the American Society for Training and Development (ASTD) reported using
some form of e-learning in their companies (Ellis, 2003). E-learning (i.e., electronic learning)
has been defined by ASTD as “a wide set of applications and processes, such as Web-based
learning, computer-based learning, virtual classrooms, and digital collaboration. It includes
the delivery of content via Internet, intranet/extranet (LAN/WAN), audio- and videotape, sat
-
ellite broadcast, interactive TV, and CD-ROM” (Kaplan-Leiserson, 2002, para. 85). As a
*Corresponding author. Tel.: 407-882-1325; fax: 407-882-1550.
E-mail address: esalas@pegasus.cc.ucf.edu
Journal of Management, Vol. 31 No. 6, December 2005 920-940
DOI: 10.1177/0149206305279815
© 2005 Southern Management Association. All rights reserved.
920
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

training medium, e-learning is a powerful tool for delivering many and varied instructional
technologies and methods. For example, e-learning can be used to present online lectures
through the use of live stream audio and video technology, textual materials in the form of elec
-
tronic PowerPoint slides, and discussions through the use of message boards and chat rooms.
Although the term e-learning is only a few years old (see Henry, 2001), it has already been
described as the next “killer application” for the Internet (Chambers, as cited in Henry, 2001).
E-learning allows organizations to deliver training consistently to all employees; to update
training content when necessary; to reduce travel costs to outside training facilities; and to pro
-
vide training to employees on demand, anytime, and anywhere (Burgess & Russell, 2003).
Because of its cost-effectiveness and just-in-time availability, e-learning was estimated to
account for approximately $11.4 billion of U.S. corporate training investments in 2003 (Moe
& Blodget, 2000).
Recently, several reviews of e-learning practice and research have been published (e.g.,
Burgess & Russell, 2003; Kosarzycki, Salas, DeRouin, & Fiore, 2003; Salas, Kosarzycki,
Burke, Fiore, & Stone, 2002; Welsh, Wanberg, Brown, & Simmering, 2003). Kosarzycki et al.
and Welsh et al. focused on the “who, what, where, when, and why” of e-learning in organiza
-
tions. Salas et al. summarized the major themes in recent e-learning research, and Burgess and
Russell reviewed research on the effectiveness of e-learning initiatives.
Describing it as the “e-learning movement,” Welsh et al. (2003) found that the use of e-
learning in organizations outpaces academic research on the topic. Thus, because e-learning is
a rapidly changing area of practice, organizational practices must be examined regularly to
keep up with the state of the art in e-learning. Thus, one goal of this article is to provide a
review of the state of the art of computer-based, e-learning programs, focusing on how manag-
ers and organizations are currently using this delivery medium for employee development.
The second goal of this article is to evaluate e-learning’s effectiveness by analyzing current
research on e-learning outcomes assessment. Finally, we present a research agenda to advance
the science of e-learning in organizations with the goal of providing suggestions that could
potentially bring science up to the speed of practice. No review, to our knowledge, has focused
specifically on these three issues.
The State of the Art of E-Learning in Organizations
In interviewing a number of distance learning subject matter experts, Welsh et al. (2003)
found that four themes will characterize the landscape of e-learning during the next several
years. Specifically, more focus will be placed on synchronous learning tools, organizations
will begin to “blend” their classroom training with e-learning, e-learning technology will
advance and make training programs more accessible, and better integration of the various
characteristics of e-learning (e.g., peer collaboration, performance support, information pre
-
sentation) will occur. The current state of the art of e-learning in organizations reveals that
many of these predictions are already coming to fruition. This section describes the ways in
which e-learning is currently being used for employee development.
DeRouin et al. / E-Learning in Organizations 921
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

Information Technology, Business, and Soft Skills
Are Taught Via E-Learning
In a recent benchmarking survey of U.S. and Canadian businesses, respondents indicated
that they are using e-learning primarily in the training of information technology (IT) skills
(e.g., programming skills) (Skillsoft, 2001). This finding is not particularly surprising because
the topic area and the training medium are so closely connected (Brennan, 2004), and most e-
learning is currently conducted in the technology/telecommunications field (Skillsoft, 2001).
In addition to using e-learning in the training of IT skills, a growing number of businesses
have used e-learning in the training of business and soft skills. At Nestlé, for example, e-learn
-
ing is used to train employees on communication, teamwork, and leadership skills (“Nestlé
Widens Course Offers,” 2004), and at Bank of America, e-learning is the delivery mode of
choice for interpersonal skills training (Dobbs, 2000). Some of the most common business
and soft skills to be taught via e-learning in organizations include management, leadership,
communication, customer service, quality management, and human resources skills
(Skillsoft, 2001). As these skills have traditionally been taught in classroom settings, there is
much debate over whether e-learning is appropriate (in particular, for soft skills training).
The debate primarily centers on whether soft skills can be effectively trained via e-learning,
because these skills involve the development of verbal and nonverbal skills that have tradition-
ally been trained face-to-face. Pure computer-based training programs cannot provide the
face-to-face interaction that classroom training offers; as a result, it has been argued that the
use of e-learning for soft skills training is unsuitable. Currently, we are unaware of any
research that has empirically examined the effectiveness of e-learning for soft skills training.
Learners Are Given Greater Control Over Learning
A new development in organizational e-learning programs is to offer learners increasingly
greater amounts of control over their own learning. For instance, a common feature of many
online training programs today is learner self-pacing (DeRouin, Fritzsche, & Salas, in press).
This type of “learner control” provides trainees with the freedom to enter and leave instruc
-
tional material as they see fit. Self-pacing, therefore, permits trainees to work on training tasks
as quickly or as slowly as they prefer.
In addition to self-pacing, learners have been given control over such various instructional
elements as the sequence of instructional material, the content of instruction (e.g., which top
-
ics to study), and the amount of instruction during training (Sims & Hedberg, 1995). In gen
-
eral, learner control appears to have a positive, albeit small, impact on learning outcomes.
Results appear to be most positive when outcomes for learner control are skill-based rather
than cognitive, when learners have no prior experience with the material, and when learner
control of pacing/sequencing is offered rather than control of content (Kraiger & Jerden, in
press).
However, despite its potential learning benefits, one downside of providing trainees with
increased control over the training program is that trainees are constantly required to make
decisions about the content to view next and the way that content should be presented. These
922 Journal of Management / December 2005
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

decisions can hamper their ability to concentrate on the training material and, thus, reduce
their learning during training (Freitag & Sullivan, 1995; Gray, 1987). As a result, organiza
-
tions implementing learner control during training need to ensure that the amount of control is
consistent with the goals of the instructional program and with the extent of control learners
can effectively manage (for guidelines on how to effectively implement learner control in e-
learning, see Brown & Ford, 2002; DeRouin, Fritzsche, & Salas, 2004).
To guide future research on learning control in computer-based instruction, Kraiger and
Jerden (in press) have created a model of learning control, in which they break down learner
control into objective (the actual amount of learner control given to learners) and perceived
(the amount of control learners believe themselves to have) learner control. Objective learner
control is proposed to be affected by certain characteristics of the training system, such as the
culture of the organization, the technological capabilities of the system, and the specific peda
-
gogical models used to develop instruction. In contrast, perceived learner control is believed to
be the direct result of objective learner control; in other words, learners’ perceptions of the
amount of control they have are a direct result of the actual learner control they are given.
Other outcomes of objective learner control that Kraiger and Jerden put forward include learn
-
ing and affect/attitudes. The relationships between objective learner control and all three out
-
come variables are proposed to be moderated by several outside variables, including training
and learner characteristics and learner preferences for control. (For more information on this
model, the readers are recommended to refer to Kraiger and Jerden, in press.)
Organizations Develop Better Ways to Engage
E-Learners and Enhance Collaboration
Finding ways to keep learners engaged is a challenging problem for any e-learning devel-
oper. In fact, lack of engagement has been shown to be one of the primary reasons why learn-
ers drop out of distance learning courses (Skipper, 2000). As a result, organizations are now
focusing their efforts on ways to increase the motivation and involvement of learners in e-
learning programs. One way that organizations have attempted to accomplish this is through
the use of learning games. Learning games are generally computer games, such as arcade or
crossword puzzle games, which are used in the presentation and/or practice of training topics.
As tools that are typically associated with entertainment and recreation, learning games can
improve trainee performance in e-learning if they increase the appeal of e-learning as a train
-
ing medium, encourage trainees to practice extensively and to discover patterns and relation
-
ships within the training material, and reduce the fears typically associated with testing and
evaluation during training (Horton, 2002).
Organizations are also engaging e-learners through customization and personalization of
the learning experience and the use of stories to present instructional material. Customization
typically involves the adaptation of various instructional elements to meet learner preferences
and needs. Personalization refers to changes that are made to the structure of the program to
give the feeling that the learner is engaged in a conversation with the program. Personalization
can be promoted in e-learning by using conversational rather than formal language in on-
screen text or audio recording (Clark & Mayer, 2003).
DeRouin et al. / E-Learning in Organizations 923
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

For example, SmartTutor is an intelligent, Web-based tutoring system that is used in adult
education in Hong Kong (Cheung, Hui, Zhang, & Yiu, 2003). SmartTutor uses information
about the student and the course content to provide personalized feedback about performance
in the course, tailored advice for future work in the course, and adaptive tests that are based on
students’ current knowledge level. Thus, SmartTutor is customized for individual learners,
provides intelligent tutoring, and can be used as part of instruction in a variety of content
domains.
The use of stories or narratives in e-learning can also promote learner engagement in e-
learning by “bringing to life” abstract concepts or concepts that might be perceived as dry or
uninteresting (Hakkaladaddi, 2005; Prensky, 2001; Shepherd, 2004). In stories, learning
objectives can be presented in dialogues, characters can be created to be similar to learners,
and learning can occur through how the characters resolve problems in the story
(Hakkaladaddi, 2005). Organizations such as Avaya, Sprint, and Volvo cars have used story
-
telling to train frontline employees on how to deliver their brand image in their customer inter
-
actions (Gronstedt, 2004). The stories are used to help employees visualize the brand image
(“living and breathing the brand”), and then learners practice communicating the brand image
through simulated customer interactions.
In addition to engaging learners in e-learning, organizations have begun to increase the col-
laboration of trainees in e-learning programs. One way in which organizations have accom-
plished this is by offering more opportunities for learners to communicate. Two tools that are
currently available for increasing communication between learners include synchronous and
asynchronous communication tools. Synchronous communication refers to the use of
threaded discussions (e.g., chat rooms) that allow for conversations between trainees to occur
in “real time” (Selix, 2001). In contrast, asynchronous communication refers to the use of
message boards and other types of communication in which comments, questions, and
answers are posted and later accessed by trainees (Selix, 2001).
Clark and Mayer (2003) recommended that organizations choose which type of collabora-
tive tool to use depending on the concurrency of the learner population (i.e., the size of the
learner population during a specified time period). In low-concurrent environments, only a
small number (or an unpredictable number) of learners participate in learning tasks. Start and
end dates for learning are not the same for all learners, and learners are typically at different
points in the learning task. As a result, in low-concurrent environments, asynchronous tools
are likely to be more effective. In contrast, in high-concurrent environments, several learners
may complete learning tasks at the same time. Start and stop dates are restricted to specific
time frames, and learners are generally at similar points in the learning task. In this type of
environment, both asynchronous and synchronous learning tools may be useful.
Organizations Combine E-Learning
With Other Training Media
One of the most popular terms in the e-learning literature to date is “blended learning.” A
Google search of the term blended learning revealed more than 900,000 hits, and a search of
this term in the scholarly databases Academic Search Premier and Business Source Premier
924 Journal of Management / December 2005
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

revealed more than 200 hits, suggesting that this concept is discussed frequently in both aca
-
demic and nonacademic settings. Blended learning can be defined as “the thoughtful combi
-
nation of training methods” (Brodsky, 2003, para. 1). With respect to the use of e-learning, in
particular, blended learning has been used to refer to “training that combines traditional class
-
room sessions with e-learning and self-study” (Kovaleski, 2004: 35). The blended-learning
approach offers organizations both the cost savings associated with e-learning and the per
-
sonal touch of classroom instruction (Goodridge, 2001; Masie, 2002).
Throughout the e-learning literature, case studies and examples can be found of how
blended learning is implemented in organizational settings. For instance, Great Britain’s
Home Office is combining e-learning with instructor-led seminars and live coaching in the
training of civil servants (“Training News,” 2004), and United Way has implemented blended
learning by incorporating into its training program a variety of different delivery strategies,
including e-learning (“The New Look of E-learning,” 2004). In fact, survey results by Balance
Learning Limited suggest that as many as 77% of all U.S. companies currently rely on blended
learning to meet their training objectives (Sparrow, 2004).
Training Programs Implement Rapid E-Learning
Because many training programs today need to be developed in short time frames and train-
ing content becomes outdated rather quickly, the movement is now toward rapid e-learning
(Bersin, 2004; “The Top 6 Trends,” 2004; “Using Rapid E-learning,” 2004). Rapid e-learning
refers to e-learning that can be created and administered quickly and requires a limited amount
of effort in its development and delivery. Training programs created under the guise of rapid e-
learning typically take less than 3 weeks to develop and often consist of Microsoft PowerPoint
presentations placed on the Web (Bersin, 2004).
An innovative way of offering rapid e-learning was recently developed by Cisco Systems.
Specifically, Cisco’s rapid e-learning program involves placing video-based instruction on the
Web. Each month, Cisco uses its video e-learning system (vSearch) to present approximately
400 video training sessions. Users themselves access videos on vSearch around six times per
week. vSearch allows topics, such as product information and specs, to be available on
demand to employees around the globe. Because product changes can be communicated
quickly via a simple change in the videos posted to the Web, Cisco’s e-learning system repre
-
sents a novel (and potentially more engaging) approach to rapid e-learning (Santosus, 2004).
Although the term rapid e-learning was only recently coined (in 2003; see Bersin 2004),
the concept is a logical extension of the general movement toward faster training development.
In the past several years, training researchers have emphasized the need for rapid instructional
design—in other words, the ability to design programs more quickly and cost-effectively. In
1990, Tripp and Bichelmeyer published a seminal work discussing the application of rapid
prototyping (a design methodology used in software development) to instructional systems
design. Rapid prototyping is based on the idea that programs can be created more effectively
and efficiently when modifications are made to models of programs (essentially, rapidly built
prototypes) rather than to final products. In rapid prototyping, construction and utilization
occur simultaneously; users are brought in after a model of the program has been created, and
DeRouin et al. / E-Learning in Organizations 925
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

their input and recommendations are critical to the way in which design and development pro
-
ceed. This process of continual development and testing allows products to be brought to users
more quickly and is likely to result in products that better meet the needs of learners (in terms
of usability and content).
Organizations Customize E-Learning
to Meet Employee Preferences
Organizations have begun to align their e-learning strategies with employee preferences.
This approach is likely to have the benefit of increasing trainee satisfaction with e-learning as
well as enhancing trainee learning. One company that has had tremendous success aligning its
e-learning strategies with employee preferences is Hewlett-Packard (HP). Because HP’s
motto for training a global workforce is “one size does not fit all,” it allows its regional trainers
in countries around the world to select the best delivery modes for training their employees
(O’Leonard, 2004: 3).
HP has found that employee preferences for e-learning (and other training media) differ
considerably around the world. In particular, HP has found that in Asia, employees prefer
instructor-presented or blended learning options. Conversely, in the United States and Europe,
employees prefer self-paced and instructor-presented learning approaches, respectively
(O’Leonard, 2004).
These variations in e-learning preferences by region suggest that a single type of e-learning
program may not meet the needs and expectations of all employees. Consequently, it may be
beneficial for organizations to consider creating a “cafeteria”-style training plan similar to that
provided by HP. This strategy will allow the training program to be more flexible and may lead
to an improvement in trainee performance during e-learning.
E-Learning’s Effectiveness
The just-in-time availability and apparent cost-effectiveness of e-learning have made it an
enticing training medium for employee development. However, research on e-learning’s
effectiveness in workplace organizations is scant and often limited with regard to the type
and level of evaluation conducted. In this section, we describe the results of our review of e-
learning’s effectiveness, breaking down our analysis into Kirkpatrick’s (1976) four levels of
training evaluation, that is, reactions, learning, employee behavior, and organizational results.
We also briefly discuss the research findings for blended learning versus e-learning alone, and
some empirically and theoretically supported principles for improving e-learning’s
effectiveness.
In our review, search terms such as online learning, distance education, and e-learning
were combined with evaluation, assessment, satisfaction, and effectiveness. These search cat
-
egories revealed hundreds of articles addressing e-learning’s effectiveness (many of them
comparing e-learning programs to traditional lecture-based college courses). In addition, an
examination of the reference lists of several e-learning review articles revealed even more arti
-
926 Journal of Management / December 2005
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

cles on the topic (e.g., Burgess & Russell, 2003; Strother, 2002; Welsh et al., 2003). Although
we cannot provide an exhaustive discussion of the studies, we focused on identifying trends
and gaps in the literature to inform readers about the current state of e-learning research. Our
review of e-learning’s effectiveness includes the outcomes of studies from both classroom and
workplace settings.
Reactions
In assessing e-learning’s effectiveness, most organizations rely on the first level of
Kirkpatrick’s (1976) evaluation framework: employee reactions. Employee reactions gener
-
ally involve attitudes toward, and satisfaction with, e-learning or preferences for e-learning
compared to other modes of instruction (generally, classroom-based training). At least two
meta-analyses examining learner responses to e-learning have been conducted. One of these, a
meta-analysis of the effectiveness of computer-based instruction (CBI) for learners aged 5 to
adult (Kulik & Kulik, 1991), revealed that students in CBI demonstrated slightly more positive
attitudes toward computers and the subject matter being taught than learners in more tradi
-
tional forms of instruction. Moreover, students in CBI rated their instructional programs more
favorably than learners in other programs. The other meta-analysis examined student satisfac-
tion levels with distance learning and traditional classroom instruction (Allen, Bourhis, Bur-
rell, & Mabry, 2002). In this meta-analysis, learners demonstrated a slight preference for more
traditional forms of instruction and had somewhat higher satisfaction levels with face-to-face
than distance education.
Employee satisfaction with e-learning has also been examined in several large-scale sur-
veys of e-learning’s effectiveness. The results of these surveys have generally indicated posi-
tive attitudes toward workplace e-learning. For instance, a recent survey by Skillsoft (2004) of
204 e-learning users in 15 organizations (e.g., AT&T, FedEx, Lloyds TSB, Nestlé,
PricewaterhouseCoopers) reported that employees generally find e-learning to be enjoyable
and would likely recommend e-learning to coworkers. Moreover, a survey of nonprofit organi
-
zations sponsored by Isoph and the Nonprofit Technology Enterprise Network found that 88%
of e-learning users are somewhat to very satisfied with their e-learning programs. Some of the
most favorable e-learning aspects reported were e-learning’s accessibility, cost-effectiveness,
and convenience (“E-News,” 2005).
1
In addition to the meta-analyses and satisfaction surveys, researchers have empirically
examined learner responses to e-learning. One study, for example, of employees in workplace
settings showed that when attitude surveys were compared before and after a computer-based
course featuring work-related literacy training, commercial truck and bus drivers appeared
more positive about their ability to pass the written portion of the commercial driver’s license
exam (Baker, 1992).
Together, these findings suggest that learners vary in their attitudes toward, and satisfaction
with, e-learning systems. On a positive note, however, the studies and surveys that we
described involving organizational employees found favorable reactions toward e-learning. If
this outcome is true for all workplace e-learners, then e-learning’s potential to provide a satis
-
fying training experience is more promising than the above findings suggest. To gain a better
DeRouin et al. / E-Learning in Organizations 927
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

assessment of whether an e-learning program is effective, however, one needs to examine e-
learning outcomes at the next level of evaluation: employee learning.
Learning
Employee learning is the second level of evaluation in Kirkpatrick’s (1976) framework.
This level is particularly important for managers and training developers to study as it indi
-
cates whether participants gained knowledge of the core topics and principles taught during
training. Fortunately, it appears that many organizations and educational institutions do con
-
duct evaluations of learning outcomes in e-learning as the research literature on this topic is
considerable.
When course test grades measuring content mastery are used as a measure of effectiveness,
the research has yielded mixed results. Several studies have found poorer learning outcomes
associated with e-learning. One study found higher course failure rates in a Web section com
-
pared with a classroom section of an introductory psychology course (Waschull, 2001).
Mottarella, Fritzsche, and Parrish (2004) found poorer course grades and standardized
achievement test scores following Web-based rather than classroom-based instruction in a
more advanced psychology course. Similarly, other researchers found lower final exam scores
for students in the instructor’s Web-based Psychology Research Methods course compared
with the classroom section (Wang & Newlin, 2000).
However, several studies examining the learning outcomes of e-learning have found no dif-
ference in the posttest performances of students in e-learning or traditional delivery modes.
For example, Stocks and Freddolino (1998) found no difference in final course grades for their
Web-based and classroom sections of a graduate Research Methods in Social Work course. A
quasi-experimental study examining the effectiveness of an online nursing course found that
grades for students in the traditional and e-learning courses were similar (Alexander,
Polyakova-Norwood, Johnston, Christensen, & Loquist, 2003). Likewise, a study of teacher
preparation courses provided online or through traditional instructional media revealed that
students performed equivalently at both pretest and posttest (Smith, Smith, & Boone, 2000).
A meta-analysis of distance education courses, specifically those involving televised
instruction, similarly found no difference in the achievement of students in traditional pro
-
grams versus telecourses (Machtmes & Asher, 2000). The results of this meta-analysis are
consistent with the findings of the well-known annotated bibliography of 248 distance educa
-
tion studies by Russell (1999). This bibliography revealed “no significant difference” between
the learning outcomes of students in distance education and traditional courses for studies
conducted between 1928 and 1996. Besides the research conducted with students, quasi-
experimental research performed with workplace employees (e.g., Wisher & Curnow, 1999)
has also indicated no differences in learning between e-learners and classroom trainees.
In contrast to the previously mentioned studies that showed poorer outcomes or no differ
-
ence between e-learning and classroom-based instruction, several studies have shown that e-
learning can improve learning and achievement scores for students and employees. Brown
(2001), for example, empirically investigated the impact on learning of a 2-day Intranet-
delivered problem-solving course. His results indicated that trainee knowledge scores
928 Journal of Management / December 2005
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

increased significantly from pre- to posttest. In a quasi-experimental study, Whetzel, Felker,
and Williams (1996) examined the effects of satellite training on the learning outcomes of
United States Postal Service (USPS) managers and supervisors. USPS employees participated
in two courses (i.e., Automation Systems and Programs; Mail Counts and Route Inspections)
that provided overviews of the automation technology being developed at the postal service
and the procedures involved in evaluating mail carrier performance. As both of these courses
were also offered in classroom settings, the authors tested whether learning outcomes differed
between the satellite and lecture-based formats. Whetzel et al. (1996) found that not only did
USPS employees in the satellite courses perform as well on the training posttest as employees
in the classroom courses, but they actually performed significantly better. This result was
found even after controlling for pretest scores.
Our review revealed at least two meta-analyses that supported e-learning’s potential for
improving learning outcomes. The earliest of the two is a meta-analysis conducted by Kulik
and Kulik in 1991, previously mentioned in our discussion of student satisfaction with e-
learning. Kulik and Kulik found that student learning in CBI programs was moderately, but
significantly, greater than student learning in conventional classes. The second meta-analysis,
by Allen et al. (2004), found that students in distance learning courses had higher exam scores
than students taught in traditional, lecture-based settings. The benefits of distance education
did not stop here, however. The authors also found that the overall course grades of students in
distance education courses were significantly greater than those of students in classroom set-
tings. These findings suggest that e-learning has the potential to enhance students’ immediate
and long-term learning outcomes. However, these findings also need to be considered in light
of the abundance of research findings showing no significant differences in learning between
traditional classroom-based instruction and e-learning.
Overall, it is difficult to conclude that e-learning is more, less, or equally effective at the
learning level than traditional classroom-based training. It is not surprising that results differ
so widely as the studies themselves differ in many ways from each other (e.g., the content,
duration, and goals of the training; the quality of the research design used). The results suggest
that there are likely to be important moderators of the relationship between e-learning and
learning outcomes. For example, our review suggests that studies conducted in organizational
settings found more positive results for e-learning than those conducted in educational set
-
tings. Regardless, improvements in learning do not necessarily mean changes in employee
behavior. To determine whether behavior change has occurred, additional tests of e-learning’s
impact on job performance and work-related skills are required.
Behavior
The third level in Kirkpatrick’s (1976) evaluation framework, employee behavior, refers to
whether the skills learned in training transfer to the job (Strother, 2002). A limited number of
studies have empirically examined the effects of e-learning on employee behavior. In general,
these studies have supported the use of e-learning for improving work behaviors. Gopher,
Weil, and Bareket (1996), for example, quasi-experimentally investigated the flight perfor
-
mance of Israeli Air Force cadets after they participated in a 10-hour training session using a
DeRouin et al. / E-Learning in Organizations 929
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

computer game. The computer game, Space Fortress II, was modified to teach both the ele
-
ments of flying and attention control. After completing the computerized training, cadets
completed eight actual flights, each around 45 to 60 minutes long. The flights were part of an
evaluation period in which cadet performance was assessed for the role of high-performance
jet trainer. A comparison of the flight performance of cadets who received game training and
cadets who did not revealed that flight performance was significantly improved as a result of
the game training. In fact, five out of the eight flights showed significant differences in perfor
-
mance outcomes, and two showed marginal significance. These findings suggest that
computer games can be useful and effective tools for improving employee behavior.
In another quasi-experiment, Whetzel et al. (1996) examined whether satellite training
affected the performance of job-relevant skills. Specifically, after completing satellite and
classroom training, the training’s participants, USPS managers and supervisors, were asked to
complete two forms used on the job. These forms were used to conduct mail counts, one way
of evaluating mail carrier performance. The authors found that employees in the satellite-
training group completed one of the forms significantly better than employees in the class
-
room-training group. For the second form, however, satellite training did not appear to have a
significant effect on employee performance.
In addition, our review of the literature revealed a third study that found benefits of e-
learning for on-the-job performance. This study, conducted by Thomson NETg, empirically
examined whether e-learning was effective in improving the performance of learners on real-
world tasks. The tasks involved a series of Microsoft Excel functions that were developed by
managers from a range of industries, and participants were assigned to conditions through
stratified random sampling. Thomson NETg (2003) found that e-learning was significantly
more effective in improving performance on these job-relevant tasks than the control (or no-
training) condition.
Successful employee behavior change as a result of e-learning has also been supported
through surveys designed to assess e-learning’s impact on actual work practice. In particular,
in a large-scale survey conducted by Skillsoft (2004), 87% of e-learning users reported using
skills and knowledge they had gained from e-learning back on the job. These users were even
able to give concrete examples of how e-learning had affected their work. For the majority of
these users, the skill and business areas most improved by e-learning were computer and IT
skills, communication with coworkers and with customers/suppliers, business outcomes (e.g.,
sales), work processes (e.g., project management), and personal skills (e.g., leadership,
assertiveness) (Skillsoft, 2004).
The results of Skillsoft’s survey and the studies described above suggest that employee
behavior can be effectively changed as a result of e-learning.
2
With corporate and government
investments in training programs rising to more than $300 billion (Moe & Blodget, 2000), it is
also important to examine how e-learning affects organizational outcomes, such as depart
-
ment- and company-wide sales performance and customer service ratings. The next level of
Kirkpatrick’s (1976) framework examines the impact of e-learning programs on these
organizational results.
930 Journal of Management / December 2005
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

Organizational Results
As Bersin so succinctly stated, “The ultimate purpose of e-learning is not to reduce the cost
of training, but to improve the way your organization does business” (2002, p. 26). Unfortu
-
nately, however, the level of evaluation most frequently overlooked by e-learning researchers
and practitioners is e-learning’s impact on organizational results. The neglect of evaluation at
this level is not unique to e-learning. In fact, across all training research, there is a paucity of
research examining the return-on-investment and business outcomes of organizational train
-
ing programs. One of the key reasons why organizational results are ignored is that the effect
of training media/methods on these outcomes is extremely challenging to measure. Neverthe
-
less, a few organizations have attempted to do so with regard to e-learning, often in some
rather creative ways.
HP, for example, evaluated how customer service was affected by its focus on e-learning
and blended instruction rather than classroom training. The famous provider of business solu
-
tions found that sales representatives were able to answer questions more quickly and accu
-
rately, enhancing customer-service provider relations (O’Leonard, 2004). In addition,
Unilever estimated the increase in product sales as a result of its online training for sales
employees; the company found that sales increased by several million dollars after e-learning
(Hoekstra, 2001).
A different way of measuring whether e-learning is affecting organizational outcomes is to
examine how e-learning contributes to the completion of organizational objectives and goals.
A recent survey of e-learning sponsors from 15 organizations in the United Kingdom revealed
that core business strategies are being accomplished through the use of e-learning. These core
strategies were said to go above and beyond the cost savings of implementing the e-learning
program (Overton, 2004).
From the results of these studies, surveys, and reports, it appears that e-learning can posi-
tively influence organizational outcomes. More research is needed, however, to definitively
determine how e-learning accomplishes this and what specific outcomes e-learning affects.
Other variables that can be used to assess e-learning’s impact on organizational effectiveness
include production levels, employee turnover, quality measures, and absenteeism (Strother,
2002).
Blended Learning Versus E-Learning Alone
In our review of the state of the art of e-learning in organizations, we mentioned that organi
-
zations are increasingly turning toward blended learning (a combination of e-learning and
other training media) rather than e-learning alone. Intuitively, the idea of a blended-learning
environment makes sense as it appeals to the needs and learning styles of a variety of trainees.
However, what are the research findings surrounding this new wave in e-learning instruction?
In our review of the literature, we found only one study that empirically examined the bene
-
fits of blended learning over e-learning alone (Thomson NETg, 2003). This study was men
-
tioned briefly in our discussion of employee behavior change as a result of e-learning and
involved as many as 200 employees from companies such as Lockheed-Martin and the Execu
-
DeRouin et al. / E-Learning in Organizations 931
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

tive Service Corps of Chicago. Participants in the blended-learning condition were divided
into three groups, each group differing in the amount of instructor support or reference mate
-
rial they were provided. Participants in the e-learning condition completed their instruction
completely online, without the benefit of a classroom environment. At the conclusion of train
-
ing, participants were asked to complete several Microsoft Excel tasks designed to mimic real-
world assignments.
Study results revealed that participants in the three blended-learning environments per
-
formed real-world tasks with between 27% and 32% better accuracy than the e-learning-only
group. In addition, they completed these tasks in between 41% to 51% less time. Therefore,
not only did the participants who received blended learning complete the real-world tasks
better and with fewer errors, but they also performed them significantly faster (Thomson
NETg, 2003).
It appears, therefore, that blended learning does offer organizations some benefits over
stand-alone e-learning programs, particularly with respect to transfer of training outcomes.
However, the results of this study have not been peer-reviewed and, therefore, need to be
viewed with caution.
Principles for Enhancing E-Learning’s Effectiveness
Despite the mixed results for e-learning’s impact on reactions, learning, behavior, and
organizational results, there are several design elements that have been shown to potentially
improve the effectiveness of this medium. Although there are tips and guidelines available for
enhancing e-learning’s effectiveness (e.g., “Ten Tactics to Make E-Learning ‘Stick,’” 2003),
we focus our discussion of e-learning principles on those proposed by Clark and Mayer
(2003). These principles are both broad in scope (i.e., they cover many different aspects of
designing e-learning) and are based on the tenets of cognitive learning theory and an extensive
program of empirical research. As a result, Clark and Mayer’s principles are widely applicable
and well grounded in theory and research findings. The following subsections are brief
descriptions of four of Clark and Mayer’s e-learning principles and a few of the research find
-
ings they use to support these principles. (For a full description of Clark and Mayer’s princi
-
ples, we urge the reader to refer to their original work, Clark & Mayer, 2003.)
The multimedia principle. The multimedia principle states that graphics and text should be
used in e-learning rather than simply text alone. Clark and Mayer (2003) proposed that when
text is combined with relevant graphics (i.e., graphics that provide further explanation of
material rather than graphics meant simply for aesthetic value), the stage is set for learners to
engage in a deeper and more active processing of learning material, as they have to mentally
construct the relationships between the written text and pictorial illustrations. Several studies
conducted by Mayer and colleagues support the use of text and graphics in explaining con
-
cepts (e.g., Mayer, 1989; Mayer & Anderson, 1991, 1992; Mayer & Gallini, 1990). Specifi
-
cally, these studies found that students given both text and pictures in the descriptions of
mechanical and scientific processes performed significantly better on a learning transfer test
than students who received text alone.
932 Journal of Management / December 2005
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

The contiguity principle. A second principle that Clark and Mayer (2003) put forth is the
contiguity principle. The contiguity principle states that when text is used to explain a graphic
or vice versa, the text and graphics should be placed near each other on the screen. Colocating
the graphics and text permits trainees to focus on the instructional material rather than on try
-
ing to match a miscellaneous set of pictures to text. Although this principle appears somewhat
obvious, it is often neglected in the design of e-learning systems. In particular, this principle is
commonly violated when pictures and words are physically separated while scrolling screens
and when graphics and text are connected through Web links. Empirical support for the conti
-
guity principle can be seen in the work by Mayer, Steinhoff, Bower, and Mars (1995) and
Moreno and Mayer (1999). In both of these studies, the authors found that when text was
placed next to the graphic it described rather than underneath it, students performed better on
problem-solving transfer tests.
The modality principle. According to the modality principle, whenever appropriate, audio
technology should be used to present information instead of onscreen text. The underlying
reason for this is that learners are likely to become overwhelmed with visual information when
only presented with text, graphs, illustrations, and figures during learning. Because cognitive
learning theory states that human visual and auditory information processing takes place in
two separate channels, learners are likely to manage and integrate greater amounts of informa-
tion when material is presented through both audio and text. This is not to say, however, that
the program should include an audio recording of the text that is simultaneously presented on
the screen. Including an audio version will only add to the cognitive workload on the learner.
Instead, the audio recording should replace the text on the screen, allowing the material to be
split among the two modalities.
The empirical basis for this principle lies in the work of researchers, such as Mayer and
Moreno (1998), Moreno and Mayer (1999), and Moreno, Mayer, Spires, and Lester (2001). In
several studies comparing animation combined with narration to animation combined with
text, these authors uniformly found that students who received the narrative-animation version
performed significantly better on a problem-solving transfer test than students who received
the text-animation version.
The personalization principle. In their personalization principle, Clark and Mayer (2003)
recommend that text for e-learning programs is written in first and second person and that
learners are given access to onscreen virtual learning coaches (i.e., characters that provide
guidance and direction to learners). Both of these strategies were mentioned briefly in our dis
-
cussion of ways in which organizations are engaging e-learners. The purpose of using first and
second person as well as onscreen coaches is to encourage the learner to see the computer as a
conversational partner rather than as an information delivery agent. In doing so, learners are
likely to exert more effort toward learning the material presented to them (Beck, McKeown,
Sandora, Kucan, & Worthy, 1996).
The personalization principle is backed by research, such as that of Moreno and Mayer
(2000) and Moreno et al. (2001). In the first of these research articles, Moreno and Mayer con
-
ducted five studies examining how performance is affected by personalized versus formal text.
The authors found that learners who received text written in the first and second person per
-
DeRouin et al. / E-Learning in Organizations 933
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

formed significantly better on transfer tests than learners who received formal text. Similarly,
in the Moreno et al. study, students who learned with an onscreen learning coach (in this case,
Herman-the-Bug for an e-learning game on botany) performed better on transfer tests than
students receiving identical text and graphics without a learning coach.
Directions for Future Research
E-learning has the potential to be an effective training medium as it offers trainees the
opportunity to receive information in various formats (e.g., graphics, text, video) and to access
this information anytime and anywhere. However, as Welsh et al. (2003) point out, research
lags behind the practice of e-learning. Thus, the following research ideas are proposed as a
way to help bridge the gap between research and practice.
More theory is needed to guide the design, delivery, and implementation of e-learning.
Research in almost every domain of management (e.g., leadership, motivation, organizational
development) relies on theories that are modified, appended, and supported over years of
observation and study. These theories are then used to make practical predictions about orga
-
nizations and workplace environments, to explain various employee behaviors, and to create
the workplace conditions necessary for maximizing performance.
In the distance learning and e-learning literatures, we are aware of no theory that has been
the major influence in the design, delivery, and implementation of e-learning systems.
Because e-learning is intended to develop knowledge, skills, and/or attitudes, cognitive and
learning science (complemented by instructional design, educational, human factors, and
industrial and organizational psychology research) should be guiding e-learning systems.
The behavioral/objectivist model and the collaborative/cooperative learning model can be
used to guide the development of e-learning programs (Salas et al., 2002), and Clark and
Mayer’s (2003) principles are based on the tenets of cognitive learning theory and a systematic
program of research on e-learning. Although these principles are an excellent step toward cre
-
ating an integrated theory of learning in e-learning, further development of these principles
and greater use of cognitive information processing models would be useful. Without theory,
practitioners have to rely on fragmented advice from researchers who do not have the research
results to inform designers on how to use various e-learning options to create an e-learning
program that effectively and efficiently promotes positive learning outcomes.
Brown and Ford (2002) presented an input-process-output model of learning to illustrate
how training design influences learning. For positive change in cognitive, skill-based, and/or
affective learning, the delivery methods must positively affect active learning states. The three
active learning states include motivation, mastery orientation (as opposed to a performance
orientation), and mindfulness (i.e., systematically trying to integrate new knowledge into
previous knowledge).
Brown and Ford (2002) noted that “the commonly accepted design principles of today are
largely predicated on traditional classroom delivery, which uses a structured and sequenced
learning environment for helping learners master training content” (p. 202). In applying the
input-process-output model to e-learning, Brown and Ford noted that there are several impor
-
934 Journal of Management / December 2005
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

tant differences between traditional classroom learning environments and e-learning. For
example, a classroom instructor controls the pace of the course, the feedback that is given, and
the sequence of events. In e-learning, feedback can be frequent and available immediately fol
-
lowing a response from the learner. How much feedback should be given? Is it ever helpful to
learning to delay feedback? What is the best mechanism for delivering the feedback online?
Orientation and “way finding” issues are also an issue in e-learning. How do you best design
the training so that learners easily understand their progress in the course, when they have
completed enough practice, and when to seek more feedback? How does blended learning
compare with stand-alone e-learning programs? Research on collaborative training protocols
has shown that there are learning benefits associated with being trained in groups (Salas &
Cannon-Bowers, 2000). How can the advantages of collaborative training protocols be inte
-
grated into e-learning? Brown and Ford’s model offers an excellent start to linking
information processing theory and training outcomes literature with e-learning.
Research should be more learner focused than technology focused. With little integrated e-
learning theory available, it is not surprising that e-learning design appears to have been driven
more by advances in technology and “bells and whistles” than by our long-standing history of
cognitive science research and learning theory (Mungai, Jones, & Wong, 2002). Technology
will continue to outpace e-learning research findings, so it is unrealistic to expect that
researchers could empirically examine the usefulness of each new technology to e-learning
outcomes prior to its use in e-learning programs. Instead, research should examine the value
and limits of the application of technology. For example, when do various kinds of technology
generally help or hinder e-learning? Under what conditions do bells and whistles enhance
motivation to learn versus interfere with learning? In general, how do various kinds of techno-
logical advances improve motivation to learn, help learners achieve and maintain a mastery
orientation, or affect mindfulness?
Some research has already examined how the amount of technology affects learners. In
their meta-analysis, Allen et al. (2002) found that increasing the number of advanced multi-
media components within a distance education program significantly decreased learner satis
-
faction with e-learning. It appears that too strong of an emphasis on technology led to a decre
-
ment in learner attitudes toward e-learning. A stronger focus on the learner rather than on the
technology in e-learning research and practice is likely to lead to the development of more
effective e-learning programs, which in turn will likely improve learner satisfaction with e-
learning.
Another finding that supports a greater learner orientation in e-learning research is that of
Wisher and Curnow (1999). Wisher and Curnow found that the inclusion of real-time video of
an instructor in Web-based training had no effect on learning outcomes. In testing for the
video’s impact on learning, the authors deliberately turned the video off and on for certain par
-
ticipants during training, comparing trainees’ performance during these times. However, the
audio portion of the instructor’s lectures was still available to all participants. Learners without
the video performed just as well as learners with it. In other words, seeing the instructor during
training was unimportant to learner performance in the online program, and, consequently, the
video could have been removed from the training without negative consequences.
DeRouin et al. / E-Learning in Organizations 935
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

The findings of both of these studies point to the necessity of considering the learner in the
design of e-learning systems. Because learners often perform as well or better in training pro
-
grams that do not involve complex and sophisticated technology, it is important that research
-
ers examine which characteristics are most conducive to workplace e-learner success so that
more effective e-learning programs can be developed.
More research on e-learning effectiveness needs to be conducted in workplace settings.
The majority of the research examining e-learning’s effectiveness takes place in educational
environments with elementary- to college-age students (Salas, DeRouin, & Littrell, in press).
Although some of the findings of these studies are likely to apply to adult populations in work
-
place training contexts, there are certain conditions (e.g., when the instructional program
needs to be brief, highly accessible, and include training on skills with immediate application)
that affect employees engaged in training differently than students engaged in educational
pursuits (Brown & Ford, 2002). As a result, it is important to conduct e-learning research in
workplace settings.
For example, organizations often use simulation-based training to build specific skills.
Game-based learning may be particularly useful for skill building as it can provide necessary
practice opportunities and feedback at the same time that it is fun, engaging, and motivating to
learners (Prensky, 2001). In addition, work teams are often used by organizations (Salas &
Cannon-Bowers, 2000), and effective approaches have been developed to train teamwork
skills (e.g., team self-correction) and to address challenges that are often faced by work teams
(e.g., team coordination, working in distributed teams). However, little is known about how to
effectively train teams in e-learning environments. For example, what is the effectiveness of
team training in e-learning environments in which learners interact with virtual team members
who were created using artificial intelligence?
Research also shows that the pretraining environment and climate for learning in organiza-
tions can affect how much learning occurs in training. If training is framed negatively (e.g., as
remedial) or if employees have had negative experiences with past training opportunities, for
example, motivation to learn can be negatively affected (Salas & Cannon-Bowers, 2000).
Compared to traditional classroom training, e-learning tends to be more self-directed and less
instructor-led. Thus, we would predict that the climate for learning would be even more
important to learning outcomes in e-learning.
We encourage researchers to begin measuring behavioral and organizational outcomes of
e-learning. E-learning is generally viewed as more convenient than more traditional training
methods. However, employee behaviors and organizational results (other than savings in
training costs) following e-learning are rarely the subject of empirical study. This information
is crucial in evaluating whether the e-learning is truly worth the financial investment. Accord
-
ing to one estimate, training vendors typically charge between $100 and $125 an hour to create
computer-based and Web-based training programs. Because developing these programs is
extremely time and labor intensive, 1 finished hour of computer- or Web-based training with
-
out multimedia (e.g., audio, video) could costs upwards of $40,000 (Kruse, 2004). Being able
to weigh the costs of e-learning against the behaviors and organizational results that result
from it will help managers and training departments to determine when and if e-learning is a
viable training strategy.
936 Journal of Management / December 2005
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

Conclusion
E-learning, an instructional strategy for imparting needed knowledge, skills, and attitudes
in organizations, is here to stay. Its viability, effectiveness, and potential to return tangible ben
-
efits to organizations depend largely on how it is designed, delivered, and evaluated. Our
review suggests that although some progress has been made in understanding the benefits of e-
learning, much remains to be done. And much remains to be learned about how to best design
the e-learning environment, how to optimize its delivery, and what works when and why. We
hope that this review energizes and motivates research that is theoretically based, methodolog
-
ically sound, and offers practical recommendations and guidelines to organizations. We look
forward to the emergence of a science of e-learning that informs organizations about how to
design, deliver, and evaluate e-learning systems.
Notes
1. We acknowledge that these firms may have a conflict of interest with regard to their findings, and their results are
not peer-reviewed. Therefore, we urge readers to view these survey results with caution.
2. Again, we acknowledge that many of these firms (e.g., Skillsoft, Thomson NETg) may have a conflict of interest
with regard to their findings, and their results are not peer-reviewed.
References
Alexander, J. W., Polyakova-Norwood, V., Johnston, L. W., Christensen, P., & Loquist, R. S. 2003. Collaborative
development and evaluation of an online nursing course. Distance Education, 24: 41-56.
Allen, M., Bourhis, J., Burrell, N., & Mabry, E. 2002. Comparing student satisfaction with distance education to tradi-
tional classrooms in higher education: A meta-analysis. American Journal of Distance Education, 16: 83-97.
Allen, M., Mabry, E., Mattrey, M., Bourhis, J., Titsworth, S. Burrell, N., et al. 2004. Evaluating the effectiveness of dis
-
tance learning: A comparison using meta-analysis. Journal of Communication, 54: 402-420.
Baker, G. 1992. Instructional design of a computer-assisted work-related literacy program. Journal of Computer-
Based Instruction, 19: 33-36.
Beck, I., McKeown, M. G., Sandora, C., Kucan, L., & Worthy, J. 1996. Questioning the author: A yearlong classroom
implementation to engage students in text. Elementary School Journal, 96: 385-414.
Bersin, J. 2002. Measure the metrics: How to link e-learning to business strategy. E-learning, 3: 26-28.
Bersin, J. 2004. Rapid e-learning has arrived. http://www.bersin.com/tips_techniques/may_04_rel_update.asp.
Accessed Nov. 17, 2004.
Brennan, M. 2004. E-learning: Proving waste is a good thing. Chief Learning Officer, 3: 50-53.
Brodsky, M. W. 2003, November. Four blended learning blunders and how to avoid them. Learning Circuits, ASTD.
http://www.learningcircuits.org/2003/nov2003/elearn.html. Accessed Feb. 9, 2004.
Brown, K. G. 2001. Using computers to deliver training: Which employees learn and why? Personnel Psychology, 54:
271-296.
Brown, K. G., & Ford, J. K. 2002. Using computer technology in training: Building an infrastructure for active learn
-
ing. In K. Kraiger (Ed.), Creating, implementing, and managing effective training and development: 192-233. San
Francisco: Jossey-Bass.
Burgess, J. R. D., & Russell, J. E. A. 2003. The effectiveness of distance learning initiatives in organizations. Journal
of Vocational Behavior, 63: 289-303.
Cheung, B., Hui, L., Zhang, J., & Yiu, S. M. 2003. SmartTutor: An intelligent tutoring system in Web-based adult edu
-
cation. Journal of Systems and Software, 68: 11-25.
DeRouin et al. / E-Learning in Organizations 937
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

Clark, R. C., & Mayer, R. E. 2003. E-learning and the science of instruction: Proven guidelines for consumers and
designers of multimedia learning. San Francisco: Jossey-Bass.
DeRouin, R. E., Fritzsche, B. A., & Salas E. 2004. Optimizing e-learning: Research-based guidelines for learner-
controlled training. Human Resource Management, 43: 147-162.
DeRouin, R. E., Fritzsche, B. A., & Salas E. In Press. Learner control and workplace e-learning: Design, person, and
organizational issues. In J. Martocchio (Ed.), Research in personnel and human resources management, Vol. 24.
Boston: JAI/Elsevier.
Dobbs, K. 2000. What the online world needs now: Quality. Training, 37: 84-94.
Ellis, R. K. 2003, November 17. E-learning trends 2003. http://www.astd.org/ASTD/Publications/LearningCircuits/
2003/nov2003/trends.htm. Accessed Jan. 30, 2004.
E-News: Non-profits successfully using e-learning. 2005. T+D, 59: 11.
Freitag, E. T., & Sullivan, H. J. 1995. Matching learner preference to amount of instruction: An alternative form of
learner control. Educational Technology, Research and Development, 43: 5-14.
Goodridge, E. 2001. E-learning blends in with classrooms. Information Week, 834: 97.
Gopher, D., Weil, M., & Bareket, T. 1996. Transfer of skill from a computer game trainer to flight. Human Factors, 36:
387-405.
Gray, S. H. 1987. The effect of sequence control on computer assisted learning. Journal of Computer-Based Instruc
-
tion, 14: 54-56.
Gronstedt, A. 2004. Living the brand. Communication World, 21(5): 14-17.
Hakkaladaddi, C. P. 2005. Bringing life to e-learning through stories. E-learning Age, April: 12-13.
Henry, P. 2001. E-learning technology, content and services. Education + Training, 43: 249-255.
Hoekstra, J. 2001. Three in one. Online Learning, 5: 28-32.
Horton, W. 2002. Games that teach: Simple computer games for adults who want to learn. In A. Rossett (Ed.), The
ASTD e-learning handbook: 139-157. New York: McGraw-Hill.
Kaplan-Leiserson, E. 2002. E-learning glossary. http://www.learningcircuits.org/glossary.html. Accessed July 1,
2002.
Kirkpatrick, D. L. 1976. Evaluation of training. In R. L. Craig (Ed.), Training and development handbook: A guide to
human resource development, 2nd ed.: 18.1-18.27. New York: McGraw-Hill.
Kosarzycki, M. P., Salas, E., DeRouin, R., & Fiore, S. M. 2003. Distance learning in organizations: A review and
assessment of future needs. In E. Salas (Series Ed.) & D. Stone (Vol. Ed.), Advances in human performance and
cognitive engineering research: Vol. 3. Human resources technology: 69-98. Boston: JAI.
Kovaleski, D. 2004. Blended learning in focus. Corporate Meetings & Incentives, 23: 35-36.
Kraiger, K., & Jerden, E. In Press. A meta-analytic investigation of learner control: Old findings and new directions. In
S. M. Fiore & E. Salas (Eds.), Where is the learning in distance learning? Towards a science of distributed learn
-
ing and training. Washington, DC: American Psychological Association.
Kruse, K. 2004. How much will it cost? Estimating e-learning budgets. http://www.e-learningguru.com/articles/
art6_1.htm. Accessed Feb. 22, 2005.
Kulik, C. C., & Kulik, J. A. 1991. Effectiveness of computer-based instruction: An updated analysis. Computers in
Human Behavior, 7: 75-94.
Machtmes, K., & Asher, J. W. 2000. A meta-analysis of the effectiveness of telecourses in distance education. Ameri
-
can Journal of Distance Education, 14: 27-46.
Masie, E. 2002. Blended learning: The magic is in the mix. In A. Rosett (Ed.), The ASTD e-learning handbook.New
York: McGraw-Hill.
Mayer, R. E. 1989. Systematic thinking fostered by illustrations in scientific text. Journal of Educational Psychology,
81: 240-246.
Mayer, R. E., & Anderson, R. B. 1991. Animations need narrations: An experimental test of dual-coding hypothesis.
Journal of Educational Psychology, 83: 484-490.
Mayer, R. E., & Anderson R. B. 1992. The instructive animation: Helping students build connections between words
and pictures in multimedia learning. Journal of Educational Psychology, 84: 444-452.
Mayer, R. E., & Gallini, J. K. 1990. When is an illustration worth ten thousand words? Journal of Educational Psy
-
chology, 88: 64-73.
Mayer, R. E., & Moreno, R. 1998. A split-attention effect in multimedia learning: Evidence for dual processing sys
-
tems in working memory. Journal of Educational Psychology, 90: 312-320.
938 Journal of Management / December 2005
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

Mayer, R. E., Steinhoff, K., Bower, G., & Mars, R. 1995. A generative theory of textbook design: Using annotated
illustrations to foster meaningful learning of science text. Educational Technology, Research and Development,
43: 31-43.
Moe, M. T., & Blodget, H. 2000. The knowledge web. Merrill Lynch & Co. Merrill Lynch—eLearning: The knowl
-
edge web Part 4 corporate e-learning—Feeding hungry minds. http://www.usc.edu/dept/education/globaled/
hentschke/documents/KW4.pdf. Accessed Feb. 25, 2004.
Moreno, R., & Mayer, R. E. 1999. Cognitive principles of multimedia learning: The role of modality and contiguity.
Journal of Educational Psychology, 91: 358-368.
Moreno, R., & Mayer, R. E. 2000. Engaging students in active learning: The case for personalized multimedia mes
-
sages. Journal of Educational Psychology, 93: 724-733.
Moreno, R., Mayer, R. E., Spires, H., & Lester, J. 2001. The case for social agency in computer-based teaching: Do stu
-
dents learn more deeply when they interact with animated pedagogical agents? Cognition and Instruction, 19:
177-214.
Mottarella, K. E., Fritzsche, B. A., & Parrish, T. J. 2004. Who learns more? Achievement scores following Web-based
versus classroom instruction in psychology courses. Psychology of Learning and Teaching, 4: 50-53.
Mungai, D., Jones, D., & Wong, L. 2002. Distance Learning, 2002. Proceedings of the Annual Conference on Teach
-
ing and Learning (18th, Madison, Wisconsin, August 14-16).
Nestlé widens course offers for all its staff. 2004. ITTraining, May: 11.
The new look of e-learning and blended learning strategies. 2004. KM Review,7:4.
O’Leonard, K. 2004. HP case study: Flexible solutions for multi-cultural learners. Oakland, CA: Bersin &
Associaties.
Overton, L. 2004. Linking learning to business: Summary report 2004. Reading, UK: Bizmedia Ltd.
Prensky, M. 2001. Digital game-based learning. New York: McGraw-Hill.
Russell, T. L. 1999. The “no significant difference” phenomenon: A comparative research annotated bibliography on
technology for distance education. Raleigh: Office of Instructional Telecommunication, North Carolina State
University.
Salas, E., & Cannon-Bowers, J. A. 2000. Design training systematically. In E. A. Locke (Ed.), The Blackwell hand-
book of principles of organizational behavior: 43-59. Oxford, UK: Basil Blackwell.
Salas, E., DeRouin, R. E., & Littrell, L. N. (in press). Research-based guidelines for designing distance learning: What
we know so far. In H. G. Guetal & D. L. Stone (Eds.), The brave new world of eHR. San Francisco: Jossey-Bass.
Salas, E., Kosarzycki, M. P., Burke, C. S., Fiore, S. M., & Stone, D. L. 2002. Emerging themes in distance learning
research and practice: Some food for thought. International Journal of Management Reviews, 4: 135-153.
Santosus, M. 2004. Cisco on video. In the know. Knowledge Management Research Center. http://www.cio.com/
research/knowledge/edit/k030804_cisco.html. Accessed Mar. 21, 2005.
Selix, G. 2001. Improving blended learning. [Electronic version]. E-learning, 2: 48.
Shepherd, C. 2004. Trainers should wear their story-telling hat. ITTraining, July/August: 28.
Sims, R., & Hedberg, J. 1995. Dimensions of learner control: A reappraisal for interactive multimedia instruction. In
J. M. Pearce & A. Ellis (Eds.), Learning with technology. Proceedings of the Twelfth Annual Conference of the
Australian Society for Computers in Learning in Tertiary Education: 468-475. Melbourne, Australia: Australian
Society for Computers in Learning in Tertiary Education.
Skipper, J. 2000, April. Distance learning barriers: ASTD study. DL News, IBM Corporation, p. 1.
Skillsoft. 2001. E-learning in USA & Canada benchmark survey. Nashua, OH: Author.
Skillsoft. 2004. EMEA e-learning benchmark survey: The user’s perspective. Middlesex, UK: Author.
Smith, S. B., Smith, S. J., & Boone, R. 2000. Increasing access to teacher preparation: The effectiveness of traditional
instructional methods in an online learning environment. Journal of Special Education Technology, 15: 37-46.
Sparrow, S. 2004. Blended is better. T+D, 58: 52-55.
Stocks, J. T., & Freddolino, P. P. 1998. Evaluation of a world wide Web-based graduate social work research methods
course. Computers in Human Services
, 15: 51-69.
Strother, J. B. 2002. An assessment of the effectiveness of e-learning in corporate training programs. International
Review of Research in Open and Distance Learning [Online], 3.
Ten tactics to make e-learning “stick” in your dept. or center. 2003, June. IOMA’s Report on Customer Relationship
Management: 4-6.
DeRouin et al. / E-Learning in Organizations 939
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from

Thomson NETg. 2003. Thomson job impact study-final results-the next generation of corporate learning: Achieving
the right blend. Naperville, IL: NETg.
The top 6 trends training managers can expect in ’05. 2004. Managing Training & Development, 4(1): 12-14.
Training news. 2004. Personnel Today, September 14: 4.
Tripp, S. D., & Bichelmeyer, B. 1990. Rapid prototyping: An alternative instructional design strategy. Educational
Technology, Research and Development, 38: 31-44.
Using rapid e-learning to deliver training now! 2004. Managing Training & Development, 4(1): 6-7, 10.
Wang, A. Y., & Newlin, M. H. 2000. Characteristics of students who enroll and succeed in psychology Web-based
classes. Journal of Educational Psychology, 92: 137-143.
Waschull, S. B. 2001. The online delivery of psychology courses: Attrition, performance, and evaluation. Teaching of
Psychology, 28: 143-146.
Welsh, E. T., Wanberg, C. R., Brown, E. G., & Simmering, M. J. 2003. E-learning: Emerging uses, empirical results
and future directions. International Journal of Training and Development, 7: 245-258.
Whetzel, D. L., Felker, D. B., & Williams, K. M. 1996. A real world comparison of the effectiveness of satellite train
-
ing and classroom training. Educational Technology, Research and Development, 44: 5-18.
Wisher, R. A., & Curnow, C. K. 1999. Perceptions and effects of image transmissions during Internet-based training.
American Journal of Distance Education, 13: 37-51.
Biographical Notes
Renée E. DeRouin is a doctoral student in the industrial and organizational psychology program at the University of
Central Florida and the recipient of both the Society for Industrial and Organizational Psychology’s Robert J. Wherry
award for 2004 and the 2005 American Psychological Association’s Distinguished Graduate Student in Professional
Psychology award. Her research interests include training, e-learning, learner control, and stereotype threat.
Barbara A. Fritzsche is director of the Ph.D. program in industrial and organizational psychology at the University of
Central Florida. Her research interests include learner control in training, decision making in job selection, diversity in
the workplace, and prosocial personality. Her work has appeared in journals such as the Journal of Personality and
Social Psychology, the Journal of Occupational and Organizational Psychology, the Journal of Vocational Behavior,
and the Journal of Applied Social Psychology.
Eduardo Salas is trustee chair and professor of psychology at the University of Central Florida and program director
of the Human Systems Integration Department at the Institute for Simulation and Training (IST). He has authored
more than 300 journal articles and book chapters and coedited 13 books. He is editor of Human Factors and is on the
editorial boards of the Journal of Applied Psychology, Military Psychology, Group Dynamics, and Journal of Organi
-
zational Behavior.
940 Journal of Management / December 2005
at SAGE Publications on January 17, 2011jom.sagepub.comDownloaded from