Blended Learning Opportunities

Full text

1-800-262-9699
•
www.amanet.org
AMA SPECIAL REPORT
BLENDED
LEARNING
OPPORTUNITIES
By Allison Rossett and Rebecca Vaughan Frazee
Copyright 2006
American Management Association
CONTENTS
Blended Learning Opportunities
What Is Blended Learning?
Why Blend?
What Goes into a Blend?
What Types of Blending Make Sense?
How Does Blended Learning Affect What People Do?
QUICKLIST 1: Are Employees Ready for Blended Learning?
QUICKLIST 2: Are Managers Ready for Blended Learning?
QUICKLIST 3: Are Executives Readying the Organization
for Blended Learning?
How Do We Look at the Effectiveness of Blends?
Blended Learning: Bridging the Classroom and the Workplace
QUICKLIST 4: I Want to Learn More About Blended Learning
References

What is blended learning?
In 1958 the milk monitors at Public School 164 in New York City
experienced a blend of approaches to help them learn to distribute the milk
to other youngsters. Loading and carrying was demonstrated. They were
coached by their supervisor on handling leaks and mashed cartons, and
moving from one classroom to the other in safe and orderly fashion. They
were paired with a more experienced monitor to try milk delivery the first
few times.
Ingrid, a German engineer, wanted to become a fluent English speaker. She
conversed in English in online chat rooms. She vacationed in English
speaking places and hung out in cafes. She studied English language tapes.
And she sought a British boyfriend who spoke not a word of German.
It’s not different for financial services representatives today. Many go to
class. They rely on a manager for coaching. And they have materials to
which they refer when they need to know more about a product or
situation. These materials are available any time, anywhere, online from a
comprehensive support system.
The point, of course, is that milk monitors, language students, financial
services professionals, and everybody else, no matter their century or age,
are well served by something not novel or radical or trendy or even
necessarily techie. They profit from a well-crafted blend.
What is blended learning?
Blended learning (BL) integrates seemingly opposite
approaches, such as formal and informal learning, face-to-face
and online experiences, directed paths and reliance on self-
direction, and digital references and collegial connections,
in order to achieve individual and organizational goals.
ឣBL is devoted to learning and performance. From the organization’s
perspective, blended learning is about improving performance and
achieving business objectives. From the employee’s perspective, blending
is about getting work done, when and where a need emerges, more
typically at a time and place of the employee’s choosing.
ឣBL takes many forms. Blending might involve structured or casual
interactions with instructors, peers, coaches, mentors, and supervisors.
It happens in classrooms, on ships, at home, and in the field, even the
BLENDED LEARNING OPPORTUNITIES
1-800-262-9699
•
www.amanet.org
2

battlefield. It might involve time spent independently with reading
material, online modules, databases, reference manuals, templates,
checklists, worked examples, or hours engaged on a group assignment or
conversation with peers. Technology is central to some blends, less a part
of others.
ឣBL addresses that nagging concern about transfer of training. BL is the
next step in a continuing commitment to systems, results, and
performance. If you are concerned about lessons that stop at the
classroom door and events limited to time and place, BL has much for
you.
ឣBL relies on compelling assets and experiences. As we move from
instructors to blends, from classroom to field, participation and results
are the hands of employees. Employees can elect to skip entire programs
or elements that feel superficial, complicated, or irrelevant, in favor of
their “real work.” Thus, BL programs and assets must present themselves
as worthwhile and manageable.
ឣBL capitalizes on the resident smarts in the organization. BL presses
people and organizations to find, store, stir, and share what they know. A
database might help sales people re-use parts of proposals. Far-flung hotel
administrators can “ask the experts” through FAQs, email, phone calls, or
live video streams. Employees may turn to their supervisors to practice a
skill or explore an idea.
ឣBL promotes connections and conversations. BL encourages the
organization to extend lessons and conversations far beyond the
classroom and into the workplace through coaching, e-coaching, and
online communities. A sales person who has learned about a new product
can chat with more experienced colleagues attempting to bring that
product to Asia. An executive can reach out for expert views from a
trusted e-coach. A researcher can reflect with others on the investment
team about how a natural disaster should influence their choices.
ឣBL guides, directs and tracks. BL must do two things: first, it must propel
action, showing employees how to benefit from the blend, how far they
have come, where else they need to go, and what else is possible; and
second, it must simultaneously encourage smart choices and involvement.
Diagnostics, assessments and feedback, menus, and sample paths can be
used to tailor experiences, assets and activities.
BLENDED LEARNING OPPORTUNITIES
1-800-262-9699
•
www.amanet.org
3

Why blend?
Blended learning has a growing presence in workforce learning and
performance. Kim and colleagues’ 2005 survey of 200 training professionals
in the United States predicted an increase in the use of BL in their
organizations. In another survey of almost 300 training professionals in the
US and UK, ASTD and Balance Learning reported that more than two-
thirds of respondents ranked blended learning as “the most effective and
cost-efficient form of training,”and indicated that “blended learning will
make up about 30% of all corporate training budgets by 2006,” (Sparrow,
2004).
That others are doing it is interesting, but not conclusive. Far more
compelling are studies and experiences that suggest BL works. What might
blended learning do for you?
• Nurture a world-class and worldwide workforce
Globalization, offshore outsourcing, and franchising are changing the
nature of organizations and the needs, location and experiences of their
employees. Executives expect workforce learning to translate into
performance, and to make contributions—big ones. Do they want their
sales people in class or out in the field? Do they want consultants with each
other or customers? Do they want knowledge acquired in class nine months
ago or access to ideas and perspectives that reflect what is happening today?
As organizations have shifted to customized and boundaryless services,
knowledge and expertise must follow and surround peripatetic employees.
• Provide consistent and updated messages
Instructors are a great resource during training, but their messages
sometimes differ from one to another, and their smarts depart after class.
Technology, on the other hand, can deliver standardized messages,
BLENDED LEARNING OPPORTUNITIES
“Everyone is overworked and no one has any time for traditional
development approaches and methods. Organizations need simpler
management-driven approaches and tools that are designed to make
continuous learning and development an integral part of the process.”
(Sullivan, 2005, July)
1-800-262-9699
•
www.amanet.org
4

instructional and otherwise, consistently, tirelessly, swiftly, repeatedly,
patiently, around the globe. Online modules, knowledge bases, and archived
presentations do not get jet lag.
• Exploit technology
Dropping prices and increasing functionality mean that more people
around the world are plugging in, with and without wires. The number of
PCs is projected to surpass 1 billion in 2007, and the number of PDAs is
anticipated to reach almost 60 million by 2008, with most boasting wireless
email and web browsing capabilities (eTForecasts, 2005). Ipsos-Insight
reported that at least two-thirds of all Internet users connect via high
speed broadband (Modi, 2005). Of course, cell phones are everywhere—
a whopping 1.5 billion and counting. They can be used for mobile training,
coaching, and performance support. Internet browser capabilities allow
employees to access web-based databases or search engines through their
cell phones. Short text messaging (SMS) can be used to send coaching tips,
quizzes and knowledge checks, or to measure training transfer. And video
clips can provide short examples of desirable performance in areas such as
negotiation, managing meetings, or customer service.
• Foster independent habits for learning and reference
Learners like choices (Reigeluth & Stein, 1983).With BL, employees can
progress at their own pace and even repeat parts of the program (Zenger &
Uehlein, 2001). They can participate in communities and relationships, and
enjoy interaction, guidance, and encouragement from peers, instructors,
supervisors, and coaches. For those who are reluctant to turn exclusively to
independent learning, blended forms anchored in the classroom can pave
the way.
BLENDED LEARNING OPPORTUNITIES
“Three factors will change the face of traditional training and
development: increased global competition, outsourcing and smart suites.
The most visible of the three, smart suites, will integrate learning at the
desktop with an employee’s other tools such as e-mail, calendaring, IM,
and document management. In this environment informal learning is
pumped-up and the line between learning and doing fades.”
—Margaret Driscoll, IBM Global Services (Neal, 2004)
1-800-262-9699
•
www.amanet.org
5

• Converge learning and work
In the traditional instructor-led world, you are either IN class or AT work.
Not surprisingly then, instructors and managers worry about transfer. That
is less of a concern in a blended situation because BL brings learning,
information and support to where the work gets done (Rossett, 2005b). Got
a question? You can look it up online. Got a problem? You can chat with
your manager or share it with an online community. Eager to get better at
personnel management? Fortunately, there’s a course you can take and a
pre-assessment that will make certain you’re ready for that course. AMA’s
blended approach capitalizes on this benefit.
• Improve performance and control costs
Studies report increased cost-effectiveness (Graham, Allen, & Ure, 2003),
and increased productivity for those using a blended approach as opposed
to e-learning alone (Thomson/NETg, 2003). Other studies have reported
enhanced employee retention (Bersin, 2004; CLO, 2005a; Nelson, 2005), and
reduced training time for blended approaches (Zenger & Uehlein, 2001). In
addition, online resources can be easier and cheaper to update and
distribute (Osguthorpe & Graham, 2003).Singh (Singh, 2003) noted that less
expensive solutions, such as virtual collaboration, coaching, recorded live
events, and self-paced materials, can be used instead of more expensive
customized computer-based content.
If studies and opinions do not attract you to BL, consider that it just plain
makes sense. Who wouldn’t benefit from additional opportunities to
practice and reflect, targeted resources, engaged supervisors, interactions
with experts and peers, and advice and learning experienced right there,
within the workflow?
BLENDED LEARNING OPPORTUNITIES
Training professionals surveyed in 2004 “predicted that knowledge
management tools, online simulations, wireless technologies, and reusable
content objects would impact the delivery of e-learning most greatly
during the next few years…[and that] authentic cases and scenario
learning, simulations or gaming, virtual team collaboration and problem
solving, and problem-based learning would be used more widely in the
coming decade.” (Kim et al., 2005)
1-800-262-9699
•
www.amanet.org
6

What goes into a blend?
A tour of assets
• The classroom
In a 2004 survey by the eLearning Guild (Pulichino, 2005), respondents
recognized classroom instruction as the most frequently used part of a
blend. According to some, it is still the best way to learn. AMA’s 2004
research made that point,“More than two-thirds of American organizations
agree with training experts that classroom training—with its opportunities
to interact with capable instructors and interested peers—is the best way for
adults to acquire the new skills and behaviors that they need to move their
careers, and their companies, forward.” The classroom remains a critical
component of blended learning and is not going to disappear any time
soon.
BLENDED LEARNING OPPORTUNITIES
1-800-262-9699
•
www.amanet.org
7

• Coaching, e-coaching, e-mentoring
Coaches can be used to demonstrate, model, remind, critique, guide, nudge
and nag, in person or using technology. The key is finding knowledgeable,
credible coaches who are willing to do what needs to be done on an as
needed basis. Today most coaches rely on some form of electronic means to
communicate, if only to arrange appointments. But e-coaches go further,
typically using the internet strategically (Rossett & Marino, 2005). Relations
and skills can flourish online too, using Instant Messaging with voice and
video, for example, to coach a sales pitch, discuss approaches to a product
launch, or rework a job description.
• Blogs and Wikis
A blog, short for web log, is a website that contains a chronological
collection of journal-style entries on a given topic, constantly and easily
updated by an individual or group. Blogs typically allow readers to
comment by adding their own entries, rather than altering existing ones,
which can provide a rich array of perspectives and opinions. Unlike text-
only threaded discussion boards, blogs look good with their embedded
hyperlinks, images, and custom layouts. See an example of a blog at the HR
Blog, where managers can congregate to discuss human resources issues.
Plant managers at DaimlerChrysler U.S., for example, use blogs to discuss
and keep track of problems and their solutions, and IBM employees
worldwide use blogs to maintain a running tab of software development
projects and business strategies (McGregor, 2004).
A wiki (quickly, quickly, in the Hawaiian language) is a website where
several authors come together online to quickly generate and modify
content. Wikipedia.org is an example. One financial services company uses
a wiki to keep experts talking about a difficult topic and to then use their
collected thoughts to educate less experienced employees.
Blogs and wikis are designed to facilitate fast and inexpensive collaboration
and information sharing. Think of them as grassroots tools because they
put more power in employees’ hands. Blogs and wikis “… allow the user to
determine the relevancy of content rather than being dependent upon a
central distribution center or a linear distribution chain. After the initial
setup, users, not administrators, control a wiki, to the benefit of both”
(Goodnoe, 2005).
BLENDED LEARNING OPPORTUNITIES
1-800-262-9699
•
www.amanet.org
8

• Online communities
Online communities can expose participants to the skills, knowledge, and
culture of their profession. Though typically informal in nature, online
communities benefit from the care of an online moderator who can manage
the flow and pace of discussions, pose and answer technical questions, and
sustain group process. In a study of an online community of coordinators
for Alberta Community Adult Learning Councils, Gray (Gray, 2004) found
that participants “perceived the role of the moderator as ‘absolutely critical’
in starting up, supporting, and sustaining the informal online
environment.”
• Performance support
Performance support puts the smarts at employees’ fingertips rather than
inside their heads (Rossett & Mohr, 2004). Support may include online help
systems and knowledge databases, decision tools, documentation, templates,
job aids, and so forth. In the U.S. Coast Guard, to complete safety
inspections aboard vessels, officers rely on their personal digital assistants
(PDA). The PDA walks them through a series of questions about the vessel
and customizes an inspection checklist.
BLENDED LEARNING OPPORTUNITIES
We must look beyond the boundaries of traditional training, and beyond
the boundaries of the course. Certainly this will take us into performance
support and knowledge management, but we must go further, bleeding
e-learning into corporate communications, workplace learning,
marketing, recruitment, customer learning, searches on the web and the
real world. This expansive view of learning delivery offers lots of scope
for exciting new approaches to blended learning.
…. The learning organization is built not on the premise of more
training. In fact, in the case of formal training, less rather than more
may be required. It is built upon the need for learners to feel motivated
towards achieving goals through continuous learning. (Clark, 2003)
1-800-262-9699
•
www.amanet.org
9

What types of blending make sense?
Blending is all about decisions. Managers must decide:
• Which assets to buy, build or adapt, and how they will be maintained;
• How and when to leverage the precious human resource provided by
experts and instructors;
• How much guidance and independence employees will enjoy;
• What kind of guidance system and support will be provided, if any at
all, and whether assistance comes from the instructor, an expert, the
manager, or an automated program.
Graham (2005) suggests that most blends today are collections of separate,
stand alone F2F and/or online components from which learners pick and
choose. Direction is minimal; freedom is maximized. Citicorp Latin
America successfully deployed such a blend with instructor-led workshops
and web-based modules for sales training. After the basic training class,
sales people may or may not elect to complete the modules. Likewise, some
who completed the modules may not have attended the F2F workshop.
In a more directed blend, components are presented within a defined
learning system. Integration might come from diagnostics that point
learners to specific lessons, resources, or assessments that culminate in a
certificate or other form of recognition. For example, students studying
project management at Defense Acquisition University can take self-
assessments that check their understanding of content from prerequisite
courses, and point them to topic areas in preparation for the next course.
Three blended models
Anchor Blend
An Anchor Blend starts with a defining and substantive classroom event,
followed by independent experiences that include interaction with online
resources, structured workplace learning activities, online learning and
reference, diagnostics, and assessments.
Schneider Electric, an international company with more than 85,000
employees worldwide, uses an anchor-blended approach for their
professional managers development program (PMDP) (Whitney, 2005).
Richardson and eCornell use an anchor blend for a sales and service
BLENDED LEARNING OPPORTUNITIES
1-800-262-9699
•
www.amanet.org
10

training program that launches with a highly interactive classroom event
followed by online modules for continuous reinforcement (CLO, 2005b).
Bookend Blend
The Bookend Blend is characterized by a three-part experience: something
introductory, an essential, substantive and meaty learning experience,
online or F2F, and then something that concludes and extends the learning
into practice at work.
As shown in Figure 1, AMA’s three-part approach to blended learning
combines “before-the-seminar” elements such as web-based pre-
assessments and a customized “seminar learning plan,” then substantive F2F
seminar experiences, concluding with “after-the-seminar” post-assessments.
If post-assessments reveal skill gaps, participants are encouraged to
participate in small “tune up” courses, additional readings and other related
materials (Leonard, 2005).
Figure 1. The AMA Blended Learning Experience
Mail Boxes Etc., now UPS, relied on substantive classroom experiences, with
on-the-job training as their bookends. The four-week blended program for
new franchisees started with one week of training on-the-job combined
with online courses. After that grounding in the field, employees attended
two weeks of classroom training at a regional training site. Finally, they
returned to their stores for more on-the-job training supported by online
resources.
Field Blend
The Field Blend is most distinct from training-as-usual. It is employee-
centric, with each individual surrounded by many kinds of assets and
continuous choices about when and where and whether to reach for them.
BLENDED LEARNING OPPORTUNITIES
1-800-262-9699
•
www.amanet.org
11
PLAN & PREPARE INTERACT APPLY
ONLINE LIVE TRAINING ONLINE/ON THE JOB

Although a classroom experience is often part of the Field mix, it is but one
method of many, with the focus on independent and persistent choices and
continuous learning and reference at work. The Field Blend requires
employees to commit to continuing growth and effort, relying on defined
expectations, engaged managers, and assessments that point to resources.
Key to the effort, of course, is provision of a treasure trove of resources and
some way to find what’s there and what’s right for you or your people.
For example, for engineers at Shell EP, blending relies on authentic
workplace problems as grist for learning. As they work through these real-
life problems, employees share their experiences, work product, and
reflections by contributing to an online repository. Engineers might
compare their situation to a colleague’s, or look for trends in the field and
relate those to the course materials. Their contributions become content
objects which can be reused for follow-up activities (Margaryan, Collis, &
Cooke, 2004).
A global technology company uses a blended approach for management
development. Once individuals become managers, they are paired with
other new managers and provided with ongoing access to learning,
communications, assessment, feedback, and information. Participants
receive basic management content through self-paced online modules, they
learn through interactive scenarios, and they use an online workspace to
collaborate with peers and experienced e-mentors. The learning
management system provides tracking and periodic assessment so that
dedicated tutors can follow and coach participants’ progress. The classroom
remains a part of the blend, with managers and their mentors meeting F2F
to advance lessons learned online.
Comparing three BL models
• Familiarity
The Anchor Blend will feel most familiar because it starts in the classroom.
In the classroom, an instructor leads the group through structured
experiences and introduces other more independent options, online and
otherwise. The classroom moments are essential because employees are
taught content and how the blended system works. New roles and
expectations are introduced. Relationships are nurtured and a taste is
developed for the continuing engagement and independent efforts to follow.
BLENDED LEARNING OPPORTUNITIES
1-800-262-9699
•
www.amanet.org
12

• Authenticity
The Field Blend will feel the most authentic because learning is there,
within the flow of work and life. When time allows or because a pressing
need has emerged, the employee seeks out a supervisor, experiences an e-
learning module, or consults an e-coach, online knowledge base, print
documentation, or FAQ.
• Independence
While all blends require independent choices about engagement by
employees, the Field Blend is predicated on it. It delivers a buffet of
resources and relies upon nutritious habits by employees over time, as they
work. In contrast, employees typically experience the Bookend and Anchor
Blends as a group. While some independent action is required, a group of
employees typically follows similar and defined learning paths.
• Simplicity
Nothing, of course, is as simple as scheduling a class, hiring an instructor,
and filling the seats. And that’s not all that simple. Still, in comparison with
each other and from the perspective of participants and managers, the
Bookend Blend is most simple of all. Purposely constrained in the number
of assets, and forthright in defining what to do and in what order, the
Bookend Blend comforts by stating, limiting, and directing choices. This
makes good sense for novice or reluctant participants. The benefits of
extension into the workplace and a continued conversation are there; the
degrees of freedom have been bounded, but not so much as to lose the
benefits of practice and learning set in the real world. AMA’s Bookend
Blend, for example, assures and guides learners through defined elements
before and after F2F experiences.
BLENDED LEARNING OPPORTUNITIES
1-800-262-9699
•
www.amanet.org
13

How do I craft a blend?
Table 2. Consider the audience
• Rely on the Anchor or Bookend Blends to guide
involvement,capitalize on instructors’ talents and
expertise, and reduce and define options. Consider
live e-learning events and e-coaching. Informal and
low cost approaches are useful, such as supervisor
coaching, lunch chats, listservs, email, documen-
tation, databases, e-coaching, and resource links.
• Technology will help meet the needs of dispersed
employees. For large numbers, and where content
is stable, consider development and maintenance
of media-rich assets such as online modules,
high-quality video and audio, scenarios,
simulations, and online help systems.
Asynchronous and synchronous events, e-
coaches, online communities and listservs with
involved experts will also work here. Ensure
guidance systems that allow users to search and
find what they need.
• If they doubt themselves or the topic, rely on
human guidance and interactions to coach and
motivate. Ensure that participants experience
success through an active instructor, live e-
learning, and supervisory coaching. Anchor and
Bookend Blends are suitable. The Field model is
less so.
• No surprise here. They are accustomed to it. The
Bookend Blend holds promise for employees
keen on the classroom. The Anchor Blend also
makes sense here. Use classroom events to
demonstrate how technology can deliver answers
as needed and nurture interpersonal connections
through synchronous events, e-coaching,
resource links, and online communities.
BLENDED LEARNING OPPORTUNITIES
1-800-262-9699
•
www.amanet.org
14
The total audience
is large and far-
flung
Employees are
hesitant or
resistant
Employees like the
classroom
The total audience
is small (<250)

• The Field Blend allows for many tastes and
preferences. The challenge is to provide a
guidance and search system, rich and updated
resources, and ways for individuals to combine
their experiences in ways that are rewarded and
recognized.
• Different content requires different amounts of
maintenance. For volatile content, develop assets
that are easier and cheaper to build and
maintain, such as online resource links,
databases, listservs, email notifications, live e-
learning, online documentation, and live
coaching and mentoring in class or via personal
relationships.
• If employees must know it by heart, any of the
three models will work. What’s critical is to invest
in extensive practice and feedback over time. Use
online tutorials, workbooks, simulations,
coaching, e-coaching, and classroom practice
sessions. Measure achievement. Track effort and
progress. Provide feedback. Consider
certification.
• If there is little time for employees to attend
training, provide them with what they need at
work, in the workflow, and linked to pressing
questions and tasks. Rely on resources that aren't
time-dependent, such as self-paced learning
modules, examples, readings, and asynchronous
BLENDED LEARNING OPPORTUNITIES
“Self-training (e.g., e-learning) faces four major challenges: The ability
to get employees motivated on their own; No access to continuous
personalized support; No other learning tools besides the course materials
employees and managers use; No extensive hands-on practice so new
skills can be immediately applied on the job.” (AMA, 2004)
1-800-262-9699
•
www.amanet.org
15
Employees must
demonstrate
that they know
their stuff
Our people need
to be at work,
not in class
Employees are
diverse and so are
their interests and
challenges
The content
changes often
Table 3. Consider results and reality

discussions. Provide short targeted lessons and knowledge resources that are
linked to their challenges and tasks. Whenever possible, minimize training
and insinuate performance support and job aids into the work. If employees
are committed and eager, the Field Blend should work. If they are less keen
or confident, get supervisors, managers and instructors involved to
encourage employees to habitually feast from accessible resources targeted
to questions.
How does BL affect what people do?
Blended learning redefines what everybody does.
Employees
Blended learning shifts responsibility for learning from the instructor to the
employee. For many, this is not an easy transition. They like what they know
– classroom experiences led by instructors – and are often uncomfortable
and not particularly adept at learning more independently and online. “An
important finding in the education literature is that many students given
BLENDED LEARNING OPPORTUNITIES
"When you have senior management doing the teaching, they have so
much credibility with the workforce. Because they take the time to teach,
it shows true sponsorship and true engagement, and I think then
employees learn even more. It gets back to the manager being the most
important person in an employee’s life," says Rita Danker, vice president
of organizational development and human resources at Schneider
Electric. (Whitney, 2005)
1-800-262-9699
•
www.amanet.org
16

control over their own learning choose to terminate the experience before
mastering the training task,” (Brown, 2001).
Provide useful, clear guidance systems that link to work and career paths.
Help employees consider their readiness to learn continuously and
independently. Think about your employees. Are they ready? What might
get in the way? Quicklist 1 can help reflect on their readiness.
Supervisors/Managers
Blended learning, because it often occurs in the workplace, depends on an
active supervisor or manager. When BL is in place, managers and
supervisors must coach, guide, track, motivate, and encourage. They can
influence F2F or online, formally and informally, synchronously and
asynchronously. Quicklist 2 raises questions for managers.
To support the development of new real estate agents at Century 21,
managers are given a coaching guide to answer questions, contextualize the
content, and provide on-the-job learning opportunities as new agents
participate in online and F2F training. General Electric and Intel define
employee development as a part of the manager’s job.
Shell EP used a ‘learning agreement’ between the learner and manager that
detailed expected performance improvement to result from the course.
While learners were pleased with the learning agreements/contracts they
developed with their managers, managers were less enthusiastic. The
researchers concluded that “It is necessary to identify additional tools and
strategies to involve the line managers, to extend their role from
“approving”the participation in the course to being a full partner in the
learning process,” (Margaryan et al., 2004).
BLENDED LEARNING OPPORTUNITIES
1-800-262-9699
•
www.amanet.org
17

Instructors
In a BL system, teaching is important, but not sufficient; instructors do
more than teach. Through online systems, they might monitor and nudge
employees’ progress, moderate a discussion board, coach managers, enliven
online communities, offer feedback on a group or individual task, analyze
workplace readiness, and post answers to frequently asked questions.
In Avaya’s blended learning courses, instructors act as experts and
consultants, helping students analyze and solve problems. They work with
students in the classroom to develop diagnostic and trouble-shooting skills,
and they provide online support by monitoring student progress. At the
Defense Acquisition University, instructors are growing into coaches and
performance consultants. They tailor services to needs and are measured, in
part, by the satisfaction of the people they serve in these new ways.
Experts
One of the nifty things about blends is that content experts can be involved
in small yet targeted ways, in person or via technology. They may be used to
enhance a specific lesson, or called upon as a “go to” person in the field with
questions, guidance, and resources. Wyndham International, Inc. uses a
system that works much like the online dating service, Match.com.
Wyndham’s system connects hotel employees to experienced people who
can answer questions.
Executives
The executive’s role is to provide an organizational environment that is
collaborative and cross-functional. Here is where learning professionals and
business leaders foster a learning culture that extends beyond classrooms
and into the workplace.
Before embarking on BL, executives should examine where they stand.
Quicklist 3 will help.
BLENDED LEARNING OPPORTUNITIES
In a 2004 survey of training professionals, “in terms of skills required by
online trainers/instructors, a majority of the respondents predicted that
by 2010, online facilitating or moderating would be the most vital skill for
online trainers/instructors, followed by online mentoring, lecturing, and
evaluating or assessing skills.” (Kim, Bonk, & Zeng, 2005)
1-800-262-9699
•
www.amanet.org
18

How do we look at the effectiveness of blends?
In a recent survey by the eLearning Guild, respondents projected an
increase in measurement of e-learning based on business impact (Pulichino,
2005). Good intentions aside, according to the ASTD 2004 State of the
Industry report, evaluation that targets field results is far from universal.
What we attempt to do here is to provide a brief overview of the
possibilities for measuring BL.
Approaching measurement: Then and now
One approach to measurement, Kirkpatrick’s four levels of evaluation
(Kirkpatrick, 1998), has enjoyed decades of popularity (ASTD, 2003;
(Pulichino, 2005)). Kirkpatrick’s model is based on four levels, commencing
with the most familiar concerns and concluding with the most strategic: (1)
Did they like it? Were employees satisfied with the experience? (2) Did they
get it? Did employees learn? Can they do it? (3) Do they do it at work? Did
training transfer into performance back on the job? (4) So what? Does
performance translate into results that matter?
Recently, Phillips (Phillips, 2003) added a fifth level to the model that asks,
(5) Was it worth it? Do results justify the investment?
Rossett (Rossett, 2005a) suggested a shift away from five-level evaluation
that is predicated on being IN training or AT work, to assessment that is
ongoing and merged with practice, as is the case with blended learning.
Integrated evaluation, thus, asks questions about satisfaction, engagement,
influence, practice, and value and it asks these questions continuously.
BLENDED LEARNING OPPORTUNITIES
1-800-262-9699
•
www.amanet.org
19

Measurement of BL is:
ឣContinuous and meaningful. In the good old days, evaluation happened
after class, typically by administering a survey that queries participants
about satisfaction. With a blend, such questions happen often, for
example, after an executive has completed 20 minutes with her coach or
15 minutes looking at recent research on a key topic. Was that useful?
Would you recommend the experience to others? What other questions
linger? How can we be more useful to you?
ឣTransparent, diverse,and technology-based. The use of technology for
data collection provides training managers, line managers, executives,
and employees themselves with continuous and ready access to data.
Do I know how to answer that question? Did I demonstrate it? How am
I doing with this material? Data can be collected using surveys, interviews,
focus groups, observations done with performance checklists, review of
work products, self-assessments, multiple choice or essay tests,
simulations, bench work, automated tracking of progress or usage, and
so on and so forth. Technology can play a part, through web-based forms,
emails, short text messaging to wireless devices or cell phones, or online
assessment with customized feedback built into learning modules and
simulations. The technology systems help leaders know what their
employees want to know, how they are faring, how much they are
participating, the resources they are tapping, and how to be more
helpful throughout the process.
What gets measured?
What indicators will you use to look inside BL programs? Here are some
options:
ឣEnrollment and Completion. What are the enrollment rates? Did
employees complete a particular module, course, or series of courses? Did
they take assessments and self-assessments? Enrollment and completion
are germane where compliance and certification are concerned, but are
less relevant in BL, because the learning and relationships are often
ongoing and even informal. No matter the case, rely on assessments that
ask participants to measure up against work-relevant performance
standards.
ឣParticipation. Because blending shifts control and decisions to
employees, their choices reveal much about the health and success of the
BLENDED LEARNING OPPORTUNITIES
1-800-262-9699
•
www.amanet.org
20

program and assets. Which assets are chosen? Which ignored? How are
various assets and activities being accessed and used? What do typical
learning paths look like? How much do employees rely on the knowledge
base to find answers? How often did they review portions of an online
module, take the self-assessment, download a template, or examine a
video example? How much time did they spend engaged in online
communities or in conversations with their supervisors or an expert?
ឣReaction. Are they pleased about the time spent on the blend? Which
assets do they favor? Which do they ignore? While few will expect the
blend to be a rollicking good time, employees must see good reasons to
take an online class, chat with a coach, look at a database, self-assess, or
check in with an online community.Are employees satisfied with the
program and assets? What about coaches, experts, or supervisors? Do they
enjoy their roles? Will employees continue to come back for more? Would
they recommend it to their colleagues?
ឣLearning. Did employees earn a passing score for an entire course or
sections of it? Can they demonstrate attainment of skills, retention,
proficiency, and mastery through some type of authentic assessment?
Does the learning and assessment combine to earn participants some
form of recognition or certification?
ឣPerformance. Are they handling work challenges differently? Better? Do
outcomes reflect new skills and knowledge? Are they using what is taught
in the blend? Do customers perceive a difference?
ឣBusiness impact. Why did the sponsors commit to the program? What
indicators of success will they accept? Establish these answers and then
seek relevant data throughout the effort.
ឣValue. Was it worth it for the organization? Was it worth it for the
employee? Do the benefits realized from the program justify the costs in
terms of money and time spent on development, implementation, and
maintenance? Would the sponsor do it all over again?
BLENDED LEARNING OPPORTUNITIES
1-800-262-9699
•
www.amanet.org
21

An example of how evaluation might work
Let’s look at how we might approach measurement of a blend devoted to
developing customer service specialists at a financial services company.
Adriana, a retirement specialist, is glued to her desk, taking customer phone
calls and using her computer to access customer records and blended assets.
She and her colleagues are motivated to continuously develop their skills
BLENDED LEARNING OPPORTUNITIES
1-800-262-9699
•
www.amanet.org
22

because tax laws and the company’s portfolio of services are constantly
changing, and also because they earn more as they acquire appropriate
certifications. Their supervisors are supportive of the program and are
themselves certified. For this situation, the Field Blend offers choices,
freedom and practicality. Below is a description of how Adriana might
experience the blend, and why these particular indicators were chosen.
BL: bridging the classroom and workplace
Years ago, after teaching a seminar in a room in a hotel, one of the authors
heard these words,“That was a great class, but…” The participant then
offered a litany of concerns, starting with her manager and proceeding to
lack of time, lack of fit with her job description, and the certainty that if she
did this, it wouldn’t elevate the measures on her performance review.
Blended learning, predicated on an active manager, engaged peers, targeted
guidance system, and rich learning and information assets, begins to tackle
some of the “buts” by accompanying her at work.
But it is just the beginning for BL.
It is important to admit that as good an idea as blended learning is, in most
settings today, it is more potential than actual. BL is a glimmer in the eyes of
people eager to transcend the limitations of classroom instruction and training
schedules – too often experiences that are set in time and bounded by place.
What we enjoy now is a blended learning opportunity. Each organization
has that opportunity, like the organizations described in this paper, to
leverage internal experts and installed technology, to reiterate classroom
messages, and to systematically stir treasured assets into the workplace and
workforce. Ask your executives. Ask yourself. If it was possible, wouldn’t
organizations be strengthened by more pervasive learning experiences,
knowledge, support and conversations in closer proximity to where the
work gets done, where customers are served, and where problems are
solved?
Perhaps most intriguing of all is the influence on employees themselves.
Blended learning asks much of them, while it establishes and applauds skills
and habits that favor continuous growth, collaboration, curiosity and
reference. What thoughtful manager wouldn’t be keen on that?
BLENDED LEARNING OPPORTUNITIES
1-800-262-9699
•
www.amanet.org
23

THE PERFECT STORM: The Future of Retention and Engagement
1-800-262-9699
•
www.amanet.org
24
Quicklist 4: I want to learn more about blended learning.
I have a few
hours
• AMA. (2004). Training for Today's Business Needs.
• Bersin, J. (2004). Blended Learning: Finding What Works. Chief Learning Officer.
• Brennan, M. (January 2004). Blended Learning and Business Change. Chief Learning
Officer.
• Rossett, A., Douglis, F., & Frazee, R. V. (2003). Strategies for Building Blended
Learning, Learning Circu its.
I have a day or
so
• Check out ASTD, eLearning Gu ild, and Chief Learning Officer for research, ar ticles, and
seminars.
• Peruse articles on blended learning at the e-Learning Centre.
• Read articles and research rel ated to busin ess, learnin g, and results at IDC.
• Examine examples at Rossett, A., & Frazee, R. V. (2003). E-learning Portfolio.
I am ready to
make a big
commitment
• Bielawski, L., & Metcalf, D. (2003). Blended eLearning: Integrating Knowledge,
Performance Support, and Online Learning. HRD Press .
• Bersin, J. (2004). The Blended Learning Book: Best Practices, Proven Methodologies, and
Lessons Learned. Pfeiffer.
• Bonk, C. J., & Graham, C. R. (Eds.). (2005). The Handbook of Blended Learning: Global
Perspective s, Local Design s. Pfeiffer.
• Driscoll, M ., & Carliner, S . (2005). Advanced Web-Based Training Strategies: Unlocking
Instructionally Sound Online Learning. John Wiley & Sons.
• Goldsmith, M., Morgan, H., & Ogg, A.J. (2004). Leading Organizational Learning:
Harnessing the Power of Knowledge. Jossey-Bass.
• Lev, B. (2001). Intangibles: Management, Measurement, and Reporting. Washington,
D.C.: Brookings Institution Press.
• Masie, E. (Ed.). (2005). Learning Rants, Raves, and Reflections. Pfeiffer.
• Piskurich, G. (Ed.). (2003). The AMA Handbook of E-Learning: Effective Design,
Implementation, and Technology Solutions. NY: AMACOM.
• Rossett A. (Ed.) (2002). The ASTD e-learning Handbook: Best Practices, Strategies, and
Case Studies for an Emerging Field. NY: McGraw-H ill.
• Rossett, A., & Sheldon, K. (2001). Beyond the Podium: Delivering Training and
Performance to a Digital World. Jossey-Bass/Pfeiffer.
• Rothwell, W.J., Lindholm, J., & Wallick, W.G. (2003). What CEOs Expect From
Corporate Training: Building Workplace Learning and Performance Initiatives That
Advance Organizational Goals. NY: AMACOM.
• Schank, R. (2005). Lessons in Learning, e-Learning, and Training: Perspectives and
Guidance for the Enlightened Trainer. Pfeiffer.
• Tobin, D. R. (1997). The Knowledge-Enabled organization: Moving from "Training" to
"Learning" to Meet Business Goals. New York, NY: AMACOM.

Dr.Allison Rossett, long time Professor of Educational Technology at
San Diego State University, is in the Training magazine HRD Hall of Fame
and is a member of the ASTD International Board of Directors. Recipient
of ASTD’s 2002 award for her contributions to workplace learning and
performance, Rossett edited The ASTD E-Learning Handbook: Best Practices,
Strategies, and Case Studies for an Emerging Field. Rossett also wrote Beyond
the Podium: Delivering Training and Performance to a Digital World and
First Things Fast: A Handbook for Performance Analysis. Prior award-
winning books are Training Needs Assessment, and A Handbook of Job Aids,
currently undergoing revision. Some of Allison recent articles are “Moving
Your Class Online,”“Confessions of a Web Dropout,”and “Training and
Organizational Development, Siblings Separated at Birth.” She has
published articles on persistence in online learning, needs analysis, and
performance support. Rossett's client list includes Microsoft, IBM, HP,
the Getty Conservation Institute, Fidelity Investments, Deloitte Consulting,
BP, the IRS, TSA, Amgen, Royal Bank of Scotland, and several elearning
start-ups. Dr. Rossett can be reached at arossett@mail.sdsu.edu.
Rebecca Vaughan Frazee, has been working as an instructional designer
and performance consultant for nearly a decade. She has managed several
large-scale projects and designed f2f and technology-based solutions for
national, global and Fortune 50 companies. Her work centers on the
development of training and education professionals, helping them expand
their focus beyond training to more systemic performance solutions,
and supporting them in the move to performance consulting, e-learning
and blended learning. She has taught graduate courses in educational
technology and conducted workshops and presentations on project
management, data analysis, needs assessment and performance
improvement. Her recent publications include “Technology adoption:
Bringing along the late-comers” in the ASTD E-Learning Handbook,
“Using Relevance to Facilitate Online Participation in a Hybrid Course” in
EDUCAUSE Quarterly, and “Begin with the end (user) in mind: Planning
for the San Diego State University campus portal” in Designing Portals:
Opportunities and Challenges. Rebecca’s client list includes Motorola,
Fidelity Investments, PricewaterhouseCoopers, Prudential Securities,
Eli Lilly, TEC International, NASSCO, the Corporation for National
Service, San Jose State University, and the IRS. She can be reached at
rebeccafrazee@cox.net.
BLENDED LEARNING OPPORTUNITIES
1-800-262-9699
•
www.amanet.org
25

References
AMA. (2004).Training for Today's Business Needs. Retrieved August 20, 2005, from
http://www.amanet.org
Bersin, J. (2004). Blended Learning: Finding What Works. Retrieved February 26, 2004, from the Chief
Learning Officer website at http://www.clomedia.com/content/templates/clo_feature.asp?arti-
cleid=357&zoneid=30
Brown, K. G. (2001). Using computers to deliver training:Which employees learn and why? Personnel
Psychology, 54(2), 271-296.
Clark, D. (2003). Blended Learning (White Paper). Retrieved August 20, 2005, from
http://www.epic.co.uk/content/resources/white_papers/blended.htm
CLO. (2005a). Industry News: Domino's Pizza Aims to Improve Retention with E-Learning. Retrieved
August 1, 2005, from http://www.clomedia.com/common/newscenter/newsdisplay.cfm?id=3970
CLO. (2005b). Industry News: eCornell Selects Richardson for Blended Training. Retrieved August 1,
2005, from http://www.clomedia.com/common/newscenter/newsdisplay.cfm?id=4157
eTForecasts. (2005). PCs In-Use Surpassed 820M in 2004: PCs In-Use Will Top 1B in 2007. Retrieved
August 20, 2005, from http://www.etforecasts.com/pr/pr305.htm
Goodnoe, E. (2005, August 8). How To Use Wikis For Business, from
http://www.informationweek.com/story/showArticle.jhtml?articleID=167600331
Graham, C. R. (2005). Blended Learning Systems: Definition, Current Trends, and Future Directions;.
In C. Bonk & C. R. Graham (Eds.), Handbook of blended learning: Global Perspectives, local designs.
Graham, C. R.,Allen, S., & Ure,D. (2003). Blended learning environments: A review of the research
literature. Unpublished manuscript, Provo, UT.
Gray, B. (2004). Informal Learning in an Online Community of Practice. Journal Of Distance Education,
19(1), 20-35.
Kim, K.-J., Bonk, C. J., & Zeng,T. (2005). Surveying the future of workplace e-learning: the rise of
blending, interactivity, and authentic learning, from
http://www.elearnmag.org/subpage.cfm?section=research&article=5-1
Kirkpatrick, D. (1998). Evaluating training programs:The four levels (2nd ed.). San Francisco, CA:
Berrett-Koehler Publishers.
Leonard, P. (2005). Blended learning provides efficient, effective, measureable learning. Retrieved
August 1, 2005, from http://www.amanet.org/blended/PatLeonardArticle.htm
Margaryan, A., Collis, B., & Cooke, A. (2004). Activity-based blended learning. Human Resource
Development International, 7(2), 265-274.
McGregor,J. (2004). It's a Blog World After All. Retrieved 81, from Fast Company at
http://www.fastcompany.com/magazine/81/blog.html
Modi, N. (2005). The Majority Of Global Internet Users Using A High-Speed Connection. Retrieved
August 20, 2005, from http://www.ipsos-na.com/news/pressrelease.cfm?id=2583
Neal, L. (2004). Predictions for 2004, from E-Learn Magazine at
http://www.elearnmag.org/subpage.cfm?section=opinion&article=39-1
Nelson, A. (2005). Deploying Media-rich Learning Systems. CLO Magazine, 4(5), 45-47.
Osguthorpe, R.T., & Graham, C. R. (2003). Blended learning systems: Definitions and
directions. Quarterly Review of Distance Education, 4(3), 227-234.
Phillips, J. (2003). Return on Investment in Training and Performance Improvement.
Boston, MA: Butterworth-Heinemann.
Pulichino, J. (2005).The eLearning Guild: Metrics and Measurement 2005 Research Report.
Reigeluth, C. M., & Stein, E. S. (1983). The elaboration theory of instruction. In C. M. Reigeluth (Ed.),
Instructional design theories and models: An overview of their current status (pp. 335-381).
Hillsdale, NJ: Lawrence Erlbaum Associates.
Rossett, A. (2005a). Are We Asking the Right Questions? Paper presented at The eLearning Guild
Instructional Design Conference, Boston, Mass.
BLENDED LEARNING OPPORTUNITIES
1-800-262-9699
•
www.amanet.org
26

Rossett, A. (2005b). Never Worry about Transfer Again—Seven Strategies for Converging Learning and
Work. Paper presented at the International Society for Performance Improvement Annual Conference,
Vancouver, British Columbia, Canada.
Rossett, A., Douglis, F., & Frazee, R. V. (2003, July). Strategies for building blended learning, Learning
Circuits. Retrieved August 1, 2004, from http://www.learningcircuits.org/2003/jul2003/rossett.htm
Rossett, A., & Marino, G. (2005). If coaching is good, then e-coaching is... T+D, 59(11).
Rossett, A., & Mohr, E. (2004). Performance support tools: where learning, work, and results converge:
Einstein was an early performance support tool user. The U.S. Coast Guard uses PSTs. Time to get on
board? T&D, 58(2).
Rossett, A., & Schafer, L. (June 2003). What to do about e-dropouts: What if it's not the e-learning but
the e-learner? T+D, 57(6), 40-46.
Singh, H. (2003). Building Effective Blended Learning Programs. Educational Technology
& Learning, 43, 51-54.
Sparrow, S. (2004). Blended is Better. By: Sparrow, Stephanie. T+D, Nov 2004,Vol. 58 Issue 11, p 52, 4p;.
Sullivan, J. (2005, July).World-Class Talent Development: How Learning Organizations Align Talent
Development with Organizational Objectives, from
http://www.whitepaper.informationweek.com/shared/write/collateral/WTP/12970_44663_87636_wpa
per_TalentDev.pdf?ksi=1061922&ksc=1222810955
Thomson/NETg. (2003).Thomson Job Impact Study. Retrieved February 17, 2003, from
http://www.netg.com/Upload/uk_ThomsonJobImpactStudy.pdf
Whitney, K. (2005, August). Schneider Electric: Lighting Up the Performance Improvement Process.
Workforce Performance Solutions, 1(3), 34-35, 56.
Zenger,J., & Uehlein, C. (2001). Why blended will win. T + D, 55(8), 54-60.
BLENDED LEARNING OPPORTUNITIES
1-800-262-9699
•
www.amanet.org
27
About American Management Association
American Management Association is the world’s leading membership-based
management development organization. Since 1923, it has provided valuable
and practical action-oriented learning programs to business professionals at every
stage of their careers. Each year, more than 500,000 AMA customers and members
learn new skills and behaviors, gain more confidence, advance their careers, and
contribute to the success of their organizations through a wide range of AMA
seminars, conferences, and executive forums, as well as through AMA books
and publications, research, online learning and self-study courses. For more
information, visit www.amanet.org