Content uploaded by Barry Kort
Author content
All content in this area was uploaded by Barry Kort
Content may be subject to copyright.
1
An Affective Model of the Interplay Between Emotions and Learning
Barry Kort, Rob Reilly, Rosalind Picard
Media Laboratory, M.I.T.
{bkort, reilly, picard}@media.mit.edu
Abstract
This article proffers a novel model by which to
conceptualize the impact of emotions upon learning.
We believe there is an interplay of emotions and
learning, but this interaction is far more complex than
previous theories have articulated. Our model goes
beyond previous research studies not just in the
emotions addressed, but also in an attempt to formalize
an analytical model that describes the dynamics of
emotional states during model-based learning
experiences.
1. Introduction
Why is there no word in English for the
art of learning? Webster says that
pedagogy means the art of teaching.
What is missing is the parallel word for
learning. In schools of education,
courses on the art of teaching are simply
listed as “methods.” Everyone
understands that the methods of
importance in education are those of
teaching—these courses supply what is
thought to be needed to become a skilled
teacher. But what about methods of
learning?
- Seymour Papert, The Children’s Machine
Educators have traditionally emphasized conveying
information and facts; rarely have they modeled the
learning process. When teachers present material to the
class, it is usually in a polished form that omits the
natural steps of making mistakes (e.g., feeling
confused), recovering from them (e.g., overcoming
frustration), deconstructing what went wrong (e.g., not
becoming dispirited), and starting over again (with
hope and perhaps enthusiasm). Those of us who work
in science, math, engineering, and technology (SMET)
as professions know that learning naturally involves
failure and a host of associated affective responses.
Yet, educators of SMET learners have rarely
illuminated these natural concomitants of the learning
experience. The unfortunate result is that when
students see that they are not getting the facts right (on
quizzes, exams, etc.), then they tend to believe that
they are either ‘not good at this,’ ‘can’t do it,’ or that
they are simply ‘stupid’ when it comes to these
subjects. What we fail to teach them is that all these
feelings associated with various levels of failure are
normal parts of learning, and that they can be actually
be helpful signals for how to learn better.
Expert teachers are very adept at recognizing and
addressing the emotional state of learners and, based
upon that observation, taking some action that
positively impacts learning. But what do these expert
teachers ‘see’ and how do they decide upon a course of
action? How do student who have strayed from
learning return to productive path, such as the one that
Csikszentmihalyi [1990] refers to as his “zone of flow”?
Preliminary research by Lepper and Chabay [1988]
indicates that “expert human tutors… devote at least as
much time and attention to the achievement of
affective and emotional goals in tutoring, as they do to
the achievement of the sorts of cognitive and
informational goal that dominate and characterize
traditional computer-based tutors.”
Skilled humans can assess emotional signals with
varying degrees of accuracy, and researchers are
beginning to make progress giving computers similar
abilities at recognizing affective expressions. Although
computers perform as well as people only in highly
restricted domains, we believe that accurately
identifying a learner’s emotional/cognitive state is a
critical indicator of how to assist the learner in
achieving an understanding of the efficiency and
pleasure of the learning process. We also assume that
computers will, much sooner than later, be more
capable of recognizing human behaviors that lead to
strong inferences about affective state.
2
Axis -1. 0 -0. 5 0 +0. 5 +1. 0
Anxiety-Confidence Anxiety Worry Discomfort Comfort Hopeful Confident
Boredom-Fascination Ennui Boredom Indifference Interest Curiosity Intrigue
Frustration-Euphoria Frustration Puzzlement Confusion Insight Enlightenment Ephipany
Dispirited-Encouraged Dispirited Disappointed Dissatisfied Satisfied Thrilled Enthusiastic
Terror-Enchantment Terror Dread Apprehension Calm Anticipatory Excited
-1. 0 -0. 5 0 +0. 5 +1. 0
Figure 1 – Emotion sets possibly relevant to learning (in contrast to traditional emotion theories)
2. Affective Computing: Emotions and
Learning
The extent to which emotional upsets
can interfere with mental life is no
news to teachers. Students who are
anxious, angry, or depressed don’t
learn; people who are caught in these
states do not take in information
efficiently or deal with it well.
- Daniel Goleman, Emotional Intelligence
In order to accomplish our goal, which is to
embody a computer with the ability to identify a
learner’s affective state and respond accordingly, we
must redefine, and in some cases, reengineer various
aspects of educational pedagogy. To this end it is
necessary for us to rethink our perspective of what is
happening in education and based upon our
hypothesis reengineer accordingly. Some of these
beliefs will be theorized, perhaps beyond a practical
level but not beyond a level needed for understanding
them. We need to explore the underpinnings of
various educational theories and evolve or revise
them. For example, we propose a model that
describes the range of various emotional states during
learning (see Figure 1). The model is inspired by
theory often used to describe complex interactions in
engineering systems, and as such is not intended to
explain how learning works, but rather is intended to
give us a framework for thinking about and posing
questions about the role of emotions in learning.
Like with any metaphor, the model has limits to its
application. In this case, the model is not intended to
fully describe all aspects of the complex interaction
between emotions and learning, but rather only to
serve as a beginning for describing some of the key
phenomena that we think are all too often overlooked
in learning pedagogy. Our model goes beyond
previous research studies not just in the emotions
addressed, but also in an attempt to formalize an
analytical model that describes the dynamics of
emotional states during model-based learning
experiences, and to do so in a language that the
SMET learner can come to understand and utilize.
3. Guiding Theoretical Frameworks:
Developing an Advanced Technology
The older [learning theories] deal with
the activity that is sometimes
caricatured by the image of a white-
coated scientist watching a rat run
through a maze…newer [thinking is]
more likely to be based upon the
theories of performance of computer
programs than on the behavior of
animals… but… they are not about the
art of learning… they do not offer
advice to the rat (or to the computer)
about how to learn.
- Seymour Papert, The Children’s Machine
Before describing the model’s dynamics, we should
say something about the space of emotions it names.
Previous emotion theories have proposed that there
are from two to twenty basic or prototype emotions
(see for example, Plutchik, 1980; Leidelmeijer,
1991). The four most common emotions appearing
on the many theorists’ lists are fear, anger, sadness,
and joy. Plutchik [1980] distinguished among eight
basic emotions: fear, anger, sorrow, joy, disgust,
acceptance, anticipation, and surprise. Ekman [1992]
has focused on a set of from six to eight basic
emotions that have associated facial expressions.
However, none of the existing frameworks seem to
address emotions commonly seen in SMET learning
experiences, some of which we have noted in Figure
1. Whether all of these are important, and whether
the axes shown in Figure 1 are the “right” ones
remains to be evaluated, and it will no doubt take
many investigations before a “basic emotion set for
learning” can be established. Such a set may be
culturally different and will likely vary with
developmental age as well. For example, it has been
argued that infants come into this world only
3
expressing interest, distress, and pleasure [Lewis,
1993] and that these three states provide sufficiently
rich initial cues to the caregiver that she or he can
scaffold the learning experience appropriately in
response. We believe that skilled observant human
tutors and mentors (teachers) react to assist students
based on a few ‘least common denominators’ of
affect as opposed to a large number of complex
factors; thus, we expect that the space of emotions
presented here might be simplified and refined
further as we tease out which states are most
important for shaping the companion’s responses.
Nonetheless, we know that the labels we attach to
human emotions are complex and can contain
mixtures of the words here, as well as many words
not shown here. The challenge, at least initially, is to
see how our model and its hypothesis can do initially
with a very small space of possibilities, since the
smaller the set, the more likely we are to have greater
classification success by the computer.
Constructive Learning
Disappointment Awe
Puzzlement Satisfaction
Confusion Curiosity
II I
Negative Positive
Affect Affect
III IV
Frustration Hopefulness
Discard Fresh research
Misconceptions
Un-learning
Figure 2a – Proposed model relating phases of
learning to emotions in Figure 1
II I
III IV
Figure 2b – Circular and helical flow of emotion
Figures 2a and 2b attempt to interweave the emotion
axes shown in Figure 1 with the cognitive dynamics
of the learning process. The horizontal axis is an
Emotion Axis. It could be one of the specific axes
from Figure 1, or it could symbolize the n-vector of
all relevant emotion axes (thus allowing multi-
dimensional combinations of emotions). The positive
valence (more pleasurable) emotions are on the right;
the negative valence (more unpleasant) emotions are
on the left. The vertical axis is what we call the
Learning Axis, and symbolizes the construction of
knowledge upward, and the discarding of
misconceptions downward. (Note: we do not see
learning as being simply a process of
constructing/deconstructing or adding/subtracting
information; this terminology is merely a projection
of one aspect of how people can think about learning.
Other aspects could be similarly included along the
Learning Axis.)
The student ideally begins in quadrant I or II: they
might be curious and fascinated about a new topic of
interest (quadrant I) or they might be puzzled and
motivated to reduce confusion (quadrant II). In either
case, they are in the top half of the space, if their
focus is on constructing or testing knowledge.
Movement happens in this space as learning
proceeds. For example, when solving a puzzle in The
Incredible Machine, a student gets an idea how to
implement a solution and then builds its simulation.
When she runs the simulation and it fails, she sees
that her idea has some part that doesn’t work – that
needs to be deconstructed. At this point it is not
uncommon for the student to move down into the
lower half of the diagram (quadrant III) where
emotions may be negative and the cognitive focus
changes to eliminating some misconception. As she
consolidates her knowledge—what works and what
does not—with awareness of a sense of making
progress, she may move to quadrant IV. Getting a
fresh idea propels the student back into the upper half
of the space, most likely quadrant I. Thus, a typical
learning experience involves a range of emotions,
moving the student around the space as they learn.
If one visualizes a version of Figures 2a and 2b for
each axis in Figure 1, then at any given instant, the
student might be in multiple quadrants with respect to
different axes. They might be in quadrant II with
respect to feeling frustrated; and simultaneously in
quadrant I with respect to interest level. It is
important to recognize that a range of emotions
occurs naturally in a real learning process, and it is
not simply the case that the positive emotions are the
good ones. We do not foresee trying to keep the
student in quadrant I, but rather to help them see that
the cyclic nature is natural in SMET learning, and
that when they land in the negative half, it is only
4
part of the cycle. Our aim is to help them to keep
orbiting the loop, teaching them how to propel
themselves especially after a setback.
A third axis (not shown), can be visualized as
extending out of the plane of the page—the
Knowledge Axis. If one visualizes the above
dynamics of moving from quadrant I to II to III to IV
as an orbit, then when this third dimension is added,
one obtains the ‘excelsior spiral that climbs the tree
of knowledge.’ In the phase plane plot, time is
parametric as the orbit is traversed in a
counterclockwise direction. In quadrant I,
anticipation and expectation are high, as the learner
builds ideas and concepts and tries them out.
Emotional mood decays over time, either from
boredom or from disappointment. In quadrant II, the
rate of construction of working knowledge
diminishes, and negative emotions emerge as
progress flags. In quadrant III, the learner discards
misconceptions and ideas that didn't pan out, as the
negative affect runs its course. In quadrant IV, the
learner recovers hopefulness and positive attitude as
the knowledge set is now cleared of unworkable and
unproductive concepts, and the cycle begins anew.
In building a complete and correct mental model
associated with a learning opportunity, the learner
may experience multiple cycles around the phase
plane until completion of the learning exercise. Each
orbit represents the time evolution of the learning
cycle. Note that the orbit doesn't close on itself, but
gradually moves up the knowledge axis.
A computerized Learning Companion, which
would track a learner through their learning journey,
be sensitive to their affective state and respond
appropriately, could, we believe, use models such as
these to assess whether or not learning is proceeding
at a healthy rate. The model could help guide it in
exploring strategies for making decisions about when
best to intervene with a hint, word of encouragement,
or observation (typically in quadrants III and IV.)
Thus, we see the computerized Learning Companion
as helping to scaffold the learning experience by
trying to keep the learner moving through this space,
e.g., not avoiding quadrant III, but helping them to
keep moving through it instead of getting stuck there.
The models may also be useful to learners in aiding
in their own metacognition about their learning
experience, especially helping them identify and
work with naturally-occurring negative emotions in a
productive and cognitively satisfying way. And, as a
vicarious outcome, this model could be utilized by
human teachers when dealing with students.
4. References
[1] Chen, L.S., T.S. Huang, T. Miyasato, and R. Nakatsu,
“Multimodal human emotion/expression recognition,” in
Proc. of Int. Conf. on Automatic Face and Gesture
Recognition, (Nara, Japan), IEEE Computer Soc., April
1998.
[2] Csikszentmihalyi, M. (1990). Flow: The Psychology of
Optimal Experience, Harper-Row: NY.
[3] DeSilva, L.C., T. Miyasato, and R. Nakatsu, Facial
emotion recognition using multi-modal information, in
Proc. IEEE Int. Conf. on Info., Comm. and Sig. Proc.,
(Singapore), pp. 397-401, Sept 1997.
[4] Donato, G., M.S. Bartlett, J.C. Hager, P.Ekman, and
T.J. Sejnowski, Classifying facial actions, IEEE Pattern
Analy. and Mach. Intell., vol. 21, pp. 974--989, October
1999.
[5] Ekman, Paul. (1992). Are there basic emotions?,
Psychological Review, 99(3): 550-553.
[6] Goleman, D., (1995). Emotional Intelligence. Bantam
Books: New York.
[7] Huang, T.S., L.S. Chen, and H. Tao, Bimodal emotion
recognition by man and machine, ATR Workshop on
Virtual Communication Environments, (Kyoto, Japan),
April 1998.
[8] Leidelmeijer, K. (1991). Emotions: An Experimental
Approach. Tilburg University Press.
[9] Lepper. M.R. and R.W. Chabay (1988). Socializing the
intelligent tutor: Bringing empathy to computer tutors. In
Heinz Mandl and Alan Lesgold (Eds.), Learning Issues for
Intelligent Tutoring Systems, pp. 242-257.
[10] Lewis M., (1993). Ch. 16: The emergence of human
emotions. In M. Lewis and J. Haviland, (Eds.), Handbook
of Emotions, pages 223-235, New York, NY. Guilford
Press.
[11] Papert, Seymour (1993). The Children’s Machine:
Rethinking School in the Age of the Computer, Basic
Books: New York.
[12] Picard, Rosalind.W., (1997). Affective Computing.
Cambridge, MA: MIT Press 1997.
[13] Plutchik, R. ‘A general psychoevolutionary theory of
emotion,’ in Emotion Theory, Research, and Experience
(R. Plutchik and H. Kellerman, eds.), vol. 1, Theories of
Emotion, Academic Press, 1980.
[14] Scheirer, J., R. Fernandez and Rosalind. W. Picard
(1999), Expression Glasses: A Wearable Device for Facial
Expression Recognition, Proceedings of CHI, February
1999.
