A Cognitive Meta-Analysis of Design Approaches to
Interruptions in Intelligent Environments
Antti Oulasvirta and Antti Salovaara
Helsinki Institute for Information Technology
PO Box 9800, 02015 HUT, Finland
Minimizing interruptions to users is a crucial and acknowl-
edged precondition for the adoption of new intelligent tech-
nologies such as ubiquitous and proactive computing. This
paper takes a step toward achieving a consensus among the
numerous existing approaches addressing the challenge
posed by interruptions. We start by explicating why inter-
ruptions are considered important. We then reveal similari-
ties and differences among the approaches from a cognitive
viewpoint. It appears that the approaches draw from differ-
ent assumptions about human cognition. Some of the ap-
proaches contain inconsistencies. The cognitive analysis
also inspires directions for future work.
Interruptions, intelligent environments, user interfaces.
ACM Classification Keywords
H5.2. User Interfaces: Theory and methods
After over 10 years of research in intelligent environments
(IE) , the field now seems to be in a state of conceptual
balkanization. Currently, there are at least 15 named design
approaches. Consider, for example, proactive, ubiquitous,
pervasive, mobile, situated, wearable, ensemble, invisible,
context-aware, peripheral, and calm computing, ambient
intelligence, disappearing computer, attentive and intelli-
gent user interfaces, and personal technologies, each having
their proponents. Consequently, it is difficult for us re-
searchers to get an overall grasp of the field.
In this paper, we argue that designing disruption-free inter-
action is a central design problem for IE and “technology
beyond the desktop” in general . The problem is shared
by many of the approaches but also allows for distinguish-
ing between them. In this paper, we first explicate the prob-
lem of interruptions and then investigate and evaluate, from
the point of view of cognitive psychology, how some of the
most prominent approaches have addressed the problem.
What We Mean by Intelligent Environments?
By intelligent environments we mean technological aug-
mentation of user's physical surroundings with systems or
devices that are able to respond to user activity. This tech-
nology aims to provide services and control over processes,
and support decision-making and other cognitive needs.
Responsiveness and adaptation are based either on pre-
programmed heuristics or real-time reasoning capabilities.
All the approaches mentioned in the introductory paragraph
fit at least partly into this characterization. For the purposes
of this paper, intelligent environment serves as a general
umbrella term that covers most of the approaches.
WHY INTERRUPTIONS IS AN IMPORTANT DESIGN
ISSUE FOR INTELLIGENT ENVIRONMENTS?
IE will be in homes, lecture halls, gardens, schools, city
streets, cards, buses, trams, shops, malls etc. In other words,
elsewhere than at the desktop. As these use contexts inher-
ently involve many sequentially and simultaneously per-
formed tasks, they can be called multitasking contexts. Fre-
quent task-switching is an unavoidable implication of such
multitasking. Because the resources of attention are limited,
we must switch back and forth between tasks and informa-
tion sources, leaving the switched-from tasks temporarily
on hold. Successful multitasking is a complex cognitive
achievement, requiring planning, timing, monitoring, and
control of action. Sometimes we cannot know, without task-
switching, whether the switched-to task is worth switching
to. These temporary shifts of attention to irrelevant or un-
important sources of information (from the user’s point of
view) are here called interruptions.
The costs of interruptions to social and cognitive perform-
ance are somewhat known. In social interaction, interrup-
tions not only delay and distract the fluent course of turn-
taking in human–human conduct, but also can render ac-
tions of people incomprehensible for others . In cogni-
tive psychology, it is known that there is cost of switching
attention between information sources or tasks that is in the
magnitude of seconds. Interruptions also hamper memory
by making memories more susceptible to omissions and
distortions. Interruptions are most harmful for higher-level
thought processes involving heavy load for working mem-
ory, for example when solving novel problems. Looking at
the social and cognitive costs of interruptions, it becomes
understandable why interruptions are associated with all
kinds of negative consequences: delays, errors, mistakes,
frustration etc (see ).
Whereas desktop-based applications could mainly interrupt
only other computer-based tasks, in intelligent environ-
ments the to-be-interrupted tasks are related more to the
psychosocial well-being and life-management of the users.
The tasks carried out at a desktop computer are but a minor
subset of the spectrum of life-management tasks and the
larger hierarchy of human and social needs. A justified and
often heard fear is that interrupting these activities can eas-
ily lead to rejection of the interruption-causing technology.
The remedy is wise design that minimizes the costs and
negative effects of interruptions.
To summarize, the logic is that interruptions are an un-
avoidable feature of interaction in intelligent environments,
and if not carefully designed, they hamper our psychosocial
well-being, which can lead to dismissal of the technology
more easily than in the traditional desktop-based HCI.
Therefore, it is justified to claim that the problem of inter-
ruptions is highlighted in intelligent environments.
REVIEW OF CONTEMPORARY APPROACHES FROM
THE POINT OF VIEW OF COGNITIVE SCIENCE
In the following, contemporary solutions to the problem are
analyzed from the point of view of how they map to differ-
ent aspects of the human cognition.
According to an interpretation of Weiser championed by
Philips (as cited in ), computers at the age of ubiquitous
computing should be invisible. Weiser’s “disappearance” is
here taken literally to mean perceptual invisibility.
Perceptual disappearance, if it worked, would, by defini-
tion, solve the problem of interruptions. Invisibility of a
user interface, however, is in many respects a non-goal and
a paradox in design. At the time of interaction, the user in-
terface must become visible somehow.
Mixed Initiative Interfaces (MIIs) assume that “intelligent
services and users may often collaborate efficiently to
achieve the user’s goals” [1, p. 159]. Instead of immediately
taking the foreground—interrupting the ongoing activity of
the user, a MII progressively signals requests for attention.
Initially this may happen through a channel peripheral to
user’s activity, but can then be achieved in turns with the
user. This is a step towards the kind of deepening and pro-
gressive turntaking in human-human interruption manage-
ment. The main idea is a promising one: the first steps in
interaction are very non-disruptive and will not create a
feeling of being interrupted, and only upon negotiation with
the user will the interaction taken further. A small signal
from the user is enough to terminate the turntaking if the
interrupting task seems irrelevant or unimportant.
In Peripheral Computing, the interface attempts to provide
peripheral awareness of people and events (e.g., [3, 10]).
Ambient channels provide a steady a flow of auditory cues
(such as a sound like a rain) or gradually changing lighting
conditions. According to Hiroshi Ishii, “The smooth transi-
tion of users’ focus of attention between background and
foreground using ambient media and graspable objects is a
key challenge of Tangible Bits” .
In practice, the promise of peripheral interfaces lies in our
capacity to preattentively and unconsciously process pe-
ripheral stimulus sources (i.e., stimuli that are not in the
center of conscious attention). By habituation to irrelevant
ambient stimuli, and sensitization to relevant and important
ambient stimuli, the subconscious cognitive system is capa-
ble of learning what is worth bringing to conscious atten-
tion and what is not. Sudden or abrupt changes in sound-
scapes, for example, typically receive immediate attention
and thus create an interruption. The amount of information
that can be conveyed in such a manner is relatively small,
which limits its generality. Moreover, internalizing the
meanings of ambient signals takes considerable time.
Stephen Intille at MIT has examined how to exploit a cog-
nitive phenomenon called change-blindness in designing
ambient displays embedded to user’s environment. The idea
is to minimize the perceived change by eliminating all at-
tention grabbing cues . If a change occurring on a dis-
play is not perceived, it cannot capture attention and inter-
rupt the user. Blanking an image, changing the view rap-
idly, displaying “mud splashes” to distract noticing
changes, changing information very slowly, using eye
blinks and saccades, and using occlusion are the proposed
A limitation in the approach is that it cannot be used to
convey critical information to the user. That is, it can be
used to decrease the possibility of uninteresting information
grabbing the attention, but not for designing how the inter-
ruption should take place.
The idea in Multimodal Interfaces is to use unreserved mo-
dalities for interaction. This obviously calls for understand-
ing what modalities are typically reserved in a use situation.
For example, in mobility, our visual attention is mostly re-
served for orienting ourselves to others and navigating
through the environment. Nomadic Radio  addressed this
problem by creating a messaging service that instead of
visual modality required only auditory attention and speech
for interaction. This made it possible for the users to not
interrupt the navigation task for doing messaging.
A limitation for the approach is posed by the fact that al-
though our attentive capacity is modular in respect to mo-
dalities, the central executive is a serial processing unit.
This implies that when the automated control of modality
specific subsystems is not possible, as in novel and unprac-
ticed situations, processing the task requires our conscious
attention and thus creates an interruption.
Attention and Task Preferences
Attentive User Interfaces (AUIs) are based on the idea that
modeling the deployment of user attention and task prefer-
ences is the key for minimizing the disruptive effects of
interruptions . By monitoring users' physical proximity,
body orientation, eye fixations, and the like, AUIs can de-
termine what device, person, or task the user is attending to.
Knowing the focus of attention makes it possible in some
situations to avoid interrupting users in tasks that are more
important or time-critical than the one interrupting.
Before taking the foreground, AUIs determine whether the
user is available for interruption, given the priority of the
request, signal the user via a non-intrusive peripheral chan-
nel, and sense user acknowledgment of the request. AUI are
focused on facilitating user’s attention efficiently, but does
not say that interruptions should be minimized. They only
need to be introduced at a right time and in a right way,
depending on the urgency, and determined partly by the
importance of the user’s present task.
Learning and Automatization
Ubiquitous Computing (ubicomp) aims to “activate the
world” with hundreds of wireless computing devices per
person, ranging in size from tiny to wall-sized. According
to its founder, Mark Weiser, ubicomp “takes into account
the human world and allows the computers themselves to
vanish into the background. Such a disappearance is a fun-
damental consequence not of technology but of human psy-
chology. Whenever people learn something sufficiently
well, they cease to be aware of it” [13, p. 66, italics added].
The idea that interaction with technological artefacts be-
comes automatized and thus unconsciously performed skill
is based on a psychological fact. Well-learned routines do
not require conscious control but can be unconsciously car-
ried out, ballistically from the beginning to the end. When
the user learns to use an artefact well enough for a mean-
ingful goal-pursuit, the interruptions it makes become a
natural, or unconscious and thus not disrupting, part of in-
teraction. In selecting this road for design, we need to know
preconditions for a skill becoming automatized. Studies of
automatization offer starting points for this (e.g., ).
Augmenting Everyday Skills
Unremarkable Computing is an approach to the design of
ubicomp suggested by the Xerox Research Centre Europe.
The focus is on designing domestic devices that are “unre-
markable” to users. Here invisibility is understood as the
use of a device being a part of a routine, since “routines are
invisible in use for those who are involved in them” [9, p.
403]. Then, technology is subservient to routines and ac-
tions: “…the key point is that the computation is in service
of actions – everyday actions – which themselves have a
significance” [9, p. 404].
Interruptions caused by a device should be designed to be a
part of a routine. “Things with a routine character may then
have many of the qualities we are aiming for by being tacit
and calm in that they are not ‘dramatic’ and do not com-
mand attention except when they need to. They are seen but
unremarked, used as resources for action” [p.403].
The authors are sympathetic to the Tangible Interfaces ap-
proach (e.g., [3, 10]) that augments traditional artifacts with
functionalities that fit to everyday routines. “Manipulating
physical objects is one of people’s everyday competencies
and more generally available than, say, abstract computer
commands and software applications” [9, p. 404]. Here the
authors, however, fail to notice that the use of abstract com-
puter commands can be automatized as well as any other
everyday skill. Thus, they can be unremarkable as well.
Augmenting routines may not always work as intended.
When a new tool is introduced, its adoption is bound to
affect the routine. If the technology does not introduce a
change, what is its benefit for users? On the other hand,
people are known to be clever at inventing opportunistically
new uses to artifacts, which alter the nature of routines in
unexpected ways. It can be that Unremarkable Computing,
by concentrating on augmenting present-day routines,
misses the potential of IE technologies and actually weak-
ens Weiser’s point of harnessing automatization.
Delegating Decision-Making Responsibility
Proactive Computing was recently introduced by Tennen-
house and colleagues [8, 12]. The enabling technologies
include sensors and actuators intimately connected to the
physical world, processors with faster-than-human operat-
ing speed, and autonomous software programs assembled to
form “knowbots” assigned for helping the user. The key
idea is using simulations of the real world to make infer-
ences and predictions that anticipate and prepare for events.
User’s role in a proactive system is to monitor and steer
processes, without actively intervening in decision-making
situations that may arise. The user is relieved from making
decisions every time when the system encounters a branch-
ing point in its activity. Thus, interruptions that would nor-
mally require decision-making are minimized and the user
is raised above the traditional interaction loop by letting
him/her take a monitoring role.
A somewhat similar approach that also attempts to delegate
decision-making responsibility to intelligent systems is
taken by the Ambient Intelligence (AmI) technology pro-
gramme of the European Union. One part of the AmI vision
entails intelligent agents that assume some of the control
responsibility from users, as in the following example of a
call mediating intelligent agent: “With a nice reproduction
of Dimitrios’ voice and typical accent, a call from his wife
is further analysed by his D-Me. In a first attempt, Dimit-
rios’ ‘avatar-like’ voice runs a brief conversation with his
wife, with the intention of negotiating a delay while ex-
plaining his current environment. [Since Dimitrios had
some pressing tasks to do, and after a while] his wife’s call
is now interpreted by his D-Me as sufficiently pressing to
mobilize Dimitrios. It ‘rings’ him using a pre-arranged call
tone” [4, p. 5]. Using human-like agents like D-Me may
prove fruitful for the delegation approach, because in the IE
use contexts we are accustomed to collaborate with humans
in pursuing goals.
NEW DIRECTIONS FOR RESEARCH
The cognitive scientific framework that was used for ana-
lyzing the approaches above, also inspires novel approaches
that have not been yet explored.
One such approach is that of memory. Whereas perception,
attention, and decision-making have been addressed in the
existing approaches, memory is not. The idea in memory-
based approach would be to design interruptions that impair
our ability to remember the interrupted task as little as pos-
sible. For example, presenting interruptions during low
working memory load would be one step towards this goal.
Another one would be providing retrieval cues adaptively in
the UI to help the user to mentally restore the cognitive
state to resume the interrupted task. The memory-based
approach build on the AUI approach, and would require
extensive tracking of user’s perception, attention, and inter-
action history to track the contents of user’s memory and
the development of memory skills.
Another approach inspired by the framework relates to
stress and inference. A possibility for designing less disrup-
tive interruptions is to make them more predictable. It is
known from the cognitive studies of stress that events that
are both unpredictable and uncontrollable cause stress.
Thus, instead of tracking users and predicting their interrup-
tability, the system could try to predict and visualize to the
user when it is going to interrupt him the next time. This is
largely a problem for UI design and the psychology of in-
ference, as the user employs his/her mental models to draw
inferences from the cues available at the user interface.
A third, and probably the most promising approach inspired
by the framework relates to human needs and preferences.
As common sense reveals us, some tasks are more impor-
tant than others, and just those tasks are the ones that de-
serve our attention and are thus not considered as interrupt-
ing or disrupting. Getting a call from a dear friend is usually
delightful, were we in a meeting or not. Thus, interrupting
user can be, and should be, beneficial, and one can ask if
the quest for minimizing interruptions is a solution without
a problem. Innovating more meaningful design concepts for
the technology of future would solve part of the problem.
HCI research has been criticized for being atheoretical.
This is definitely true of research in intelligent environ-
ments. The only way to systematize and bring consensus to
this atheoretical and balkanized field is by constructing
concepts and theories. As shown in this paper, interruptions
is such a concept. It makes visible similarities and differ-
ences among research approaches, and helps future work by
revealing possibly important omissions. The cognitive sci-
entific approach to interaction and interruptions is, of
course, but one conceptualization of only one key problem
in intelligent environments. Future research must search for
similar emerging frameworks elsewhere and attempt to ex-
plicate and evaluate them.
This work has been funded by the Academy of Finland.
1. Horvitz, E. Principles of mixed-initiative user interfaces.
Proc. CHI 1999, ACM Press (1999), 156-166.
2. Intille, S. S. Change blind information display for ubiq-
uitous environments. Proc. UbiComp 2002, Lecture
Notes in Computer Science 2498 (2002), 91-106.
3. Ishii, H. and Ullmer, B. Tangible bits: towards seamless
interfaces between people, bits and atoms. Proc. CHI
1997, ACM Press (1997), 234-241.
4. Scenarios for Ambient Intelligence in 2010. Information
Society Technology Advisory Group (ISTAG), Euro-
pean Union, Sixth Framework Programme (FP6).
5. Logan, G. D. Toward an instance theory of automatisa-
tion. Psychological Review 95, 4. 492-527.
6. McFarlane D. C., and Latorella K. A. The scope and
importance of human interruption in human-computer
interaction design, Human-Computer Interaction 17, 1
7. Sawhney, N., and Schmandt, C. Nomadic radio:speech
& audio interaction for contextual messaging in no-
madic environments. ACM TOCHI 7, 3 (2000), 353-383.
8. Tennenhouse, D. Proactive computing. Communications
of the ACM 43, 5 (2000), 43–50.
9. Tolmie, P., Pycock, J., Diggins, T., MacLean, A., and
Karsenty, A. Unremarkable Computing. Proc. CHI
2002, CHI Letters 1(1), 399-406.
10. Ullmer, B., and Ishii, H. Emerging frameworks for tan-
gible user interfaces. IBM Systems Journal 39, 3-4
11. Vertegaal, R. Attentive user interfaces. Communications
of the ACM 46, 3 (2003), 31-33. Special issue on atten-
tive user interfaces.
12. Want, R., Pering, T., and Tennenhouse, D. Comparing
autonomic and proactive computing. IBM Systems Jour-
nal 42, 1 (2003), 129–135.
13. Weiser, M. The computer for the 21st century. Scientific
American 265, 3 (1991), 66–73.