Conference PaperPDF Available

A cognitive meta-analysis of design approaches to interruptions in intelligent environments

Authors:

Abstract

Minimizing interruptions to users is a crucial and acknowledged precondition for the adoption of new intelligent technologies such as ubiquitous and proactive computing. This paper takes a step toward achieving a consensus among the numerous existing approaches addressing the challenge posed by interruptions. We start by explicating why interruptions are considered important. We then reveal similarities and differences among the approaches from a cognitive viewpoint. It appears that the approaches draw from different assumptions about human cognition. Some of the approaches contain inconsistencies. The cognitive analysis also inspires directions for future work.
1
A Cognitive Meta-Analysis of Design Approaches to
Interruptions in Intelligent Environments
Antti Oulasvirta and Antti Salovaara
Helsinki Institute for Information Technology
PO Box 9800, 02015 HUT, Finland
{oulasvir,asalovaa}@hiit.fi
ABSTRACT
Minimizing interruptions to users is a crucial and acknowl-
edged precondition for the adoption of new intelligent tech-
nologies such as ubiquitous and proactive computing. This
paper takes a step toward achieving a consensus among the
numerous existing approaches addressing the challenge
posed by interruptions. We start by explicating why inter-
ruptions are considered important. We then reveal similari-
ties and differences among the approaches from a cognitive
viewpoint. It appears that the approaches draw from differ-
ent assumptions about human cognition. Some of the ap-
proaches contain inconsistencies. The cognitive analysis
also inspires directions for future work.
Author Keywords
Interruptions, intelligent environments, user interfaces.
ACM Classification Keywords
H5.2. User Interfaces: Theory and methods
INTRODUCTION
After over 10 years of research in intelligent environments
(IE) [13], the field now seems to be in a state of conceptual
balkanization. Currently, there are at least 15 named design
approaches. Consider, for example, proactive, ubiquitous,
pervasive, mobile, situated, wearable, ensemble, invisible,
context-aware, peripheral, and calm computing, ambient
intelligence, disappearing computer, attentive and intelli-
gent user interfaces, and personal technologies, each having
their proponents. Consequently, it is difficult for us re-
searchers to get an overall grasp of the field.
In this paper, we argue that designing disruption-free inter-
action is a central design problem for IE and “technology
beyond the desktop” in general [13]. The problem is shared
by many of the approaches but also allows for distinguish-
ing between them. In this paper, we first explicate the prob-
lem of interruptions and then investigate and evaluate, from
the point of view of cognitive psychology, how some of the
most prominent approaches have addressed the problem.
What We Mean by Intelligent Environments?
By intelligent environments we mean technological aug-
mentation of user's physical surroundings with systems or
devices that are able to respond to user activity. This tech-
nology aims to provide services and control over processes,
and support decision-making and other cognitive needs.
Responsiveness and adaptation are based either on pre-
programmed heuristics or real-time reasoning capabilities.
All the approaches mentioned in the introductory paragraph
fit at least partly into this characterization. For the purposes
of this paper, intelligent environment serves as a general
umbrella term that covers most of the approaches.
WHY INTERRUPTIONS IS AN IMPORTANT DESIGN
ISSUE FOR INTELLIGENT ENVIRONMENTS?
IE will be in homes, lecture halls, gardens, schools, city
streets, cards, buses, trams, shops, malls etc. In other words,
elsewhere than at the desktop. As these use contexts inher-
ently involve many sequentially and simultaneously per-
formed tasks, they can be called multitasking contexts. Fre-
quent task-switching is an unavoidable implication of such
multitasking. Because the resources of attention are limited,
we must switch back and forth between tasks and informa-
tion sources, leaving the switched-from tasks temporarily
on hold. Successful multitasking is a complex cognitive
achievement, requiring planning, timing, monitoring, and
control of action. Sometimes we cannot know, without task-
switching, whether the switched-to task is worth switching
to. These temporary shifts of attention to irrelevant or un-
important sources of information (from the user’s point of
view) are here called interruptions.
The costs of interruptions to social and cognitive perform-
ance are somewhat known. In social interaction, interrup-
tions not only delay and distract the fluent course of turn-
taking in humanhuman conduct, but also can render ac-
tions of people incomprehensible for others [11]. In cogni-
tive psychology, it is known that there is cost of switching
attention between information sources or tasks that is in the
magnitude of seconds. Interruptions also hamper memory
by making memories more susceptible to omissions and
distortions. Interruptions are most harmful for higher-level
2
thought processes involving heavy load for working mem-
ory, for example when solving novel problems. Looking at
the social and cognitive costs of interruptions, it becomes
understandable why interruptions are associated with all
kinds of negative consequences: delays, errors, mistakes,
frustration etc (see [6]).
Whereas desktop-based applications could mainly interrupt
only other computer-based tasks, in intelligent environ-
ments the to-be-interrupted tasks are related more to the
psychosocial well-being and life-management of the users.
The tasks carried out at a desktop computer are but a minor
subset of the spectrum of life-management tasks and the
larger hierarchy of human and social needs. A justified and
often heard fear is that interrupting these activities can eas-
ily lead to rejection of the interruption-causing technology.
The remedy is wise design that minimizes the costs and
negative effects of interruptions.
To summarize, the logic is that interruptions are an un-
avoidable feature of interaction in intelligent environments,
and if not carefully designed, they hamper our psychosocial
well-being, which can lead to dismissal of the technology
more easily than in the traditional desktop-based HCI.
Therefore, it is justified to claim that the problem of inter-
ruptions is highlighted in intelligent environments.
REVIEW OF CONTEMPORARY APPROACHES FROM
THE POINT OF VIEW OF COGNITIVE SCIENCE
In the following, contemporary solutions to the problem are
analyzed from the point of view of how they map to differ-
ent aspects of the human cognition.
Invisibility
According to an interpretation of Weiser championed by
Philips (as cited in [9]), computers at the age of ubiquitous
computing should be invisible. Weiser’s “disappearance” is
here taken literally to mean perceptual invisibility.
Perceptual disappearance, if it worked, would, by defini-
tion, solve the problem of interruptions. Invisibility of a
user interface, however, is in many respects a non-goal and
a paradox in design. At the time of interaction, the user in-
terface must become visible somehow.
Progressive Negotiation
Mixed Initiative Interfaces (MIIs) assume that “intelligent
services and users may often collaborate efficiently to
achieve the user’s goals” [1, p. 159]. Instead of immediately
taking the foregroundinterrupting the ongoing activity of
the user, a MII progressively signals requests for attention.
Initially this may happen through a channel peripheral to
user’s activity, but can then be achieved in turns with the
user. This is a step towards the kind of deepening and pro-
gressive turntaking in human-human interruption manage-
ment. The main idea is a promising one: the first steps in
interaction are very non-disruptive and will not create a
feeling of being interrupted, and only upon negotiation with
the user will the interaction taken further. A small signal
from the user is enough to terminate the turntaking if the
interrupting task seems irrelevant or unimportant.
Preattentive Processing
In Peripheral Computing, the interface attempts to provide
peripheral awareness of people and events (e.g., [3, 10]).
Ambient channels provide a steady a flow of auditory cues
(such as a sound like a rain) or gradually changing lighting
conditions. According to Hiroshi Ishii, “The smooth transi-
tion of users’ focus of attention between background and
foreground using ambient media and graspable objects is a
key challenge of Tangible Bits” [3].
In practice, the promise of peripheral interfaces lies in our
capacity to preattentively and unconsciously process pe-
ripheral stimulus sources (i.e., stimuli that are not in the
center of conscious attention). By habituation to irrelevant
ambient stimuli, and sensitization to relevant and important
ambient stimuli, the subconscious cognitive system is capa-
ble of learning what is worth bringing to conscious atten-
tion and what is not. Sudden or abrupt changes in sound-
scapes, for example, typically receive immediate attention
and thus create an interruption. The amount of information
that can be conveyed in such a manner is relatively small,
which limits its generality. Moreover, internalizing the
meanings of ambient signals takes considerable time.
Change Blindness
Stephen Intille at MIT has examined how to exploit a cog-
nitive phenomenon called change-blindness in designing
ambient displays embedded to user’s environment. The idea
is to minimize the perceived change by eliminating all at-
tention grabbing cues [2]. If a change occurring on a dis-
play is not perceived, it cannot capture attention and inter-
rupt the user. Blanking an image, changing the view rap-
idly, displaying “mud splashes” to distract noticing
changes, changing information very slowly, using eye
blinks and saccades, and using occlusion are the proposed
methods.
A limitation in the approach is that it cannot be used to
convey critical information to the user. That is, it can be
used to decrease the possibility of uninteresting information
grabbing the attention, but not for designing how the inter-
ruption should take place.
Unreserved Modalities
The idea in Multimodal Interfaces is to use unreserved mo-
dalities for interaction. This obviously calls for understand-
ing what modalities are typically reserved in a use situation.
For example, in mobility, our visual attention is mostly re-
served for orienting ourselves to others and navigating
through the environment. Nomadic Radio [7] addressed this
problem by creating a messaging service that instead of
visual modality required only auditory attention and speech
for interaction. This made it possible for the users to not
interrupt the navigation task for doing messaging.
A limitation for the approach is posed by the fact that al-
3
though our attentive capacity is modular in respect to mo-
dalities, the central executive is a serial processing unit.
This implies that when the automated control of modality
specific subsystems is not possible, as in novel and unprac-
ticed situations, processing the task requires our conscious
attention and thus creates an interruption.
Attention and Task Preferences
Attentive User Interfaces (AUIs) are based on the idea that
modeling the deployment of user attention and task prefer-
ences is the key for minimizing the disruptive effects of
interruptions [11]. By monitoring users' physical proximity,
body orientation, eye fixations, and the like, AUIs can de-
termine what device, person, or task the user is attending to.
Knowing the focus of attention makes it possible in some
situations to avoid interrupting users in tasks that are more
important or time-critical than the one interrupting.
Before taking the foreground, AUIs determine whether the
user is available for interruption, given the priority of the
request, signal the user via a non-intrusive peripheral chan-
nel, and sense user acknowledgment of the request. AUI are
focused on facilitating user’s attention efficiently, but does
not say that interruptions should be minimized. They only
need to be introduced at a right time and in a right way,
depending on the urgency, and determined partly by the
importance of the user’s present task.
Learning and Automatization
Ubiquitous Computing (ubicomp) aims to “activate the
world” with hundreds of wireless computing devices per
person, ranging in size from tiny to wall-sized. According
to its founder, Mark Weiser, ubicomp “takes into account
the human world and allows the computers themselves to
vanish into the background. Such a disappearance is a fun-
damental consequence not of technology but of human psy-
chology. Whenever people learn something sufficiently
well, they cease to be aware of it” [13, p. 66, italics added].
The idea that interaction with technological artefacts be-
comes automatized and thus unconsciously performed skill
is based on a psychological fact. Well-learned routines do
not require conscious control but can be unconsciously car-
ried out, ballistically from the beginning to the end. When
the user learns to use an artefact well enough for a mean-
ingful goal-pursuit, the interruptions it makes become a
natural, or unconscious and thus not disrupting, part of in-
teraction. In selecting this road for design, we need to know
preconditions for a skill becoming automatized. Studies of
automatization offer starting points for this (e.g., [5]).
Augmenting Everyday Skills
Unremarkable Computing is an approach to the design of
ubicomp suggested by the Xerox Research Centre Europe.
The focus is on designing domestic devices that are “unre-
markable” to users. Here invisibility is understood as the
use of a device being a part of a routine, since “routines are
invisible in use for those who are involved in them” [9, p.
403]. Then, technology is subservient to routines and ac-
tions: “…the key point is that the computation is in service
of actions everyday actions which themselves have a
significance” [9, p. 404].
Interruptions caused by a device should be designed to be a
part of a routine. “Things with a routine character may then
have many of the qualities we are aiming for by being tacit
and calm in that they are not ‘dramatic’ and do not com-
mand attention except when they need to. They are seen but
unremarked, used as resources for action” [p.403].
The authors are sympathetic to the Tangible Interfaces ap-
proach (e.g., [3, 10]) that augments traditional artifacts with
functionalities that fit to everyday routines. “Manipulating
physical objects is one of people’s everyday competencies
and more generally available than, say, abstract computer
commands and software applications” [9, p. 404]. Here the
authors, however, fail to notice that the use of abstract com-
puter commands can be automatized as well as any other
everyday skill. Thus, they can be unremarkable as well.
Augmenting routines may not always work as intended.
When a new tool is introduced, its adoption is bound to
affect the routine. If the technology does not introduce a
change, what is its benefit for users? On the other hand,
people are known to be clever at inventing opportunistically
new uses to artifacts, which alter the nature of routines in
unexpected ways. It can be that Unremarkable Computing,
by concentrating on augmenting present-day routines,
misses the potential of IE technologies and actually weak-
ens Weiser’s point of harnessing automatization.
Delegating Decision-Making Responsibility
Proactive Computing was recently introduced by Tennen-
house and colleagues [8, 12]. The enabling technologies
include sensors and actuators intimately connected to the
physical world, processors with faster-than-human operat-
ing speed, and autonomous software programs assembled to
form “knowbots” assigned for helping the user. The key
idea is using simulations of the real world to make infer-
ences and predictions that anticipate and prepare for events.
User’s role in a proactive system is to monitor and steer
processes, without actively intervening in decision-making
situations that may arise. The user is relieved from making
decisions every time when the system encounters a branch-
ing point in its activity. Thus, interruptions that would nor-
mally require decision-making are minimized and the user
is raised above the traditional interaction loop by letting
him/her take a monitoring role.
A somewhat similar approach that also attempts to delegate
decision-making responsibility to intelligent systems is
taken by the Ambient Intelligence (AmI) technology pro-
gramme of the European Union. One part of the AmI vision
entails intelligent agents that assume some of the control
responsibility from users, as in the following example of a
call mediating intelligent agent: “With a nice reproduction
of Dimitrios’ voice and typical accent, a call from his wife
4
is further analysed by his D-Me. In a first attempt, Dimit-
rios’ ‘avatar-like’ voice runs a brief conversation with his
wife, with the intention of negotiating a delay while ex-
plaining his current environment. [Since Dimitrios had
some pressing tasks to do, and after a while] his wife’s call
is now interpreted by his D-Me as sufficiently pressing to
mobilize Dimitrios. It ‘rings’ him using a pre-arranged call
tone” [4, p. 5]. Using human-like agents like D-Me may
prove fruitful for the delegation approach, because in the IE
use contexts we are accustomed to collaborate with humans
in pursuing goals.
NEW DIRECTIONS FOR RESEARCH
The cognitive scientific framework that was used for ana-
lyzing the approaches above, also inspires novel approaches
that have not been yet explored.
One such approach is that of memory. Whereas perception,
attention, and decision-making have been addressed in the
existing approaches, memory is not. The idea in memory-
based approach would be to design interruptions that impair
our ability to remember the interrupted task as little as pos-
sible. For example, presenting interruptions during low
working memory load would be one step towards this goal.
Another one would be providing retrieval cues adaptively in
the UI to help the user to mentally restore the cognitive
state to resume the interrupted task. The memory-based
approach build on the AUI approach, and would require
extensive tracking of user’s perception, attention, and inter-
action history to track the contents of user’s memory and
the development of memory skills.
Another approach inspired by the framework relates to
stress and inference. A possibility for designing less disrup-
tive interruptions is to make them more predictable. It is
known from the cognitive studies of stress that events that
are both unpredictable and uncontrollable cause stress.
Thus, instead of tracking users and predicting their interrup-
tability, the system could try to predict and visualize to the
user when it is going to interrupt him the next time. This is
largely a problem for UI design and the psychology of in-
ference, as the user employs his/her mental models to draw
inferences from the cues available at the user interface.
A third, and probably the most promising approach inspired
by the framework relates to human needs and preferences.
As common sense reveals us, some tasks are more impor-
tant than others, and just those tasks are the ones that de-
serve our attention and are thus not considered as interrupt-
ing or disrupting. Getting a call from a dear friend is usually
delightful, were we in a meeting or not. Thus, interrupting
user can be, and should be, beneficial, and one can ask if
the quest for minimizing interruptions is a solution without
a problem. Innovating more meaningful design concepts for
the technology of future would solve part of the problem.
CONCLUSIONS
HCI research has been criticized for being atheoretical.
This is definitely true of research in intelligent environ-
ments. The only way to systematize and bring consensus to
this atheoretical and balkanized field is by constructing
concepts and theories. As shown in this paper, interruptions
is such a concept. It makes visible similarities and differ-
ences among research approaches, and helps future work by
revealing possibly important omissions. The cognitive sci-
entific approach to interaction and interruptions is, of
course, but one conceptualization of only one key problem
in intelligent environments. Future research must search for
similar emerging frameworks elsewhere and attempt to ex-
plicate and evaluate them.
ACKNOWLEDGMENTS
This work has been funded by the Academy of Finland.
REFERENCES
1. Horvitz, E. Principles of mixed-initiative user interfaces.
Proc. CHI 1999, ACM Press (1999), 156-166.
2. Intille, S. S. Change blind information display for ubiq-
uitous environments. Proc. UbiComp 2002, Lecture
Notes in Computer Science 2498 (2002), 91-106.
3. Ishii, H. and Ullmer, B. Tangible bits: towards seamless
interfaces between people, bits and atoms. Proc. CHI
1997, ACM Press (1997), 234-241.
4. Scenarios for Ambient Intelligence in 2010. Information
Society Technology Advisory Group (ISTAG), Euro-
pean Union, Sixth Framework Programme (FP6).
5. Logan, G. D. Toward an instance theory of automatisa-
tion. Psychological Review 95, 4. 492-527.
6. McFarlane D. C., and Latorella K. A. The scope and
importance of human interruption in human-computer
interaction design, Human-Computer Interaction 17, 1
(2002), 1-61.
7. Sawhney, N., and Schmandt, C. Nomadic radio:speech
& audio interaction for contextual messaging in no-
madic environments. ACM TOCHI 7, 3 (2000), 353-383.
8. Tennenhouse, D. Proactive computing. Communications
of the ACM 43, 5 (2000), 4350.
9. Tolmie, P., Pycock, J., Diggins, T., MacLean, A., and
Karsenty, A. Unremarkable Computing. Proc. CHI
2002, CHI Letters 1(1), 399-406.
10. Ullmer, B., and Ishii, H. Emerging frameworks for tan-
gible user interfaces. IBM Systems Journal 39, 3-4
(2000), 915-931.
11. Vertegaal, R. Attentive user interfaces. Communications
of the ACM 46, 3 (2003), 31-33. Special issue on atten-
tive user interfaces.
12. Want, R., Pering, T., and Tennenhouse, D. Comparing
autonomic and proactive computing. IBM Systems Jour-
nal 42, 1 (2003), 129–135.
13. Weiser, M. The computer for the 21st century. Scientific
American 265, 3 (1991), 6673.
... The user's emotional and internal state, here referred as mood, has been considered an influential aspect in a user's interruptibility and availability in several previous works [11,19,26,28,29]. For Dabbish and Baker [11] the relation is less direct and made through a connection of their observed variable of "interruption threshold" defined as varying in accordance to context and external cues. ...
... literature of interruptibility [19,26,29], is defined as to "encompass the interruptee's cognitive level of involvement in tasks and how it affects task performances" [16]. However, the definition of social context as "the interruptee' physical environment as understood in a social sense" -relating to the place the user is in, the people around him/her and the nature of the task [16] -is closer to "social expectation". ...
... From the perspective of usertechnology interaction, the Technology Acceptance Model (TAM) [12] has added the factor of subjective norm since it became apparent that the acceptance of technology seems to be influenced by others [32]. The influence of the broad social context includes culture and social norms [5,7,34], the presence and opinions of others [5] and social interactions [26]. This has been studied in regard to the effect on user behavior, decisions and expectations towards privacy, including how much control over data sharing may be desired [23], as well as its effects on the user's interruptibility [9,19,25,28] and the decision to interrupt somebody else [17]. ...
Conference Paper
This paper presents an organized set of variables that can aid intelligent privacy agents in predicting the best and necessary moments to interrupt users in order to give them control and awareness over their privacy, avoiding information overload or over choice.
... For example, Agarwal and Karahanna (2000) developed a construct called cognitive absorption which includes focused immersion, which is "the experience of total engagement where other attentional demands are, in essence, ignored." Thus, as digital technologies have increasingly mediated and enabled our work, focus has become a vital construct in studies of human computer interaction (O'Conaill and Frohlich, 1995;McFarlane and Latorella, 2002;González and Mark, 2004;Oulasvirta and Salovaara, 2004;Mark et al., 2005Mark et al., , 2008. ...
... The effects of interruptions on human cognition are fairly well-documented (e.g., O'Conaill and Frohlich, 1995;McFarlane and Latorella, 2002;Rennecker and Godwin, 2005). Although, some research has suggested that interruptions may have some positive effects (Gillie and Broadbent, 1989;O'Conaill and Frohlich, 1995;Jett and George, 2003), by and large, findings suggests interruptions lead to delays, errors, mistakes, and frustration (Oulasvirta and Salovaara, 2004;Mark et al., 2008). These negative effects are especially pronounced when the task being interrupted is one requiring heavier cognitive resources -such as when performing unstructured or novel problem solving (McFarlane and Latorella, 2002). ...
Article
Full-text available
Until the turn of the millennium, switching tasks required moving locations or reconfiguring physical workspaces. However, contemporary digital tools and workspaces allow knowledge workers to perform an increasingly diverse set of tasks, with an increasingly extending arm of influence, all from the same physical location without any physical reconfigurations or traversing of physical space. Along with this increased ambidexterity comes an increase in the quantity and frequency of demands on the time of knowledge workers. This digitalization of work now tethers their ability to perform largely to their ability to intensely focus in small chunks, and then “hyper-jump” that focus to another task without traversing the cognitive cool downs or warm ups required to reconfigure their train of thought from one task to another. Accordingly, they must become more like the hyper-functioning tools they employ if they are to avoid becoming the bottleneck resource in the configuration of socio-technical elements comprising their work routines. In order to better understand how knowledge workers manage their time and maintain focus across multiple and interrupting tasks, we leverage current time prioritization literature to propose and theorize around two key constructs: time-chunking and hyper-refocusing. By combining the possible values for these two constructs, we hypothesize the emergence of six forms of knowledge workers and their relative expected performance. The effects of digitalization on these new worker forms are discussed. We conclude by discussing opportunities for new research questions regarding time-chunking strategies and the hyper-refocusing ability.
... The aim of the researchers is to develop intelligent technologies that control electronic interruptions based on topic relevancy, priority or user focus at the moment of the interruption. These technologies are monitoring interruptibility using different kinds of sensors [25]- [28]. ...
Conference Paper
Full-text available
On the one hand, new technologies and communication tools improve employee productivity and accelerate information and knowledge transfer, while on the other hand, information overload and continuous interruptions make it even harder to concentrate at work. It is a great challenge for companies to find the right balance, while there is also an ongoing demand to recruit and retain the talented employees who are able to adopt the modern work style and effectively use modern communication tools. For this reason, this research does not focus on the objective measures of office interruptions, but aims to find those disruption factors which influence the comfort and job satisfaction of employees, and the way how they feel generally at work. The focus of this research is on how employees feel about the different types of interruptions, which are those they themselves identify as hindering factors, and those they feel as stress factors. By identifying and then reducing these destructive factors, job satisfaction can reach a higher level and employee turnover can be reduced. During the research, we collected information from depth interviews and questionnaires asking about work environment, communication channels used in the workplace, individual communication preferences, factors considered as disruptions, and individual steps taken to avoid interruptions. The questionnaire was completed by 141 office workers from several types of workplaces based in Hungary. Even though 66 respondents are working at Hungarian offices of multinational companies, the research is about the characteristics of the Hungarian labor force. The most important result of the research shows that while more than one third of the respondents consider office noise as a disturbing factor, personal inquiries are welcome and considered useful, even if in such cases the work environment will not be convenient to solve tasks requiring concentration. Analyzing the sizes of the offices, in an open-space environment, the rate of those who consider office noise as a disturbing factor is surprisingly lower than in smaller office rooms. Opinions are more diverse regarding information communication technologies. In addition to the interruption factors affecting the employees' job satisfaction, the research also focuses on the role of the offices in the 21 st century.
... Interfețele bazate pe senzori de urmărire observă acțiunile (verbale și/sau non- verbale) utilizatorului, care sunt considerate intrări [Turk, 2000]. Scopul acestor interfețe [Oulasvirta, 2004] este de a culege informațiile de la utilizator și de a le procesa cât mai rapid pentru a răspunde utilizatorului într-un timp scurt. ...
... It is important to understand and master human internal affective experience, because it can improve the rationality and collaboration of HCI and achieve a vivid and seamless interaction [3]. agent's proactive behavior or action to participants according to affective computing [7]. In the beginning of this century, for example, European Union countries proposed a plan of health care services using the technology of proactive computing and affective computing [8,9]. ...
Article
Affective computing is important in human–computer interaction. Especially in interactive cloud computing within big data, affective modeling and analysis have extremely high complexity and uncertainty for emotional status as well as decreased computational accuracy. In this paper, an approach for affective experience evaluation in an interactive environment is presented to help enhance the significance of those findings. Based on a person-independent approach and the cooperative interaction as core factors, facial expression features and states as affective indicators are applied to do synergetic dependence evaluation and to construct a participant’s affective experience distribution map in interactive Big Data space. The resultant model from this methodology is potentially capable of analyzing the consistency between a participant’s inner emotional status and external facial expressions regardless of hidden emotions within interactive computing. Experiments are conducted to evaluate the rationality of the affective experience modeling approach outlined in this paper. The satisfactory results on real-time camera demonstrate an availability and validity comparable to the best results achieved through the facial expressions only from reality big data. It is suggested that the person-independent model with cooperative interaction and synergetic dependence evaluation has the characteristics to construct a participant’s affective experience distribution, and can accurately perform real-time analysis of affective experience consistency according to interactive big data. The affective experience distribution is considered as the most individual intelligent method for both an analysis model and affective computing, based on which we can further comprehend affective facial expression recognition and synthesis in interactive cloud computing.
Article
Undoubtedly, sustainable development has inspired a generation of scholars and practitioners in different disciplines into a quest for the immense opportunities created by the development of sustainable urban forms for human settlements that will enable built environments to function in a more constructive and efficient way. However, there are still significant challenges that need to be addressed and overcome. The issue of such forms has been problematic and difficult to deal with, particularly in relation to the evaluation and improvement of their contribution to the goals of sustainable development. As it is an urban world where the informational and physical landscapes are increasingly being merged, sustainable urban forms need to embrace and leverage what current and future ICT has to offer as innovative solutions and sophisticated methods so as to thrive—i.e. advance their contribution to sustainability. The need for ICT of the new wave of computing to be embedded in such forms is underpinned by the recognition that urban sustainability applications are deemed of high relevance to the contemporary research agenda of computing and ICT. To unlock and exploit the underlying potential, the field of sustainable urban planning is required to extend its boundaries and broaden its horizons beyond the ambit of the built form of cities to include technological innovation opportunities. This paper explores and substantiates the real potential of ICT of the new wave of computing to evaluate and improve the contribution of sustainable urban forms to the goals of sustainable development. This entails merging big data and context–aware technologies and their applications with the typologies and design concepts of sustainable urban forms to achieve multiple hitherto unrealized goals. In doing so, this paper identifies models of smart sustainable city and their technologies and applications and models of sustainable urban form and their design concepts and typologies. In addition, it addresses the question of how these technologies and applications can be amalgamated with these design concepts and typologies in ways that ultimately evaluate and improve the contribution of sustainable urban forms to the goals of sustainable development. The overall aim of this paper suits a mix of three methodologies: literature review, thematic analysis, and secondary (qualitative) data analysis to achieve different but related objectives. The study identifies four technologies and two classes of applications pertaining to models of smart sustainable city as well as three design concepts and four typologies related to models of sustainable urban form. Finally, this paper proposes a Matrix to help scholars and planners in understanding and analyzing how the contribution of sustainable urban forms to sustainability can be improved through ICT of the new wave of computing and its novel technologies and applications, as well as a data–centric approach into evaluating this contribution and a simulation method for strategically optimizing it.
Chapter
The aim of this chapter is to give insights into the origin and context of the AmI vision; to shed light on the customary assumptions behind the dominant vision of AmI, underlying many of its envisioned scenarios, and provide an account on its current status; to outline and describe a generic typology for AmI; to provide an overview on technological factors behind AmI and the many, diverse research topics and areas associated with AmI; to introduce and describe human-directed sciences as well as artificial intelligence and their relationships and contributions to AmI; and to discuss key paradigmatic, non-paradigmatic, pre-paradigmatic, and post-paradigmatic dimensions of AmI. Moreover, this chapter intends to provide essential underpinning conceptual tools for exploring the subject of AmI further in the remaining chapters.
Article
With the theme Ambient Intelligence (AmI) industry, designers and scientists explore a vision of future daily life-a vision of humans being accompanied and surrounded by computerised devices, intelligent interfaces, wireless networking technology and software agents. Computing resources and computing services will be present everywhere and interconnected anytime. The focus of AmI is to bring to life the everyday objects and tools of our daily environment. The purposes of this technology are circumambient ways of monitoring the actions of humans and the changes in their environment. Sensors of many types and physical actors will be used to react and pre-act in a way that is articulated as desirable and pleasant. AmI as a 'crossover approach' is strongly related to several other Computer Science topics (Punie 2003: 6; Schmidt 2004; Oulasvirta 2004). This technology is not new. A lot of ambient technology is already available, like monitoring analogue physical processes, describing them with digital data and analysing these data using knowledge-based interpretation models. New is that the public and the private environment of humans is permeated by an overwhelming number of autonomous active devices. This will cause the inevitability of the employment of artificial intelligent agents to automate routine decisions and to provide against stupefying read and write collisions15 of the artificial devices. There is no guarantee that these artificial agents can cooperate appropriately and safely. This penetration process has already started with remote recognition systems for facial expression and body tracking (Turk 2004). © 2007 VS Verlag für Sozialwissenschaften | GWV Fachverlage GmbH, Wiesbaden.
Article
This paper aims to define users ’ information expectations as web technologies continue to improve in loading time and uninterrupted interface interactivity. Do web technologies like Ajax – or, more abstractly, a quicker fulfilling of user needs – change these needs, or do they merely fulfill preexisting expectations? Users navigated through a mock e-commerce site where each page that loads has a 50% chance of implementing Ajax technology, from functions of the shopping cart to expanding categories of products. Users were observed through eye tracking and measuring their pulse and respiratory effort. Questionnaires were administered before and after these tasks to assess their thoughts about the study. Qualitative and quantitative observation found users almost unanimously favored the Ajax functions over the non-Ajax. Users emphasized the usability concerns of switching to Ajax, especially concerning feedback. Headings:
Conference Paper
Ajax, as one of the technological pillars of Web 2.0, has revolutionized the way that users access content and interact with each other on the Web. Unfortunately, many developers appear to be inspired by what is technologically possible through Ajax disregarding good design practice and fundamental usability theories. The key usability challenges of Ajax have been noted in the research literature with some technical solutions and design advice available on developer forums. What is unclear is how commercial Ajax developers respond to these issues. This paper presents the results of an empirical study of four commercial web sites that utilize Ajax technologies. The study investigated two usability issues in Ajax with the results contrasted in relation to the general usability principles of consistency, learnability and feedback. The results of the study found inconsistencies in how the sites managed the usability issues and demonstrated that combinations of the issues have a detrimental effect on user performance and satisfaction. The findings also suggest that developers may not be consistently responding to the available advice and guidelines. The paper concludes with several recommendations for Ajax developers to improve the usability of their Web applications.
Article
Full-text available
At first glance it seems absurd that busy people doing important jobs should want their computers to interrupt them. Interruptions are disruptive and people need to concentrate to make good decisions. However, successful job perfor- mance also frequently depends on people's abilities to (a) constantly monitor their dynamically changing information environments, (b) collaborate and communi- cate with other people in the system, and (c) supervise background autonomous services. These critical abilities can require people to simultaneously query a large set of information sources, continuously monitor for important events, and re- spond to and communicate with other human operators. Automated monitoring
Article
Full-text available
We present steps toward a conceptual framework for tangible user interfaces. We introduce the MCRpd interaction model for tangible interfaces, which relates the role of physical and digital representations, physical control, and underlying digital models. This model serves as a foundation for identifying and discussing several key characteristics of tangible user interfaces. We identify a number of systems exhibiting these characteristics, and situate these within 12 application domains. Finally, we discuss tangible interfaces in the context of related research themes, both within and outside of the human-computer interaction domain.
Article
Full-text available
This paper discusses the interaction techniques developed for Nomadic Radio, a wearable computing platform for managing voice and text-based messages in a nomadic environment. Nomadic Radio employs an auditory user interface, which synchronizes speech recognition, speech synthesis, non-speech audio and spatial presentation of digital audio, for navigating among messages as well as asynchronous alerting of newly arrived messages. Emphasis is placed on an auditory modality as Nomadic Radio is designed to be used while performing other tasks in a user's everyday environment; a range of auditory cues provide peripheral awareness of incoming messages. Notification is adaptive and context sensitive; messages are presented as more or less obtrusive based on importance inferred from content filtering, whether the user is engaged in conversation and her recent responses to prior messages. Auditory notifications are dynamically scaled from ambient sound through recorded voice cues up to message
Article
Full-text available
Although the trend to use more computing devices may provide an opportunity for increased productivity, such benefit comes at a cost. That cost is the requirement to swiftly adapt to changes in our information environment. Rather than mitigate this cost, our computing devices currently exacerbate it. This is because their user interfaces have not fundamentally changed in 20 years. As a consequence, devices bombard users with requests for attention, regardless of the cost of their interruptions.
Article
Full-text available
This paper presents our vision of Human Computer Interaction (HCI): "Tangible Bits." Tangible Bits allows users to "grasp & manipulate" bits in the center of users' attention by coupling the bits with everyday physical objects and architectural surfaces. Tangible Bits also enables users to be aware of background bits at the periphery of human perception using ambient display media such as light, sound, airflow, and water movement in an augmented space. The goal of Tangible Bits is to bridge the gaps between both cyberspace and the physical environment, as well as the foreground and background of human activities. This paper describes three key concepts of Tangible Bits: interactive surfaces; the coupling of bits with graspable physical objects; and ambient media for background awareness. We illustrate these concepts with three prototype systems -- the metaDESK, transBOARD and ambientROOM -- to identify underlying research issues. Keywords tangible user interface, ambient media, gras...
Article
This article presents a theory in which automatization is construed as the acquisition of a domain-specific knowledge base, formed of separate representations, instances, of each exposure to the task. Processing is considered automatic if it relies on retrieval of stored instances, which will occur only after practice in a consistent environment. Practice is important because it increases the amount retrieved and the speed of retrieval; consistency is important because it ensures that the retrieved instances will be useful. The theory accounts quantitatively for the power-function speed-up and predicts a power-function reduction in the standard deviation that is constrained to have the same exponent as the power function for the speed-up. The theory accounts for qualitative properties as well, explaining how some may disappear and others appear with practice. More generally, it provides an alternative to the modal view of automaticity, arguing that novice performance is limited by a lack of knowledge rather than a scarcity of resources. The focus on learning avoids many problems with the modal view that stem from its focus on resource limitations.
Article
Specialized elements of hardware and software, connected by wires, radio waves and infrared, will be so ubiquitous that no one will notice their presence.
Article
This paper provides an overview of the relationship between proactive computing and autonomic computing, considering the design of systems that are beyond the scope of our existing computational infrastructure. Autonomic computing, as described by the IBM's manifesto on the subject, is a clear statement of the difficulties and challenges facing the computing industry today. In particular, autonomic computing addresses the problem of managing complexity. Intel Research is exploring computing futures that overlap autonomic computing but also explore new application domains that require principles we call proactive computing, enabling the transition from today's interactive systems to proactive environments that anticipate our needs and act on our behalf.
Article
Recent debate has centered on the relative promise of focusing user-interface research on developing new metaphors and tools that enhance users' abilities to directly manipulate objects versus directing effort toward developing interface agents that provide automation. In this paper, we review principles that show promise for allowing engineers to enhance human---computer interaction through an elegant coupling of automated services with direct manipulation. Key ideas will be highlighted in terms of the LookOut system for scheduling and meeting management. Keywords Intelligent agents, direct manipulation, user modeling, probability, decision theory, UI design INTRODUCTION There has been debate among researchers about where great opportunities lay for innovating in the realm of human--- computer interaction [10]. One group of researchers has expressed enthusiasm for the development and application of new kinds of automated services, often referred to as interface "agents." The effor...
Conference Paper
Occupants of future computing environments with ubiquitous display devices may feel inundated with changing digital information. One solution is to create a reasoning module that accepts requests to display information from multiple applicatins and controls how the information is presented to minimize visual disruptions to users. Such a system might use information about what activity is occurring in the space to exploit a powerful phenomenon of the human visual system: change blindness.