No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Using context-aware computing to reduce the perceived burden of interruptions from mobile devices
Abstract
The potential for sensor-enabled mobile devices to proactively present information when and where users need it ranks among the greatest promises of ubiquitous computing. Unfortunately, mobile phones, PDAs, and other computing devices that compete for the user's attention can contribute to interruption irritability and feelings of information overload. Designers of mobile computing interfaces, therefore, require strategies for minimizing the perceived interruption burden of proactively delivered messages. In this work, a context-aware mobile computing device was developed that automatically detects postural and ambulatory activity transitions in real time using wireless accelerometers. This device was used to experimentally measure the receptivity to interruptions delivered at activity transitions relative to those delivered at random times. Messages delivered at activity transitions were found to be better received, thereby suggesting a viable strategy for context-aware message delivery in sensor-enabled mobile computing devices.
... However, sharp increases in the average number of such applications installed on each phone have inescapably resulted in people receiving an increasing number of notifcations, which has prompted debates over whether notifcations have become distraction or even damaging to smartphone users [14,15,23,56,108]. As a result, a strand of research has attempted to identify moments for deferring notifcations to, including opportune moments [35,89], breakpoints [2,45,51,81,85], and task transitions [45,53], for the purpose of reducing the distraction or disturbance they cause. Research has also shown, however, that smartphone users have a propensity to initiate notifcationchecking themselves [19], and that this behavior can occur even while a person is in the middle of a task [5,27,33,36,105]: a phenomenon that has been widely presumed to harm task performance. ...
... However, sharp increases in the average number of such applications installed on each phone have inescapably resulted in people receiving an increasing number of notifcations, which has prompted debates over whether notifcations have become distraction or even damaging to smartphone users [14,15,23,56,108]. As a result, a strand of research has attempted to identify moments for deferring notifcations to, including opportune moments [35,89], breakpoints [2,45,51,81,85], and task transitions [45,53], for the purpose of reducing the distraction or disturbance they cause. Research has also shown, however, that smartphone users have a propensity to initiate notifcationchecking themselves [19], and that this behavior can occur even while a person is in the middle of a task [5,27,33,36,105]: a phenomenon that has been widely presumed to harm task performance. ...
... The most common reason our participants mentioned for wanting to interact with their notifcations after completing a task was to protect their performance of the primary task, which can be further broken down into task progress and task quality [58]. This motivation resonates strongly with prior research that sought to identify task transitions and opportune moments for delivering notifcations [45]. Our participants often mentioned this motivation in combination with an assessment that the task was continuous and could not be paused (i.e., that they would need to start over if interrupted), and/or that they themselves needed to concentrate on it to complete it efciently. ...
... While the definition of receptivity involves multiple aspects, specifically the ability of a person to receive, process, and use the support provided, most of the previous studies have limited their focus solely on receiving support. Moreover, before states of receptivity became an important construct in the field of health intervention research motivated by the increasing interest in JITAIs, the topic was already highly relevant and investigated in the domain of ubiquitous computing under the umbrella of interruptibility research (Ho & Intille, 2005). This stream of research investigates the use of push notifications to attract the attention of individuals while reducing user burden and disruption through notifications. ...
... Much of the early research on interruptibility focused on the timing, content, and context of notifications and how interruption burden could be reduced, or indeed, receptivity increased. Research has shown that messages delivered during activity transitions (e.g., a transition from standing to sitting) (Ho & Intille, 2005) or during naturally occurring breakpoints, for example, when finishing an episode of mobile interaction (Fischer, Greenhalgh, & Benford, 2011), can decrease perceived burden and improve user receptivity. Specific times of day, namely morning and evening hours, are also linked to higher receptivity since people tend to be at home during these times (Mashhadi, Mathur, & Kawsar, 2014). ...
... 2 Breakpoints Fischer et al., 2011;Ho and Intille, 2005;Okoshi et al., 2015aOkoshi et al., , 2015bOkoshi et al., , 2017 (1) Breakpoints may indicate opportune moments to trigger notifications (e.g., after calls, after receiving SMS). ...
Intervention authors can likely increase the effectiveness of mobile health interventions (MHIs) by determining conditions under which individuals are able to receive, process, and use support. In this chapter, we will therefore first introduce and motivate the relevance of receptivity to MHIs. Second, we will describe the anatomy of an “ideal” MHI before key processes involved in the detection and prediction of receptivity to MHIs are discussed. Thereafter, we will review research on receptivity and summarize factors that may carry relevant signals for determining receptive states and thus, offer guidance for the design of receptivity-capable MHIs. We will conclude with challenges intervention authors and engineers face and offer opportunities for future work.
... This information is then used to infer the most appropriate time to send a notification. Several studies have shown that notifications sent based on time, users' activity, location, mobile application usage, cognitive context or ringer mode are more likely to be accepted and acted upon by users [9,15,22,30,31,33,39,40,43,44,48]. ...
... The most prominent factors that impact or are associated with interruptibility are time [30,33,39,43,50,53], physical activity [22,24,30,33,34,36,53], location [8,9,13,30,39,43,46,48], application usage [33,34,36,40,44], and ringer mode [8,31,33,40,50]. ...
... One of the significant features for detecting available moments is physical activity. It has been found that users are more interruptible during activity transitions [22,24]. In recent works [30,33], activity type (e.g., walking, standing, running) obtained from mobile devices has been used for an interruptibility management system. ...
Understanding in which circumstances office workers take rest breaks is important for delivering effective mobile notifications and make inferences about their daily lifestyle, e.g., whether they are active and/or have a sedentary life. Previous studies designed for office workers show the effectiveness of rest breaks for preventing work-related conditions. In this paper, we propose a hybrid personalised model involving a kernel density estimation model and a generalised linear mixed model to model office workers’ available moments for rest breaks during working hours. We adopt the experience-based sampling method through which we collected office workers’ responses regarding their availability through a mobile application with contextual information extracted by means of the mobile phone sensors. The experiment lasted 10 workdays and involved 19 office workers with a total of 528 responses. Our results show that time, location, ringer mode, and activity are effective features for predicting office workers’ availability. Our method can address sparse sample issues for building individual predictive behavioural models based on limited and unbalanced data. In particular, the proposed method can be considered as a potential solution to the “cold-start problem”, i.e., the negative impact of the lack of individual data when a new application is installed.
... Pejovic et al. [46] considered four physical activities related to interruptibility (being still, being on foot, being on a bicycle, and being inside a vehicle) and observed that the degree of interruptibility differs for each activity. Ho et al. [21] found that a user is likely to be interruptible when transitioning between two physical activities (e.g., from sitting to walking). Based on these observations, we aim to find whether physical movements signal opportune moments for smart speakers to engage with users. ...
... ESM Trigger Strategies: In our study, ESM prompts were delivered randomly or when movements were detected. Previous studies [21] reported that users were likely to be interruptible to mobile devices delivering messages during activity transition moments (e.g. from sitting to walking); these messages were more positively received during activity transitions than when delivered at random times. We wanted to investigate if the transition period between physical activities is an opportune time to interrupt users. ...
... Users were generally more interruptible after entering a room and were transitioning between physical activities, but were less interruptible during departure; participants were interruptible in 96% of cases upon returning to their rooms, but only 35% of departures were reported as interruptible. High interruptibility during activity transition is consistent with previous research [21,46]. During activity transitions, as the users were not occupied with any activities, they could easily shift their attention to the smart speakers [67]. ...
Increasing number of researchers and designers are envisioning a wide range of novel proactive conversational services for smart speakers such as context-aware reminders and restocking household items. When initiating conversational interactions proactively, smart speakers need to consider users' contexts to minimize disruption. In this work, we aim to broaden our understanding of opportune moments for proactive conversational interactions in domestic contexts. Toward this goal, we built a voice-based experience sampling device and conducted a one-week field study with 40 participants living in university dormitories. From 3,572 in-situ user experience reports, we proposed 19 activity categories to investigate contextual factors related to interruptibility. Our data analysis results show that the key determinants for opportune moments are closely related to both personal contextual factors such as busyness, mood, and resource conflicts for dual-tasking, and the other contextual factors associated with the everyday routines at home, including user mobility and social presence. Based on these findings, we discuss the need for designing context-aware proactive conversation management features that dynamically control conversational interactions based on users' contexts and routines.
... Interruptibility research has been carried on for decades, with the wider aim of reducing interruptions in workplaces [70,107] and desktop environments [23,40,43]. In recent years, considerable research attention has shifted to mobile receptivity [14,38,106]. In particular, opportuneness describes whether a given moment is suitable for a person to perform a particular action [78]. ...
... In particular, opportuneness describes whether a given moment is suitable for a person to perform a particular action [78]. Various other terms for and sub-types of opportune moments have been proposed, including interruptible moments [14,38,56,59,93,106], break-points [1, 14, 33, 42, 72-75, 77, 85, 89], transitions [92], receptive moments [34,61,94], and moments when users would be attentive [28] or responsive [51]. However, all of this research shares the aim of identifying, characterizing, and predicting good moments for people to receive notifications [80,81,95], including but not limited to messaging notifications [78] and ads [80,99]. ...
News notifications on smartphones provide a convenient way to stay informed, but their delivery timing can influence user engagement. Despite this, research on the impact of notification timing on reading behavior remains limited. Therefore, we developed NewsMoment, a news aggregation app that monitors user reading patterns and sends news notifications. Our experience sampling study with 46 NewsMoment users revealed four distinct reading modes: typical, comprehensive, scanning, and unengaged. Deep reading, encompassing typical and comprehensive modes, more often occurred during self-initiated browsing rather than through pushed news. Interestingly, shallow reading modes - unengaged and scanning - showed varying prevalence, associated triggers, and engagement, despite their similarities. Importantly, unengaged reading persisted regardless of users' perceived moment opportuneness, whereas scanning reading was more common during inopportune moments. These findings suggest that identifying opportune moments for news reading may primarily reduce scanning reading, without substantially impacting unengaged reading.
... Previous studies have shown that users' receptivity to notifications is influenced by many factors: (1) informational qualities of the notifications, e.g. interest, entertainment, relevance and actionability [242]; (2) mobile usage, such as the time of the interruption and the type of app pushing the notification [243,242]; (3) demographics, such as personality traits [244]; and (4) personal dynamics, such as location [245], transitions between activities [246] and social roles [247]. ...
... Okoshi et al. [289,290,291] examined breakpoints in physical activities and app usage and found that notifications delivered at breakpoints, denoted as transitions between apps and physical activities, could lower individuals' mental burden. Ho and Intille [246] also suggested that notifications delivered during activity transitions produced more favourable outcomes than those delivered randomly. As described earlier, we used the Android Google API to record the current physical activities of the participants. ...
... Kern et al. [39] showed that users' personal and social interruptibility could be determined with an accuracy of up to 91%. Ho et al. [27] used wireless accelerometers to detect postural and ambulatory activity transitions in real time with an average accuracy of 91.2%; their results showed that participants were more receptive to the delivered messages. Haapalainen et al. [26] used psycho-physiological sensors and found electrocardiogram median absolute deviation and median heat fux measurements were the most accurate at distinguishing between low and high levels of cognitive load, with an overall accuracy of 80% when used together. ...
... For the sensor data, we focused on the features that captured users' movement in VR. Previous research found that interruptibility could be predicted through velocity [27,39,41,76] and the rotation of the device users wear [70,88]; as such, we used the velocity and angular velocity of the HMD and the controllers, as well as the rotation of the HMD, as features. We also used the angle between the HMD and gaze, as well as gaze-shift speed over time, as features since previous research has found that diferent tasks afect eye-head dynamics [13] and gaze-shift dynamics [14]. ...
... The app was implemented for the Android mobile operating system, embraces the Android Material design patterns [24] to ensure "contemporary" look and feel. It relies on mobile sensing to detect activity breakpoints (further elaborated in Section 3.1) in order to deliver messages at the most appropriate times [25]. Sample screens of our applications are displayed in Figure 1. ...
... Finally, previous research has examined the importance of the timing of prompts [27] with the goal of recognizing the "right" moments to send the message. While numerous factors may affect a user's readiness to react to, and indeed even notice, a notification, a breakpoint, such as change in activity, e.g., from sitting to walking or from walking to sitting, is often found to be a favorable moment for interrupting a user [25]. Thus, in our application we harness a built-in Google Activity Recognition classifier to detect activity transitions in a battery-friendly manner and, in case sufficient time has passed since the last notification (1.5 h), a new message is delivered. ...
The growing ubiquity of smartphones and the ease of creating and distributing applications render the mobile platform an attractive means for facilitating positive behavior change at scale. Within the smartphone as a behavior change support system, mobile notifications play a critical role as they enable timely and relevant information distribution. In this paper we describe our preliminary investigation of the persuasiveness of mobile notifications delivered within a real-world behavior change intervention mobile app, which enabled users to set goals and define tasks related to those goals. The application aimed to motivate the users with notifications belonging to one of two groups—tailored and non-tailored, seeing them as sparks in the Fogg Behavior Model and personalizing them according to the users’ Big Five personality traits. Results indicate that customized messages may work for some individuals while working poorly for others. When analyzing users as a single group, no significant differences were observed, but when proceeding with the analysis on the individual level we found seven users whose personality traits notifications interact with in interesting ways. Our results offer two general insights: (1) Using personality-tailored messaging in a dynamic mobile domain as opposed to a static domain leads to different outcomes, and it seems that there is no one-to-one mapping between domains; (2) A major reason for most of our hypotheses being false may be that messages that are deemed as persuasive on their own are not what persuades people to perform an action. Unlike the clear-cut findings observed in other domains, we discover a rather nuanced relationship between the personalization and persuasiveness that calls for further exploration at the individual participant level.
... Various studies have focused on user behavior and user state to control the timing of notifications. Some studies attempted to control notifications by acquiring user actions from web camera information, mouse movement, keyboard typing information, and page or application switching timing [12] [2] [7] [10]. Furthermore, a study developed a middleware, "Attelia," that detected the breakpoints and delays in notification timing [16]. ...
... There are several methods for obtaining user behavior based on the information obtained from a web camera [12], mouse movement [2], or keyboard timing [7]. User behavior can be obtained from the page/application timing switch [10]. Furthermore, a middleware called "Attelia" has been used to delay the notification timing by detecting the breakpoints [16]. ...
The various notifications received on computers while being used interrupt users’ reading experience, negatively affect their emotions, and increase the cognitive load. Many studies have focused on user behavior and user state to control the notification timing. Blinking has been applied as a type of physical activity to detect users’ interests and emotions, detect driver fatigue, and design interactive robots. In this study, we focused on breakpoints while reading text information depending on the blink frequency during concentrated reading as a method to control notification timing. We constructed a system that controls notification timing based on the detected breakpoint. In the experiment, we simulated a real reading environment using the prototype of the system. We evaluated the detection times of the breakpoints and the effectiveness of the system. Although we have not proven the hypothesis that the expected breakpoint is detected based on the blink frequency, we found that the users’ browsing experience was improved when they used the control system.
... Applying the concept of bounded deferral, OASIS deferred push notifications for a limited period of time regardless of if a breaking point was detected. Ho and Intille (2005) studied the use of body-worn sensors available on mobile devices to effectively predict interruptibility. Also, using EEG (Electroencephalogram) sensors, Mathur, Lane and Kawsar (2016) studied which sensor information from mobile devices is relevant to anticipating a suitable moment for delivery. ...
Mobile apps market is a growing market and the main technological enabler of apps are push notifications (PN). Today, users are currently receiving a daily average of 63 PN.
After an introduction that highlights the relevance of PN, this chapter covers the background of its topic –pop-up messages that emerge on the smartphone screen- and its characterization: (1) proactive communication with the user; (2) explicitly authorized through an opt-in request; (3) wide range of content, private and social, sent by social networks, commercial companies, or news publishers from apps or web site; (4) targeted according to the users’ interests, previous behaviors, or time of day; (6) always prompting the user to click on the PN that will land on the sender’s app / web site.
There is a steep competition for the user's attention to click through the PN message. Thus the chapter moves through to discuss the factors that influence the choice of whether to open or ignore a PN: (1) Timing in the delivery, disruption, and systems for managing PN in a non-disruptive way. (2) Top-down factors in PN usage, such as user profile, user reaction times, and user interest in the content (3) Bottom-up factors, such as message textual and visual features as an antecedent of click-through rate.
Before concluding, the chapter suggests future directions for researchers and practitioners: how to increase opt-in rates, user experience of PN, reasons to opt-out …
The chapter ends with a conclusion and a list of references.
... The authors find that notifications delivered at breakpoints denoted as transitions between applications and physical activities can lower the individuals' mental burden. Ho and Intille [23] also suggest that notifications delivered during activity transitions produce more favorable outcomes than those delivered randomly. The number of different activities detected was also used as a feature in the first stage classification. ...
Notifications are one of the most prevailing mechanisms on smartphones and personal computers to convey timely and important information. Despite these benefits, smartphone notifications demand individuals' attention and can cause stress and frustration when delivered at inopportune timings. This paper investigates the effect of individuals' smartphone usage behavior and mood on notification response time. We conduct an in-the-wild study with more than 18 participants for five weeks. Extensive experiment results show that the proposed regression model is able to accurately predict the response time of smartphone notifications using current user's mood and physiological signals. We explored the effect of different features for each participant to choose the most important user-oriented features in order to to achieve a meaningful and personalised notification response prediction. On average, our regression model achieved over all participants an MAE of 0.7764 ms and RMSE of 1.0527 ms. We also investigate how physiological signals (collected from E4 wristbands) are used as an indicator for mood and discuss the individual differences in application usage and categories of smartphone applications on the response time of notifications. Our research sheds light on the future intelligent notification management system.
... However, interruptive messages result in productivity loss, increased stress, and time pressure [151], implying that less opportune delivery of intervention may lead to a low level of intervention adherence [152]. A user's behavioral routines can be leveraged to find opportune moments (e.g., activity transition times) [153]. A user's contextual model based on temporal and location contexts can be further used to define the complex rules for delivery timing. ...
With the advent of Digital Therapeutics (DTx), the development of software as a medical device (SaMD) for mobile and wearable devices has gained significant attention in recent years. Existing DTx evaluations, such as randomized clinical trials, mostly focus on verifying the effectiveness of DTx products. To acquire a deeper understanding of DTx engagement and behavioral adherence, beyond efficacy, a large amount of contextual and interaction data from mobile and wearable devices during field deployment would be required for analysis. In this work, the overall flow of the data-driven DTx analytics is reviewed to help researchers and practitioners to explore DTx datasets, to investigate contextual patterns associated with DTx usage, and to establish the (causal) relationship of DTx engagement and behavioral adherence. This review of the key components of data-driven analytics provides novel research directions in the analysis of mobile sensor and interaction datasets, which helps to iteratively improve the receptivity of existing DTx.
... Therefore, connection to different cloud services gains from a division of processing between the cloud and the device, imposing a bigger load on local assets along with storage. Third, mobile devices are more and more treated as the main computing device, performing more complicated operations than formerly thought up (Ho & Intille, 2005). However, storage has commonly been ignored as an important aspect of phones, and tablets at least in terms of capabilities (Zhong et al., 2014;Son, Lee, Kim, Yoo & Lee, 2015;Zhong et al., 2016). ...
p>The decision to buy smartphones comes with the consideration of several factors, especially among millennials who have been known to be very painstaking when it comes to smartphone specs. With this, this study was conducted to provide a scenario of how millennials behave when it comes to buying smartphones and identify the relative importance of the features or attributes (battery, screen size, storage, storage space and main camera resolution) in determining millennials’ best and least-preferred models of a smartphone. A total of N=400 millennials responded to a market survey that employed twenty (20) software-generated plancards. Using metric conjoint analysis (a marketing research technique), this study found out that as far as millennials in Digos City are concerned, the screen size of a smartphone is the most important attribute, while the battery of a smartphone is the least important attribute. On aggregate, a smartphone with 6 inches screen size is most preferred, while a smartphone that has a non-removable battery is the least preferred attribute level. The additive model of conjoint analysis further revealed that the most preferable smartphone is one that has a removable battery, with 6” screen size, has a microSD slot storage, with 64 GB storage space and 12 megapixels as main camera resolution, while the least preferred combination is one that has a removable battery, 4” screen size, no microSD slot storage, 16 GB storage space and 16-megapixel main camera resolution. Implications of this market research are also discussed.
JEL: D01; D40; D83; L63
Article visualizations:
</p
... Furthermore, prior works studied infuences of transition lags due to task interruptions on users' emotions, cognition, and contextual determinants and reported that interruptions could induce negative feelings (e.g., stress [46], annoyance [3], anxiety [4]) and increase workload [46] in a variety of environments (e.g., ofce work [75] and driving [35]). In addition, many studies have reported that task interruptions are infuenced by various contextual factors, such as changes in physical activity [25], changes in conversation [33], calendar information [71], messages from diferent social relationships [49], and diference in personality traits [50]. ...
... Plusieurs raisons pourraient expliquer ce constat. Des auteurs et autrices ont par exemple souligné qu'un des freins à l'adoption des chatbots réside dans la difficulté de ces derniers à identifier le contexte de la conversation (Ho & Intille, 2005 ;Coniam, 2014 ;Q. V. Liao et al., 2016). ...
In what way are chatbots new resources for humans? What are the conditions for their successful implementation? How do chatbots transform human activity? To answer these questions, this paper investigates how four chatbots were implemented and utilized in a professional context based on real user activity. We first show that the current design approach does not take sufficient account of the real activity and its multiple determinants, or of the users' point of view on their activity. Yet these interactive devices participate in a pre-existing socio-technical system made up of a diversity of subjects engaged in activities with multiple purposes. Through the prism of instrumental geneses, we therefore propose to identify how these chatbots can help or hinder human activity, so as to document, for design purposes, the conditions leading to the emergence of chatbot uses. After a synthetic presentation of the empirical results, we discuss the relevance of the instrumental approach when characterizing the contributions and limitations of chatbots. Finally, we argue that a design perspective supporting the postulate of human-machine asymmetry is prolific in supporting a design approach that is more focused on human power to act than on innovation.
... In contrast, because of the physical absence of the interrupter notification interruptions are a kind of flexible with the inclusion of many options for the user. So, the handling of the interruption can be negotiated (Ho and Intille 2005). In Fig. 1, three states in which a user might interact with notification in a normal way are shown. ...
This paper devises the issue of machine learning rule-based methodology for uncovering the behavior-based rules of respective smart-phone users for purpose of providing context wise individualized notification services. Nowadays, the number of notifications arrived at an inappropriate moment of time or carried unrelated material, which can cause disruption. Previously Rule-based Classifier and Association Rule Mining (ARM) Techniques have been used to solve those problems. However, the classifier approach processes accuracy and reliability problems because of small data instances. ARM creates a vast number of redundant rules, which are pointless for creating context-aware decisions. Redundant rules can make the approach non-efficient and also make the dataset unnecessarily large, making decision-based problems more complicated. For those problems solution in this article, we propose a new Behavioral Adversarial Traversal Tree approach for extracting user behavioral rules with respect to different contexts. A real-world dataset is collected to make this approach more relevant. The Proposed approach effectively identifies and removes the redundant rules with individual behavior-oriented time slots, which are used in the proposed approach to make it more exact and efficient. Our experiments and comparisons on each individual contextual dataset exhibit that the following rule discovery approach is more adequate and more exact for context-aware notification services.
... Balancing intrusiveness and noticeability and finding appropriate moments for notification delivery constitute popular research topics in the field of human-computer interaction (HCI) [18,19,20], in which the exploration on auditory notifications makes an important part and has also been covered in previous ICAD conferences [8,11]. To provide auditory notifications effectively but in a less distracting manner, Kilander and Lönnqvist [8] proposed to render an artificial audio ambience using pre-designed sound signals (e.g., a thunderstorm sequence), and they added reverb effect to create an ambient atmosphere. ...
Less intrusive information delivery has been a popular research topic for auditory displays. While most research has addressed this issue by creating new notification cues such as rendering ambient soundscapes or modifying background music, we present a novel method to gently deliver artificial notification sounds that have been commonly used in digital devices and for popular applications. We propose to play a notification sound by embedding it into the music that a user is listening to, after changing the musical timbre, amplitude, tempo, and octave of the notification to match these features of the music. To implement this concept, we extend a melody extraction algorithm for notification timbre transfer, and we present a pipeline that algorithmically selects a proper time spot and harmoniously embeds the notification into music. To validate our design concept, we present a user study comparing our method with the standard method of playing notification sounds on digital devices. Through an extensive analysis of 96 tasks performed by 32 participants, we demonstrate that our method can deliver notification sounds in a less intrusive but adequately noticeable manner and is preferred by most participants.
... Several researchers have tried to identify context factors, create context frameworks and models for different application areas (e.g. Dey and Abowd 1999, , Shilit and Theimer 1994, Petrelli et al. 2001, Pascoe 1998, Schmidt 2001, Hall 1976, Becker and Nicklas 2004, Jung et al. 2005, Raptis et al. 2005, Ho and Intille 2005, Hinze and Buchanan 2005, Mihalic et al. 2006. A reflection can also be found in Murer et al. 2015. ...
Austria has a mature RTI funding system today. Being a late-comer
in terms of institutionalisation, Austria developed councils and agencies as the standard organisational models since the 1960s. Institutional path dependencies, fragmentation and contradicting claims of autonomy and political steering led to persistent tensions. Recent changes were therefore driven by the need for better coordination and rebalancing principal-agent relations.
... Choosing the opportune moments to prompt participants while adhering to the overall study protocol can be a way to mitigate these challenges. Numerous works examined how to do this by leveraging the capabilities of smartphones to gather contextual information [18,[35][36][37][38]. ...
The Experience Sampling Method (ESM) is gaining ground for collecting self-reported data from human participants during daily routines. An important methodological challenge is to sustain sufficient response rates, especially when studies last longer than a few days. An obvious strategy is to deliver the experiential questions on a device that study participants can access easily at different times and contexts (e.g., a smartwatch). However, responses may still be hampered if the prompts are delivered at an inconvenient moment. Advances in context sensing create new opportunities for improving the timing of ESM prompts. Specifically, we explore how physiological sensing on commodity-level smartwatches can be utilized in triggering ESM prompts. We have created Experiencer, a novel ESM smartwatch platform that allows studying different prompting strategies. We ran a controlled experiment (N=71) on Experiencer to study the strengths and weaknesses of two sampling regimes. One group (N=34) received incoming notifications while resting (e.g., sedentary), and another group (N=37) received similar notifications while being active (e.g., running). We hypothesized that response rates would be higher when experiential questions are delivered during lower levels of physical activity. Contrary to our hypothesis, the response rates were found significantly higher in the active group, which demonstrates the relevance of studying dynamic forms of experience sampling that leverage better context-sensitive sampling regimes. Future research will seek to identify more refined strategies for context-sensitive ESM using smartwatches and further develop mechanisms that will enable researchers to easily adapt their prompting strategy to different contextual factors.
... Recognizing the activity a human is doing at any given moment is a technique that has a wide range of areas of application. From health and care applications [1][2][3] to automation and context-awareness [4][5][6], sports [7,8], social media [9] and even security surveillance [10]. Dependent on the use case, the applicability of a certain approach in regards to sensors, attributes or models can vary. ...
Human Activity Recognition (HAR) is a field with many contrasting application domains, from medical applications to ambient assisted living and sports applications. With ever-changing use cases and devices also comes a need for newer and better HAR approaches. Machine learning has long been one of the predominant techniques to recognize activities from extracted features. With the advent of deep learning techniques that push state of the art results in many different domains like natural language processing or computer vision, researchers have also started to build deep neural nets for HAR. With this increase in complexity, there also comes a necessity to compare the newer approaches to the previous state of the art algorithms. Not everything that is new is also better. Therefore, this paper aims to compare typical machine learning models like a Random Forest (RF) or a Support Vector Machine (SVM) to two commonly used deep neural net architectures, Convolutional Neural Nets (CNNs) and Recurrent Neural Nets (RNNs). Not only in regards to performance but also in regards to the complexity of the models. We measure complexity as the memory consumption, the mean prediction time and the number of trainable parameters of the models. To achieve comparable results, the models are all tested on the same publicly available dataset, the UCI HAR Smartphone dataset. With this combination of prediction performance and model complexity, we look for the models achieving the best possible performance/complexity tradeoff and therefore being the most favourable to be used in an application. According to our findings, the best model for a strictly memory limited use case is the Random Forest with an F1-Score of 88.34%, memory consumption of only 0.1 MB and mean prediction time of 0.22 ms. The overall best model in terms of complexity and performance is the SVM with a linear kernel with an F1-Score of 95.62%, memory consumption of 2 MB and a mean prediction time of 0.47 ms. The two deep neural nets are on par in terms of performance, but their increased complexity makes them less favourable to be used.
... Okoshi et al. [82] determined accurate application-specific break points, during which the user can be interrupted while she is using an application. Ho et al. [49] determined when the user is transitioning from one physical activity to another (e.g., from sitting to walking) using body-worn accelerometers and used those moments to deliver notifications. These prior systems do not consider the importance of the notification, nor do they consider the possibility of interrupting an activity when user has spare attention to handle a new secondary task. ...
We live in a world where mobile computing systems are increasingly integrated with our day-to-day activities. People use mobile applications virtually everywhere they go, executing them on mobile devices such as smartphones, tablets, and smart watches. People commonly interact with mobile applications while performing other primary tasks such as walking and driving (e.g., using turn-by-turn directions while driving a car). Unfortunately, as an application becomes more mobile, it can experience resource scarcity (e.g., poor wireless connectivity) that is atypical in a traditional desktop environment. When critical resources become scarce, the usability of the mobile application deteriorates significantly. In this dissertation, I create system support that enables users to interact smoothly with mobile applications when wireless network connectivity is poor and when the user’s attention is limited. First, I show that speculative execution can mitigate user-perceived delays in application responsiveness caused by high-latency wireless network connectivity. I focus on cloud-based gaming, because the smooth usability of such application is highly dependent on low latency. User studies have shown that players are sensitive to as little as 60 ms of additional latency and are aggravated at latencies in excess of 100ms. For cloud-based gaming, which relies on powerful servers to generate high-graphics quality gaming content, a slow network frustrates the user, who must wait a long time to see input actions reflected in the game. I show that by predicting the user’s future gaming inputs and by performing visual misprediction compensation at the client, cloud-based gaming can maintain good usability even with 120 ms of network latency. Next, I show that the usability of mobile applications in an attention-limited environment (i.e., driving a vehicle) can be improved by automatically checking whether interfaces meet best-practice guidelines and by adding attention-aware scheduling of application interactions. When a user is driving, any application that demands too much attention is an unsafe distraction. I first develop a model checker that systematically explores all reachable screens for an application and determines whether the application conforms to best-practice vehicular UI guidelines. I find that even well- known vehicular applications (e.g., Google Maps and TomTom) can often demand too much of the driver’s attention. Next, I consider the case where applications run in the background and initiate interactions with the driver. I show that by quantifying the driver’s available attention and the attention demand of an interaction, real-time scheduling can be used to prevent attention overload in varying driving conditions.
... However, data is captured about work experiences in order to support learning; rather than data being captured about learning experiences and processes. Secondly, this research contributes to research on adaptive and contextualized reflection guidance (Fischer et al., 1993;McCall et al., 1990;Kocielnik et al., 2018); and more broadly to research on context-aware prompting (e.g., Ho & Intille, 2005;Pejovic & Musolesi, 2014). ...
Professional and lifelong learning are a necessity for workers. This is true both for re-skilling from disappearing jobs, as well as for staying current within a professional domain. AI-enabled scaffolding and just-in-time and situated learning in the workplace offer a new frontier for future impact of AIED. The hallmark of this community’s work has been i) data-driven design of learning technology and ii) machine-learning enabled personalized interventions. In both cases, data are the foundation of AIED research and data-related ethics are thus central to AIED research. In this paper we formulate a vision how AIED research could address data-related ethics issues in informal and situated professional learning. The foundation of our vision is a secondary analysis of five research cases that offer insights related to data-driven adaptive technologies for informal professional learning. We describe the encountered data-related ethics issues. In our interpretation, we have developed three themes: Firstly, in informal and situated professional learning, relevant data about professional learning – to be used as a basis for learning analytics and reflection or as a basis for adaptive systems - is not only about learners. Instead, due to the situatedness of learning, relevant data is also about others (colleagues, customers, clients) and other objects from the learner’s context. Such data may be private, proprietary, or both. Secondly, manual tracking comes with high learner control over data. Thirdly, learning is not necessarily a shared goal in informal professional learning settings. From an ethics perspective, this is particularly problematic as much data that would be relevant for use within learning technologies hasn’t been collected for the purposes of learning. These three themes translate into challenges for AIED research that need to be addressed in order to successfully investigate and develop AIED technology for informal and situated professional learning. As an outlook of this paper, we connect these challenges to ongoing research directions within AIED – natural language processing, socio-technical design, and scenario-based data collection - that might be leveraged and aimed towards addressing data-related ethics challenges.
... Fischer et al. showed that participants react faster to probes when they are delivered immediately after completing a task on mobile (e.g., reading a text message) [7]. Ho et al. demonstrated that placing the probe between two physical activities (like sitting and walking) may attract user attention quickly [14]. In [25], authors demonstrated that last survey response, phone's ringer mode can be leveraged to identify suitable probing moments. ...
... Many studies have found a significant correlation between physical activity and interruptibility [14,26,33,39]. The type of physical activity was also found to be significant; for example, people driving in a vehicle replied more slowly than people walking outside [39]. ...
Just-In-Time Adaptive Intervention (JITAI) is an emerging technique with great potential to support health behavior by providing the right type and amount of support at the right time. A crucial aspect of JITAIs is properly timing the delivery of interventions, to ensure that a user is receptive and ready to process and use the support provided. Some prior works have explored the association of context and some user-specific traits on receptivity, and have built post-study machine-learning models to detect receptivity. For effective intervention delivery, however, a JITAI system needs to make in-the-moment decisions about a user’s receptivity. To this end, we conducted a study in which we deployed machine-learning models to detect receptivity in the natural environment, i.e., in free-living conditions.
We leveraged prior work regarding receptivity to JITAIs and deployed a chatbot-based digital coach – Ally – that provided physical-activity interventions and motivated participants to achieve their step goals. We extended the original Ally app to include two types of machine-learning model that used contextual information about a person to predict when a person is receptive: a static model that was built before the study started and remained constant for all participants and an adaptive model that continuously learned the receptivity of individual participants and updated itself as the study progressed. For comparison, we included a control model that sent intervention messages at random times. The app randomly selected a delivery model for each intervention message. We observed that the machine-learning models led up to a 40% improvement in receptivity as compared to the control model. Further, we evaluated the temporal dynamics of the different models and observed that receptivity to messages from the adaptive model increased over the course of the study.
... [26,27,31], context-aware systems that would defer notification until opportune moments (e.g. [19,21,43,46], content-aware filtering of notifications based on importance (e.g. [12,59], and ambient notification systems (e.g. ...
... Many studies have found a significant correlation between physical activity and interruptibility [33,38,14,25]. The type of physical activity was also found to be significant; for example, people driving in a vehicle replied more slowly than people walking outside [38]. ...
JITAI is an emerging technique with great potential to support health behavior by providing the right type and amount of support at the right time. A crucial aspect of JITAIs is properly timing the delivery of interventions, to ensure that a user is receptive and ready to process and use the support provided. Some prior works have explored the association of context and some user-specific traits on receptivity, and have built post-study machine-learning models to detect receptivity. For effective intervention delivery, however, a JITAI system needs to make in-the-moment decisions about a user's receptivity. To this end, we conducted a study in which we deployed machine-learning models to detect receptivity in the natural environment, i.e., in free-living conditions. We leveraged prior work regarding receptivity to JITAIs and deployed a chatbot-based digital coach -- Walkie -- that provided physical-activity interventions and motivated participants to achieve their step goals. The Walkie app included two types of machine-learning model that used contextual information about a person to predict when a person is receptive: a static model that was built before the study started and remained constant for all participants and an adaptive model that continuously learned the receptivity of individual participants and updated itself as the study progressed. For comparison, we included a control model that sent intervention messages at random times. The app randomly selected a delivery model for each intervention message. We observed that the machine-learning models led up to a 40% improvement in receptivity as compared to the control model. Further, we evaluated the temporal dynamics of the different models and observed that receptivity to messages from the adaptive
... Many researchers already investigated interruptibility, especially in terms of finding opportune moments to deliver smartphone notifications [45,53]. They found that interruptibility depends on different features such as the user activity [26], being in company [45], location [14], and engagement with the smartphone [16], but also on the content of notification [34] such as its importance [18,47]. Such features are also known as situationally-induced impairments [48]. ...
... Recognizing surfaces based on their vibration signatures is useful as it can enable tagging of different locations without requiring any additional hardware such as Near Field Communication (NFC) tags. Such tagging of locations can provide us with indirect information about the user activities and intentions without any dedicated infrastructure, based on which we can enable useful services such as context aware notifications/alarms [5]. For example, a user can set their phone to automatically go into silent mode when it is placed on their bed side table. ...
Recognizing surfaces based on their vibration signatures is useful as it can enable tagging of different locations without requiring any additional hardware such as Near Field Communication (NFC) tags. However, previous vibration based surface recognition schemes either use custom hardware for creating and sensing vibration, which makes them difficult to adopt, or use inertial (IMU) sensors in commercial off-the-shelf (COTS) smartphones to sense movements produced due to vibrations, which makes them coarse-grained because of the low sampling rates of IMU sensors. The mainstream COTS smartphones based schemes are also susceptible to inherent hardware based irregularities in vibration mechanism of the smartphones. Moreover, the existing schemes that use microphones to sense vibration are prone to short-term and constant background noises (e.g. intermittent talking, exhaust fan, etc.) because microphones not only capture the sounds created by vibration but also other interfering sounds present in the environment. In this paper, we propose VibroTag, a robust and practical vibration based sensing scheme that works with smartphones with different hardware, can extract fine-grained vibration signatures of different surfaces, and is robust to environmental noise and hardware based irregularities. We implemented VibroTag on two different Android phones and evaluated in multiple different environments where we collected data from 4 individuals for 5 to 20 consecutive days. Our results show that VibroTag achieves an average accuracy of 86.55% while recognizing 24 different locations/surfaces, even when some of those surfaces were made of similar material. VibroTag's accuracy is 37% higher than the average accuracy of 49.25% achieved by one of the state-of-the-art IMUs based schemes, which we implemented for comparison with VibroTag.
... For instance, Bailey et al. [5] suggested that an attention-aware system that defers presenting peripheral information until coarse boundaries are reached during task execution could mitigate the negative impact of such information's arrival. Some studies have used system-usage features to predict opportune moments for notification [17][18][19], while others have investigated mobile activity [15,24] and mental workload [28] for this purpose. Users have also been found to perceive varying levels of disruption depending on what tasks they are performing [12,13], and that mobile interruptibility is influenced by their levels of task engagement [40,45]. ...
... One of the commonly adopted approaches is based on devising smart ESM strategies, which identify the opportune probing moments, minimizing the user interruption. This approach leverages on different on-device sensors to infer user context and issue the probes at favorable moments, when the high-quality self-reports can be elicited [28,9,20,1,7,10]. Additionally, attempts have been made to improve the participant retention rate and data quality based on incentives, which can be in different forms like monetary, providing community benefit, increasing reputation, rewarding participants [17,11,6]. ...
In an Experience Sampling Method (ESM) based emotion self-report collection study, engaging participants for a long period is challenging due to the repetitiveness of answering self-report probes. This often impacts the self-report collection as participants dropout in between or respond with arbitrary responses. Self-reflection (or commonly known as analyzing past activities to operate more efficiently in the future) has been effectively used to engage participants in logging physical, behavioral, or psychological data for Quantified Self (QS) studies. This motivates us to apply self-reflection to improve the emotion self-report collection procedure. We design, develop, and deploy a self-reflection interface and augment it with a smartphone keyboard-based emotion self-report collection application. The interface provides feedback to the users regarding the relation between typing behavior and self-reported emotions. We validate the proposed approach using a between-subject study, where one group (control group) is not exposed to the self-reflection interface and the other group (study group) is exposed to it. Our initial results demonstrate that using self-reflection it is possible to engage the participants in the long-term and collect more self-reports.
... Additional important research questions in the development of the HeartSteps activity suggestion component concern the potential effects of habituation (Rankin et al., 2009) and/or treatment burden (Clawson et al., 2015;Eysenbach, 2005;Ho & Intille, 2005;Klasnja et al., 2008;Shaw et al., 2013;Yardley et al., 2016). If individuals habituate to the activity suggestions or find the intervention burdensome, the causal effect would be expected to deteriorate over time. ...
Although there is much excitement surrounding the use of mobile and wearable technology for the purposes of delivering interventions as people go through their day-to-day lives, data analysis methods for constructing and optimizing digital interventions lag behind. Here, we elucidate data analysis methods for primary and secondary analyses of micro-randomized trials (MRTs), an experimental design to optimize digital just-in-time adaptive interventions. We provide a definition of causal "excursion" effects suitable for use in digital intervention development. We introduce the weighted and centered least-squares (WCLS) estimator which provides consistent causal excursion effect estimators for digital interventions from MRT data. We describe how the WCLS estimator along with associated test statistics can be obtained using standard statistical software such as SAS or R. Throughout we use HeartSteps, an MRT designed to increase physical activity among sedentary individuals, to illustrate potential primary and secondary analyses.
... Similarly, their interaction with computers (which applications they use and how long) might also reveal boredom [7,41]. The context and activity of users are other important factors [31,45,55,70]. When people are moving or during an activity transition, they are more likely to respond to messages. ...
In this study, we investigate the effects of social context, personal and mobile phone usage on the inference of work engagement/challenge levels of knowledge workers and their responsiveness to well-being related notifications. Our results show that mobile application usage is associated to the responsiveness and work engagement/challenge levels of knowledge workers. We also developed multi-level (within- and between-subjects) models for the inference of attentional states and
engagement/challenge levels with mobile application usage indicators as inputs, such as the number of applications used prior to notifications, the number of switches between applications, and application category usage. The results of our analysis show that the following features are effective for the inference of attentional states and engagement/challenge levels: the number of switches between mobile applications in the last 45 minutes and the duration of application usage in the last 5 minutes before users’ response to ESM messages.
With the advent of digital therapeutics (DTx), the development of software as a medical device (SaMD) for mobile and wearable devices has gained significant attention in recent years. Existing DTx evaluations, such as randomized clinical trials, mostly focus on verifying the effectiveness of DTx products. To acquire a deeper understanding of DTx engagement and behavioral adherence, beyond efficacy, a large amount of contextual and interaction data from mobile and wearable devices during field deployment would be required for analysis. In this work, the overall flow of the data-driven DTx analytics is reviewed to help researchers and practitioners to explore DTx datasets, to investigate contextual patterns associated with DTx usage, and to establish the (causal) relationship between DTx engagement and behavioral adherence. This review of the key components of data-driven analytics provides novel research directions in the analysis of mobile sensor and interaction datasets, which helps to iteratively improve the receptivity of existing DTx.
This chapter offers a conceptual framework that ties together two domains of design decisions for digital therapeutics—those related to intervention design (i.e., which components or treatments to include) and those related to study design (i.e., how to test whether components or treatments work as intended). This framework is intended to help researchers in identifying and testing key digital therapeutics design considerations across the intervention development lifespan—that is, from formative work through longer-term implementation. The framework encourages intervention developers to make design decisions by considering three motivating questions: first, what health outcomes are we trying to impact; second, what are we trying to learn; and third, cross-cutting both prior questions to inform potential evidence, how much is at stake in terms of health impacts if we get this decision wrong? We consider how these questions can help to inform design decisions across three phases of the intervention lifespan—preparation and formative work, optimization of intervention components, and evaluation of effectiveness and implementation—to maximize the likelihood of positive health impact and accumulation of evidence about the intervention's functioning.
Quality of life (QoL) is a subjective term often determined by various aspects of living, such as personal well-being, health, family, and safety. QoL is challenging to capture objectively but can be anticipated through a person’s emotional state; especially positive emotions indicate an increased QoL and may be a potential indicator for other QoL aspects (such as health, safety). Affective computing is the study of technologies that can quantitatively assess human emotions from external clues. It can leverage different modalities including facial expression, physiological responses, or smartphone usage patterns and correlate them with the person’s life quality assessments. Smartphones are emerging as a main modality, mostly because of their ubiquitous availability and use throughout daily life activities. They include a plethora of onboard sensors (e.g., accelerometer, gyroscope, GPS) and can sense different user activities passively (e.g., mobility, app usage history). This chapter presents a research study (here referred to as the TapSense study) that focuses on assessing the individual’s emotional state from the smartphone usage patterns. In this TapSense study, the keyboard interaction of n = 22 participants was unobtrusively monitored for 3 weeks to determine the users’ emotional state (i.e., happy, sad, stressed, relaxed) using a personalized machine learning model. TapSense can assess emotions with an average AUCROC of 78%(±7% std). We summarize the findings and reflect upon these in the context of the potential developments within affective computing at large, in the long term, indicating a person’s quality of life.
Voice assistants, such as Amazon's Alexa and Google Home, increasingly find their way into consumer homes. Their functionality, however, is currently limited to being passive answer machines rather than proactively engaging users in conversations. Speakers' proactivity would open up a range of important application scenarios, including health services, such as checking in on patient states and triggering medication reminders. It remains unclear how passive speakers should implement proactivity. To better understand user perceptions, we ran a 3-week field study with 13 participants where we modified the off-the-shelf Google Home to become proactive. During the study, our speaker proactively triggered conversations that were essentially Experience Sampling probes allowing us to identify when to engage users. Applying machine-learning, we are able to predict user responsiveness with a 71.6% accuracy and find predictive features. We also identify self-reported factors, such as boredom and mood, that are significantly correlated with users' perceived availability. Our prototype and findings inform the design of proactive speakers that verbally engage users at opportune moments and contribute to the design of proactive application scenarios and voice-based experience sampling studies.
The growing use of mobile devices that are available everywhere can blur the boundaries between life domains work and life. The increasing number of notifications on smartphones leads to interruptions that might be unrelated to the current life domain and task and are therefore disruptive. Despite some tools there is a gap between the preferred and the actual separation of life domains. In this paper we show how concepts from the field of boundary management can be applied for notification management on mobile devices. We present a formal model of the semantic structure of life domains, which is based on the concepts of integration and segmentation from boundary theory. We introduce an app for the management of notifications on Android smartphones that leverages on this formal model. In a field study we evaluated the app with real-life notifications. The results show a significant reduction of the gap between actual and preferred boundary management.
Mobile ecological momentary assessments (mEMAs) require substantial user efforts to complete, resulting in low user compliance. One major source of incompliance is triggering mEMAs at inopportune moments. In this work, we propose a framework for implementing adaptive mEMAs using reinforcement learning (RL) to address the timing and context challenge, aiming to improve long term response compliance. To effectively model user state, we also propose a two-level user model with both momentary and routine state features. A novel k-routine mining algorithm is developed to extract routine state from passive sensing data. Using real mobile sensing data collected from 220 participants for over two weeks, we show that our proposed RL strategies consistently outperform the baseline methods including a random strategy and a supervised strategy in user compliance.
The goal of this in-the-wild study was to understand how different patient, provider, and environment contexts affected the use of a tablet-based checklist in a dynamic medical setting. Fifteen team leaders used the digital checklist in 187 actual trauma resuscitations. The measures of checklist interactions included the number of unchecked items and the number of notes written on the checklist. Of the 10 contexts we studied, team leaders’ arrival after the patient and patients with penetrating injuries were both associated with more unchecked items. We also found that the care of patients with external injuries contributed to more notes written on the checklist. Finally, our results showed that more experienced leaders took significantly more notes overall and more numerical notes than less experienced leaders. We conclude by discussing design implications and steps that can be achieved with context-aware computing towards adaptive checklists that meet the needs of dynamic use contexts.
Recognizing surfaces based on their vibration signatures is useful as it can enable tagging of different locations without requiring any additional hardware such as Near Field Communication (NFC) tags. However, previous vibration based surface recognition schemes either use custom hardware for creating and sensing vibration, which makes them difficult to adopt, or use intertial (IMU) sensors in commercial off-the-shelf (COTS) smartphones to sense movements produced due to vibrations, which makes them coarse-grained because of the low sampling rates of IMU sensors. The mainstream COTS smartphones based schemes are also susceptible to inherent hardware based irregularities in vibration mechanism of the smartphones. Moreover, the existing schemes that use microphones to sense vibration are prone to short-term and constant background noises (e.g. intermittent talking, exhaust fan, etc.) because microphones not only capture the sounds created by vibration but also other interfering sounds present in the environment. In this paper, we propose VibroTag, a practical vibration based sensing scheme that works with smartphones with different hardware, can extract fine-grained vibration signatures of different surfaces, and is robust to environmental noise and hardware based irregularities. We evaluated in multiple different environments where we collected data from 4 individuals for 5 to 20 consecutive days.
Learning on a mobile device in everyday settings makes users particularly susceptible for interruptions. Guidance (memory) cues can be implemented to support users in resuming a learning task after a distraction. These cues can take a wide range of forms and designs and, to work effectively, need to be carefully adapted to the mobile learning use case. In this work, we present a structured in-depth literature review on task resumption support for mobile devices. In particular, we propose a design space based on 30 carefully chosen publications to highlight well-evaluated design ideas as well as currently underrepresented research directions. Furthermore, we evaluate the causes of interruptions in the domain of mobile learning and derive design ideas for task resumption support on mobile devices. To this end, we conducted two focus groups with HCI experts (\(N=4\)) and users of mobile learning applications (\(N=3\)). Based on the literature review, focus groups, and further related work, we discuss ideas and research gaps for task resumption cues in mobile learning. We derive six design guidelines to support researchers and designers of mobile learning applications and emphasize promising research directions and open questions.
Prospective memory lapses, which involve forgetting to perform intended actions, affect independent living in older adults. Although memory training using smartphone applications could address them, users are sometimes unaware of available times for training or forget about it, presenting a need for proactive prompts. Existing applications mostly provide time-based prompts and prompts based on users' cognitive contexts remain an under-explored area. We developed Prompto, a conversational memory coach that detects physiological signals to suggest training sessions when users are relaxed and potentially more receptive. Our study with 21 older adults showed that users were more receptive to prompts and memory training under low cognitive load than under high cognitive load. Interviews and an in-the-wild deployment of Prompto indicated that majority of users appreciated the concept, found it helpful and were likely to respond to its prompts. We contribute towards developing technologies with cognitive context-aware prompting based on users' physiological readings.
The need to better understand the role of context has emerged after the revolution of mobile computing, as such devices are used in heterogeneous circumstances. However, it is difficult to say what context of use in mobile human-computer interaction actually means. This study summarises past research in mobile contexts of use and not only provides a deeper understanding of the characteristics associated with it, but also indicates a path for future research. This article presents an extensive and systematic literature review of more than 100 papers published in five high-quality journals and one main conference in the field of HCI during the years 2000-2007. The authors’ results show that context of use is still explored as a relatively static phenomenon in mobile HCI. Its most commonly mentioned characteristics are linked to social, physical, and technical components, while transitions between the contexts were rarely listed. Based on this review, a descriptive model of context of use for mobile HCI (CoU-HMCI) summarising five components, their subcomponents and descriptive properties is presented. The model can help both practitioners and academics to identify broadly relevant contextual factors when designing, experimenting with, and evaluating, mobile contexts of use.
While mobile instant messaging (MIM) facilitates ubiquitous interpersonal communication, its constant connectivity could build the expectation of an immediate response to messages, and its notifications flood could cause interruptions at inopportune moments. We examine two design concepts for MIM-private status sharing and sender-controlled notifications-that aim to lower the pressure for an immediate reply and reduce unnecessary interruptions by untimely notifications. Private status sharing reactively reveals a customized status with a selected partner(s) only when the partner has sent a message. Sender-controlled notifications give senders the control of choosing whether to send a notification for their own messages. We built MyButler, an Android app prototype that instantiates these two concepts and integrated it with KakaoTalk, a commercial MIM app. During a two-week field study with 11 pairs (5 couples and 6 friend pairs), participants expressed themselves through a total of 210 different statuses, 64.3% of which indicated the current activity or task of the user. Participants reported that private status sharing enabled them to explain their unavailability and relieved the pressure and expectations for timely attendance. We reveal more findings on the types of privately shared statuses and their roles in MIM communication; the in-situ behaviors and patterns of using sender-controlled notifications; and the motivations of MIM users in choosing whether to alert their messages. In terms of message notifications, senders chose to send 25.4% of the messages without any notification. We found that senders' decisions to alert are affected by the receiver's status, their own status to chat, and the possibility of message content exposure to others through notifications. Based on our findings, we draw insights into how the concepts of private status sharing and sender-controlled notifications can be applied in future designs and explorations.
The significance of Attentional User Interfaces that help create computing and communication systems to sense and reason about human attention by fusing together information from multiple streams, is discussed. Attentional cues are central in decisions about when to initiate or to make an effective contribution to a conversation or project. Computers with an ability to track and to understand attentional patterns among people engaged in conversations can provide new kinds of services and facilities. It is noted that continuing refinement of methods for recognizing, reasoning and communicating about attention will change in a qualitative manner the way working with computing systems is perceived.
We introduce utility-directed procedures for mediating the flow of
potentially distracting alerts and communications to computer users. We present
models and inference procedures that balance the context-sensitive costs of
deferring alerts with the cost of interruption. We describe the challenge of
reasoning about such costs under uncertainty via an analysis of user activity
and the content of notifications. After introducing principles of
attention-sensitive alerting, we focus on the problem of guiding alerts about
email messages. We dwell on the problem of inferring the expected criticality
of email and discuss work on the Priorities system, centering on prioritizing
email by criticality and modulating the communication of notifications to users
about the presence and nature of incoming email.
We describe the design and functionality of the Scope, a glanceable notification summarizer. The Scope is an information visualization designed to unify notifications and minimize distractions. It allows users to remain aware of notifications from multiple sources of information, including e-mail, instant messaging, information alerts, and appointments. The design employs a circular radar-like screen divided into sectors that group different kinds of notifications. The more urgent a notification is, the more centrally it is placed. Visual emphasis and annotation is used to reveal important properties of notifications. Several natural gestures allow users to zoom in on particular regions and to selectively drill down on items. We present key aspects of the Scope design, review the results of an initial user study, and describe the motivation and outcome of an iteration on the visual design. Figure 1 Scope (Figure 2 shows same in color). ABSTRACT We describe the design and functionality of the Scope, a glanceable notification summarizer. The Scope is an information visualization designed to unify notifications and minimize distractions. It allows users to remain aware of notifications from multiple sources of information, including e-mail, instant messaging, information alerts, and appointments. The design employs a circular radar-like screen divided into sectors that group different kinds of notifications. The more urgent a notification is, the more centrally it is placed. Visual emphasis and annotation is used to reveal important properties of notifications. Several natural gestures allow users to zoom in on particular regions and to selectively drill down on items. We present key aspects of the Scope design, review the results of an initial user study, and describe the motivation and outcome of an iteration on the visual design.
A person seeking someone else's attention is normally able to quickly assess how interruptible they are. This assessment allows for behavior we perceive as natural, socially appropriate, or simply polite. On the other hand, today's computer systems are almost entirely oblivious to the human world they operate in, and typically have no way to take into account the interruptibility of the user. This paper presents a Wizard of Oz study exploring whether, and how, robust sensor-based predictions of interruptibility might be constructed, which sensors might be most useful to such predictions, and how simple such sensors might be.The study simulates a range of possible sensors through human coding of audio and video recordings. Experience sampling is used to simultaneously collect randomly distributed self-reports of interruptibility. Based on these simulated sensors, we construct statistical models predicting human interruptibility and compare their predictions with the collected self-report data. The results of these models, although covering a demographically limited sample, are very promising, with the overall accuracy of several models reaching about 78%. Additionally, a model tuned to avoiding unwanted interruptions does so for 90% of its predictions, while retaining 75% overall accuracy.
Current systems often create socially awkward interruptions or unduly demand attention because they have no way of knowing if a person is busy and should not be interrupted. Previous work has examined the feasibility of using sensors and statistical models to estimate human interruptibility in an office environment, but left open some questions about the robustness of such an approach. This paper examines several dimensions of robustness in sensor-based statistical models of human interruptibility. We show that real sensors can be constructed with sufficient accuracy to drive the predictive models. We also create statistical models for a much broader group of people than was studied in prior work. Finally, we examine the effects of training data quantity on the accuracy of these models and consider tradeoffs associated with different combinations of sensors. As a whole, our analyses demonstrate that sensor-based statistical models of human interruptibility can provide robust estimates for a variety of office workers in a range of circumstances, and can do so with accuracy as good as or better than people. Integrating these models into systems could support a variety of advances in human computer interaction and computer-mediated communication.
We present methods for inferring the cost of interrupting users based on multiple streams of events including information generated by interactions with computing devices, visual and acoustical analyses, and data drawn from online calendars. Following a review of prior work on techniques for deliberating about the cost of interruption associated with notifications, we introduce methods for learning models from data that can be used to compute the expected cost of interruption for a user. We describe the Interruption Workbench, a set of event-capture and modeling tools. Finally, we review experiments that characterize the accuracy of the models for predicting interruption cost and discuss research directions.
Although cell phones are extremely useful, they can be annoying and distracting to owners and others nearby. We describe sensing techniques intended to help make mobile phones more polite and less distracting. For example, our phone's ringing quiets as soon as the user responds to an incoming call, and the ring mutes if the user glances at the caller ID and decides not to answer. We also eliminate the need to press a TALK button to answer an incoming call by recognizing if the user picks up the phone and listens to it.
For wearable computing applications, human activity is a central part of the user's context. In order to avoid user annoyance it should be acquired automatically using body-worn sensors. We propose to use multiple acceleration sensors that are distributed over the body, because they are lightweight, small and cheap. Furthermore activity can best be measured where it occurs. We present a hardware platform that we developed for the investigation of this issue and results as to where to place the sensors and how to extract the context information.
suggest some challenges for this emerging research community. The paramount challenge of notification is preventing unwanted distraction to the primary task, while still delivering information in an accurate and timely manner. In many cases, very little distraction can be tolerated. For example, a typical in-vehicle information system may notify the user about navigation instructions, incoming communications, and other information secondary from the main task of the user---driving the car. Such systems should be designed to ensure notification is provided without diverting attention from driving-related tasks. In other cases, a user is willing to accept some distraction in exchange for valued information. Desktop computer users may perform daily word-processing tasks while casually alert Why is the attentive user interface paradigm important for humancomputer interaction? The human attention system is so sensitive to various methods of notification that traditi
Software development is a highly abstract process that requires intense concentration. The authors show that interrupting this process can significantly reduce a developer's efficiency and can even contribute to project delays
Self-report data are ubiquitous in behavioral and medical research. Retrospective assessment strategies are prone to recall bias and distortion. New techniques for assessing immediate experiences in respondents' natural environments (e.g., Ecological Momentary Assessment, Experience Sampling) are being used by many researchers to reduce reporting bias. This article discusses seven aspects of momentary research that are often overlooked or minimized in the presentation of momentary research reports, yet that are critical to the success of the research: (a) the rationale for the momentary sampling design, (b) the details of momentary sampling procedures, (c) the data acquisition interface, (d) rates of compliance with the sampling plan, (e) the procedures used to train and monitor participants, (f) data management procedures, and (g) the data analytic approach. Attention to these areas in both the design and reporting of momentary research studies will not only improve momentary research protocols but also allow for the successful replication of research findings by other investigators.
Many CSCW projects dealing with individual availability and interruption filtering achieve only limited success. Perhaps this is because designers of such systems have limited evidence to draw upon; most data on interruption management is at least a decade old. This study uses an empirical sampling method and qualitative interviews to examine attitudes toward availability and interruption. Specifically, we analyze how corporate research managers spend their time and look at how their attitudes toward interruption relate to their various activities. Attitudes toward interruption are marked by a complex tension between wanting to avoid interruption and appreciating its usefulness. We conclude by discussing the implications of these findings for design, suggesting that the notion of socially translucent systems may be a fruitful approach.
This paper describes a study that probes the cost of interrupting users with instant messages during different "phases" of a computing task. We found that interrupting users during the "evaluation phase" of the task resulted in significantly longer completion times than interruptions in other phases. We also found that interruptions that were irrelevant to the task resulted in longer times to process the message and longer task resumption times than relevant messages. These initial results have implications for the principled design of intelligent interrupters and instant messages.
People have cognitive limitations that make them sensitive to interruption. These limitations can cause people to make serious mistakes when they are interrupted. Unfortunately, interruption of people is a side effect of systems that allow users to delegate tasks to active background processes, like intelligent software agents. Delegation carries the costs of supervision, and that often includes being interrupted by subordinates. User interfaces for these kinds of computer systems must be designed to accommodate people's limitations relative to being interrupted. A theory-based taxonomy of human interruption was used to identify the four known methods for deciding when to interrupt people. An experiment was conducted with 36 subjects to compare these four different design approaches within a common context. The results show important differences between the four user interface design solutions to the problem of interrupting people in human-computer interaction (HCI). KEYWORDS interrup...
A variety of legacy and emerging health applications are designed to monitor sensed information about a person's physiological signals over time. Such applications include systems for tracking heart conditions that have been in use by cardiologists for decades to recent prototypes for monitoring elders in cognitive decline. This thesis focuses on addressing the challenges inherent in an interactive monitoring system - how and when to interact with a user. This research aims to improve these systems in two main ways: 1) explore how to interact through social-emotional, relational dialogue, and 2) explore when to interact by adjusting the timing of these interruptions. An interactive, health application has been developed for data collection, annotation, and feedback that is part of a longer-term research plan for gathering data to understand more about stress, the physiological signals involved in its expression, and the interplay between stress and interruptibility. The system has been developed on a mobile platform and uses affect and interruption-sensitive strategies to engage users and allow for real-time annotation of stress, activity and timing information through text and audio input. The platform supports continuous, wireless, and non-intrusive collection of heart signal data, accelerometer, and pedometer information, as well as automatic labeling of location information from context beacons. This system is the first of its kind to be affect and interruption-responsive - to use physiological data to adjust the timing of interruptions, as well as to adaptively respond with dialogue and relational strategies that specifically address the user's stress levels and the disruption the device may be incurring upon the user.
This experiment examined the influence of the style of thecomputer user interface on the extent to which performance wasdecreased by task interruption. Performance on data entry of apersonnel database was compared using two different styles ofinterfaces, under three different forms of interruption. Tensubjects were given a graphical user interface with a mouse andscreen buttons; and ten were given a character-based interface withtab and function keys. Interruptions came at unexpected times. Eachsubject was interrupted three times, once with each form ofinterruption: telephone call, visitor, and on-screen message.During each interruption, the subject was asked to answer a simplequestion. The number of fields entered minus the number of fieldsin error, per minute, was measured during the two-minutepost-interruption period. Eye motion data was analyzed in terms ofthe amount of time and the average durations of eye fixations onthe screen. An exit questionnaire solicited the subjects'perceptions of their performance.Compared to the control, the screen interruption was disruptive,the telephone interruption was not, and the walk-in visitorinterruption produced intermediate performance. Performance wasworse in the first minute than in the second minute immediatelyfollowing the interruptions. Lower performance was directly relatedto looking more at the screen. The duration of the interruption wasweakly correlated to the performance. Subjects' perceptions oftheir performance did not correlate with performance measures.The disruptive effect of the screen interruption compared to thelack of disruption after the telephone interruption was a surprise.This result is contrary to the commonly held belief that telephoneinterruptions are a problem to computer users. It may be importantto avoid disruption caused by screen windows popping up in designof multi-window user interfaces. Trends in office automation havethe screen being used for more and more functions, with demands onscreen displays being further aggravated by information availablethrough integrated networks.The disruption of the screen interruption may be due to theabruptness of the on-set of the interruption which prohibited thesubject from completing the current action. The message box lockedout the main task until the subject responded to the question. Thisdiffered from the onset of the walk-in and telephone interruptionswhich did not lock out interaction with the main task. Anotherpossible explanation for the disruption of the screen interruptioncompared to the telephone interruption may be the similarity insense modality. This work is seen as preliminary to furtherstudy.
Two multi-session experiments are described in which a complex problem-solving task was interrupted at different stages of practice. In Experiment 1, subjects practiced the main problem-solving task for three sessions, with intermittent interruptions during each session. By the end of Session 3, interruptions which were similar to the main task, in terms of type of material processed and processing demands, no longer disrupted performance as they had in Sessions 1 and 2. In Experiment 2, subjects practiced the same problem-solving task for two sessions without interruptions. The same types of interruptions used in Experiment 1 were introduced in Session 3. Although the main task was well learned by the third session, the interruptions disrupted subjects' main-task accuracies dramatically. These results suggest that training tasks under uninterrupted conditions can lead to excellent performance, but may not allow subjects to develop the kinds of strategies needed to flexibly recover from interruptions when they occur.
We present MITes (MIT Environmental Sensors): a portable kit of ubiquitous wireless sensing devices for real-time data collection of human activities in natural settings. The sensors designed to be used in two ways: (1) determining people's interaction with objects in the environment, and (2) measuring acceleration on different parts of the body. The sensors have been designed to permit low-cost research studies where data is acquired simultaneously from hundreds of objects in an environment and multiple parts of the body.
Classic work on interruptions by Zeigarnik showed that tasks that were interrupted were more likely to be recalled after a delay than tasks that were not interrupted. Much of the literature on interruptions has been devoted to examining this effect, although more recently interruptions have been used to choose between competing designs for interfaces to complex devices. However, none of this work looks at what makes some interruptions disruptive and some not. This series of experiments uses a novel computer-based adventure-game methodology to investigate the effects of the length of the interruption, the similarity of the interruption to the main task, and the complexity of processing demanded by the interruption. It is concluded that subjects make use of some form of non-articulatory memory which is not affected by the length of the interruption. It is affected by processing similar material however, and by a complex mentalarithmetic task which makes large demands on working memory.
This paper describes a distributed, multi-sensor system architecture designed to provide a wearable computer with a wide range
of complex context information. Starting from an analysis of useful high level context information we present a top down design
that focuses on the peculiarities of wearable applications. Thus, our design devotes particular attention to sensor placement,
system partitioning as well as resource requirements given by the power consumption, computational intensity and communication
overhead. We describe an implementation of our architecture and initial experimental results obtained with the system.
This paper describes exploratory studies of interruption modalities and disruptiveness. Five interruption modalities were compared: heat, smell, sound, vibration, and light. Much more notable than the differences between modalities was the differences between people. We found that subjects' sensitiveness depended on their previous life exposure to the modalities. Individual differences greatly control the effect of interrupting stimuli. We show that is possible to build a multimodal adaptive interruption interface, such interfaces would dynamically select the output interruption modality to use based on its effectiveness on a particular user.
In this work, algorithms are developed and evaluated to de- tect physical activities from data acquired using five small biaxial ac- celerometers worn simultaneously on different parts of the body. Ac- celeration data was collected from 20 subjects without researcher su- pervision or observation. Subjects were asked to perform a sequence of everyday tasks but not told specifically where or how to do them. Mean, energy, frequency-domain entropy, and correlation of acceleration data was calculated and several classifiers using these features were tested. De- cision tree classifiers showed the best performance recognizing everyday activities with an overall accuracy rate of 84%. The results show that although some activities are recognized well with subject-independent training data, others appear to require subject-specific training data. The results suggest that multiple accelerometers aid in recognition because conjunctions in acceleration feature values can effectively discriminate many activities. With just two biaxial accelerometers - thigh and wrist - the recognition performance dropped only slightly. This is the first work to investigate performance of recognition algorithms with multiple, wire-free accelerometers on 20 activities using datasets annotated by the subjects themselves.
A variety of legacy and emerging health applications are designed to monitor sensed information about a person's physiological signals over time. Such applications include systems for tracking heart conditions that have been in use by cardiologists for decades to recent prototypes for monitoring elders in cognitive decline. This thesis focuses on addressing the challenges inherent in an interactive monitoring system - how and when to interact with a user. This research aims to improve these systems in two main ways: 1) explore how to interact through social-emotional, relational dialogue, and 2) explore when to interact by adjusting the timing of these interruptions. An interactive, health application has been developed for data collection, annotation, and feedback that is part of a longer-term research plan for gathering data to understand more about stress, the physiological signals involved in its expression, and the interplay between stress and interruptibility. The system has been developed on a mobile platform and uses affect and interruption-sensitive strategies to engage users and allow for real-time annotation of stress, activity and timing information through text and audio input. The platform supports continuous, wireless, and non-intrusive collection of heart signal data, accelerometer, and pedometer information, as well as automatic labeling of location information from context beacons. This system is the first of its kind to be affect and interruption-responsive - to use physiological data to adjust the timing of interruptions, as well as to adaptively respond with dialogue and relational strategies that specifically address the user's stress levels and the disruption the device may be incurring upon the user. (Cont.) The system has been evaluated with seven subjects who used either the responsive or non- responsive system for four days, then used the opposite system for another four days, and finally, were asked to choose which system to continue interacting with for the last four days. The affect and interruption responsive system was rated as significantly less stressful on users, and five out of seven of the subjects chose to continue working with the responsive system. This study has demonstrated that designing platforms that are relational and responsive to a person's affect can facilitate a less frustrating and more enjoyable experience over time, even in tasks that are highly disruptive. Overall, this thesis has contributed not only a new system for gathering annotations useful for studies of stress, but also provided new insights into the value of using relational and attentional strategies in a task that involves a large number of interruptions. Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2004. Includes bibliographical references (p. 53-56).
SenSay is a context-aware mobile phone that adapts to dynamically changing environmental and physiological states. In addition to manipulating ringer volume, vibration, and phone alerts, SenSay can provide remote callers with the ability to communicate the urgency of their calls, make call suggestions to users when they are idle, and provide the caller with feedback on the current status of the SenSay user. A number of sensors including accelerometers, light, and microphones are mounted at various points on the body to provide data about the user's context. A decision module uses a set of rules to analyze the sensor data and manage a state machine composed of uninterruptible, idle, active and normal states. Results from our threshold analyses show a clear delineation can be made among several user states by examining sensor data trends. SenSay augments its contextual knowledge by tapping into applications such as electronic calendars, address books, and task lists.
In this paper we propose to use context information obtained from body-worn sensors to mediate notifications for a wearable computer In particular we introduce a model which uses two axes, namely personal and social interruptability of the user in order to decide both whether or not to notify the user and to decide which notification modality to use. Rather than to model and recognize the complete context of the user we argue that personal and social interruptability can be derived directly from various sensors by the combination of tendencies. First experimental results show the feasibility of the approach using acceleration, audio, and location sensors.
As users continue offloading more control and responsibility to
the computer, coordinating the asynchronous interactions between the
user and computer is becoming increasingly important. Without proper
coordination, an application attempting to gain the user's attention
risks interrupting the user in the midst of performing another task. To
justify why an application should avoid interrupting the user whenever
possible, we designed an experiment measuring the disruptive effect of
an interruption on a user's task performance. The experiment utilized
six Web-based task categories and two categories of interruption tasks.
The results of the experiment demonstrate that: (i) a user performs
slower on an interrupted task than a non-interrupted task, (ii) the
disruptive effect of an interruption differs as a function of the task
category, and (iii) different interruption tasks cause similar
disruptive effects on task performance. These results empirically
validate the need to better coordinate user interactions among
applications that are competing for the user's attention
Just because something is new and clever does not necessarily mean that it is good. The utility of a technology must be evaluated strictly in terms of whether it actually helps people succeed. A new technology that is useful for isolated tasks may carry an invisible and costly sideeffect like interrupting people. However, the side-effect of human interruption by machine only manifests itself in real world contexts where people normally perform several complex heterogeneous tasks in parallel. The telephone and email are examples. They are useful by themselves, but in a real work environment they also create annoying interruptions. A newer and more problematic example is the technology of semi-autonomous computer systems such as intelligent agents. These systems can be assigned to do useful things in the background while their human users work on other tasks. However, delegating a task requires supervising a task; and whenever an intelligent agent must initiate an interaction with its us...
The majority of reminder systems are inflexible; reminders are issued at static, prespecified times. To be effective, cognitive orthotics should reason about what reminders should be issued and when. This paper describes the personalized cognitive orthotic (PCO), a system that uses plan-based reasoning to attain flexibility. PCO relies on local search techniques to generate high-quality reminder plans based on knowledge of the user's plans and her typical behavior. PCO is being developed in concert with other technologies aimed at improved plan management, including systems that update a user's plans and track action execution. We describe the PCO as it is implemented in the Nursebot application: where it provides timely and relevant reminders to elderly people who have cognitive decline that necessitates assistance in managing their daily activities.
: We describe a study on the influence of instant messaging (IM) on ongoing computing tasks. The study both replicates and extends earlier work on the cost of sending notifications at different times and the sensitivity of different tasks to interruption. We investigate alternative hypotheses about the nature of disruption for a list evaluation task, an activity identified as being particularly costly to interrupt. Our findings once again show the generally disruptive effects of IM, especially during fast, stimulus-driven search tasks. In addition, we show that interruptions coming early during a search task are more likely to result in the user forgetting the primary task goal than interruptions that arrive later on. These findings have implications for the design of user interfaces and notification policies that minimize the disruptiveness of notifications. Keywords: Notifications, user study, interruptions, information overload, divided attention 1 Introduction With the adv...
Predicting M. Czerwinski. Scope: providing awareness of multiple notifications at a glance
- S E Hudson
- J Fogarty
- C G Atkeson
- D Avrahami
- J Forlizzi
- S Kiesler
- J C Lee
- J Yang
S.E. Hudson, J. Fogarty, C.G. Atkeson, D. Avrahami, J. Forlizzi, S. Kiesler, J.C. Lee, and J. Yang. Predicting M. Czerwinski. Scope: providing awareness of multiple notifications at a glance. In Proceedings of AVI 2002, ACM Conference on Advanced Visual Interfaces. ACM Press, 2002.