Article

Smartphone Interruptibility Using Density-Weighted Uncertainty Sampling with Reinforcement Learning

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

We present the In-Context application for smart-phones, which combines signal processing, active learning, and reinforcement learning to autonomously create a personalized model of interruptibility for incoming phone calls. We empirically evaluate the system, and show that we can obtain an average of 96.12% classification accuracy when predicting interruptibility after a week of training. In contrast to previous work, we leverage density-weighted uncertainty sampling combined with a reinforcement learning framework applied to passively collected data to achieve comparable or superior classification accuracy using many fewer queries issued to the user.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... In [30], the authors use KNN classifier while building their recommendation system. Fisher et al. [31] also use KNN technique in their smartphone interruptibility analysis. ...
... Fogarty et al. [35] use Naive Bayes classifier for predicting human interruptibility with sensors. Fisher et al. [31] use this classifier in their smartphone interruptibility analysis. Recently, Sarker et al. [36] propose a robust prediction model based on Naive Bayes classifier for context-aware mobile services. ...
... Fogarty et al. [35] use SVM classifier for predicting human interruptibility with sensors. Fisher et al. [31] use this technique in their smartphone interruptibility analysis. Turner et al. [37,38] also use SVM classifier in their interruptibility prediction and management analysis. ...
Article
Full-text available
Due to the increasing popularity of recent advanced features and context-awareness in smart mobile phones, the contextual data relevant to users’ diverse activities with their phones are recorded through the device logs. Modeling and predicting individual’s smartphone usage based on contexts, such as temporal, spatial, or social information, can be used to build various context-aware personalized systems. In order to intelligently assist them, a machine learning classifier based usage prediction model for individual users’ is the key. Thus, we aim to analyze the effectiveness of various machine learning classification models for predicting personalized usage utilizing individual’s phone log data. In our context-aware analysis, we first employ ten classic and well-known machine learning classification techniques, such as ZeroR, Naive Bayes, Decision Tree, Random Forest, Support Vector Machine, K-Nearest Neighbors, Adaptive Boosting, Repeated Incremental Pruning to Produce Error Reduction, Ripple Down Rule Learner, and Logistic Regression classifiers. We also present the empirical evaluations of Artificial Neural Network based classification model, which is frequently used in deep learning and make comparative analysis in our context-aware study. The effectiveness of these classifier based context-aware models is examined by conducting a range of experi-ments on the real mobile phone datasets collected from individual users. The overall experimental results and discussions can help both the researchers and applications developers to design and build intelligent context-aware systems for smartphone users.
... In a short lab and field study with software developers, a combination of HR, HRV, EDA, and EEG sensors has been used to predict interruptibility [88]. Furthermore, accelerometer data has been used in several studies to detect physical activity and to show that interruptions are better delivered during moments recognized as activity transitions, e.g. when walking to another location [30,18,45]. A further and not yet fully studied factor of interruptibility is sleep, which has been shown to have a big impact on productivity and mood [68,80,49]. ...
... Biometric Sensors. Based on prior research as well as invasiveness, we chose to use two biometric sensors for our field study: the Polar H7 for recording HR and HRV data, which both have been linked to stress and cognitive load by previous research [1,28], and the Fitbit Charge 2 for recording HR (sampled every 10s), physical activity (sampled every 1min), and sleep (duration and quality metrics), which have been linked to interruptibility [30,18,45] and productivity [68,80]. ...
... Fitbit + Polar + Computer Monitoring 79% 76% 79% 74% 69% 72% 74% 85% 86% 78% 74% 72% 62% 75.3% _int _int -comp_int Interruptibility Prediction Accuracy (2 States, Individual Models) Table 3: Prediction results using different sensors and combinations thereof per participant and averaged over all (the darker the color the higher the accuracy). 37], as well as with the features related to being idle, calendar entries and physical movement, e.g. when changing location and coming back from a meeting [30,18,45,73]. Participants further mentioned that their interruptibility depends on internal states such as sleepiness (85%), focus (77%), mood (46%), challenge (38%), productivity (38%), stress (38%), health (23%), and engagement (15%). ...
Conference Paper
Full-text available
Knowledge workers experience many interruptions during their work day. Especially when they happen at inopportune moments, interruptions can incur high costs, cause time loss and frustration. Knowing a person's interruptibility allows optimizing the timing of interruptions and minimize disruption. Recent advances in technology provide the opportunity to collect a wide variety of data on knowledge workers to predict interruptibility. While prior work predominantly examined interruptibility based on a single data type and in short lab studies, we conducted a two-week field study with 13 professional software developers to investigate a variety of computer interaction, heart-, sleep-, and physical activity-related data. Our analysis shows that computer interaction data is more accurate in predicting interruptibility at the computer than biometric data (74.8% vs. 68.3% accuracy), and that combining both yields the best results (75.7% accuracy). We discuss our findings and their practical applicability also in light of collected qualitative data.
... In general, studies typically use a single channel for interruptions. This ranges from audio recordings (e.g., [13,12]); to messaging communications (e.g., instant messaging [46,16] or email [27]); to tasks in other PC application windows (e.g., [14]); to phone calls (e.g., [11,55]); and to smartphone notifications (e.g., [43,47,51]). In reality, our daily lives involve multiple devices that can interact with us in more than one way. ...
... The objective for a study dictates to what extent different priorities are considered for making or assessing interruptions. For example, some papers have considered classifying all moments as either interruptible or not (e.g., [11,47,43]), while others have considered finding breakpoints within or between activities for interruptions to occur (e.g., [26,58,25]). There are also instances with specific focus, such as predicting the timeliness instant messages being read (e.g., [46]). ...
... Context Smartphone Sensors: e.g., hardware sensors [57,32,46,47,53,43,11,50] or software APIs [34,57,46,55,11,8,50] Physiological sensors: e.g., physical state [32,53] or activity [53,30,19] Environmental sensors: e.g., sound or motion in a room [41,13,24,6,20,21] or car [32] Software events: e.g., active windows, keyboard and mouse activity [25,26,29,41,13,58,39,18,6,20,22,21,14] Calendar schedules [56,57,20] Temporal logs: e.g., of user actions [29,34,46,21] Spatial logs: e.g., GPS [56,57,55,53,11,50] or connections to antennas [41,47,55,43,22] Latent Self reports: e.g., experience sampling [10,42,41,59,53,43,23,26,30,19] or post-experiment surveys [1,16,28] Qualitative feedback: e.g., post-interviews [10,23] Third party observer reports: e.g., in situ observation [28] or video annotations [24,30,12] Physiological sensors: e.g., mental state or workload [35,2,53,4] interruptible, or whether these cases should be classed separately. ...
Conference Paper
Full-text available
When should a machine attempt to communicate with a user? This is a historical problem that has been studied since the rise of personal computing. More recently, the emergence of pervasive technologies such as the smartphone have extended the problem to be ever-present in our daily lives, opening up new opportunities for context awareness through data collection and reasoning. Complementary to this there has been increasing interest in techniques to intelligently synchronise interruptions with human behaviour and cognition. However, it is increasingly challenging to categorise new developments, which are often scenario specific or scope a problem with particular unique features. In this paper we present a meta-analysis of this area, decomposing and comparing historical and recent works that seek to understand and predict how users will perceive and respond to interruptions. In doing so we identify research gaps, questions and opportunities that characterise this important emerging field for pervasive technology.
... For instance, a smoking urge may occur outside of office setting, where most of the above used sensors (e.g., video, keystrokes, etc.) may not be available. In addition, the approach of probing users at regular intervals to gauge their interruptibility may not indicate their true availability due to subjective biases as pointed out in [12]. ...
... The first work to use an objective metric was [12] that conducted a week-long study with 5 users. It collected the moments when users changed their ring tones themselves and also in response to a prompt generated every 2 hours. ...
... First, [42] recruited volunteers without any compensation. Other works in the area of interruptibility also either used no compensation [12,42] or a fixed compensation [20,22,31,32] for participation. Micro-incentives are now being used in scientific studies to achieve better compliance with protocols [36]. ...
Conference Paper
Full-text available
Wearable wireless sensors for health monitoring are enabling the design and delivery of just-in-time interventions (JITI). Critical to the success of JITI is to time its delivery so that the user is available to be engaged. We take a first step in modeling users' availability by analyzing 2,064 hours of physiological sensor data and 2,717 self-reports collected from 30 participants in a week-long field study. We use delay in responding to a prompt to objectively measure availability. We compute 99 features and identify 30 as most discriminating to train a machine learning model for predicting availability. We find that location, affect, activity type, stress, time, and day of the week, play significant roles in predicting availability. We find that users are least available at work and during driving, and most available when walking outside. Our model finally achieves an accuracy of 74.7% in 10-fold cross-validation and 77.9% with leave-one-subject-out.
... Siworiek et al. [61] used a hip mounted accelerometer and ambient noise microphone to determine when a person could be interrupted for a phone call and Fogarty et al. used external sensors placed in-situ in the office environment and Naive Bayes classifiers to predict office worker interruptibility [3,25,26,29]. Variations on this work have been performed by Chen and Vertegaal [16] using personal sensors to profile cognitive availability, by Fisher and Simmons [24] using reinforcement learning on smartphone data to profile interruptibility for incoming phone calls, and by Goyal & Fussell [27] and Züger et al. [74] who both used biometric sensors to predict knowledge worker's interruptibility. As sensors become more widespread and large amounts of data easier to collect, machine-learning based approaches are becoming more accurate at predicting interruptibility. ...
... Although there is a reduction in the interruption window, the average interrupt time is still reasonable for many kinds of informational interactions that may take tens of seconds. Additionally, the number of opportunities (24) for the driving period remains sufficiently large for engagement throughout the drive. Table 6. ...
Article
Full-text available
This paper examines sensor fusion techniques for modeling opportunities for proactive speech-based in-car interfaces. We leverage the Is Now a Good Time (INAGT) dataset, which consists of automotive, physiological, and visual data collected from drivers who self-annotated responses to the question "Is now a good time?," indicating the opportunity to receive non-driving information during a 50-minute drive. We augment this original driver-annotated data with third-party annotations of perceived safety, in order to explore potential driver overconfidence. We show that fusing automotive, physiological, and visual data allows us to predict driver labels of availability, achieving an 0.874 F1-score by extracting statistically relevant features and training with our proposed deep neural network, PazNet. Using the same data and network, we achieve an 0.891 F1-score for predicting third-party labeled safe moments. We train these models to avoid false positives---determinations that it is a good time to interrupt when it is not---since false positives may cause driver distraction or service deactivation by the driver. Our analyses show that conservative models still leave many moments for interaction and show that most inopportune moments are short. This work lays a foundation for using sensor fusion models to predict when proactive speech systems should engage with drivers.
... While in some tasks, personalized models have been shown to outperform a general model, this is not always the case [8]. Further, we investigate the data requirements associated with training accurate personalized models of attentiveness as different tasks have shown different data requirements [6,15]. Our results show that on average, with seven days of training data, the personalized model outperforms the general model, confirming that a personalized model can capture a user's messaging behavior in varying contexts more accurately. ...
... It has been shown that an individuals' characteristics such as demographics are associated with different smartphone usage patterns [1]. Previously, personalized models for tasks like call-availability prediction [6] and interruptibility prediction [15] have been shown to outperform generic models. Capitalizing on this prior trend of research, we investigated the potential gain of a personalized model of prediction based on users' own prior data in comparison to prediction using all available data from the population. ...
Conference Paper
Full-text available
Delays in response to mobile messages can cause negative emotions in message senders and can affect an individual's social relationships. Recipients, too, feel a pressure to respond even during inopportune moments. A messaging assistant which could respond with relevant contextual information on behalf of individuals while they are unavailable might reduce the pressure to respond immediately and help put the sender at ease. By modelling attentiveness to messaging, we aim to (1) predict instances when a user is not able to attend to an incoming message within reasonable time and (2) identify what contextual factors can explain the user's attentiveness---or lack thereof---to messaging. In this work, we investigate two approaches to modelling attentiveness: a general approach in which data from a group of users is combined to form a single model for all users; and a personalized approach, in which an individual model is created for each user. Evaluating both models, we observed that on average, with just seven days of training data, the personalized model can outperform the generalized model in terms of both accuracy and F-measure for predicting inattentiveness. Further, we observed that in majority of cases, the messaging patterns identified by the attentiveness models varied widely across users. For example, the top feature in the generalized model appeared in the top five features for only 41% of the individual personalized models.
... One possible way to resolve this dilemma is to start with a general model and then to use machine learning techniques to reduce the amount of personalized data required to achieve a high performance. That said, in our study, collecting each individual participant's data took, on average, 30 minutes per activity, and the time and efort to collect personalized data may be further reduced through techniques such as user clustering (e.g., [90]) and active learning (e.g., [18]). Developers can consider the trade-of between cost and performance based on their own needs. ...
... Another approach is to adapt notifications that users receive to the situation. While a classic study has investigated this idea by adding additional information about the call to the generic 'ring' of the phone (Milewski, 2006), recent applications have employed user preferences and machine learning to automatically detect and silence unwanted calls based on the devices' sensors and usage data (De Russis & Monge Roffarello, 2017;Fisher & Simmons, 2011;Oh, Jalali, & Jain, 2015;Schulze & Groh, 2014, 2016Smith, Lavygina, Ma, Russo, & Dulay, 2014;. ...
Article
Full-text available
Notifications are one of the core functionalities of smartphones. Previous research suggests they can be a major disruption to the professional and private lives of users. This paper presents evidence from a mixed-methods study using first-person wearable video cameras, comprising 200 h of audio-visual first-person, and self-confrontation interview footage with 1130 unique smartphone interactions (N = 37 users), to situate and analyse the disruptiveness of notifications in real-world contexts. We show how smartphone interactions are driven by a complex set of routines and habits users develop over time. We furthermore observe that while the duration of interactions varies, the intervals between interactions remain largely invariant across different activity and location contexts, and for being alone or in the company of others. Importantly, we find that 89% of smartphone interactions are initiated by users, not by notifications. Overall this suggests that the disruptiveness of smartphones is rooted within learned user behaviours, not devices.
... In [30], the authors use the k-nearest neighbor classifier while designing their recommendation system. Similarly, the naive Bayes classification technique is used in Ayu et al. [31], Fisher et al. [32] in their analysis. To analyze contextual mobile phone data, Pielot et al. [33], Bedogni et al. [27], Bayat et al. [34] have used support vector machines in their context-aware analysis to build context-aware models. ...
Article
Full-text available
Abstract Nowadays, machine learning classification techniques have been successfully used while building data-driven intelligent predictive systems in various application areas including smartphone apps. For an effective context-aware system, context pre-modeling is considered as a key issue and task, as the representation of contextual data directly influences the predictive models. This paper mainly explores the role of major context pre-modeling tasks, such as context vectorization by defining a good numerical measure through transformation and normalization, context generation and extraction by creating new brand principal components, context selection by taking into account a subset of original contexts according to their correlations, and eventually context evaluation, to build effective context-aware predictive models utilizing multi-dimensional contextual data. For creating models, various popular machine learning classification techniques such as decision tree, random forest, k-nearest neighbor, support vector machines, naive Bayes classifier, and deep learning by constructing a neural network of multiple hidden layers, are used in our study. Based on the context pre-modeling tasks and classification methods, we experimentally analyze user-centric smartphone usage behavioral activities utilizing their contextual datasets. The effectiveness of these machine learning context-aware models is examined by considering prediction accuracy, in terms of precision, recall, f-score, and ROC values, and has been made an empirical discussion in various dimensions within the scope of our study.
... According to R. Fisher and R. Simmons, "Smartphone Interruptibility Using Density-Weighted Uncertainty Sampling with Reinforcement Learning," [8], introduced the In-Context smartphone application, which used an amalgamation of signal processing, active learning, and supervise machine learning to engender a personalized policy for transmuting user's ringtone autonomously. This application leverages a smartphone's GPS, accelerometer, microphone, proximity sensor, and computing power to identify homogeneous contexts and act according to the user's observed historical predilections. ...
... Sensor data is used to either construct the participants' context, or to trigger notifications based on the occurrence of a specified event. For the most commonly used sensor, this for example includes inferring interruptibility based on location [103], sending a questionnaire at a specified location [297], or distributing participant efforts over a geographic area [210]. ...
Thesis
Full-text available
The widespread availability of technologically-advanced mobile devices has brought researchers the opportunity to observe human life in day-to-day circumstances. Rather than studying human behaviour through extensive surveys or in artificial laboratory situations, this research instrument allows us to systematically capture human life in naturalistic settings. Mobile devices can capture two distinct data streams. First, the data from sensors embedded within these devices can be appropriated to construct the context of study participants. Second, participants can be asked to actively and repeatedly provide data on phenomena which cannot be reliably collected using the aforementioned sensor streams. This method is known as Experience Sampling. Researchers employing this method ask participants to provide observations multiple times per day, across a range of contexts, and to reflect on current rather than past experiences. This approach brings a number of advantages over existing methods, such as the ability to observe shifts in participant experiences over time and context, and reducing reliance on the participant’s ability to accurately recall past events. As the onus of data collection lies with participants rather researchers, there is a firm reliance on the reliability of participant contributions. While previous work has focused on increasing the number of participant contributions, the quality of these contributions has remained relatively unexplored. This thesis focuses on improving the quality and quantity of participant data collected through mobile Experience Sampling. Assessing and subsequently improving the quality of participant responses is a crucial step towards increasing the reliability of this increasingly popular data collection method. Previous recommendations for researchers are based primarily on anecdotal evidence or personal experience in running Experience Sampling studies. While such insights are valuable, it is challenging to replicate these recommendations and quantify their effect. Furthermore, we evaluate the application of this method in light of recent developments in mobile devices. The opportunities and challenges introduced by smartphone-based Experience Sampling studies remain underexplored in the current literature. Such devices can be utilised to infer participants’ context and optimise questionnaire scheduling and presentation to increase data quality and quantity. By deploying our studies on these devices, we explore the opportunities of mobile sensing and interaction in the context of mobile Experience Sampling studies. Our findings illustrate the feasibility of assessing and quantifying participant accuracy through the use of peer assessment, ground truth questions, and the assessment of cognitive skills. We empirically evaluate these approaches across a variety of study goals. Furthermore, our results provide recommendations on study design, motivation and data collection practices, and appropriate analysis techniques of participant data concerning response accuracy. Researchers can use our findings to increase the reliability of their data, to collect participant responses more evenly across different contexts in order to reduce the potential for bias, and to increase the total number of collected responses. The goal of this thesis is to improve the collection of human-labelled data in ESM studies, thereby strengthening the role of smartphones as valuable scientific instruments. Our work reveals a clear opportunity in the combination of human and sensor data sensing techniques for researchers interested in studying human behaviour in situ.
... The advent of smartphones equipped with a wide variety of sensors has made it possible for interruptibility to be modeled on-line and in-situ. Fisher and Simmons used reinforcement learning on smartphones to create a personalized model of interruptibility for incoming phone calls [18]. Similarly, Anguita et al. used the inertial sensors on smartphones to perform human activity recognition using a Support Vector Machine [3]. ...
Conference Paper
Full-text available
Advances in automotive sensing systems and speech interfaces provide new opportunities for smarter driving assistants or infotainment systems. For both safety and consumer satisfaction reasons, any new system which interacts with drivers must do so at appropriate times. We asked 63 drivers, ”Is now a good time?” to receive non-driving information during a 50-minute drive. We analyzed 2,734 responses and synchronized automotive and video data, and show that while the chances of choosing a good time can be determined with better success using easily accessible automotive data, certain nuances in the problem require a richer understanding of the driver and environment states in order to achieve higher performance. We illustrate several of these nuances with quantitative and qualitative analyses to contribute to the understanding of how to design a system that might simultaneously minimize the risk of interacting at a bad tim
... 5 . It is worth to note that not only physical activities [46,77,99] or interactions [35,91], but also more expressive concepts such as complex activities [54], engagement levels [84], even personal traits [64,119] are frequently used in attention management systems. To infer such contextual information, various machine learning algorithms e.g., J48 [77,99], K-Means [54], or Support Vector Machines [99] are typically trained and 6 . ...
Article
Full-text available
Today's information and communication devices provide always-on connectivity, instant access to an endless repository of information, and represent the most direct point of contact to almost any person in the world. Despite these advantages, devices such as smartphones or personal computers lead to the phenomenon of attention fragmentation, continuously interrupting individuals' activities and tasks with notifications. Attention management systems aim to provide active support in such scenarios, managing interruptions, for example, by postponing notifications to opportune moments for information delivery. In this article, we review attention management system research with a particular focus on ubiquitous computing environments. We first examine cognitive theories of attention and extract guidelines for practical attention management systems. Mathematical models of human attention are at the core of these systems, and in this article, we review sensing and machine learning techniques that make such models possible. We then discuss design challenges towards the implementation of such systems, and finally, we investigate future directions in this area, paving the way for new approaches and systems supporting users in their attention management.
... Sensor data is used to either construct the participants' context, or to trigger notifications based on the occurrence of a specified event. For the most commonly used sensor, this for example includes inferring interruptibility based on location (Fisher and Simmons 2011), sending a questionnaire at a specified location (Sabra et al. 2015), or distributing participant efforts over a geographic area (Linnap and Rice 2014). ...
Article
Full-text available
The Experience Sampling Method (ESM) is used by scientists from various disciplines to gather insights into the intra-psychic elements of human life. Researchers have used the ESM in a wide variety of studies, with the method seeing increased popularity. Mobile technologies have enabled new possibilities for the use of the ESM, while simultaneously leading to new conceptual, methodological, and technological challenges. In this survey, we provide an overview of the history of the ESM, usage of this methodology in the computer science discipline, as well as its evolution over time. Next, we identify and discuss important considerations for ESM studies on mobile devices, and analyse the particular methodological parameters scientists should consider in their study design. We reflect on the existing tools that support the ESM methodology and discuss the future development of such tools. Finally, we discuss the effect of future technological developments on the use of the ESM and identify areas requiring further investigation.
... An activity-aware system utilizes knowledge of the current activity being performed, in addition to other contextual information such as time and location, to adapt the system and its related services. Identifying current activities providers a rich source of information for systems that can adapt to each user's behavioral patterns [47]. In addition to the current activity label, Thyme utilizes eight additional features in order to learn user-sensitive times to prompt for activity labels. ...
Conference Paper
Full-text available
Smartphone prompts and notifications are popular because they provide users with timely and important information. However, they can also be an annoyance if they pop up at inopportune times and interrupt important tasks. In this paper we introduce Thyme, an intelligent notification front end that uses activity recognition and machine learning to identify the best times to prompt smartphone users. We evaluate performance of an activity-aware prompting approach based on 47 participants with fixed time and Thyme-based prompts. Our results show that responsiveness improves from 12.8% to 93.2% using this intelligent approach to timing of smartphone-based prompts.
... Smith et al. [71] considered dataset imbalance, error costs, user behaviors to recognize disruptive incoming calls, and developed RingLearn [70] to mitigate disruptive phone calls. Fisher et al. [20] built an in-context application for smartphone to create personalized interruptibility prediction model for phone calls. Although they achieved high prediction accuracy (96.12%), similar to other works [66,71,70], their model only predicts phone's ringer modes (on and off), which is a rough measurement of interruptibility. ...
Conference Paper
Smartphones frequently notify users about newly available messages or other notifications. It can be very disruptive when these notifications interrupt users while they are busy. Our work here is based on the observation that people usually exhibit different levels of busyness at different contexts. This means that classifying users' interruptibility as a binary status, interruptible or not interruptible, is not sufficient to accurately measure their availability towards smartphone interruptions. In this paper, we propose, implement and evaluate a two-stage hierarchical model to predict people's interruptibility intensity. Our work is the first to introduce personality traits into interruptibility prediction model, and we found that personality data improves the prediction significantly. Our model bootstraps the prediction with similar people's data, and provides a good initial prediction for users whose individual models have not been trained on their own data yet. Overall prediction accuracy of our model can reach 66.1%.
... Diverse approaches have been proposed to facilitate intelligent interruption for the smartphone [4], across phone calls [5,6] and notifications [7]. This has included determining the influence of contextual factors [8,9], exploring methods of labelling interruptibility [10,11,12] and training predictive models [13,7]. However, interruption is challenging to observe in isolation because interrupting the user to ask how interruptible they are is itself an interruption. ...
Article
Full-text available
Smartphone notifications frequently interrupt our daily lives, often at inopportune moments. We propose the decision-on-information-gain model, which extends the existing data collection convention to capture a range of interruptibility behaviour implicitly. Through a six-month in-the-wild study of 11,346 notifications, we find that this approach captures up to 125% more interruptibility cases. Secondly, we find different correlating contextual features for different behaviour using the approach and find that predictive models can be built with >80% precision for most users. However we note discrepancies in performance across labelling, training, and evaluation methods, creating design considerations for future systems.
... Activity Engagement: Activity engagement represents a combination of two relevant contexts: social and cognitive [14,16]. In this work, the user's cognitive context, frequently considered in the ...
Conference Paper
This paper presents an organized set of variables that can aid intelligent privacy agents in predicting the best and necessary moments to interrupt users in order to give them control and awareness over their privacy, avoiding information overload or over choice.
... Smith et al. [71] considered dataset imbalance, error costs, user behaviors to recognize disruptive incoming calls, and developed RingLearn [70] to mitigate disruptive phone calls. Fisher et al. [20] built an in-context application for smartphone to create personalized interruptibility prediction model for phone calls. Although they achieved high prediction accuracy (96.12%), similar to other works [66,71,70], their model only predicts phone's ringer modes (on and off), which is a rough measurement of interruptibility. ...
Conference Paper
Full-text available
Digital currencies represent a new method for exchange -- a payment method with no physical form, made real by the Internet. This new type of currency was created to ease online transactions and to provide greater convenience in making payments. However, a critical component of a monetary system is the people who use it. Acknowledging this, we present results of our interview study (N=20) with two groups of participants (users and non-users) about how they perceive the most popular digital currency, Bitcoin. Our results reveal: non-users mistakenly believe they are incapable of using Bitcoin, users are not well-versed in how the protocol functions, they have misconceptions about the privacy of transactions, and that Bitcoin satisfies properties of ideal payment systems as defined by our participants. Our results illustrate Bitcoin's tradeoffs, its uses, and barriers to entry.
... In recent years, increased instrumentation in consumer technology has driven new research into contextual prediction across a variety of domains, from mobile robots to consumer smartphones and sensor equipped power wheelchairs. Some of this work has leveraged decision-theoretic prediction algorithms [4], [5], while other work has formulated these tasks as a sequential decision making process, leveraging spectral latent variable models [6]. ...
... Approaches have also been demonstrated that mitigate the interruption caused by phone calls by silencing the notification but leaving the visual alert. The Ringlearn system [9] learned preferences for when to silence phone call notifications in different settings whilst the In-Context system [3] learnt when to silence phone call notifications taking account of different levels of user interruptibility. ...
Conference Paper
Full-text available
Smartphones are capable of alerting their users to different kinds of digital interruption using different modalities and with varying modulation. Smart notification is the capability of a smartphone for selecting the user's preferred kind of alert in particular situations using the full vocabulary of notification modalities and modulations. It therefore goes well beyond attempts to predict if or when to silence a ringing phone call. We demonstrate smart notification for messages received from a document retrieval system while the user is attending a meeting. The notification manager learns about their notification preferences from users' judgements about videos of meetings. It takes account of the relevance of the interruption to the meeting, whether the user is busy and the sensed location of the smartphone. Through repeated training, the notification manager learns to reliably predict the preferred notification modes for users and this learning continues to improve with use.
... It is also unclear if the integrity of smartphone sensor data relies on the phone being in a fixed position (i.e., the person keeping the phone in the same position all day). Although recent studies have attempted to circumvent these issues, solutions appear largely experimental or prototypical (30,31). Importantly, there have been few studies in the transportation literature that have included accelerometer data to improve prediction of transportation mode. ...
Article
Full-text available
Background: Active travel is an important area in physical activity research, but objective measurement of active travel is still difficult. Automated methods to measure travel behaviors will improve research in this area. In this paper, we present a supervised machine learning method for transportation mode prediction from global positioning system (GPS) and accelerometer data. Methods: We collected a dataset of about 150 h of GPS and accelerometer data from two research assistants following a protocol of prescribed trips consisting of five activities: bicycling, riding in a vehicle, walking, sitting, and standing. We extracted 49 features from 1-min windows of this data. We compared the performance of several machine learning algorithms and chose a random forest algorithm to classify the transportation mode. We used a moving average output filter to smooth the output predictions over time. Results: The random forest algorithm achieved 89.8% cross-validated accuracy on this dataset. Adding the moving average filter to smooth output predictions increased the cross-validated accuracy to 91.9%. Conclusion: Machine learning methods are a viable approach for automating measurement of active travel, particularly for measuring travel activities that traditional accelerometer data processing methods misclassify, such as bicycling and vehicle travel.
... Reinforcement learning (RL) is a formal framework for optimizing a defined reward model. In this context, RL in the Markov decision process (MDP) framework has been investigated in several applications such as dialogue management [1], autonomous navigation [2], Smartphone interruptibility [3], etc. However, defining the reward model is not straightforward in practical domains. ...
Data
Full-text available
In this paper, we propose an algorithm for learn-ing a reward model from an expert policy in partially ob-servable Markov decision processes (POMDPs). The problem is formulated as inverse reinforcement learning (IRL) in the POMDP framework. The proposed algorithm then uses the expert trajectories to find an unknown reward model-based on the known POMDP model components. Similar to previous IRL work in Markov Decision Processes (MDPs), our algorithm maximizes the sum of the margin between the expert policy and the intermediate candidate policies. However, in contrast to previous work, the expert and intermediate candidate policy values are approximated using the beliefs recovered from the expert trajectories, specifically by approximating expert belief transitions. We apply our IRL algorithm to a healthcare dialogue POMDP where the POMDP model components are estimated from real dialogues. Our experimental results show that the proposed algorithm is able to learn a reward model that accounts for the the expert policy.
Article
Smartphones provide various functions supporting users in their daily lives. However, the temptation of getting distracted and tuning out is high leading to so-called rabbit holes. To quantify rabbit hole behavior, we developed an Android tracking application that collects smartphone usage enriched with experience sampling questionnaires. We analyzed 14,395 smartphone use sessions from 21 participants, collected over two weeks, showing that rabbit hole sessions are significantly longer and contain more user interaction, revealing a certain level of restlessness in use. The context of rabbit hole sessions and subjective results revealed different triggers for spending more time on the phone. Next, we conduct an expert focus group (N=6) to put the gained insights into perspective and formulate a definition of the mobile phone rabbit hole. Our results form the foundation for predicting and communicating the mobile phone rabbit hole, especially when prolonged smartphone use results in regret.
Chapter
Catching the attention of users, via their smartphone, is most often achieved through push notifications. Such an approach is already in place within applications such as Facebook, Google, Instagram, TikTok, and Twitter. Emails and messaging apps are also used to send notifications, which can assist users who may otherwise miss important events. As a result of these multiple services the user may be overwhelmed with notifications and frequently interrupted during key task. To address this concern, a mobile app has been developed for collecting different phone events and sensors data “in-the-wild”. This data which is then used as the basis for behavioural modeling to profile a user and subsequently detect the best time to send a notification. Within the mobile app, a service is running continuously in the background, collecting sensor data (such as accelerometer and gyroscope data), notification data such as notification title, and user response data relating to engagement with notifications and phone events such as foreground app name, battery percentage etc. Additionally, the app collects user cognitive state data through self-reporting. This paper analyse a dataset contributed by five users. In total, 2766 notifications were received over period of up to 10 days. 78% percent of the time, users ignored or remove notifications based on the data analysis. It is therefore a serious issue in the realm of ubiquitous computing that there is a constant bombardment of push notifications.
Chapter
Notifications received on the smartphone have become a vital part of typical work and social life. Notifications provide information ranging from important incoming business emails and meeting prompts to home deliveries and social interaction. But they can also become burden some on the user as the average number of incoming notifications can be in the region of 100 per day. To cope with this overload a user might completely disable the notifications which will result in missing important information. This paper evaluates the user challenges, methods used to manage these challenges and discusses key challenges for data collection. In particular, we assess an overlap between user behavior and situational context. This has led to the development of a hybrid model based on both behavioral and situational context to detect the best time to send the notification to the user.
Article
Notifications are an indispensable feature of mobile devices, but their delivery can interrupt and distract users. Prior work has examined interventions, such as deferring notification delivery to opportune moments, but has not systematically studied how users might prefer an intelligent system to manage their notifications. Hence, we directly probed Android smartphone users’ notification preferences via a one-week experience-sampling study ( N = 35). We found that users prefer mitigating undesired interruptions by suppressing alerts over deferring them and referred to notification content factors more frequently than contextual factors for explaining their preferences. Then we demonstrated the challenges and potentials of leveraging user actions to help predict notification preferences. Specifically, we showed that a model personalized using user actions achieved a performance gain of 39% than a generic model. This improvement is similar to the 42% performance gain using labels solicited from the user while using observable user actions causes no extra disruption.
Article
Voice assistants, such as Amazon's Alexa and Google Home, increasingly find their way into consumer homes. Their functionality, however, is currently limited to being passive answer machines rather than proactively engaging users in conversations. Speakers' proactivity would open up a range of important application scenarios, including health services, such as checking in on patient states and triggering medication reminders. It remains unclear how passive speakers should implement proactivity. To better understand user perceptions, we ran a 3-week field study with 13 participants where we modified the off-the-shelf Google Home to become proactive. During the study, our speaker proactively triggered conversations that were essentially Experience Sampling probes allowing us to identify when to engage users. Applying machine-learning, we are able to predict user responsiveness with a 71.6% accuracy and find predictive features. We also identify self-reported factors, such as boredom and mood, that are significantly correlated with users' perceived availability. Our prototype and findings inform the design of proactive speakers that verbally engage users at opportune moments and contribute to the design of proactive application scenarios and voice-based experience sampling studies.
Article
Full-text available
Artificial intelligence (AI) techniques have grown rapidly in recent years in the context of computing with smart mobile phones that typically allows the devices to function in an intelligent manner. Popular AI techniques include machine learning and deep learning methods, natural language processing, as well as knowledge representation and expert systems, can be used to make the target mobile applications intelligent and more effective. In this paper, we present a comprehensive view on "mobile data science and intelligent apps" in terms of concepts and AI-based modeling that can be used to design and develop intelligent mobile applications for the betterment of human life in their diverse day-today situation. This study also includes the concepts and insights of various AI-powered intelligent apps in several application domains, ranging from personalized recommendation to healthcare services, including COVID-19 pandemic management in recent days. Finally, we highlight several research issues and future directions relevant to our analysis in the area of mobile data science and intelligent apps. Overall, this paper aims to serve as a reference point and guidelines for the mobile application developers as well as the researchers in this domain, particularly from the technical point of view.
Article
Notifications on mobile devices punctuate our daily lives to provide information and prompt for further engagement. Investigations into the cognitive processes involved in consuming notifications are common across the literature, however most research to date investigates notifications in isolation of one another. In reality, notifications often coexist together, forming a “stack”, however the behavioural implications of this on the response towards individual notifications has received limited attention. Through an in-the-wild study of 1889 Android devices, we observe user behaviour in a stream of 30 million notifications from over 6000 applications. We find distinct strategies for user management of the notification stack within usage sessions, beyond the behaviour patterns observable from responses to individual notifications. From the analysis, we make recommendations for collecting and reporting data from mobile applications to improve validity through timely responses, and capture potential confounding features.
Conference Paper
In this paper, we show that characterizing user behaviour from location or smartphone usage traces, without accounting for the interaction of individuals in physical-world groups, can lead to erroneous results. We conducted one of the largest studies in the UbiComp domain thus far, involving indoor location traces of more than 6,000 users, collected over a 4-month period at our university campus, and further studied fine-grained App usage of a subset of 156 Android users. We apply a state-of-the-art group detection algorithm to annotate such location traces with group vs. individual context, and then show that individuals vs. groups exhibit significant differences along three behavioural traits: (1) the mobility pattern, (2) the responsiveness to calls / SMSs and (3) application usage. We show that these significant differences are robust to underlying errors in the group detection technique and that the use of such group context leads to behavioural results that differ from those reported in prior popular work.
Conference Paper
Pressure ulcers are a common and devastating condition faced by users of power wheelchairs. However, proper use of power wheelchair tilt and recline functions can alleviate pressure and reduce the risk of ulcer occurrence. In this work, we show that when using data from a sensor instrumented power wheelchair, we are able to predict with an average accuracy of 92% whether a subject will successfully complete a repositioning exercise when prompted. We present two models of compliance prediction. The first, a spectral Hidden Markov Model, uses fast, optimal optimization techniques to train a sequential classifier. The second, a decision tree using information gain, is computationally efficient and produces an output that is easy for clinicians and wheelchair users to understand. These prediction algorithms will be a key component in an intelligent reminding system that will prompt users to complete a repositioning exercise only in contexts in which the user is most likely to comply.
Conference Paper
Short term studies in controlled environments have shown that user behaviour is consistent enough to predict disruptive smartphone notifications. However, in practice, user behaviour changes over time (concept drift) and individual user preferences need to be considered. There is a lack of research on which methods are best suited for predicting disruptive smartphone notifications longer-term, taking into account varying error costs. In this paper we report on a 16 week field study comparing how well different learners perform at mitigating disruptive incoming phone calls.
Article
This paper is prompted by the overall question 'what is the most effective way to recognise disruptive smartphone interruptions?'. We design our experiments to answer 3 questions: 'Do users revise what they perceive as disruptive incoming calls as time goes by?', 'How do different types of machine-learners (lazy, eager, evolutionary, ensemble) perform on this task?' and 'Can we restrict the initial amount of data and/or the number of features we need to make predictions without degrading performance?'. We consider these questions using Cambridge University's Device Analyzer dataset.
Conference Paper
Mitigating the consequences of disruptive smartphone interruptions remains a challenging problem for smartphone designers. Proposed solutions often incorporate machine-learning techniques with remedies that include delaying user notifications until an opportune moment or changing the intensity and/or mode of the notification (fewer rings, vibration mode). This paper describes a new machine-learning approach that aims to maintain the quality of mitigation under concept drift - unforeseen changes in context or the user's behaviour over time. We demonstrate our approach by developing an application for Android phones to mitigate disruptive phone calls (RingLearn). We report on a field trial of the application conducted over 2 months with 10 users and suggest that long-term mitigation can be practical with careful design that addresses concept drift.
Article
In this paper we promote the idea of analyzing the character of a conversation that the user is engaged in to help determine her receptivity to mobile notifications. Given that most previous mobile notification management systems incorporate only general awareness of speech or just consider the mere presence of the user's voice, we argue for the use of speaker tracking to completely capture presence as well as dynamics and other characteristics of personal conversations in order to assess personal interruptibility.
Conference Paper
In this paper, we proposed an intrusion detection system for detecting anomaly on Android smartphones. The intrusion detection system continuously monitors and collects the information of smartphone under normal conditions and attack state. It extracts various features obtained from the Android system, such as the network traffic of smartphones, battery consumption, CPU usage, the amount of running processes and so on. Then, it applies Bayes Classifying Algorithm to determine whether there is an invasion. In order to further analyze the Android system abnormalities and locate malicious software, along with system state monitoring the intrusion detection system monitors the process and network flow of the smartphone. Finally, experiments on the system which was designed in this paper have been carried out. Empirical results suggest that the proposed intrusion detection system is effective in detecting anomaly on Android smartphones.
Article
Full-text available
We define voice activity detection (VAD) as a binary classification problem and solve it using the support vector machine (SVM). Challenges in SVM-based approach include selection of representative training segments, selection of features, normalization of the features, and post-processing of the frame-level decisions. We propose to construct a SVM-VAD using MFCC features because they capture the most relevant information of speech, and they are widely used in speech and speaker recognition making the proposed method easy to integrate with existing applications. Practical usability is our driving motivation: the proposed SVM-VAD should be easily adapted into new conditions.
Conference Paper
Full-text available
Experience sampling has been employed for decades to col- lect assessments of subjects' intentions, needs, and affective states. In recent years, investigators have employed auto- mated experience sampling to collect data to build predic- tive user models. To date, most procedures have relied on random sampling or simple heuristics. We perform a com- parative analysis of several automated strategies for guiding experience sampling, spanning a spectrum of sophistication, from a random sampling procedure to increasingly sophisti- cated active learning. The more sophisticated methods take a decision-theoretic approach, centering on the computation of the expected value of information of a probe, weighing the cost of the short-term disruptiveness of probes with their benefits in enhancing the long-term performance of predic- tive models. We test the different approaches in a field study, focused on the task of learning predictive models of the cost of interruption. Author Keywords
Conference Paper
Full-text available
Many applications require users to specify preferences. We support users in this task by letting them define preferences relative to their personal history or that of other users. We implement this idea using a graphical technique called control shadows, which we have implemented on both a desktop computer and on a cell phone with a small, grayscale display. An empirical study compared user performance on the graphical interface and a text table interface with identical functionality. On the desktop, users completed their tasks more quickly and effectively and strongly preferred the graphical interface. On the cell phone, there was no significant difference between the graphical and table interfaces. Finally, personal history proved useful in specifying preferences, but history of other users was not helpful.
Conference Paper
Full-text available
Inference and decision making with probabilistic user models may be infeasible on portable devices such as cell phones. We highlight the opportunity for storing and using precomputed inferences about ideal actions for future situations, based on offline learning and reasoning with the user models. As a motivating example, we focus on the use precomputation of call-handling policies for cell phones. The methods hinge on the learning of Bayesian user models for predicting whether users will attend meetings on their calendar and the cost of being interrupted by incoming calls should a meeting be attended.
Conference Paper
Full-text available
We present methods for inferring the cost of interrupting users based on multiple streams of events including information generated by interactions with computing devices, visual and acoustical analyses, and data drawn from online calendars. Following a review of prior work on techniques for deliberating about the cost of interruption associated with notifications, we introduce methods for learning models from data that can be used to compute the expected cost of interruption for a user. We describe the Interruption Workbench, a set of event-capture and modeling tools. Finally, we review experiments that characterize the accuracy of the models for predicting interruption cost and discuss research directions.
Conference Paper
Full-text available
We contribute a method for approximating users’ interruptibility costs to use for experience sampling and validate the method in an application that learns when to automatically turn off and on the phone volume to avoid embarrassing phone interruptions. We demonstrate that users have varying costs associated with interruptions which indicates the need for personalized cost approximations. We compare different experience sampling techniques to learn users’ volume preferences and show those that ask when our cost approximation is low reduce the number of embarrassing interruptions and result in more accurate volume classifiers when deployed for long-term use.
Conference Paper
Full-text available
A significant problem with telephone communication is that callers do not have enough awareness about the Personal Presence of people they want to call. The result can be unwanted, interrupting calls. Thelive addressbookis an application that helps users make more informed telephone calls and teleconferences, from anywhere, via their wireless PDA or desktop browser. Unlike other network-based address books, which maintain static information, the live addressbook can display dynamic information about where the recipient currently is (i.e., reach number), and how available he/she currently is for calls. The system accomplishes this by applying to telephony the "Buddy List" concepts made popular in Instant Messaging applications. User trials assess the applicability of Personal Presence information in a telephone context.
Conference Paper
Full-text available
Interest has been growing in opportunities to build and deploy statistical models that can infer a computer user's current interruptability from computer activity and relevant contextual information. We describe a system that intermittently asks users to assess their perceived interruptability during a training phase and that builds decision-theoretic models with the ability to predict the cost of interrupting the user. The models are used at run-time to compute the expected cost of interruptions, providing a mediator for incoming notifications, based on a consideration of a user's current and recent history of computer activity, meeting status, location, time of day, and whether a conversation is detected.
Article
Full-text available
A person seeking another person's attention is normally able to quickly assess how interruptible the other person currently is. Such assessments allow behavior that we consider natural, socially appropriate, or simply polite. This is in sharp contrast to current computer and communication systems, which are largely unaware of the social situations surrounding their usage and the impact that their actions have on these situations. If systems could model human interruptibility, they could use this information to negotiate interruptions at appropriate times, thus improving human computer interaction.This article presents a series of studies that quantitatively demonstrate that simple sensors can support the construction of models that estimate human interruptibility as well as people do. These models can be constructed without using complex sensors, such as vision-based techniques, and therefore their use in everyday office environments is both practical and affordable. Although currently based on a demographically limited sample, our results indicate a substantial opportunity for future research to validate these results over larger groups of office workers. Our results also motivate the development of systems that use these models to negotiate interruptions at socially appropriate times.
Article
Full-text available
People have cognitive limitations that make them sensitive to interruption. These limitations can cause people to make serious mistakes when they are interrupted. Unfortunately, interruption of people is a side effect of systems that allow users to delegate tasks to active background processes, like intelligent software agents. Delegation carries the costs of supervision, and that often includes being interrupted by subordinates. User interfaces for these kinds of computer systems must be designed to accommodate people's limitations relative to being interrupted. A theory-based taxonomy of human interruption was used to identify the four known methods for deciding when to interrupt people. An experiment was conducted with 36 subjects to compare these four different design approaches within a common context. The results show important differences between the four user interface design solutions to the problem of interrupting people in human-computer interaction (HCI). KEYWORDS interrup...
Article
Full-text available
The ability to cheaply train text classifiers is critical to their use in information retrieval, content analysis, natural language processing, and other tasks involving data which is partly or fully textual. An algorithm for sequential sampling during machine learning of statistical classifiers was developed and tested on a newswire text categorization task. This method, which we call uncertainty sampling, reduced by as much as 500-fold the amount of training data that would have to be manually classified to achieve a given level of effectiveness.
Article
The key idea behind active learning is that a machine learning algorithm can achieve greater accuracy with fewer labeled training instances if it is allowed to choose the data from which is learns. An active learner may ask queries in the form of unlabeled instances to be labeled by an oracle (e.g., a human annotator). Active learning is well-motivated in many modern machine learning problems, where unlabeled data may be abundant but labels are difficult, time-consuming, or expensive to obtain. This report provides a general introduction to active learning and a survey of the literature. This includes a discussion of the scenarios in which queries can be formulated, and an overview of the query strategy frameworks proposed in the literature to date. An analysis of the empirical and theoretical evidence for active learning, a summary of several problem setting variants, and a discussion of related topics in machine learning research are also presented.
Article
Mobile phones currently represent the most pervasive tech-nology for enabling social networking. They collect a wide range of so-cially meaningful data, such as address book contacts, pictures, call logs and exchanged messages. This data, however, is highly underutilized as it is managed by separate applications which are typically unaware of the user's social setting. This "unawareness" makes the user a slave of his device instead of helping him achieve his goals or manage his everyday activities. Enhancing next generation mobile phone with socially-aware features will provide significant benefits. In this paper, we present real data collected from a user study about mobile phone usage, with the aim of providing evidence for the need of socially-aware phone applications. As a relevant example of socially-aware capability, we analyze the case of interruption management, i.e., how and why users respond to an incom-ing call or message by interrupting their current activity. To automate interruption management on mobile phones, we suggest the adoption of a policy-based approach to express socially-aware policies based on Seman-tic Web technologies.
Conference Paper
The potential for sensor-enabled mobile devices to proactively present information when and where users need it ranks among the greatest promises of ubiquitous computing. Unfortunately, mobile phones, PDAs, and other computing devices that compete for the user's attention can contribute to interruption irritability and feelings of information overload. Designers of mobile computing interfaces, therefore, require strategies for minimizing the perceived interruption burden of proactively delivered messages. In this work, a context-aware mobile computing device was developed that automatically detects postural and ambulatory activity transitions in real time using wireless accelerometers. This device was used to experimentally measure the receptivity to interruptions delivered at activity transitions relative to those delivered at random times. Messages delivered at activity transitions were found to be better received, thereby suggesting a viable strategy for context-aware message delivery in sensor-enabled mobile computing devices.
Conference Paper
Active learning is well-suited to many prob- lems in natural language processing, where unlabeled data may be abundant but annota- tion is slow and expensive. This paper aims to shed light on the best active learning ap- proaches for sequence labeling tasks such as information extraction and document segmen- tation. We survey previously used query selec- tion strategies for sequence models, and pro- pose several novel algorithms to address their shortcomings. We also conduct a large-scale empirical comparison using multiple corpora, which demonstrates that our proposed meth- ods advance the state of the art.
Article
To understand the dynamics of mental health, it is essential to develop measures for the frequency and the patterning of mental processes in every-day-life situations. The Experience-Sampling Method (ESM) is an attempt to provide a valid instrument to describe variations in self-reports of mental processes. It can be used to obtain empirical data on the following types of variables: a) frequency and patterning of daily activity, social interaction, and changes in location; b) frequency, intensity, and patterning of psychological states, i.e., emotional, cognitive, and conative dimensions of experience; c) frequency and patterning of thoughts, including quality and intensity of thought disturbance. The article reviews practical and methodological issues of the ESM and presents evidence for its short- and long-term reliability when used as an instrument for assessing the variables outlined above. It also presents evidence for validity by showing correlation between ESM measures on the one hand and physiological measures, one-time psychological tests, and behavioral indices on the other. A number of studies with normal and clinical populations that have used the ESM are reviewed to demonstrate the range of issues to which the technique can be usefully applied.
Conference Paper
SenSay is a context-aware mobile phone that adapts to dynamically changing environmental and physiological states. In addition to manipulating ringer volume, vibration, and phone alerts, SenSay can provide remote callers with the ability to communicate the urgency of their calls, make call suggestions to users when they are idle, and provide the caller with feedback on the current status of the SenSay user. A number of sensors including accelerometers, light, and microphones are mounted at various points on the body to provide data about the user's context. A decision module uses a set of rules to analyze the sensor data and manage a state machine composed of uninterruptible, idle, active and normal states. Results from our threshold analyses show a clear delineation can be made among several user states by examining sensor data trends. SenSay augments its contextual knowledge by tapping into applications such as electronic calendars, address books, and task lists.
Article
Shown is an effective speech endpoint detection algorithm using a trained support vector machine (SVM) and a feature vector including contextual information speech features. With this and other innovations the proposed algorithm yields high discrimination and reports significant improvements over standard methods and algorithms defining the decision rule in terms of averaged subband speech features.
Article
A new software tool for user-interface development and assessment of ubiquitous computing applications is available for CHI researchers. The software permits researchers to use common PDA mobile computing devices for experience sampling studies. The basic tool offers options not currently available in any other open-source sampling package. However, the tool also has one a completely new type of functionality: context-aware experience sampling. This feature permits researchers to acquire feedback from users only in particular situations that are detected by sensors connected to a mobile computing device.
Article
Context-aware computing is a mobile computing paradigm in which applications can discover and take advantage of contextual information (such as user location, time of day, nearby people and devices, and user activity). Since it was proposed about a decade ago, many researchers have studied this topic and built several context-aware applications to demonstrate the usefulness of this new technology. Context-aware applications (or the system infrastructure to support them), however, have never been widely available to everyday users. In this survey of research on context-aware systems and applications, we looked in depth at the types of context used and models of context information, at systems that support collecting and disseminating context, and at applications that adapt to the changing context. Through this survey, it is clear that context-aware research is an old but rich area for research. The difficulties and possible solutions we outline serve as guidance for re...
Bayesphone: Precomputation of context-sensitive policies for inquiry and action in mobile devices
  • E Horvitz
  • P Koch
  • R Sarin
  • J Apacible
  • M Subramani
E. Horvitz, P. Koch, R. Sarin, J. Apacible, and M. Subramani. Bayesphone: Precomputation of context-sensitive policies for inquiry and action in mobile devices. User Modeling 2005, pages 251-260, 2005.
Coordinating the interruption of people in human-computer interaction
  • A Sasse
  • C Johnson
A. Sasse, C. Johnson, et al. Coordinating the interruption of people in human-computer interaction. In Human-computer interaction, INTER-ACT'99: IFIP TC. 13 International Conference on Human-Computer Interaction, 30th August-3rd September 1999, Edinburgh, UK, volume 1, page 295. IOS Press, 1999.
Towards socially aware mobile phones
  • A Toninelli
  • D Khushraj
  • O Lassila
  • R Montanari
A. Toninelli, D. Khushraj, O. Lassila, and R. Montanari. Towards socially aware mobile phones. In First Workshop on Social Data on the Web (SDoW). Citeseer, 2008.