No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Mercury: Empowering Programmers' Mobile Work Practices with Microproductivity
Abstract
There has been considerable research on how software can enhance programmers' productivity within their workspace. In this paper, we instead explore how software might help programmers make productive use of their time while away from their workspace. We interviewed 10 software engineers and surveyed 78 others and found that while programmers often do work while mobile, their existing mobile work practices are primarily exploratory (e.g., capturing thoughts or performing online research). In contrast, they want to be doing work that is more grounded in their existing code (e.g., code review or bug triage). Based on these findings, we introduce Mercury, a system that guides programmers in making progress on-the-go with auto-generated microtasks derived from their source code's current state. A study of Mercury with 20 programmers revealed that they could make meaningful progress with Mercury while mobile with little effort or attention. Our findings suggest an opportunity exists to support the continuation of programming tasks across devices and help programmers resume coding upon returning to their workspace.
... Microtask programming reduces the effort required to (1) onboard developers onto a project, including understanding the project structure and code and setting up a workspace (Steinmacher et al. 2015;Wang and Sarma 2011;Britto et al. 2020;Von Krogh et al. 2003;Fagerholm et al. 2013;Fagerholm et al. 2014). To reduce this traditionally lengthy process, developers work within a specialized preconfigured programming environment (Aghayi et al. 2021;Lasecki et al. 2015;LaToza et al. 2018;Goldman et al. 2011a;Nebeling et al. 2012;Schiller and Ernst 2012;LaToza et al. 2014;LaToza et al. 2013;Williams et al. 2019). Preconfigured environments offer an environment in which code and dependencies are already set up and developers may immediately begin editing code. ...
... Many studies have identified factors that influence developers' productivity, such as scope and schedule (Jensen 2014;Kamma and Jalote 2013;Maxwell 2003;Oliveira et al. 2017). Mercury (Williams et al. 2019) a mobile microtasking system that allows programmers to continue their work on-the-go is introduced and evaluated by a controlled experiment. Micro-producitivities in Mercury help programmers continue their work on-the-go and instill comfort in pausing work unexpectedly. ...
... In CrowdDesign, crowd workers may build a web app (Nebeling et al. 2012). In Mercury (Williams et al. 2019), a programmer works individually to continue work onthe-go. All microtasks are completed in a mobile setting. ...
In microtask programming, developers complete short self-contained microtasks through the use of a specialized programming environment. For example, given only a short description of the purpose of a function and a partially complete implementation, a developer might be asked to identify, test, and implement an additional behavior in the function. Adopting a microtask approach to programming tasks has been envisioned to offer a number of potential benefits, including reducing the onboarding time necessary for new developers to contribute to a project and achieving higher project velocity by enabling larger project teams and greater parallelism. To investigate the potential benefits and drawbacks of microtask programming we conducted a controlled experiment. We focused our investigation on the context in which microtasking is most widely used, implementing and debugging function bodies, and investigated the impact of microtasking with respect to onboarding, project velocity, code quality, and developer productivity. 28 developers worked to implement microservice endpoints, either in the form of traditional programming tasks described in an issue tracker or as programming microtasks. Our study did not examine the design effort necessary to prepare for microtask programming or consider how microtask programming might be applied to maintenance tasks. We found that, compared to traditional programming, microtask programming reduced onboarding time by a factor of 3.7, increased project velocity by a factor of 5.76 by increasing team size by a factor of 7, and decreased individual developer productivity by a factor of 1.3. The quality of code did not significantly differ. Through qualitative analysis of how developers worked, we explore potential explanations of these differences. These findings offer evidence for the potential benefits that adopting microtask programming might bring, particularly in cases where increasing project velocity is paramount.
... Systems for supporting the recovery of interruptions have been explored at depth, ranging from simple note-taking tools [99] to complex software for recording and reestablishing task history and context [22,35]. Recent systems, such as PlayWrite [43] and Mercury [103], allow users to partially avoid interruptions in information work by continuing their work in low-attention effort format called "microtasks" [96]. Research has demonstrated the practical utility of microtasks by comparing them to their macrotask counterparts, showing microtasks are more resilient to interruptions [16], yield higher quality work [16], and can be used to scaffold the cognitive process of maintaining and rebuilding context for complex tasks [14,87]. ...
... In this work, we explore crowdworkers' tooling practices by conducting an interview study aimed at understanding crowdworkers' perceptions and opinions about the role that tooling currently plays in their work. Our approach is heavily inspired by prior studies of information work [79,103] that share the goal of identifying opportunities for improving peoples' efficiency in their work [51]. ...
... Prior research has taken first steps toward supporting crowdworkers' decision-making processes by building and evaluating tools that help workers make more informed decisions about the work pursuit [33], work acceptance [86], and work completion [90], signalling next steps toward task recommendation systems that handle work far more strategically than the HIT-finding tools that crowdworkers currently rely on. Our findings speak directly to the importance of the systems' ability to surface the right type of work and the right amount of work at the right time -all of which have been studied in prior information work settings [8,45,53,103]. Per our findings, the most fruitful avenue of impact is enabling workers to further customize the cueing mechanisms (i.e., alerts and notifications) that enable them to sustain their engagement with work. ...
Crowdworkers regularly support their work with scripts, extensions, and software to enhance their productivity. Despite their evident significance, little is understood regarding how these tools affect crowdworkers' quality of life and work. In this study, we report findings from an interview study (N=21) aimed at exploring the tooling practices used by full-time crowdworkers on Amazon Mechanical Turk. Our interview data suggests that the tooling utilized by crowdworkers (1) strongly contributes to the fragmentation of microwork by enabling task switching and multitasking behavior; (2) promotes the fragmentation of crowdworkers' work-life boundaries by relying on tooling that encourages a 'work-anywhere' attitude; and (3) aids the fragmentation of social ties within worker communities through limited tooling access. Our findings have implications for building systems that unify crowdworkers' work practice in support of their productivity and well-being.
... Similar to the desktop scenario, the system adds a dotted blue border to the target content to provide feedback that the wiggling is being detected (Figure 1d0-4). Note that due to the limitations of the large size of the finger with respect to an individual word [15] as well as the unique use cases of mobile devices (e.g., quickly consuming and collecting blocks of information on the go [46,91]), Wigglite for mobile only supports selecting block-level content such as paragraphs or images. (3) Collection: As soon as the user makes at least five up-anddown motions, the system will commit to the collection by giving the target a darker blue background ( Figure 1d5). ...
... While the current implementation of Wigglite mobile application does enable users to collect information and triage it with valence as well as to review their collected information, extending it to support more complex operations such as creating and curating topics could be an interesting direction for future work. We anticipate two challenges: (1) how to reasonably leverage the limited screen real estate to design user interfaces and interactions that feel native to a mobile device while also being functionally lightweight and intuitive enough to perform, and (2) exploring the realistic role of mobile devices in the sensemaking ecosystem [46,91], i.e., striking a balance between pursuing feature parity with the Wigglite desktop counterpart and designing to specifically support practical use cases (e.g., reviewing and triaging information during a commute). ...
Consumers conducting comparison shopping, researchers making sense of competitive space, and developers looking for code snippets online all face the challenge of capturing the information they find for later use without interrupting their current flow. In addition, during many learning and exploration tasks, people need to externalize their mental context, such as estimating how urgent a topic is to follow up on, or rating a piece of evidence as a "pro" or "con," which helps scaffold subsequent deeper exploration. However, current approaches incur a high cost, often requiring users to select, copy, context switch, paste, and annotate information in a separate document without offering specific affordances that capture their mental context. In this work, we explore a new interaction technique called "wiggling," which can be used to fluidly collect, organize, and rate information during early sensemaking stages with a single gesture. Wiggling involves rapid back-and-forth movements of a pointer or up-and-down scrolling on a smartphone, which can indicate the information to be collected and its valence, using a single, light-weight gesture that does not interfere with other interactions that are already available. Through implementation and user evaluation, we found that wiggling helped participants accurately collect information and encode their mental context with a 58% reduction in operational cost while being 24% faster compared to a common baseline.
... On the other end of the spectrum, researchers also commonly recruit developers on an international level for remote studies, using a variety of platforms. One advantage of recruiting locally is that researchers can be fairly certain that most of their sample actually has a computer science/programming background (e.g., [1], [2], [3], [4], [5], [6]). Unfortunately, finding enough willing participants locally can be difficult; Naiakshina et al. [2] reported that they could only get 40 out of 1600 CS students to take part in a study where the compensation was 100 euros. ...
... Thirty-two participants could correctly select all the even binary numbers (Q9). Further, only 30 participants selected all the valid hexadecimal numbers (Q10), while 17 failed or selected "I don't know" (5). ...
Recruiting professional programmers in sufficient numbers for research studies can be challenging because they often cannot spare the time, or due to their geographical distribution and potentially the cost involved. Online platforms such as Clickworker or Qualtrics do provide options to recruit participants with programming skill; however, misunderstandings and fraud can be an issue. This can result in participants without programming skill taking part in studies and surveys. If these participants are not detected, they can cause detrimental noise in the survey data. In this paper, we develop screener questions that are easy and quick to answer for people with programming skill but difficult to answer correctly for those without. In order to evaluate our questionnaire for efficacy and efficiency, we recruited several batches of participants with and without programming skill and tested the questions. In our batch 42% of Clickworkers stating that they have programming skill did not meet our criteria and we would recommend filtering these from studies. We also evaluated the questions in an adversarial setting. We conclude with a set of recommended questions which researchers can use to recruit participants with programming skill from online platforms.
... In a collaborative teamwork setting, MixTAPE [40] helps generate action plans for design teams by converting call notes and client briefs into prioritized, assigned tasks using a synthesis algorithm and machine learning models. While not directly supporting action planning, many other works, particularly in the crowdsourced field, focus on decomposing a large task into manageable micro-tasks that help people make progress with limited time and resources [7,46,49]. ...
Everyone aspires to achieve personal goals. However, getting started is often complex and daunting, especially for large projects. AI has the potential to create plans and help jumpstart progress, but it often lacks sufficient personal context to be useful. We introduce JumpStarter, a system that uses AI-powered context curation to create action plans and draft personalized working solutions. JumpStarter assists users by posing questions to elicit relevant context, breaking down goals into manageable steps, and selecting appropriate context to draft working solutions for each step. A technical evaluation indicates that context curation results in plans and working solutions of higher quality. A user study demonstrates that compared to ChatGPT, JumpStarter significantly reduces users' mental load and increases their efficiency in kickstarting personal projects. We discuss the design implications of AI-powered context curation to facilitate the use of generative AI in complex problem-solving.
... Some researchers distributed the effort needed to curate resources by inserting microtasks when users are interrupted [43]. Inserting such microtasks was helpful for task resumption as well [49]. Capturing context and transferring resources across devices can be challenging, as workers may use a variety of devices for a task [41]. ...
Modern knowledge workers typically need to use multiple resources, such as documents, web pages, and applications, at the same time. This complexity in their computing environments forces workers to restore various resources in the course of their work. However, conventional curation methods like bookmarks, recent document histories, and file systems place limitations on effective retrieval. Such features typically work only for resources of one type within one application, ignoring the interdependency between resources needed for a single task. In addition, text-based handles do not provide rich cues for users to recognize their associated resources. Hence, the need to locate and reopen relevant resources can significantly hinder knowledge workers' productivity. To address these issues, we designed and developed Scrapbook, a novel application for digital resource curation across applications that uses screenshot-based bookmarks. Scrapbook extracts and stores all the metadata (URL, file location, and application name) of windows visible in a captured screenshot to facilitate restoring them later. A week-long field study indicated that screenshot-based bookmarks helped participants curate digital resources. Additionally, participants reported that multimodal -- visual and textual -- data helped them recall past computer activities and reconstruct working contexts efficiently.
... There is an increasing interest in how people use different devices for their work-related tasks [e.g., 9,14,16,35]. Martelaro et al. [24] show that in-car assistants can help users to be more productive while commuting, yet in easy, non-distracting traffic scenarios. ...
The role of conversational assistants has become more prevalent in helping people increase their productivity. Document-centered assistance, for example to help an individual quickly review a document, has seen less significant progress, even though it has the potential to tremendously increase a user's productivity. This type of document-centered assistance is the focus of this paper. Our contributions are three-fold: (1) We first present a survey to understand the space of document-centered assistance and the capabilities people expect in this scenario. (2) We investigate the types of queries that users will pose while seeking assistance with documents, and show that document-centered questions form the majority of these queries. (3) We present a set of initial machine learned models that show that (a) we can accurately detect document-centered questions, and (b) we can build reasonably accurate models for answering such questions. These positive results are encouraging, and suggest that even greater results may be attained with continued study of this interesting and novel problem space. Our findings have implications for the design of intelligent systems to support task completion via natural interactions with documents.
From automated customer support to virtual assistants, conversational agents have transformed everyday interactions, yet despite phenomenal progress, no agent exists for programming tasks. To understand the design space of such an agent, we prototyped PairBuddy—an interactive pair programming partner—based on research from conversational agents, software engineering, education, human-robot interactions, psychology, and artificial intelligence. We iterated PairBuddy’s design using a series of Wizard-of-Oz studies. Our pilot study of six programmers showed promising results and provided insights toward PairBuddy’s interface design. Our second study of 14 programmers was positively praised across all skill levels. PairBuddy’s active application of soft skills—adaptability, motivation, and social presence—as a navigator increased participants’ confidence and trust, while its technical skills—code contributions, just-in-time feedback, and creativity support—as a driver helped participants realize their own solutions. PairBuddy takes the first step towards an Alexa-like programming partner.
Information systems, such as task management applications and digital assistants, can help people keep track of tasks of different types and different time durations, ranging from a few minutes to days or weeks. Helping people better manage their tasks and their time are core capabilities of assistive technologies, situated within a broader context of supporting more effective information access and use. Throughout the course of a day, there are typically many short time periods of downtime (e.g., five minutes or less) available to individuals. Microtasks are simple tasks that can be tackled in such short amounts of time. Identifying microtasks in task lists could help people utilize these periods of low activity to make progress on their task backlog. We define actionable tasks as self-contained tasks that need to be completed or acted on. However, not all to-do tasks are actionable. Many task lists are collections of miscellaneous items that can be completed at any time (e.g., books to read, movies to watch), notes (e.g., names, addresses), or the individual items are constituents in a list that is itself a task (e.g., a grocery list). In this article, we introduce the novel challenge of microtask detection, and we present machine-learned models for automatically determining which tasks are actionable and which of these actionable tasks are microtasks. Experiments show that our models can accurately identify actionable tasks, accurately detect actionable microtasks, and that we can combine these models to generate a solution that scales microtask detection to all tasks. We discuss our findings in detail, along with their limitations. These findings have implications for the design of systems to help people make the most of their time.
Many software development organizations strive to enhance the productivity of their developers. All too often, efforts aimed at improving developer productivity are undertaken without knowledge about how developers spend their time at work and how it influences their own perception of productivity. To fill in this gap, we deployed a monitoring application at 20 computers of professional software developers from four companies for an average of 11 full workdays in situ. Corroborating earlier findings, we found that developers spend their time on a wide variety of activities and switch regularly between them, resulting in highly fragmented work. Our findings extend beyond existing research in that we correlate developers’ work habits with perceived productivity and also show productivity is a personal matter. Although productivity is personal, developers can be roughly grouped into morning, low-at-lunch and afternoon people. A stepwise linear regression per participant revealed that more user input is most often associated with a positive, and emails, planned meetings and work unrelated websites with a negative perception of productivity. We discuss opportunities of our findings, the potential to predict high and low productivity and suggest design approaches to create better tool support for planning developers’ workdays and improving their personal productivity.
As people possess increasing numbers of information devices, situations where several devices are combined and used together have become more common. We present a user study on people's current practices in combining multiple information devices in their everyday lives, ranging from pragmatic tasks to leisure activities. Based on diaries and interviews of 14 participants, we characterize the usage practices of the most common devices, including smartphones, computers, tablets, and home media centers. We analyze 123 real-life multi-device use cases and identify the main usage patterns, including Sequential Use, Resource Lending, Related Parallel Use, and Unrelated Parallel Use. We discuss the practical challenges of using several information devices together. Finally, we identify three levels of decisions that determine which devices are used in a particular situation, including acquiring, making available, and selecting the devices for use.
Research has shown that recovery processes in general and psychological detachment in particular are important for work engagement. We argue that work engagement additionally benefits from reattachment to work in the morning (i.e., mentally reconnecting to work before actually starting to work) and that the gains derived from psychological detachment and reattachment are stronger in the morning than in the afternoon. We tested our hypotheses in a daily diary study with a sample of 167 employees who completed 2 surveys per day over the period of 2 workweeks. Hierarchical linear modeling showed that work engagement was higher in the morning than in the afternoon. Evening psychological detachment and morning reattachment positively predicted work engagement throughout the day. The association between reattachment and work engagement was stronger in the morning than in the afternoon. This study demonstrates that not only psychological detachment from work during leisure time, but also reattachment to work when coming back to work are crucial for daily engagement at work. (PsycINFO Database Record
Mobile devices are becoming the prevalent computing platform for most people. TouchDevelop is a new mobile development environment that enables anyone with a Windows Phone to create new apps directly on the smartphone, without a PC or a traditional keyboard. At the core is a new mobile programming language and editor that was designed with the touchscreen as the only input device in mind. Programs written in TouchDevelop can leverage all phone sensors such as GPS, cameras, accelerometer, gyroscope, and stored personal data such as contacts, songs, pictures. Thousands of programs have already been written and published with TouchDevelop.
Crowd development is a development process designed for transient workers of varying skill. Work is organized into microtasks, which are short, self-descriptive, and modular. Microtasks recursively spawn microtasks and are matched to workers, who accrue points reflecting value created. Crowd development might help to reduce time to market and software development costs, increase programmer productivity, and make programming more fun.
Paid crowd work offers remarkable opportunities for improving productivity, social mobility, and the global economy by engaging a geographically distributed workforce to complete complex tasks on demand and at scale. But it is also possible that crowd work will fail to achieve its potential, focusing on assembly-line piecework. Can we foresee a future crowd workplace in which we would want our children to participate? This paper frames the major challenges that stand in the way of this goal. Drawing on theory from organizational behavior and distributed computing, as well as direct feedback from workers, we outline a framework that will enable crowd work that is complex, collaborative, and sustainable. The framework lays out research challenges in twelve major areas: workflow, task assignment, hierarchy, real-time response, synchronous collaboration, quality control, crowds guiding AIs, AIs guiding crowds, platforms, job design, reputation, and motivation.
In recent studies of the structure of affect, positive and negative affect have consistently emerged as two dominant and relatively independent dimensions. A number of mood scales have been created to measure these factors; however, many existing measures are inadequate, showing low reliability or poor convergent or discriminant validity. To fill the need for reliable and valid Positive Affect and Negative Affect scales that are also brief and easy to administer, we developed two 10-item mood scales that comprise the Positive and Negative Affect Schedule (PANAS). The scales are shown to be highly internally consistent, largely uncorrelated, and stable at appropriate levels over a 2-month time period. Normative data and factorial and external evidence of convergent and discriminant validity for the scales are also presented. (PsycINFO Database Record (c) 2010 APA, all rights reserved)
The world is experiencing a technology shift. In 2011, more touchscreen-based mobile devices like smartphones and tablets will be sold than desktops, laptops, and netbooks combined. In fact, in many cases incredibly powerful and easy-to-use smart phones are going to be the first and, in less developed countries, possibly the only computing devices which virtually all people will own, and carry with them at all times. Furthermore, mobile devices do not only have touchscreens, but they are also equipped with a multitude of sensors, such as location information and acceleration, and they are always connected to the cloud. TouchDevelop is a novel application creation environment for anyone to script their smartphones anywhere -- you do not need a separate PC. TouchDevelop allows you to develop mobile device applications that can access your data, your media, your sensors and allows using cloud services including storage, computing, and social networks. TouchDevelop targets students, and hobbyists, not necessarily the professional developer. Typical TouchDevelop applications are written for fun, or for personalizing the phone.
TouchDevelop's typed, structured programming language is built around the idea of only using a touchscreen as the input device to author code. It has built-in primitives which make it easy to access the rich sensor data available on a mobile device. In our vision, the state of the program is automatically distributed between mobile clients and the cloud, with automatic synchronization of data and execution between clients and cloud, liberating the programmer from worrying (or even having to know about) the details. We report on our experience with our first prototype implementation for the Windows Phone 7 platform, which already realizes a large portion of our vision. It is available on the Windows Phone Marketplace.
We offer an introduction to contextual design as an emergent method for building effective systems. Contextual design addresses a number of the inadequacies in previous methods by emphasizing: interview methods conducted in the context of the user's work, codesigning with the user, building an understanding of work in context, and summarizing conclusions through out the research. We contrast this design method to usability engineering and artifact examination.
The impact of interruptions on workflow and productivity has been extensively studied in the PC domain, but while fragmented user attention is recognized as an inherent aspect of mobile phone usage, little formal evidence exists of its effect on mobile productivity. Using a survey and a screenshot-based diary study we investigated the types of barriers people face when performing tasks on their mobile phones, the ways they follow up with such suspended tasks, and how frustrating the experience of task disruption is for mobile users. From 386 situated samples provided by 12 iPhone and 12 Pocket PC users, we distill a classification of barriers to the completion of mobile tasks. Our data suggest that moving to a PC to complete a phone task is common, yet not inherently problematic, depending on the task. Finally, we relate our findings to prior design guidelines for desktop workflow, and discuss how the guidelines can be extended to mitigate disruptions to mobile taskflow.
The rapid increase in smart phone capabilities has introduced new opportunities for mobile information access and computing. However, smart phone use may still be constrained by both device affordances and work environments. To understand how current business users employ smart phones and to identify opportunities for improving business smart phone use, we conducted two studies of actual and perceived performance of standard work tasks. Our studies involved 243 smart phone users from a large corporation. We intentionally chose users who primarily work with desktops and laptops, as these "non-mobile" users represent the largest population of business users. Our results go beyond the general intuition that smart phones are better for consuming than producing information: we provide concrete measurements that show how fast reading is on phones and how much slower and more effortful text entry is on phones than on computers. We also demonstrate that security mechanisms are a significant barrier to wider business smart phone use. We offer design suggestions to overcome these barriers.
In this paper, we explore whether sharing a user's web browsing activity across their computing devices can make it easier to find and access web sites on a mobile device. We first surveyed 175 smartphone users about their web use across multiple devices. We found that users shared web information between devices, but generally used cumbersome manual methods to do so. In a second study, we tracked the web sites visited by 14 participants on their PC and mobile phone, and used experience-sampling surveys to determine whether sharing sites across devices would be useful. We found that participants visited many of the same sites on both their mobile device and PC, and that participants were interested in viewing additional sites from their PC on their mobile device. Our results suggest that automatically sharing web activity information between devices has potential to improve the usability of the mobile web.
Designing interfaces or applications that move beyond the bounds of a single device screen enables new ways to engage with digital content. Research addressing the opportunities and challenges of interactions with multiple devices in concert is of continued focus in HCI research. To inform the future research agenda of this field, we contribute an analysis and taxonomy of a corpus of 510 papers in the cross-device computing domain. For both new and experienced researchers in the field we provide: an overview, historic trends and unified terminology of cross-device research; discussion of major and under-explored application areas; mapping of enabling technologies; synthesis of key interaction techniques spanning across multiple devices; and review of common evaluation strategies. We close with a discussion of open issues. Our taxonomy aims to create a unified terminology and common understanding for researchers in order to facilitate and stimulate future cross-device research.
While having a step-by-step breakdown for a task-an action plan-helps people complete tasks, prior work has shown that people prefer not to make action plans for their own tasks. Getting planning support from others could be beneficial, but it is limited by how much domain knowledge people have about the task and how available they are. Our goal is to incorporate the benefits of having action plans in the complex domain of writing, while mitigating the time and effort costs of creating plans. To mitigate these costs, we introduce a vocabulary-a finite set of functions pertaining to writing tasks-as a cognitive scaffold that enables people with necessary context (e.g. collaborators) to generate action plans for others. We develop this vocabulary by analyzing 264 comments, and compare plans created using it with those created without any aid, in an online study with 768 comments (N=145) and a lab study with 96 comments (N=8). We show that using a vocabulary reduces planning time and effort and improves plan quality compared to unstructured planning, and opens the door for automation and task sharing for complex tasks.
Mobile devices offer people the opportunity to get useful tasks done during time previously thought to be unusable. Because mobile devices have small screens and are often used in divided attention scenarios, people are limited to using them for short, simple tasks; complex tasks like editing a document present significant challenges in this environment. In this paper we demonstrate how a complex task requiring focused attention can be adapted to the fragmented way people work while mobile by decomposing the task into smaller, simpler microtasks. We introduce Play Write, a microproductivity tool that allows people to edit Word documents from their phones via such microtasks. When participants used Play Write while simultaneously watching a video, we found that they strongly preferred its microtask-based editing approach to the traditional editing experience offered by Mobile Word. Play Write made participants feel more productive and less stressed, and they completed more edits with it. Our findings suggest microproductivity tools like Play Write can help people be productive in divided attention scenarios.
Research Methods in Human-Computer Interaction is a comprehensive guide to performing research and is essential reading for both quantitative and qualitative methods. Since the first edition was published in 2009, the book has been adopted for use at leading universities around the world, including Harvard University, Carnegie-Mellon University, the University of Washington, the University of Toronto, HiOA (Norway), KTH (Sweden), Tel Aviv University (Israel), and many others. Chapters cover a broad range of topics relevant to the collection and analysis of HCI data, going beyond experimental design and surveys, to cover ethnography, diaries, physiological measurements, case studies, crowdsourcing, and other essential elements in the well-informed HCI researcher's toolkit. Continual technological evolution has led to an explosion of new techniques and a need for this updated 2nd edition, to reflect the most recent research in the field and newer trends in research methodology. This Research Methods in HCI revision contains updates throughout, including more detail on statistical tests, coding qualitative data, and data collection via mobile devices and sensors. Other new material covers performing research with children, older adults, and people with cognitive impairments.
Software developers typically use multiple large screens in their office setup. However, they often work away from the office where such a setup is not available, instead only working with a laptop computer and drastically reduced screen real estate. We explore how developers can be better supported in ad-hoc scenarios, for example when they work in a cafe, an airport, or at a client's site. We present insights into current work practices and challenges when working away from the usual office desk sourced from a survey of professional software developers. Based on these insights, we introduce an IDE that makes use of additional personal devices, such as a phone or a tablet. Parts of the IDE can be offloaded to these mobile devices, for example the application that is being developed, a debugging console or navigational elements. A qualitative evaluation with professional software developers showed that they appreciate the increased screen real estate.
Research has shown that productivity is mediated by an individual's ability to detach from their work at the end of the day and reattach with it when they return the next day. In this paper we explore the extent to which structured dialogues, focused on individuals' work-related tasks or emotions, can help them with the detachment and reattachment processes. Our inquiry is driven with SwitchBot, a conversational bot which engages with workers at the start and end of their work day. After preliminarily validating the design of a detachment and reattachment dialogue frame-work with 108 crowdworkers, we study SwitchBot's use in-situ for 14 days with 34 information workers. We find that workers send fewer e-mails after work hours and spend a larger percentage of their first hour at work using productivity applications than they normally would when using SwitchBot. Further, we find that productivity gains were better sustained when conversations focused on work-related emotions. Our results suggest that conversational bots can be effective tools for aiding workplace detachment and reattachment and help people make successful use of their time on and off the job.
Traditional forms of crowdsourcing such as open source software development harness crowd contributions to democratize the creation of software. However, potential contributors must first overcome joining barriers forcing casually committed contributors to spend days or weeks onboarding and thereby reducing participation. To more effectively harness potential contributions from the crowd, we propose a method for programming in which work occurs entirely through microtasks, offering contributors short, self-contained tasks such as implementing part of a function or updating a call site invoking a function to match a change made to the function. In microtask programming, microtasks involve changes to a single artifact, are automatically generated as necessary by the system, and nurture quality through iteration. A study examining the feasibility of microtask programming to create small programs found that developers were able to complete 1008 microtasks, onboard and submit their first microtask in less than 15 minutes, complete all types of microtasks in less than 5 minutes on average, and create 490 lines of code and 149 unit tests. The results demonstrate the potential feasibility as well as revealing a number of important challenges to address to successfully scale microtask programming to larger and more complex programs. IEEE
Multi-device product designers need tools to better address ecologically valid constraints in naturalistic settings early in their design process. To address this need, we created a reusable design kit of scenarios, "hint" cards, and a framework that codifies insights from prior work and our own field study. We named the kit the Moving Context Kit, or McKit for short, because it helps designers focus on context shifts that we found to be highly influential in everyday multi-device use. Specifically, we distilled the following findings from our field study in the McKit: (1) devices are typically specialized into one of six roles during parallel use notifier, broadcaster, collector, gamer, remote, and hub, and (2) device roles are influenced by context shifts between private and shared situations. Through a workshop, we validated that the McKit enables designers to engage with complex user needs, situations, and relationships when incorporating novel multi-device techniques into the products they envision.
This paper introduces flash organizations: crowds structured like organizations to achieve complex and open-ended goals. Microtask workflows, the dominant crowdsourcing structures today, only enable goals that are so simple and modular that their path can be entirely pre-defined. We present a system that organizes crowd workers into computationally-represented structures inspired by those used in organizations - roles, teams, and hierarchies - which support emergent and adaptive coordination toward open-ended goals. Our system introduces two technical contributions: 1) encoding the crowd's division of labor into de-individualized roles, much as movie crews or disaster response teams use roles to support coordination between on-demand workers who have not worked together before; and 2) reconfiguring these structures through a model inspired by version control, enabling continuous adaptation of the work and the division of labor. We report a deployment in which flash organizations successfully carried out open-ended and complex goals previously out of reach for crowdsourcing, including product design, software development, and game production. This research demonstrates digitally networked organizations that flexibly assemble and reassemble themselves from a globally distributed online workforce to accomplish complex work.
Several recent studies have highlighted the need to support parallel usage of multiple devices for cross-device use. Yet, most interfaces today are still designed for single-device use and require re-authoring to enable cross-device interaction. This paper presents two studies to inform the design of a new web browser with support for semi-automatic generation of cross-device interfaces. Based on the results of a recent study in which users manually customized web pages for cross-device use, our first study elicits from users how they might want to trigger popular cross-device patterns to transform single-device designs with relatively little effort. Our second study then examines how the emerging design patterns could be applied to the Alexa top 50 sites from 10 different genres. Based on these studies, we design semi-automatic techniques for page segmentation and distribution between multiple devices that can work on many existing web sites and require only minimal user input to switch between different cross-device designs. Finally, we discuss possible extensions to the Chrome web browser to make the techniques available for a wide range of desktop, mobile, and wearable devices, and successfully test them on popular web sites.
Although information workers may complain about meetings, they are an essential part of their work life. Consequently, busy people spend a significant amount of time scheduling meetings. We present Calendar.help, a system that provides fast, efficient scheduling through structured workflows. Users interact with the system via email, delegating their scheduling needs to the system as if it were a human personal assistant. Common scheduling scenarios are broken down using well-defined workflows and completed as a series of microtasks that are automated when possible and executed by a human otherwise. Unusual scenarios fall back to a trained human assistant who executes them as unstructured macrotasks. We describe the iterative approach we used to develop Calendar.help, and share the lessons learned from scheduling thousands of meetings during a year of real-world deployments. Our findings provide insight into how complex information tasks can be broken down into repeatable components that can be executed efficiently to improve productivity.
Crowdsourcing has become an increasingly popular means to acquire human input on demand. Microtask crowdsourcing market-places facilitate the access to millions of people (called workers) who are willing to participate in tasks in return for monetary rewards or other forms of compensation. This paradigm presents a unique learning context where workers have to learn to complete tasks on-the-fly by applying their learning immediately through the course of tasks. However, most workers typically dropout early in large batches of tasks, depriving themselves of the opportunity to learn on-the-fly through the course of batch completion. By doing so workers squander a potential chance at improving their performance and completing tasks effectively. In this paper, we propose a novel method to engage and retain workers, to improve their learning in crowdsourced information finding tasks by using achievement priming. Through rigorous experimental findings, we show that it is possible to retain workers in long batches of tasks by triggering their inherent motivation to achieve and excel. As a consequence of increased worker retention, we find that workers learn to perform more effectively, depicting relatively more stable accuracy and lower task completion times in comparison to workers who drop out early.
Crowd work is typically limited to simple, context-free tasks because they are easy to describe and understand. In contrast, complex tasks require communication between the requester and workers to achieve mutual understanding, which can be more work than it is worth. This paper explores the notion of structured communication: using structured microtasks to support communication in the domain of complex writing. Our studies compare a variety of communication mechanisms with respect to the costs to the requester in providing information and the value of that information to workers while performing the task. We find that different mechanisms are effective at different stages of writing. For early drafts, asking the requester to state the biggest problem in the current write-up is valuable and low cost, while later it is more useful for the worker if the requester highlights the text that needs to be improved. These findings can be used to enable richer, more interactive crowd work than what currently seems possible. We incorporate the findings in a workflow for crowdsourcing written content using appropriately timed mechanisms for communicating with the crowd.
Programming work is inherently interdependent, requiring developers to share and coordinate decisions that crosscut the structure of code. This is particularly challenging for programming in a microtasking context, in which developers are assumed to be transient and thus cannot rely on traditional learning and coordination mechanisms such as an extended onboarding process and code ownership. In this paper, we explore scaffolding coordination and knowledge sharing through a question and answer system, structuring project knowledge and coordination into questions and answers. To investigate its potential for enabling coordination in a microtask setting, we implemented a Q&A system for microtask programming work and conducted a user study where a crowd used it to coordinate their work on a software project over a 30-hour period. The results reveal both the potential for the use of Q&A systems for within project coordination and challenges that this approach brings.
To inform the design of cross-device applications, it is important to know what devices users typically have within reach in various settings in the home, in the workplace and on the move. We report on a survey carried out using an online questionnaire and discuss the most significant findings that should be taken into account in cross-device scenarios. The results are based on 293 participants covering a wide range of ages, nationalities, professions and living arrangements.
As the number of computing devices available to users continues to grow, personal computing increasingly involves using multiple devices together. However, support for multi-device interactions has fallen behind users' desire to leverage the diverse capabilities of the devices that surround them. In this paper, we report on an interview study of 29 designers and developers in which we investigate the barriers to creating useful, usable, and delightful multi-device experiences. We uncovered three key challenges: 1) the difficulty in designing the interactions between devices, 2) the complexity of adapting interfaces to different platform UI standards, and 3) the lack of tools and methods for testing multi-device user experiences. We discuss the technological and business factors behind these challenges and potential ways to lower the barriers they impose.
Microtasks are small units of work designed to be completed individually, eventually contributing to a larger goal. Although microtasks can be performed in isolation, in practice people often complete a chain of microtasks within a single session. Through a series of crowd-based studies, we look at how various microtasks can be chained together to improve efficiency and minimize mental demand, focusing on the writing domain. We find that participants completed low-complexity microtasks faster when they were preceded by the same type of microtask, whereas they found high-complexity microtasks less mentally demanding when pre-ceded by microtasks on the same content. Furthermore, participants were faster at starting high-complexity microtasks after completing lower-complexity microtasks, but completion time and quality were not affected. These findings provide insight into how microtasks can be ordered to optimize transitions from one microtask to another.
This paper presents the MicroWriter, a system that decomposes the task of writing into three types of microtasks to produce a single report: 1) generating ideas, 2) labeling ideas to organize them, and 3) writing paragraphs given a few related ideas. Because each microtask can be completed individually with limited awareness of what has been already done and what others are doing, this decomposition can change the experience of collaborative writing. Prior work has used microtasking to support collaborative writing with unaffiliated crowd workers. To instead study its impact on collaboration among writers with context and investment in the writing project, we asked six groups of co-workers (or 19 people in total) to use the MicroWriter in a synchronous, collocated setting to write a report about a shared work goal. Our observations suggest ways that recent advances in microtasking and crowd work can be used to support collaborative writing within preexisting groups.
There is a significant gap in the body of research on cross-device interfaces. Research has largely focused on enabling them technically, but when and how users want to use cross-device interfaces is not well understood. This paper presents an exploratory user study with XDBrowser, a cross-device web browser we are developing to enable non-technical users to adapt existing single-device web interfaces for cross-device use while viewing them in the browser. We demonstrate that an end-user customization tool like XDBrowser is a powerful means to conduct user-driven elicitation studies useful for understanding user preferences and design requirements for cross-device interfaces. Our study with 15 participants elicited 144 desirable multi-device designs for five popular web interfaces when using two mobile devices in parallel. We describe the design space in this context, the usage scenarios targeted by users, the strategies used for designing cross-device interfaces, and seven concrete mobile multi-device design patterns that emerged. We discuss the method, compare the cross-device interfaces from our users and those defined by developers in prior work, and establish new requirements from observed user behavior. In particular, we identify the need to easily switch between different interface distributions depending on the task and to have more fine-grained control over synchronization.
The physical constraints of smartwatches limit the range and complexity of tasks that can be completed. Despite interface improvements on smartwatches, the promise of enabling productive work remains largely unrealized. This paper presents WearWrite, a system that enables users to write documents from their smartwatches by leveraging a crowd to help translate their ideas into text. WearWrite users dictate tasks, respond to questions, and receive notifications of major edits on their watch. Using a dynamic task queue, the crowd receives tasks issued by the watch user and generic tasks from the system. In a week-long study with seven smartwatch users supported by approximately 29 crowd workers each, we validate that it is possible to manage the crowd writing process from a watch. Watch users captured new ideas as they came to mind and managed a crowd during spare moments while going about their daily routine. WearWrite represents a new approach to getting work done from wearables using the crowd.
Microtask crowdsourcing organizes complex work into workflows, decomposing large tasks into small, relatively independent microtasks. Applied to software development, this model might increase participation in open source software development by lowering the barriers to contribu-tion and dramatically decrease time to market by increasing the parallelism in development work. To explore this idea, we have developed an approach to decomposing programming work into microtasks. Work is coordinated through tracking changes to a graph of artifacts, generating appropriate microtasks and propagating change notifications to artifacts with dependencies. We have implemented our approach in CrowdCode, a cloud IDE for crowd development. To evaluate the feasibility of microtask programming, we performed a small study and found that a small crowd of 12 workers was able to successfully write 480 lines of code and 61 unit tests in 14.25 person-hours of time.
We present Weave, a framework for developers to create cross-device wearable interaction by scripting. Weave provides a set of high-level APIs, based on JavaScript, for developers to easily distribute UI output and combine sensing events and user input across mobile and wearable devices. Weave allows developers to focus on their target interaction behaviors and manipulate devices regarding their capabilities and affordances, rather than low-level specifications. Weave also contributes an integrated authoring environment for developers to program and test cross-device behaviors, and when ready, deploy these behaviors to its runtime environment on users' ad-hoc network of devices. An evaluation of Weave with 12 participants on a range of tasks revealed that Weave significantly reduced the effort of developers for creating and iterating on cross-device interaction.
Almost surreptitiously, crowdsourcing has entered software engineering practice. In-house development, contracting, and outsourcing still dominate, but many development projects use crowdsourcing-for example, to squash bugs, test software, or gather alternative UI designs. Although the overall impact has been mundane so far, crowdsourcing could lead to fundamental, disruptive changes in how software is developed. Various crowdsourcing models have been applied to software development. Such changes offer exciting opportunities, but several challenges must be met for crowdsourcing software development to reach its potential.
Empirical studies, often in the form of controlled experiments, have been widely adopted in software engineering research as a way to evaluate the merits of new software engineering tools. However, controlled experiments involving human participants actually using new tools are still rare, and when they are conducted, some have serious validity concerns. Recent research has also shown that many software engineering researchers view this form of tool evaluation as too risky and too difficult to conduct, as they might ultimately lead to inconclusive or negative results. In this paper, we aim both to help researchers minimize the risks of this form of tool evaluation, and to increase their quality, by offering practical methodological guidance on designing and running controlled experiments with developers. Our guidance fills gaps in the empirical literature by explaining, from a practical perspective, options in the recruitment and selection of human participants, informed consent, experimental procedures, demographic measurements, group assignment, training, the selecting and design of tasks, the measurement of common outcome variables such as success and time on task, and study debriefing. Throughout, we situate this guidance in the results of a new systematic review of the tool evaluations that were published in over 1,700 software engineering papers published from 2001 to 2011.
Large tasks can be overwhelming. For example, many people have thousands of digital photographs that languish in unorganized archives because it is difficult and time consuming to gather them into meaningful collections. Such tasks are hard to start because they seem to require long uninterrupted periods of effort to make meaningful progress. We propose the idea of selfsourcing as a way to help people to perform large personal information tasks by breaking them into manageable microtasks. Using ideas from crowdsourcing and task management, selfsourcing can help people take advantage of existing gaps in time and recover quickly from interruptions. We present several achievable selfsourcing scenarios and explore how they can facilitate information work in interruption-driven environments.
To lower the threshold to participation in crowdsourcing, we present twitch crowdsourcing: crowdsourcing via quick contributions that can be completed in one or two seconds. We introduce Twitch, a mobile phone application that asks users to make a micro-contribution each time they unlock their phone. Twitch takes advantage of the common habit of turning to the mobile phone in spare moments. Twitch crowdsourcing activities span goals such as authoring a census of local human activity, rating stock photos, and extracting structured data from Wikipedia pages. We report a field deployment of Twitch where 82 users made 11,240 crowdsourcing contributions as they used their phone in the course of everyday life. The median Twitch activity took just 1.6 seconds, incurring no statistically distinguishable costs to unlock speed or cognitive load compared to a standard slide-to-unlock interface.
With the selection of devices encompassing a wider range of computing surfaces, along with the near ubiquity of wireless networks, the nature of the workspace has become distributed over multiple locations and digital artifacts. We interviewed 22 professionals across a wide range of industries about their use of artifacts in their workflows to dis-cover new cross-device interaction paradigms and issues. We explore the impact of today's cloud services and app-based computing on how devices are used together. Gaps in data management and cross-device interactions were identified as the main obstacles and opportunities for improvement for multi-device interaction.
Taxonomies are a useful and ubiquitous way of organizing information. However, creating organizational hierarchies is difficult because the process requires a global understanding of the objects to be categorized. Usually one is created by an individual or a small group of people working together for hours or even days. Unfortunately, this centralized approach does not work well for the large, quickly changing datasets found on the web. Cascade is an automated workflow that allows crowd workers to spend as little at 20 seconds each while collectively making a taxonomy. We evaluate Cascade and show that on three datasets its quality is 80-90% of that of experts. Cascade has a competitive cost to expert information architects, despite taking six times more human labor. Fortunately, this labor can be parallelized such that Cascade will run in as fast as four minutes instead of hours or days.
Debugging mobile phone applications is hard, as current debugging techniques either require multiple computing devices or do not support graphical debugging. To address this problem we present GROPG, the first graphical on-phone debugger. We implement GROPG for Android and perform a preliminary evaluation on third-party applications. Our experiments suggest that GROPG can lower the overall debugging time of a comparable text-based on-phone debugger by up to 2/3.
The basic principles of good code documentation include the avoidance of repetition, the coexistence of code and its documentation, the documentation of all artifacts, and a preference for clear code and automated processes to explanatory documentation. Existing tools can help us by extracting documentation from specially formatted comments and by formatting code listed in documents. We can also automate the generation of bespoke documentation with a custom-made tool. When you document code, strive for completeness, consistency, effortless accessibility, and an automated low-overhead generation process.
Goal-directed cognition is often discussed in terms of specialized memory structures like the “goal stack.” The goal-activation model presented here analyzes goal-directed cognition in terms of the general memory constructs of activation and associative priming. The model embodies three predictive constraints: (1) the interference level, which arises from residual memory for old goals; (1) the strengthening constraint, which makes predictions about time to encode a new goal; and (3) the priming constraint, which makes predictions about the role of cues in retrieving pending goals. These constraints are formulated algebraically and tested through simulation of latency and error data from the Tower of Hanoi, a means-ends puzzle that depends heavily on suspension and resumption of goals. Implications of the model for understanding intention superiority, postcompletion error, and effects of task interruption are discussed.
A study at a large IT company shows that mobile informa- tion workers frequently migrate work across devices (here: smartphones, desktop PCs, laptops). While having multiple devices provides new opportunities to work in the face of changing resource deprivations, the management of devices is often problematic. The most salient problems are posed by 1) the physical effort demanded by various management tasks, 2) anticipating what data or functionality will be needed, and 3) aligning these efforts with work, mobility, and social situations. Workers' strategies of coping with these problems center on two interwoven activities: the physical handling of devices and cross-device synchroniza- tion. These aim at balancing risk and effort in immediate and subsequent use. Workers also exhibit subtle ways to handle devices in situ, appropriating their physical and op- erational properties. The design implications are discussed.
Sensors are becoming increasingly important in interaction design. Authoring a sensor-based interaction comprises three steps: choosing and connecting the appropriate hard- ware, creating application logic, and specifying the relationship between sensor values and application logic. Recent research has successfully addressed the first two issues. However, linking sensor input data to application logic remains an exercise in patience and trial-and-error testing for most designers. This paper introduces techniques for authoring sensor-based interactions by demonstration. A combination of direct manipulation and pattern recognition techniques enables designers to control how demonstrated examples are generalized to interaction rules. This approach emphasizes design exploration by enabling very rapid iterative demonstrate-edit-review cycles. This paper de- scribes the manifestation of these techniques in a design tool, Exemplar, and presents evaluations through a first-use lab study and a theoretical analysis using the Cognitive Dimensions of Notation framework.
The number of computing devices that people use is growing. To gain a better understanding of why and how people use multiple devices, we interviewed 27 people from academia and industry. From these interviews we distill four primary findings. First, associating a user's activities with a particular device is problematic for multiple device users because many activities span multiple devices. Second, device use varies by user and circumstance; users assign different roles to devices both by choice and by constraint. Third, users in industry want to separate work and personal activities across work and personal devices, but they have difficulty doing so in practice Finally, users employ a variety of techniques for accessing information across devices, but there is room for improvement: participants reported managing information across their devices as the most challenging aspect of using multiple devices. We suggest opportunities to improve the user experience by focusing on the user rather than the applications and devices; making devices aware of their roles; and providing lighter-weight methods for transferring information, including synchronization services that engender more trust from users.