Brad A. Myers

• PhD
• Professor (Full) at Carnegie Mellon University

Generative pre-trained models power intelligent software features used by millions of users controlled by developer-written natural language prompts. Despite the impact of prompt-powered software, little is known about its development process and its relationship to programming. In this work, we argue that some prompts are programs and that the dev...

Software testing is difficult, tedious, and may consume 28%–50% of software engineering labor. Automatic test generators aim to ease this burden but have important trade-offs. Fuzzers use an implicit oracle that can detect obviously invalid results, but the oracle problem has no general solution, and an implicit oracle cannot automatically evaluate...

Developers often face challenges in code understanding, which is crucial for building and maintaining high-quality software systems. Code comments and documentation can provide some context for the code, but are often scarce or missing. This challenge has become even more pressing with the rise of large language model (LLM) based code generation to...

A key goal of software engineering research is to improve the environments, tools, languages, and techniques programmers use to efficiently create quality software. Successfully designing these tools and demonstrating their effectiveness involves engaging with tool users — software engineers. Researchers often want to conduct user studies of softwa...

Consumers conducting comparison shopping, researchers making sense of competitive space, and developers looking for code snippets online all face the challenge of capturing the information they find for later use without interrupting their current flow. In addition, during many learning and exploration tasks, people need to externalize their mental...

Developers perform online sensemaking on a daily basis, such as researching and choosing libraries and APIs. Prior research has introduced tools that help developers capture information from various sources and organize it into structures useful for subsequent decision-making. However, it remains a laborious process for developers to manually ident...

Modern software development requires developers to find and effectively utilize new APIs and their documentation, but documentation has many well-known issues. Despite this, developers eventually overcome these issues but have no way of sharing what they learned. We investigate sharing this documentation-specific information through \textit{annotat...

Artificial Intelligence (AI) and End-User Development (EUD) look at automation from two different perspectives. The former tends to provide fully automatic solutions, the latter aims to empower users to directly create what they want. We need both, but it is still unclear how to combine them to obtain effective every-day automations that meet the f...

Programming language design requires making many usability-related design decisions. However, existing HCI methods can be impractical to apply to programming languages: languages have high iteration costs, programmers require significant learning time, and user performance has high variance. To address these problems, we adapted both formative and...

Text entry makes up about one-fourth of the smartphone interaction events, and is known to be challenging and difficult. However, there has been little study about the characteristics of text entry in the context of smartphone app usage. In this paper, we present a mixed-method in-situ study conducted in 2016 with 17 active smartphone users to bett...

As the amount of information online continues to grow, a correspondingly important opportunity is for individuals to reuse knowledge which has been summarized by others rather than starting from scratch. However, appropriate reuse requires judging the relevance, trustworthiness, and thoroughness of others' knowledge in relation to an individual's g...

As the amount of information online continues to grow, a correspondingly important opportunity is for individuals to reuse knowledge which has been summarized by others rather than starting from scratch. However, appropriate reuse requires judging the relevance, trustworthiness, and thoroughness of others' knowledge in relation to an individual's g...

Representing the semantics of GUI screens and components is crucial to data-driven computational methods for modeling user-GUI interactions and mining GUI designs. Existing GUI semantic representations are limited to encoding either the textual content, the visual design and layout patterns, or the app contexts. Many representation techniques also...

Rationale: Breakdowns in clinician-family communication in ICUs are common, yet there are no easily scaled interventions to prevent this problem. Objective: To assess the feasibility, usability, acceptability, and perceived effectiveness of a communication intervention that pairs proactive family meetings with an interactive, web-based tool to h...

Blockchain platforms are coming into use for processing critical transactions among participants who have not established mutual trust. Many blockchains are programmable, supporting smart contracts , which maintain persistent state and support transactions that transform the state. Unfortunately, bugs in many smart contracts have been exploited by...

Some blockchain programs (smart contracts) have included serious security vulnerabilities. Obsidian is a new typestate-oriented programming language that uses a strong type system to rule out some of these vulnerabilities. Although Obsidian was designed to promote usability to make it as easy as possible to write programs, strong type systems can c...

An important concern in end user development (EUD) is accidentally embedding personal information in program artifacts when sharing them. This issue is particularly important in GUI-based programming-by-demonstration (PBD) systems due to the lack of direct developer control of script contents. Prior studies reported that these privacy concerns were...

An important concern in end user development (EUD) is accidentally embedding personal information in program artifacts when sharing them. This issue is particularly important in GUI-based programming-by-demonstration (PBD) systems due to the lack of direct developer control of script contents. Prior studies reported that these privacy concerns were...

Some blockchain programs (smart contracts) have included serious security vulnerabilities. Obsidian is a new typestate-oriented programming language that uses a strong type system to rule out some of these vulnerabilities. Although Obsidian was designed to promote usability to make it as easy as possible to write programs, strong type systems can c...

We argue that a key challenge in enabling usable and useful interactive task learning for intelligent agents is to facilitate effective Human-AI collaboration. We reflect on our past 5 years of efforts on designing, developing and studying the SUGILITE system, discuss the issues on incorporating recent advances in AI with HCI principles in mixed-in...

Programming language designers commonly guess what language designs would be best for their users and create languages accordingly. The outcome of this is languages that are difficult to use and error-prone. In particular, blockchain programs have been plagued by serious bugs. Although techniques from the theory of programming languages can detect...

Introduction: Although family members of incapacitated, critically ill patients often struggle in the role of surrogate decision maker, there are no low-cost, easily-scaled interventions to address this problem. Aim of the study: To develop and pilot-test the Family Support Tool, an interactive, web-based tool to help individuals navigate the co...

Developers spend a significant portion of their time searching for solutions and methods online. While numerous tools have been developed to support this exploratory process, in many cases the answers to developers' questions involve trade-offs among multiple valid options and not just a single solution. Through interviews, we discovered that devel...

Natural language programming is a promising approach to enable end users to instruct new tasks for intelligent agents. However, our formative study found that end users would often use unclear, ambiguous or vague concepts when naturally instructing tasks in natural language, especially when specifying conditionals. Existing systems have limited sup...

Almost all software development revolves around the discovery and use of application programming interfaces (APIs). Once a suitable API is selected, programmers must begin the process of determining what functionality in the API is relevant to a programmer's task and how to use it. Our work aims to understand how API functionality is discovered by...

Blockchain platforms are coming into broad use for processing critical transactions among participants who have not established mutual trust. Many blockchains are programmable, supporting smart contracts, which maintain persistent state and support transactions that transform the state. Unfortunately, bugs in many smart contracts have been exploite...

Natural language programming is a promising approach to enable end users to instruct new tasks for intelligent agents. However, our formative study found that end users would often use unclear, ambiguous or vague concepts when naturally instructing tasks in natural language, especially when specifying conditionals. Existing systems have limited sup...

Since programming is a human activity, we can look to fields which have already developed methods for better understanding the details of human interactions with technologies. In particular, the field of human-computer interaction (HCI) has dozens, if not hundreds, of methods that have been validated for answering a wide range of questions about hu...

Data scientists are responsible for the analysis decisions they make, but it is hard for them to track the process by which they achieved a result. Even when data scientists keep logs, it is onerous to make sense of the resulting large number of history records full of overlapping variants of code, output, plots, etc. We developed algorithmic and v...

Despite almost all software development involving application programming interfaces (APIs), there is surprisingly little work on how people use APIs and how to evaluate and improve the usability of an API. One possible way of investigating the usability of APIs is through the user's mental model of the API. Through discussions with the developers...

Approaches for programming language design used commonly in the research community today center around theoretical and performance-oriented evaluation. Recently, researchers have been considering more approaches to language design, including the use of quantitative and qualitative user studies that examine how different designs might affect program...

Application Programming Interfaces (APIs) are a rapidly growing industry and the usability of the APIs is crucial to programmer productivity. Although prior research has shown that APIs commonly suffer from significant usability problems, little attention has been given to studying how APIs are designed and created in the first place. We interviewe...

This chapter introduces an end user development (EUD) approach for handling common types of failures encountered by goal-oriented conversational agents. We start with identifying three common sources of failures in human-agent conversations: unknown concepts, out-of-domain tasks and wrong fulfillment means or level of generalization in task executi...

Literate programming tools are used by millions of programmers today, and are intended to facilitate presenting data analyses in the form of a narrative. We interviewed 21 data scientists to study coding behaviors in a literate programming environment and how data scientists kept track of variants they explored. For participants who tried to keep a...

Moonstone is a new plugin for Eclipse that supports developers in understanding exception flow and in writing exception handlers in Java. Understanding exception control flow is paramount for writing robust exception handlers, a task many developers struggle with. To help with this understanding, we present two new kinds of information: ghost comme...

When end users approach a development task, they bring with them a set of techniques, expressions, and knowledge, which can be leveraged in order to make the process easier. The Natural Programming Project has been working for over twenty years to better understand how end users think about their tasks, and to develop new ways for users to express...

The revolutionary advances of Internet of Things (IoT) devices and applications have helped IoT emerge as an increasingly important domain for end-user development (EUD). Past research has shown that end users desire to create various customized automations, which would often utilize multiple IoT devices. Many solutions exist to support EUD across...

Brad Myers has been doing research in the area of human-computer interaction for the last 40 years. Along the way, he and his students created over 60 software systems, whose names are often acronyms for gemstones. This talk's acronym, Ruby, is fittingly the gemstone associated with a 40th anniversary. Myers's MIT Master's thesis, Incense, was one...

How do people ideate through code? Using semi-structured interviews and a survey, we studied data scientists who program, often with small scripts, to experiment with data. These studies show that data scientists frequently code new analysis ideas by building off of their code from a previous idea. They often rely on informal versioning interaction...

In this position paper, we first summarize our work on designing the conversational interface for SUGILITE – a multimodal programming by demonstration system that enables a virtual agent to learn how to handle out-of-domain commands and perform the tasks using available third-party mobile apps in task-oriented dialogs from the user's demonstrations...

In this position paper, we first summarize our work on designing the conversational interface for SUGILITE – a multimodal programming by demonstration system that enables a virtual agent to learn how to handle out-of-domain commands and perform the tasks using available third-party mobile apps in task-oriented dialogs from the user's demonstrations...

SUGILITE is a new programming-by-demonstration (PBD) system that enables users to create automation on smartphones. SUGILITE uses Android's accessibility API to support automating arbitrary tasks in any Android app (or even across multiple apps). When the user gives verbal commands that SUGILITE does not know how to execute, the user can demonstrat...

User studies have revealed that programmers face several obstacles when learning application programming interfaces (APIs). A considerable part of such difficulties relate to discovery of API elements and the relationships among them. To address discoverability problems, we show how to complement APIs with design annotations, which document design...

Context: Critical software systems developed for the government continue to be of lower quality than expected, despite extensive literature describing best practices in software engineering. Goal: We wanted to better understand the extent of certain issues in the field and the relationship to software quality. Method: We surveyed fifty software dev...

Web services offer a more reliable and efficient way to access online data than scraping web pages. However, interacting with web services to retrieve data often requires people to write a lot of code. Moreover, many web services return data in complex hierarchical structures that make it difficult for people to perform any further data manipulatio...

Lung transplant recipients are encouraged to perform self-management behaviors including: 1) monitoring health indicators, 2) adhering to their regimen, and 3) reporting abnormal health indicators to the transplant coordinator, yet performance is suboptimal. When hospital discharge was imminent, this two-group trial randomized 201 recipients to use...

Human-centered methods can help researchers better understand and meet programmers' needs. Because programming is a human activity, many of these methods can be used without change. However, some programmer needs require new methods, which can also be applied to domains other than software engineering. This article features five Web extras. The vid...

APPLICATION PROGRAMMING INTERFACES (APIs), including libraries, frameworks, toolkits, and software development kits, are used by virtually all code. If one includes both internal APIs (interfaces internal to software projects) and public APIs (such as the Java Platform SDK, the Windows .NET Framework, jQuery for JavaScript, and Web services like Go...

Programming languages can restrict state change by preventing it entirely (immutability) or by restricting which clients may modify state (read-only restrictions). The benefits of immutability and read-only restrictions in software structures have been long-argued by practicing software engineers, researchers, and programming language designers. Ho...

Many have argued that the current try/catch mechanism for handling exceptions in Java is flawed. A major complaint is that programmers often write minimal and low quality handlers. We used the Boa tool to examine a large number of Java projects on GitHub to provide empirical evidence about how programmers currently deal with exceptions. We found th...

More and more data nowadays exist in hierarchical formats such as JSON due to the increasing popularity of web applications and web services. While many end-user systems support getting hierarchical data from databases without programming, they provide very little support for using hierarchical data beyond turning the data into a flat string or tab...

Programming languages form the interface between programmers (the users) and the computation that they desire the computer to execute. Although studies exist for some aspects of programming language design (such as conditionals), other aspects have received little or no human factors evaluations. Designers thus have little they can rely on if they...

Purpose: Although barriers to shared decision making in intensive care units are well documented, there are currently no easily scaled interventions to overcome these problems. We sought to assess stakeholders' perceptions of the acceptability, usefulness, and design suggestions for a tablet-based tool to support communication and shared decision...

Many programming languages provide features that express restrictions on which data structures can be changed. For example, C++ includes const and Java includes final. Languages that are in widespread use typically provide non-transitive immutability: when a reference is specified to be immutable or read-only, the object referenced can still refere...

Cilk Plus and OpenMP are parallel language extensions for the C and C++ programming languages. The CPLEX Study Group of the ISO/IEC C Standards Committee is developing a proposal for a parallel programming extension to C that combines ideas from Cilk Plus and OpenMP. We conducted a preliminary comparison of Cilk Plus and OpenMP in a master’s level...

Previously, we presented our technique for visualizing fine-grained code changes in a timeline view, designed to facilitate reviewing and interacting with the code change history. During user evaluations, it became evident that users often wanted to see the code changes at a higher level of abstraction. Therefore, we developed a novel approach to a...

Programmers often need to revert some code to an earlier state, or restore a block of code that was deleted a while ago. However, support for this backtracking in modern programming environments is limited. Many of the backtracking tasks can be accomplished by having a selective undo feature in code editors, but this has major challenges: there can...

Today's widely deployed painting applications use a linear undo model that allows users to backtrack previous operations in reverse chronological order. This undo model is not useful if the user has performed desired operations after undesired ones. Selective undo, in contrast, allows users to select specific operations in the past and only undo th...

We present a spreadsheet model for working with streaming data. Our prototype tool presents techniques to let the user stream data from web services and web input elements to a spreadsheet without preprogramming those sources into the tool. Spreadsheet cells record metadata about where and when the data came from, allowing the user to view and mani...

This is a pretend publication from a joke article.

Designers of serious games have an interest in understanding if their games are well-aligned, i.e., whether in-game rewards incentivize behaviors that will lead to learning. Few existing serious games analytics solutions exist to serve this need. Open-ended games in particular run into issues of alignment due to their affordances for wide player fr...

Static types may be used both by the language implementation and directly by the user as documentation. Though much existing work focuses primarily on the implications of static types on the semantics of programs, relatively little work considers the impact on usability that static types provide. Though the omission of static type information may d...

Researchers have paid increasing attention in recent years to the fact that much development occurs though example modification. Helping programmers with some of the pit-falls and vagaries of working with example code is the goal of our tool, called Euklas. It helps developers to integrate JavaScript example code into their own projects by using fa...

While more and more data are available through web services, it remains difficult for end-users to create web applications that make use of these data without having to write complex code. We present Gneiss, a live programming environment that extends the spreadsheet metaphor to support creating interactive web applications that dynamically use loc...

InterState is a new programming language and environment that addresses the challenges of writing and reusing user interface code. InterState represents interactive behaviors clearly and concisely using a combination of novel forms of state machines and constraints. It also introduces new language features that allow programmers to easily modulariz...

Programming often involves reverting source code to an earlier state, which we call backtracking. We performed a longitudinal study of programmers' backtracking, analyzing 1,460 hours of fine-grained code editing logs collected from 21 people. Our analysis method keeps track of the change history of each abstract syntax tree node and looks for back...

Programmers often need to backtrack, but backtracking support in modern programming environments is limited. Previously, we have conducted a series of studies, which discovered that backtracking in programming is in fact prevalent and programmers need better backtracking tools. In this demonstration, we will present our backtracking tool called AZU...

Web services offer a more reliable and efficient way to access online data than scraping web pages. However, web service data are often in complex hierarchical structures that make it difficult for people to extract the desired parts or to perform any further data manipulation without writing a significant amount of surprisingly intricate code. In...

End-user software engineering (EUSE) is a research area that aims to invent new kinds of technologies that collaborate with end users to improve the quality of their software. The practice that EUSE research aims to support is end users using new tools and methods to improve the quality of the software that they and other end users have created. Th...

As educational games have become a larger field of study, there has been a growing need for analytic methods that can be used to assess game design and inform iteration. While much previous work has focused on the measurement of student engagement or learning at a gross level, we argue that new methods are necessary for measuring the alignment of a...

Structure makes data more useful, but also makes data entry more cumbersome. Studies have found that this is especially true on mobile devices, as mobile users often reject structured personal information management tools because the structure is too restrictive and makes entering data slower. To overcome these problems, we introduce a new data ent...

Conventional version control systems save code changes at each check-in. Recently, some development environments retain more fine-grain changes. However, providing tools for developers to use those histories is not a trivial task, due to the difficulties in visualizing the history. We present two visualizations of fine-grained code change history,...

A key use for mobile devices is to search and view online information while on the go. As a result, many mobile applications serve as front ends for online databases. While there are many thousands of data sources that provide web service APIs giving access to their databases, creating mobile applications to use those sources requires significant m...

Despite the proliferation of health information technology (IT) interventions, descriptions of the unique considerations for conducting randomized trials of health IT interventions intended for patient use are lacking. Our purpose is to describe the protocol to evaluate Pocket PATH(®) (Personal Assistant for Tracking Health), a novel health IT inte...

The rich interaction space of many educational games presents a challenge for designers and researchers who strive to help players achieve specific learning outcomes. Giving players a large amount of freedom over how they perform a complex game task makes it difficult to anticipate what they will do. In order to address this issue designers must as...

Euclase is a live development environment focused on creating interactive web applications. It uses a programming model that combines constraints and finite state machines to specify interactive behaviors. Euclase is “live” in the sense that while the user is developing code, their program is always executing. Changes made to the source of the prog...

My Natural Programming Project is working on making software development easier to learn, more effective, and less error prone. An important focus over the last few years has been to discover what are the hard-to-answer questions that developers ask while they are trying to comprehend their programs, and then to develop tools to help answer those q...