Matthew Kay

• BCS, MMath, MS, PhD
• Professor (Assistant) at University of Michigan

A core tradition of HCI lies in the experimental evaluation of the effects of techniques and interfaces to determine if they are useful for achieving their purpose. However, our individual analyses tend to stand alone, and study results rarely accrue in more precise estimates via meta-analysis: in a literature search, we found only 56 meta-analyses...

Users often rely on realtime predictions in everyday contexts like riding the bus, but may not grasp that such predictions are subject to uncertainty. Existing uncertainty visualizations may not align with user needs or how they naturally reason about probability. We present a novel mobile interface design and visualization of uncertainty for trans...

Models of human perception - including perceptual "laws" - can be valuable tools for deriving visualization design recommendations. However, it is important to assess the explanatory power of such models when using them to inform design. We present a secondary analysis of data previously used to rank the effectiveness of bivariate visualizations fo...

Many HCI and ubiquitous computing systems are characterized by two important properties: their output is uncertain—it has an associated accuracy that researchers attempt to optimize—and this uncertainty is user-facing—it directly affects the quality of the user experience. Novel classifiers are typically evaluated using measures like the F1 score—b...

Information environments have the power to affect people’s perceptions and behaviors. In this paper, we present the results of studies in which we characterize the gender bias present in image search results for a variety of occupations. We experimentally evaluate the effects of bias in image search results on the images people choose to represent...

Visualization literacy is the ability to both interpret and construct visualizations. Yet existing assessments focus solely on visualization interpretation. A lack of construction-related measurements hinders efforts in understanding and improving literacy in visualizations. We design and develop AVEC, an assessment of a person's visual encoding ab...

Users often have access to multiple forecasts regarding an event. Different forecasts incorporate different assumptions and epistemic information. A growing body of work argues against decisionmaking solely based on expected utility maximisation strategies in multiple forecasts scenarios, in favour of other strategies such as the maximin expected u...

Visualizations play a critical role in validating and improving statistical models. However, the design space of model check visualizations is not well understood, making it difficult for authors to explore and specify effective graphical model checks. VMC defines a model check visualization using four components: (1) samples of distributions of ch...

This position paper critically examines the graphical inference framework for evaluating visualizations using the lineup task. We present a re-analysis of lineup task data using signal detection theory, applying four Bayesian non-linear models to investigate whether color ramps with more color name variation increase false discoveries. Our study ut...

A year ago, we submitted an IEEE VIS paper entitled “Swaying the Public? Impacts of Election Forecast Visualizations on Emotion, Trust, and Intention in the 2022 U.S. Midterms” [50], which was later bestowed with the honor of a best paper award. Yet, studying such a complex phenomenon required us to explore many more design paths than we could coun...

Visualization items—factual questions about visualizations that ask viewers to accomplish visualization tasks—are regularly used in the field of information visualization as educational and evaluative materials. For example, researchers of visualization literacy require large, diverse banks of items to conduct studies where the same skill is measur...

Visualizations play a critical role in validating and improving statistical models. However, the design space of model check visualizations is not well understood, making it difficult for authors to explore and specify effective graphical model checks. VMC defines a model check visualization using four components: (1) samples of distributions of ch...

Empirical studies in visualisation often compare visual representations to identify the most effective visualisation for a particular visual judgement or decision making task. However, the effectiveness of a visualisation may be intrinsically related to, and difficult to distinguish from, individual-level factors such as visualisation literacy. Com...

In 2023, we submitted an IEEE VIS paper entitled “Swaying the Public? Impacts of Election Forecast Visualizations on Emotion, Trust, and Intention in the 2022 U.S. Midterms”, which was later bestowed with the honor of a best paper award. Yet, studying such a complex phenomenon required us to explore many more design paths than we could count, and c...

Visualization items—factual questions about visualizations that ask viewers to accomplish visualization tasks—are regularly used in the field of information visualization as educational and evaluative materials. For example, researchers of visualization literacy require large, diverse banks of items to conduct studies where the same skill is measur...

Existing data visualization design guidelines focus primarily onconstructing grammatically-correct visualizations that faithfullyconvey the values and relationships in the underlying data. However,a designer may create a grammatically-correct visualizationthat still leaves audiences susceptible to reasoning misleaders, e.g.by failing to normalize d...

Trust in high-profile election forecasts influences the public’s confidence in democratic processes and electoral integrity. Yet, maintaining trust after unexpected outcomes like the 2016 U.S. presidential election is a significant challenge. Our work confronts this challenge through three experiments that gauge trust in election forecasts. We gene...

Recent studies have shown that users of visual analytics tools can have difficulty distinguishing robust findings in the data from statistical noise, but the true extent of this problem is likely dependent on both the incentive structure motivating their decisions, and the ways that uncertainty and variability are (or are not) represented in visual...

Multiverse analyses involve conducting all combinations of reasonable choices in a data analysis process. A reader of a study containing a multiverse analysis might question—are all the choices included in the multiverse reasonable and equally justifiable? How much do results vary if we make different choices in the analysis process? In this work,...

We conducted a longitudinal study during the 2022 U.S. midterm elections, investigating the real-world impacts of uncertainty visualizations. Using our forecast model of the governor elections in 33 states, we created a website and deployed four uncertainty visualizations for the election forecasts: single quantile dotplot (1-Dotplot), dual quantil...

Visualization literacy is an essential skill for accurately interpreting data to inform critical decisions. Consequently, it is vital to understand the evolution of this ability and devise targeted interventions to enhance it, requiring concise and repeatable assessments of visualization literacy for individuals. However, current assessments, such...

Visualization literacy is an essential skill for accurately interpreting data to inform critical decisions. Consequently, it is vital to understand the evolution of this ability and devise targeted interventions to enhance it, requiring concise and repeatable assessments of visualization literacy for individuals. However, current assessments, such...

Graphical perception studies typically measure visualization encoding effectiveness using the error of an "average observer", leading to canonical rankings of encodings for numerical attributes: e.g., position > area > angle > volume. Yet different people may vary in their ability to read different visualization types, leading to variance in this r...

Graphical perception studies typically measure visualization encoding effectiveness using the error of an “average observer”, leading to canonical rankings of encodings for numerical attributes: e.g. , position $>$ area $>$ angle $>$ volume. Yet different people may vary in their ability to read different visualization types, leading to var...

Analysing data from experiments is a complex, multi‐step process, often with multiple defensible choices available at each step. While analysts often report a single analysis without documenting how it was chosen, this can cause serious transparency and methodological issues. To make the sensitivity of analysis results to analytical choices transpa...

There are myriad ways to analyze any given dataset. But which one to trust? In the face of such uncertainty, analysts adopt multiverse analysis: running all reasonable analyses on the dataset. Yet this is cognitively and technically difficult with existing tools—how does one specify and execute all combinations of reasonable analyses of a dataset?—...

While uncertainty is present in most data analysis pipelines, reasoning with uncertainty is challenging for novices and experts alike. Fortunately, researchers are making significant advancements in the communication of uncertainty. In this article, we detail new visualization methods and emerging cognitive theories that describe how we reason with...

When forecasting events, multiple types of uncertainty are often inherently present in the modeling process. Various uncertainty typologies exist, and each type of uncertainty has different implications a scientist might want to convey. In this work, we focus on one type of distinction between direct quantitative uncertainty and indirect qualitativ...

Uncertainty visualizations often emphasize point estimates to support magnitude estimates or decisions through visual comparison. However, when design choices emphasize means, users may overlook uncertainty information and misinterpret visual distance as a proxy for effect size. We present findings from a mixed design experiment on Mechanical Turk...

Uncertainty visualizations often emphasize point estimates to support magnitude estimates or decisions through visual comparison. However, when design choices emphasize means, users may overlook uncertainty information and misinterpret visual distance as a proxy for effect size. We present findings from a mixed design experiment on Mechanical Turk...

While uncertainty is present in most data analysis pipelines, reasoning with uncertainty is challenging for novices and experts alike. Fortunately, researchers are making significant advancements in the communication of uncertainty. In this chapter, we detail new visualization methods and emerging cognitive theories that describe how we reason with...

Bayesian statistical analysis is steadily growing in popularity and use. Choosing priors is an integral part of Bayesian inference. While there exist extensive normative recommendations for prior setting, little is known about how priors are chosen in practice. We conducted a survey (N = 50) and interviews (N = 9) where we used interactive visualiz...

Bayesian statistical analysis has gained attention in recent years, including in HCI. The Bayesian approach has several advantages over traditional statistics, including producing results with more intuitive interpretations. Despite growing interest, few papers in CHI use Bayesian analysis. Existing tools to learn Bayesian statistics require signif...

We present explorable multiverse analysis reports, a new approach to statistical reporting where readers of research papers can explore alternative analysis options by interacting with the paper itself. This approach draws from two recent ideas: i) multiverse analysis, a philosophy of statistical reporting where paper authors report the outcomes of...

To make evidence-based recommendations to decision-makers, researchers conducting systematic reviews and meta-analyses must navigate a garden of forking paths: a series of analytical decision-points, each of which has the potential to influence findings. To identify challenges and opportunities related to designing systems to help researchers manag...

The rise of affordable sensors and apps has enabled people to monitor various health indicators via self-tracking. This trend encourages self-experimentation, a subset of self-tracking in which a person systematically explores potential causal relationships to try to answer questions about their health. Although recent research has investigated how...

To make evidence-based recommendations to decision-makers, researchers conducting systematic reviews and meta-analyses must navigate a garden of forking paths: a series of analytical decision-points, each of which has the potential to influence findings. To identify challenges and opportunities related to designing systems to help researchers manag...

Understanding and accounting for uncertainty is critical to effectively reasoning about visualized data. However, evaluating the impact of an uncertainty visualization is complex due to the difficulties that people have interpreting uncertainty and the challenge of defining correct behavior with uncertainty information. Currently, evaluators of unc...

Animated representations of outcomes drawn from distributions (hypothetical outcome plots, or HOPs) are used in the media and other public venues to communicate uncertainty. HOPs greatly improve multivariate probability estimation over conventional static uncertainty visualizations and leverage the ability of the visual system to quickly, accuratel...

Everyday predictive systems typically present point predic­tions, making it hard for people to account for uncertainty when making decisions. Evaluations of uncertainty displays for transit prediction have assessed people’s ability to extract probabilities, but not the quality of their decisions. In a controlled, incentivized experiment, we had sub...

Transparent statistics is a philosophy of statistical reporting whose purpose is scientific advancement rather than persuasion. At our CHI 2017 workshop, "Moving Transparent Statistics Forward", we identified that an important first step is to develop detailed guidelines for authors and reviewers in order to help them practice and promote transpare...

People often have erroneous intuitions about the results of uncertain processes, such as scientific experiments. Many uncertainty visualizations assume considerable statistical knowledge, but have been shown to prompt erroneous conclusions even when users possess this knowledge. Active learning approaches been shown to improve statistical reasoning...

Many people who want to build mobile applications based on sensing systems have to decide what type of sensing approaches to use, often trading off factors like accuracy, time, and battery life. If you're building an app that uses location, motion data, or inferred data about usage, there will necessarily be inaccuracy in the output of those system...

End-users are often exposed to uncertain data in interactive systems such as personal health apps, intelligent navigation systems, and systems driven by machine learning. On one hand, communicating uncertainty may improve the understanding of data and predictions. On the other hand, communicating uncertainty can greatly confuse users and decrease t...

Transparent statistics is a philosophy of statistical reporting whose purpose is scientific advancement rather than persuasion. We ran a SIG at CHI 2016 to discuss problems and limitations in statistical practices in HCI and options for moving the field towards clearer and more reliable ways of writing about experiments, and received an overwhelmin...

Desirable outcomes such as health are tightly linked to behaviors, thus inspiring research on technologies that support people in changing those behaviors. Many behavior-change technologies are designed by HCI experts but this approach can make it difficult to personalize support to each user's unique goals and needs. This paper reports on the iter...

Introduction The Psychomotor Vigilance Task (PVT) is a gold-standard, widely used, and highly sensitive tool to quantify effects of sleep loss. Current portable versions are either costly, not validated, or based on out-of-date platforms. Our goal was to develop a low-cost, widely accessible version of the PVT that runs on touchscreen devices. This...

The authors present an approach for designing self-monitoring technology called "semi-automated tracking," which combines both manual and automated data collection methods. Through this approach, they aim to lower the capture burdens, collect data that is typically hard to track automatically, and promote awareness to help people achieve their self...

Throughout the day, our alertness levels change and our cognitive performance fluctuates. The creation of technology that can adapt to such variations requires reliable measurement with ecological validity. Our study is the first to collect alertness data in the wild using the clinically validated Psychomotor Vigilance Test. With 20 participants ov...

Our body clock causes considerable variations in our behavioral, mental, and physical processes, including alertness, throughout the day. While much research has studied technology usage patterns, the potential impact of underlying biological processes on these patterns is under-explored. Using data from 20 participants over 40 days, this paper pre...

Transparent statistics is a philosophy of statistical reporting whose purpose is scientific advancement rather than persuasion. We propose a SIG to discuss problems and limitations in statistical practices in HCI and options for moving the field towards clearer and more reliable ways of writing about experiments.

Data not suitable for classic parametric statistical analyses arise frequently in human–computer interaction studies. Various nonparametric statistical procedures are appropriate and advantageous when used properly. This chapter organizes and illustrates multiple nonparametric procedures, contrasting them with their parametric counterparts. Guidanc...

ARTool is an R package implementing the Aligned Rank Transform for conducting nonparametric analyses of variance on factorial models. This implementation is based on the ART procedure as used in the original implementation of ARTool by Wobbrock et al.

Manual tracking of health behaviors affords many benefits, including increased awareness and engagement. However, the capture burden makes long-term manual tracking challenging. In this study on sleep tracking, we examine ways to reduce the capture burden of manual tracking while leveraging its benefits. We report on the design and evaluation of Sl...

To review sleep related consumer technologies, including mobile electronic device "apps," wearable devices, and other technologies. Validation and methodological transparency, the effect on clinical sleep medicine, and various social, legal, and ethical issues are discussed. We reviewed publications from the digital libraries of the Association for...

Increasingly, personal health data can be tracked and integrated from numerous streams quickly and easily, but our feedback lingers in the land of “show the user a graph and hope”. How can we help people make sense of personal health data?

Though never a desirable outcome, failure is an inevitable part of research. Too often, however, the tried but failed paths are lost in the translation of work to publication. With the pragmatics of publishing (e.g., page limits) and the academic emphasis on positive outcomes, failed processes, methodologies, study designs, and technologies are fre...

Biological rhythms enable living organisms to adapt and live with periodical environmental changes, such as variation in the relative position of the earth and the sun. Internal rhythms, like body temperature and sleep-wake cycle, are driven by numerous biological processes and can be maintained even in the absence of external environmental cues. T...

The weight scale is perhaps the most ubiquitous health sensor of all and is important to many health and lifestyle decisions, but its fundamental interface--a single numerical estimate of a person's current weight--has remained largely unchanged for 100 years. An opportunity exists to impact public health by re-considering this pervasive interface....

Learn about papers and topics of interest from the 14th ACM International Conference on Ubiquitous Computing (Ubicomp 2012). Much discussion centered on the future of robotics, the state of ubicomp, and the latest in sensor research.

The Psychomotor Vigilance Task (PVT) is a validated reaction time (RT) test used to assess aspects of sleep loss including alertness and sleepiness. PVT typically requires a physical button to assess RT, which minimizes the effect of execution time (the time taken to perform a gesture) on RT. When translating this application to mobile devices, a t...

The bedroom environment can have a significant impact on the quality of a person's sleep. Experts recommend sleeping in a room that is cool, dark, quiet, and free from disruptors to ensure the best quality sleep. However, it is sometimes difficult for a person to assess which factors in the environment may be causing disrupted sleep. In this paper,...

This paper presents results from a study examining percep- tions and practices of usability in the free/open source soft- ware (FOSS) community. 27 individuals associated with 11 different FOSS projects were interviewed to understand how they think about, act on, and are motivated to address usability issues. Our results indicate that FOSS project...

We present narrative pictograms, illustrative diagrams designed to convey the abstract concepts of software agreements. Narrative pictograms arose out of a need to create software agreements that are comprehensible without written language. We first present example diagrams designed to describe the data collection policies of research software, and...

Research indicates that less than 2% of the population reads license agreements during software installation [12]. To address this problem, we developed textured agreements, visually redesigned agreements that employ factoids, vignettes, and iconic symbols to accentuate information and highlight its personal relevance. Notably, textured agreements...

Open source projects are gradually incorporating usability methods into their development practices, but there are still many unmet needs. One particular need for nearly any open source project is data that describes its user base, including information indicating how the software is actually used in practice. This paper presents the concept of ope...