Kristin Divis’s research while affiliated with Sandia National Laboratories and other places

This study addressed the cognitive impacts of providing correct and incorrect machine learning (ML) outputs in support of an object detection task. The study consisted of five experiments that manipulated the accuracy and importance of mock ML outputs. In each of the experiments, participants were given the T and L task with T-shaped targets and L-shaped distractors. They were tasked with categorizing each image as target present or target absent. In Experiment 1, they performed this task without the aid of ML outputs. In Experiments 2–5, they were shown images with bounding boxes, representing the output of an ML model. The outputs could be correct (hits and correct rejections), or they could be erroneous (false alarms and misses). Experiment 2 manipulated the overall accuracy of these mock ML outputs. Experiment 3 manipulated the proportion of different types of errors. Experiments 4 and 5 manipulated the importance of specific types of stimuli or model errors, as well as the framing of the task in terms of human or model performance. These experiments showed that model misses were consistently harder for participants to detect than model false alarms. In general, as the model’s performance increased, human performance increased as well, but in many cases the participants were more likely to overlook model errors when the model had high accuracy overall. Warning participants to be on the lookout for specific types of model errors had very little impact on their performance. Overall, our results emphasize the importance of considering human cognition when determining what level of model performance and types of model errors are acceptable for a given task.

People frequently make decisions based on uncertain information. Prior research has shown that visualizations of uncertainty can help to support better decision making. However, research has also shown that different representations of the same information can lead to different patterns of decision making. It is crucial for researchers to develop a better scientific understanding of when, why and how different representations of uncertainty lead viewers to make different decisions. This paper seeks to address this need by comparing geospatial visualizations of wildfire risk to verbal descriptions of the same risk. In three experiments, we manipulated the specificity of the uncertain information as well as the visual cues used to encode risk in the visualizations. All three experiments found that participants were more likely to evacuate in response to a hypothetical wildfire if the risk information was presented verbally. When the risk was presented visually, participants were less likely to evacuate, particularly when transparency was used to encode the risk information. Experiment 1 showed that evacuation rates were lower for transparency maps than for other types of visualizations. Experiments 2 and 3 sought to replicate this effect and to test how it related to other factors. Experiment 2 varied the hue used for the transparency maps and Experiment 3 manipulated the salience of the borders between the different risk levels. These experiments showed lower evacuation rates in response to transparency maps regardless of hue. The effect was partially, but not entirely, mitigated by adding salient borders to the transparency maps. Taken together, these experiments show that using transparency to encode information about risk can lead to very different patterns of decision making than other encodings of the same information.

Small multiples are a popular visualization method, displaying different views of a dataset using multiple frames, often with the same scale and axes. However, there is a need to address their potential constraints, especially in the context of human cognitive capacity limits. These limits dictate the maximum information our mind can process at once. We explore the issue of capacity limitation by testing competing theories that describe how the number of frames shown in a display, the scale of the frames, and time constraints impact user performance with small multiples of line charts in an energy grid scenario. In two online studies (Experiment 1 n = 141 and Experiment 2 n = 360) and a follow-up eye tracking analysis (n = 5), we found a linear decline in accuracy with increasing frames across seven tasks, which was not fully explained by differences in frame size, suggesting visual search challenges. Moreover, the studies demonstrate that highlighting specific frames can mitigate some visual search difficulties but, surprisingly, not eliminate them. This research offers insights into optimizing the utility of small multiples by aligning them with human limitations.

Small multiples are a popular visualization method, displaying different views of a dataset using multiple frames, often with the same scale and axes. However, there is a need to address their potential constraints, especially in the context of human cognitive capacity limits. These limits dictate the maximum information our mind can process at once. We explore the issue of capacity limitation by testing competing theories that describe how the number of frames shown in a display, the scale of the frames, and time constraints impact user performance with small multiples of line charts in an energy grid scenario. In two online studies (Experiment 1 n = 141 and Experiment 2 n = 360) and a follow-up eye-tracking analysis ( n = 5), we found a linear decline in accuracy with increasing frames across seven tasks, which was not fully explained by differences in frame size, suggesting visual search challenges. Moreover, the studies demonstrate that highlighting specific frames can mitigate some visual search difficulties but, surprisingly, not eliminate them. This research offers insights into optimizing the utility of small multiples by aligning them with human limitations.

The scientific community heavily relies on data visualizations for communication, precipitating the need to better understand what makes for a “good” data visualization (e.g., informing data visualization evaluation or saliency tools). In addition to the underlying mathematical or bottom-up properties of a visualization, designers must also account for the influence from top-down factors such as the viewer’s goal or perceptual biases. In the current study, we asked participants to compare two clusters of data points in a scatterplot (similar to a multidimensional data reduction comparison task). We manipulated both the underlying mathematical properties of the data set and the decision-making task. We found evidence for visual–spatial biases and differences in overt attention patterns (eye movements), even when the compared clusters were mathematically equivalent. These results demonstrate how task and perceptual biases may impact viewers’ understanding of relationships between variables in a multidimensional space, possibly leading to error or systematic biases in analysts’ interpretation of the plotted data.

One of the major challenges for evaluating the effectiveness of data visualizations and visual analytics tools arises from the fact that different users may be using these tools for different tasks. In this chapter, we present a simple example of how different tasks lead to different patterns of attention to the same underlying data visualizations. The experiment used eye tracking to record where people looked in scatterplot visualizations when given different tasks. We argue that the general approach used in this experiment could be applied systematically to task and feature taxonomies that have been developed by visualization researchers. Using eye tracking to study the impact of common tasks on how humans attend to common types of visualizations will support a deeper understanding of visualization cognition and the development of more robust methods for evaluating the effectiveness of visualizations.

Although visualizations are a useful tool for helping people to understand information, they can also have unintended effects on human cognition. This is especially true for uncertain information, which is difficult for people to understand. Prior work has found that different methods of visualizing uncertain information can produce different patterns of decision making from users. However, uncertainty can also be represented via text or numerical information, and few studies have systematically compared these types of representations to visualizations of uncertainty. We present two experiments that compared visual representations of risk (icon arrays) to numerical representations (natural frequencies) in a wildfire evacuation task. Like prior studies, we found that different types of visual cues led to different patterns of decision making. In addition, our comparison of visual and numerical representations of risk found that people were more likely to evacuate when they saw visualizations than when they saw numerical representations. These experiments reinforce the idea that design choices are not neutral: seemingly minor differences in how information is represented can have important impacts on human risk perception and decision making.