Stall Catcher interface showing feedback after a decision was made during Stage 1 and 3.

Stall Catcher interface showing feedback after a decision was made during Stage 1 and 3.

Source publication
Preprint
Full-text available
Human-AI collaboration for decision-making strives to achieve team performance that exceeds the performance of humans or AI alone. However, many factors can impact success of Human-AI teams, including a user's domain expertise, mental models of an AI system, trust in recommendations, and more. This work examines users' interaction with three simula...

Contexts in source publication

Context 1
... experimentation platform provides a proxy for problems relating to Human-AI interaction in medical decision-making. Several modifications were made to the experimental version of the Stall Catchers interface: 1) all gamification accoutrements (e.g., leaderboard, score, in-game chat, etc.) were stripped from the interface to reduce bias that might originate from point incentives, 2) a progress bar was added to the top of the interface to indicate the current stage and their progression through that stage (Figure 2), 3) the requirement and interface for indicating the location of an identified stall was removed, and 4) a small icon of a robot, with a hand pointing to either the "Flowing" or "Stalled" button, indicated the AI-Assistant's recommendation ( Figure 3). We describe the details of these changes and setup in the following section. ...
Context 2
... videos were sampled from a database with a validation set of 7,693 videos containing ground truth labels verified by expert players (i.e., players whose sensitivity was 1) not involved in the study. Each video contained cross-sectional views of blood flowing through vessels in mice brains (Figure 2). Each video also had an associated difficulty metric (0,1) which was defined based on average response classification accuracy from historical Stall Catchers data. ...
Context 3
... did not receive any form of assistance. After providing a response to a video, participants were shown the correct answer ( Figure 2). This stage served as a practice stage. ...

Similar publications

Preprint
Full-text available
People spend a significant amount of time trying to make sense of the internet, collecting content from a variety of sources and organizing it to make decisions and achieve their goals. While humans are able to fluidly iterate on collecting and organizing information in their minds, existing tools and approaches introduce significant friction into...