Matthew B. Weinger’s research while affiliated with Vanderbilt University and other places

Introduction: As the number of available treatment options for breast cancer increases, decision-making for patients has become complex. Patients often struggle to make decisions as treatment options can vary in terms of short- and long-term side effects, risks of recurrence, and impact on daily life.1 Numerous decision aids have been developed to support patient decision-making.2 However, sustained implementation and use of these tools remains limited. We propose that cognitive engineering approaches, such as naturalistic decision making (NDM), can provide a deeper understanding of how patients make treatment decisions, which can improve the design of decision support tools. Naturalistic decision making (NDM) is a theoretical perspective and methodological approach used to understand how people make decisions in the real world. Originally developed to understand decision-making of expert firefighters during crises, NDM approaches have been used to understand complex decision-making across domains including the military, offshore oil rigs, and healthcare.3,4 In this study, we used an NDM approach, the critical decision method (CDM), to gain an in-depth understanding of how breast cancer patients make treatment decisions following diagnosis. Methods: We conducted CDM interviews,5 with breast cancer patients diagnosis in the last 12 years. CDM interviews aim to understand critical or difficult events by unpacking the event using structured probes. One researcher conducted each interview over Zoom. We started each interview by asking the patient to reflect back on the beginning of their cancer journey and what they remember about their diagnosis. We then drew a timeline and asked the patient to relay the different treatments they considered or underwent for breast cancer. We then asked “Can you think of a time during your breast cancer journey when you had to make a difficult decision?” and probed patients about that decision. We continued asking patients about their treatment decision-making as time allowed. Each interview was audio-recorded and transcribed. A researcher and a patient advocate coded each interview and created a decision requirements table,6 which detailed the decisions made by the patient, what made that decision challenging, what strategies and information they used, and what their goals were at the time. We then met to discuss and come to consensus. Once a decision requirements table was created for each transcript, we developed aggregate tables and identified key themes. Results: We conducted 20 interviews, averaging 57 minutes each; patient age ranged from 42 to 81 years. Patients described an average of 8 decisions that they made following breast cancer diagnosis. Despite many patients facing the same decisions (e.g., mastectomy vs. lumpectomy), we found variability in which decisions were most difficult for patients. We identified 11 categories of difficult decisions for patients including whether to receive chemotherapy, getting genetic testing, stopping a medication due to side effects, and deciding where to receive treatment. Patients reported feeling time pressure and urgency to make treatment decisions and a fear of regretting their decisions. We found that patients’ firsthand experiences from friends who had cancer influenced their treatment decision-making. Given the heterogeneous nature of breast cancer treatment, this often presented a barrier to decision-making as patients expected to have the same experience and treatment options as their friends. Patients expressed variable goals when making treatment decisions, which often changed throughout their treatment journey. Conclusion: In this study, we explored how breast cancer patients made treatment decisions using NDM methods. This cognitive engineering approach revealed intricacies in the decision-making process of patients that will be valueable for improving the design of decision support tools. Next steps include collaborative design with patients to develop a tool that supports the broad spectrum of treatment decisions made across the patient journey. Citation Format: Megan Salwei, Barbara Voigtman, Janelle Faiman, Carrie Reale, Shilo Anders, Matthew Weinger. Harnessing Cognitive Engineering to Understand Breast Cancer Patient Decision Making [abstract]. In: Proceedings of the San Antonio Breast Cancer Symposium 2024; 2024 Dec 10-13; San Antonio, TX. Philadelphia (PA): AACR; Clin Cancer Res 2025;31(12 Suppl):Abstract nr P4-04-07.

Background A common cause of preventable harm is the failure to detect and appropriately respond to clinical deterioration. Timely intervention is needed, particularly in medically complex patients, to mitigate the effects of adverse events, disease progression, and medical error. This challenging problem requires clinical surveillance, early recognition, timely notification of the appropriate clinicians, and effective intervention. Objectives We determined the feasibility of designing, developing, and implementing the tools and processes to create a surveillance-and-risk prediction system to detect clinical deterioration in cancer outpatients. Methods We used systems engineering and iterative human-centered design to develop a functional prototype of a surveillance-and-risk prediction system. The system includes passive surveillance involving wearable sensors, active surveillance involving patient event and symptom reporting as well as extraction of selected patient data from the electronic health record (EHR), a predictive model, and communication of estimated risk to clinicians. System usability was evaluated using patient and clinician interviews and clinician ratings using the System Usability Scale (SUS). Results Fifty of 71 recruited patients enrolled in the feasibility study. Patient-reported outcome measures and clinical data extracted from the EHR were the best predictors of a patient's 7-day risk of experiencing unplanned treatment events (UTEs, i.e., emergency room visits, hospital admissions, or major treatment changes). Deep learning neural network models using these predictors demonstrated modest performance in predicting 7-day UTE risk (PROMS, F-measure: 0.900, area under the receiver operating characteristic curve [AUC-ROC]: 0.983; clinical data from EHR F-measure: 0.625, AUC-ROC: 0.983). Patient risk scores were communicated to clinicians using a risk communication prototype rated favorably by clinicians with a SUS score of 76 out of 100 (median = 80; range: 60–85). Conclusion We demonstrate the feasibility of a surveillance-and-risk prediction system for detecting and reporting clinical deterioration in cancer outpatients. Future research is needed to fully implement and evaluate system adoption and effectiveness under different clinical situations.

BACKGROUND Only 15% of the nearly 30 million Americans with hearing loss use hearing aids, partly due to high cost, stigma, and limited access to required prior medical evaluations. Hearing impairment in adults can lead to social isolation and depression and is associated with an increased risk of falls as well as dementia. Given the persistent barriers to hearing aid use, the Food and Drug Administration (FDA) recently issued a final rule to allow over-the-counter hearing aids to be sold directly to adult consumers with perceived mild-to-moderate hearing loss at pharmacies, stores, and online retailers without seeing a physician or licensed hearing health care professional. OBJECTIVE We evaluated the safety and usability of an over-the-counter hearing aid prior to FDA approval and market release. METHODS We first conducted a formative usability test of the device and associated App with 5 intended users to identify outstanding safety and usability issues. Following design modifications, we performed a second round with 15 intended users of the device. Participants were asked to complete 2-5 tasks, as if they were using the hearing aid in real life. After each task, participants rated the task difficulty. At the end of each session, participants completed a 10-question knowledge assessment and the system usability scale (SUS) and then participated in debriefing interviews to gather qualitative feedback. All sessions were video recorded and analyzed to identify use errors and design improvement opportunities. We concurrently conducted a test with 21 non-intended users (i.e., users with contraindications to use) to ascertain if consumers could determine when they should not use the device, based on the packaging, instructions, and labeling. RESULTS In all three usability tests, usability issues were identified. There were minimal safety-related issues with the device. Round 1 testing led to several design modifications which then increased task success in Round 2 testing. Participants had the most difficulty with the task of pairing the hearing aids to the cellphone. Participants also had difficulty distinguishing the right and left earbuds. Non-intended users did not always understand device contraindications (e.g., tinnitus, severe hearing loss). Overall, test findings informed eight actionable design modifications that improved device usability and safety. CONCLUSIONS This study evaluated the usability and safety of an over-the-counter hearing aid for adults with mild-to-moderate hearing loss. Human factors engineering methods identified opportunities to improve the safety and usability of this direct-to-consumer medical device for individuals with perceived mild-moderate hearing loss.

Background Only 15% of the nearly 30 million Americans with hearing loss use hearing aids, partly due to high cost, stigma, and limited access to professional hearing care. Hearing impairment in adults can lead to social isolation and depression and is associated with an increased risk of falls. Given the persistent barriers to hearing aid use, the Food and Drug Administration issued a final rule to allow over-the-counter hearing aids to be sold directly to adult consumers with perceived mild to moderate hearing loss at pharmacies, stores, and online retailers without seeing a physician or licensed hearing health care professional. Objective We evaluated the safety and usability of an over-the-counter hearing aid prior to Food and Drug Administration approval and market release. Methods We first conducted a formative usability test of the device and associated app with 5 intended users to identify outstanding safety and usability issues (testing round 1). Following design modifications, we performed a summative usability test with 15 intended users of the device (testing round 2). We concurrently conducted a test with 21 nonintended users (ie, users with contraindications to use) to ascertain if consumers could determine when they should not use the device, based on the packaging, instructions, and labeling (testing round 3). Participants were asked to complete 2‐5 tasks, as if they were using the hearing aid in real life. After each task, participants rated the task difficulty. At the end of each session, participants completed a 10-question knowledge assessment and the System Usability Scale and then participated in debriefing interviews to gather qualitative feedback. All sessions were video recorded and analyzed to identify use errors and design improvement opportunities. Results Usability issues were identified in all 3 usability testing rounds. There were minimal safety-related issues with the device. Round 1 testing led to several design modifications which then increased task success in round 2 testing. Participants had the most difficulty with the task of pairing the hearing aids to the cell phone. Participants also had difficulty distinguishing the right and left earbuds. Nonintended users did not always understand device contraindications (eg, tinnitus and severe hearing loss). Overall, test findings informed 9 actionable design modifications (eg, clarifying pairing steps and increasing font size) that improved device usability and safety. Conclusions This study evaluated the usability and safety of an over-the-counter hearing aid for adults with mild to moderate hearing loss. Human factors engineering methods identified opportunities to improve the safety and usability of this direct-to-consumer medical device for individuals with perceived mild-moderate hearing loss.

Background: Hospitalization rates for childhood pneumonia vary widely. Risk-based clinical decision support (CDS) interventions may reduce unwarranted variation. Methods: We conducted a pragmatic randomized trial in two US pediatric emergency departments (EDs) comparing electronic health record (EHR)-integrated prognostic CDS versus usual care for promoting appropriate ED disposition in children (<18 years) with pneumonia. Encounters were randomized 1:1 to usual care versus custom CDS featuring a validated pneumonia severity score predicting risk for severe in-hospital outcomes. Clinicians retained full decision-making authority. The primary outcome was inappropriate ED disposition, defined as early transition to lower- or higher-level care. Safety and implementation outcomes were also evaluated. Results: The study enrolled 536 encounters (269 usual care and 267 CDS). Baseline characteristics were similar across arms. Inappropriate disposition occurred in 3% of usual care encounters and 2% of CDS encounters (adjusted odds ratio: 0.99, 95% confidence interval: [0.32, 2.95]) Length of stay was also similar and adverse safety outcomes were uncommon in both arms. The tool's custom user interface and content were viewed as strengths by surveyed clinicians (>70% satisfied). Implementation barriers include intrinsic (e.g., reaching the right person at the right time) and extrinsic factors (i.e., global pandemic). Conclusions: EHR-based prognostic CDS did not improve ED disposition decisions for children with pneumonia. Although the intervention's content was favorably received, low subject accrual and workflow integration problems likely limited effectiveness. Clinical Trials Registration: NCT06033079.

Objective: To support a pragmatic, electronic health record (EHR)-based randomized controlled trial, we applied user-centered design (UCD) principles, evidence-based risk communication strategies, and interoperable software architecture to design, test, and deploy a prognostic tool for children in emergency departments (EDs) with pneumonia. Methods: Risk for severe in-hospital outcomes was estimated using a validated ordinal logistic regression model to classify pneumonia severity. To render the results usable for ED clinicians, we created an integrated SMART on FHIR web application built for interoperable use in two pediatric EDs using different EHR vendors: Epic and Cerner. We followed a UCD framework, including problem analysis and user research, conceptual design and early prototyping, user interface development, formative evaluation, and post-deployment summative evaluation. Results: Problem analysis and user research from 39 clinicians and nurses revealed user preferences for risk aversion, accessibility, and timing of risk communication. Early prototyping and iterative design incorporated evidence-based design principles, including numeracy, risk framing, and best-practice visualization techniques. After rigorous unit and end-to-end testing, the application was successfully deployed in both EDs, which facilitatd enrollment, randomization, model visualization, data capture, and reporting for trial purposes. Conclusions: The successful implementation of a custom application for pneumonia prognosis and clinical trial support in two health systems on different EHRs demonstrates the importance of UCD, adherence to modern clinical data standards, and rigorous testing. Key lessons included the need for understanding users' real-world needs, regular knowledge management, application maintenance, and the recognition that FHIR applications require careful configuration for interoperability.

Purpose of review This article explores the impact of recent applications of artificial intelligence on clinical anesthesiologists’ decision-making. Recent findings Naturalistic decision-making, a rich research field that aims to understand how cognitive work is accomplished in complex environments, provides insight into anesthesiologists’ decision processes. Due to the complexity of clinical work and limits of human decision-making (e.g. fatigue, distraction, and cognitive biases), attention on the role of artificial intelligence to support anesthesiologists’ decision-making has grown. Artificial intelligence, a computer's ability to perform human-like cognitive functions, is increasingly used in anesthesiology. Examples include aiding in the prediction of intraoperative hypotension and postoperative complications, as well as enhancing structure localization for regional and neuraxial anesthesia through artificial intelligence integration with ultrasound. Summary To fully realize the benefits of artificial intelligence in anesthesiology, several important considerations must be addressed, including its usability and workflow integration, appropriate level of trust placed on artificial intelligence, its impact on decision-making, the potential de-skilling of practitioners, and issues of accountability. Further research is needed to enhance anesthesiologists’ clinical decision-making in collaboration with artificial intelligence.

Background Electronic health record (EHR) system transitions are challenging for healthcare organizations. High-volume, safety–critical tasks like barcode medication administration (BCMA) should be evaluated, yet standards for ensuring safety during transition have not been established. Objective Identify risks in common and problem-prone medication tasks to inform safe transition between BCMA systems and establish benchmarks for future system changes. Design Staff nurses completed simulation-based usability testing in the legacy system (R1) and new system pre- (R2) and post-go-live (R3). Tasks included (1) Hold/Administer, (2) IV Fluids, (3) PRN Pain, (4) Insulin, (5) Downtime/PRN, and (6) Messaging. Audiovisual recordings of task performance were systematically analyzed for time, navigation, and errors. The System Usability Scale measured perceived usability and satisfaction. Post-simulation interviews captured nurses’ qualitative comments and perceptions of the systems. Participants Fifteen staff nurses completed 2–3-h simulation sessions. Eleven completed both R1 and R2, and seven completed all three rounds. Clinical experience ranged from novice (< 1 year) to experienced (> 10 years). Practice settings included adult and pediatric patient populations in ICU, stepdown, and acute care departments. Main Measures Task completion rates/times, safety and non-safety-related use errors (interaction difficulties), and user satisfaction. Key Results Overall success rates remained relatively stable in all tasks except two: IV Fluids task success increased substantially (R1: 17%, R2: 54%, R3: 100%) and Downtime/PRN task success decreased (R1: 92%, R2: 64%, R3: 22%). Among the seven nurses who completed all rounds, overall safety-related errors decreased 53% from R1 to R3 and 50% from R2 to R3, and average task times for successfully completed tasks decreased 22% from R1 to R3 and 38% from R2 to R3. Conclusions Usability testing is a reasonable approach to compare different BCMA tasks to anticipate transition problems and establish benchmarks with which to monitor and evaluate system changes going forward.