PosterPDF Available

Problem Solving Prerequisites for Bayesian Reasoning

Authors:

Abstract

Solving Bayesian reasoning problems requires correctly identifying, computing, and applying values from the problem text to the solution. Identification refers to understanding the intended meaning of the values. Computation refers to the mathematical manipulation of those values. Application goes one-step further by utilizing those identified and/or computed values in the solution. We evaluated performance on eight Bayesian reasoning problems using probing questions that separate out the extent to which uninitiated reasoners can identify, compute, and apply values from problem to solution. The results suggest that reasoners are generally proficient at identifying values, but struggle with computation and application.
Introduction
Solving Bayesian reasoning problems requires correctly identifying,
computing, and applying values from the problem text to the solution.
Identification refers to understanding the intended meaning of the values.
Computation refers to the mathematical manipulation of those values.
Application goes one-step further by utilizing those identified and/or
computed values in the solution. We evaluated performance on eight
Bayesian reasoning problems using probing questions that separate out the
extent to which uninitiated reasoners can identify, compute, and apply values
from problem to solution. The results suggest that reasoners are generally
proficient at identifying values, but struggle with computation and application.
Bayesian Reasoning Problems
Summary of Findings
Cosmides, L. & Tooby, J. (1996). Are humans good intuitive statisticians after all? Rethinking
some conclusions from the literature on judgment under uncertainty. Cognition, 58, 1-73.
Talboy, A. N., & Schneider, S. L. (in press). Focusing on what matters: Rewriting Bayesian
reasoning problems. Journal of Experimental Psychology: Applied.
Weller, J. A., Dieckmann, N. F., Tusler, M., Mertz, C. K., Burns, W. J., & Peters, E. (2012).
Development and testing of an abbreviated numeracy scale: A Rasch analysis approach.
Journal of Behavioral Decision Making, 26(2), 198-212.
Study Design
Problem Solving Prerequisites for Bayesian Reasoning
Alaina N. Talboy, M.A. and Sandra L. Schneider, Ph.D.
Judgment and Decision Making Lab, Department of Psychology, University of South Florida, Tampa, FL, USA
Identification
Key References
Every participant correctly identified the focal reference class
from the problem presentation (C+; 100% accuracy)
Identification: As more information was requested, more
participants made errors (no difference based on numeracy; Fisher Exact
Test [FET] all comparisons p> .05)
Application of Identified Subset (C+T+): Errors were made by 18%
with Higher Numeracy vs 38% with Low Numeracy (FET p= .01,
Cramer’s V= .23)
Computation of Competing Reference Class (T+): Errors were
made by 13% with Higher Numeracy vs 29% with Low Numeracy
at this step (FET p= .06, Cramers V= .14)
Application of Computed Reference Class (T+): Errors were made
by 23% with Higher Numeracy vs 34% with Low Numeracy (FET p<
.001, Cramers V= .27)
Note. T± indicates positive or negative test result. C± indicates presence or
absence of the condition. Problems available in Talboy & Schneider, 2018.
8 Bayesian Reasoning Problems Presented in the Condition-Focused Format
Abbreviated Numeracy Scale (Cronbach’s α = .60; M= 3.97, SD = 1.70)
Score 4-8: Higher Numeracy (n= 83)
Score 0-3: Low Numeracy (n= 56)
Quasi-Experimental Design
Topic N C+T+ C+T- C-T+ C-T-
Mammogram
10,000 80 20 990 8,910
Diabetes
10,000 48 2 4,975 4,972
Polygraph
1,000 47 3 47 903
Recidivism
1,000 130 26 220 624
Tennis
10,000 2,000 800 1,100 6,100
Baseball
250 130 55 15 50
Employment
200 70 70 10 50
Exam Prep
500 275 75 25 125
Table 1. Characteristics of Bayesian Reasoning Problems
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Identify C+ Identify C+T+ or C-T+ Identify Both C+T+ and
C-T+
Percent of Participants
Identification Step
Higher Numeracy
Low Numeracy
Identification and Application
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Identify Identify and Apply C+T+
Percent of Participants
Identification and Application of C+T+
Higher Numeracy
Low Numeracy
Figure 2. Application of the identified target subgroup (C+T+) was difficult for some in both
numeracy groups, especially the lower numeracy group, leading to incorrect answers.
Figure 1. There were no differences in identification of values between those with low versus
higher numeracy. There was a general decline in accuracy as number of questions increased.
Calculation and Application
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Calculate T+ Calculate and Apply T+
Percent of Participants
Calculation and Application of T+
Higher Numeracy
Low Numeracy
Figure 3. Calculating and applying the appropriate reference class (T+) was a challenge. Those
with low numeracy had more difficulty with both than those with higher numeracy.
Understanding
the intended
meaning of
values
Identification
Mathematical
manipulation
of identified
values
Computation Use of identified
and computed
values in the
problem
solution
Application
Identification of Necessary Values form Problem
How many have the condition? (C+)
How many have the condition and test positive? (C+T+)
How many do not have the condition and test positive? (C-T+)
Computation Using Problem Values to Move toward Solution
How many will test positive, whether they have the condition or not? (T+)
Application of Identified or Computed Values to Solution
Based on the number of people who test positive, how many who test positive
actually have the condition? (C+T+ out of T+ people)
Based on the number of people who test positive, what is the probability that
a person who tests positive actually has the condition? (_____%)
Probing questions can be used to determine how well reasoners can identify
versus calculate different numeric values from the problem (e.g., Cosmides &
Tooby, 1996; Talboy & Schneider, 2018), which are prerequisites to application.
Dependent Variable: Probing Questions
Implications
Identification becomes increasingly more difficult as more
complex (i.e., subset) information is requested
Those with low numeracy have more difficult with:
Application of Identified Values
Computation of Values
Application of Computed Values
ResearchGate has not been able to resolve any citations for this publication.
ResearchGate has not been able to resolve any references for this publication.