Content uploaded by Alaina Talboy

Author content

All content in this area was uploaded by Alaina Talboy on Nov 20, 2018

Content may be subject to copyright.

Introduction

Solving Bayesian reasoning problems requires correctly identifying,

computing, and applying values from the problem text to the solution.

Identification refers to understanding the intended meaning of the values.

Computation refers to the mathematical manipulation of those values.

Application goes one-step further by utilizing those identified and/or

computed values in the solution. We evaluated performance on eight

Bayesian reasoning problems using probing questions that separate out the

extent to which uninitiated reasoners can identify, compute, and apply values

from problem to solution. The results suggest that reasoners are generally

proficient at identifying values, but struggle with computation and application.

Bayesian Reasoning Problems

Summary of Findings

Cosmides, L. & Tooby, J. (1996). Are humans good intuitive statisticians after all? Rethinking

some conclusions from the literature on judgment under uncertainty. Cognition, 58, 1-73.

Talboy, A. N., & Schneider, S. L. (in press). Focusing on what matters: Rewriting Bayesian

reasoning problems. Journal of Experimental Psychology: Applied.

Weller, J. A., Dieckmann, N. F., Tusler, M., Mertz, C. K., Burns, W. J., & Peters, E. (2012).

Development and testing of an abbreviated numeracy scale: A Rasch analysis approach.

Journal of Behavioral Decision Making, 26(2), 198-212.

Study Design

Problem Solving Prerequisites for Bayesian Reasoning

Alaina N. Talboy, M.A. and Sandra L. Schneider, Ph.D.

Judgment and Decision Making Lab, Department of Psychology, University of South Florida, Tampa, FL, USA

Identification

Key References

•Every participant correctly identified the focal reference class

from the problem presentation (C+; 100% accuracy)

•Identification: As more information was requested, more

participants made errors (no difference based on numeracy; Fisher Exact

Test [FET] all comparisons p> .05)

•Application of Identified Subset (C+T+): Errors were made by 18%

with Higher Numeracy vs 38% with Low Numeracy (FET p= .01,

Cramer’s V= .23)

•Computation of Competing Reference Class (T+): Errors were

made by 13% with Higher Numeracy vs 29% with Low Numeracy

at this step (FET p= .06, Cramer’s V= .14)

•Application of Computed Reference Class (T+): Errors were made

by 23% with Higher Numeracy vs 34% with Low Numeracy (FET p<

.001, Cramer’s V= .27)

Note. T± indicates positive or negative test result. C± indicates presence or

absence of the condition. Problems available in Talboy & Schneider, 2018.

•8 Bayesian Reasoning Problems Presented in the Condition-Focused Format

•Abbreviated Numeracy Scale (Cronbach’s α = .60; M= 3.97, SD = 1.70)

•Score 4-8: Higher Numeracy (n= 83)

•Score 0-3: Low Numeracy (n= 56)

Quasi-Experimental Design

Topic N C+T+ C+T- C-T+ C-T-

Mammogram

10,000 80 20 990 8,910

Diabetes

10,000 48 2 4,975 4,972

Polygraph

1,000 47 3 47 903

Recidivism

1,000 130 26 220 624

Tennis

10,000 2,000 800 1,100 6,100

Baseball

250 130 55 15 50

Employment

200 70 70 10 50

Exam Prep

500 275 75 25 125

Table 1. Characteristics of Bayesian Reasoning Problems

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Identify C+ Identify C+T+ or C-T+ Identify Both C+T+ and

C-T+

Percent of Participants

Identification Step

Higher Numeracy

Low Numeracy

Identification and Application

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Identify Identify and Apply C+T+

Percent of Participants

Identification and Application of C+T+

Higher Numeracy

Low Numeracy

Figure 2. Application of the identified target subgroup (C+T+) was difficult for some in both

numeracy groups, especially the lower numeracy group, leading to incorrect answers.

Figure 1. There were no differences in identification of values between those with low versus

higher numeracy. There was a general decline in accuracy as number of questions increased.

Calculation and Application

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Calculate T+ Calculate and Apply T+

Percent of Participants

Calculation and Application of T+

Higher Numeracy

Low Numeracy

Figure 3. Calculating and applying the appropriate reference class (T+) was a challenge. Those

with low numeracy had more difficulty with both than those with higher numeracy.

•Understanding

the intended

meaning of

values

Identification

•Mathematical

manipulation

of identified

values

Computation •Use of identified

and computed

values in the

problem

solution

Application

Identification of Necessary Values form Problem

•How many have the condition? (C+)

•How many have the condition and test positive? (C+T+)

•How many do not have the condition and test positive? (C-T+)

Computation Using Problem Values to Move toward Solution

•How many will test positive, whether they have the condition or not? (T+)

Application of Identified or Computed Values to Solution

•Based on the number of people who test positive, how many who test positive

actually have the condition? (C+T+ out of T+ people)

•Based on the number of people who test positive, what is the probability that

a person who tests positive actually has the condition? (_____%)

Probing questions can be used to determine how well reasoners can identify

versus calculate different numeric values from the problem (e.g., Cosmides &

Tooby, 1996; Talboy & Schneider, 2018), which are prerequisites to application.

Dependent Variable: Probing Questions

Implications

•Identification becomes increasingly more difficult as more

complex (i.e., subset) information is requested

•Those with low numeracy have more difficult with:

•Application of Identified Values

•Computation of Values

•Application of Computed Values