Foundations of intuitive power analyses in children and adults

Abstract

Decades of research indicate that some of the epistemic practices that support scientific enquiry emerge as part of intuitive reasoning in early childhood. Here, we ask whether adults and young children can use intuitive statistical reasoning and metacognitive strategies to estimate how much information they might need to solve different discrimination problems, suggesting that they have some of the foundations for ‘intuitive power analyses’. Across five experiments, both adults (N = 290) and children (N = 48, 6–8 years) were able to precisely represent the relative difficulty of discriminating populations and recognized that larger samples were required for populations with greater overlap. Participants were sensitive to the cost of sampling, as well as the perceptual nature of the stimuli. These findings indicate that both young children and adults metacognitively represent their own ability to make discriminations even in the absence of data, and can use this to guide efficient and effective exploration. Adults and children can represent the relative difficulty of discriminating two populations and recognize that larger samples are required for populations with greater overlap. This suggests that they have foundations for ‘intuitive power analyses’.

Full text

Articles
https://doi.org/10.1038/s41562-022-01427-2
Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA. ✉e-mail: lschulz@mit.edu
In science, we use power analyses to estimate how much evidence
is needed to test the hypotheses we are considering. For instance,
if we suspect that a drug will cure 90% of patients while a placebo
will work 30% of the time, we will conclude that a relatively small
sample of patients will enable us to test the drug’s efficacy; by con-
trast, if we believe the drug will cure only 50% of patients, we know
we will need a larger sample to show an effect. Exact power analy-
ses, of course, require formal training in statistics and an estimate
of the size of the effect the researcher aims to demonstrate. But the
intuitive foundation of these analyses—that it takes more evidence
to distinguish some hypotheses than others—may be part of com-
mon sense reasoning more broadly. Decades of research suggest
that some of the epistemic practices that support scientific enquiry
emerge as part of intuitive reasoning in early childhood. Here we
look at whether lay adults and young children can use these intui-
tive metacognitive abilities to represent how much information they
need to distinguish between populations depending on the degree
to which the populations overlap.
This question is grounded in a long tradition of work suggest-
ing that children are sensitive to statistics in the environment and
engage in selective exploration to maximize expected information
gain. Infants attend to the transitional probability of events in both
auditory1 and visual2 stimuli and can infer abstract rules from pat-
terns of data (for example, the difference between ABA and ABB
patterns3). Infants can also infer probable outcomes simply from
the number of modal possibilities; thus, for instance, if three blue
objects and one red one are spinning in an open container, infants
look longer if the red object falls out than if a blue one does4.
Additionally, infants are sensitive to the relationship between sam-
ples and populations: If most of the objects in a box are red, babies
expect most of the objects pulled from the box to be red but suspend
this inference if the objects are drawn from somewhere other than
the box (that is, the experimenter’s lap) or are drawn selectively by
an experimenter searching through the box rather than randomly5.
Infants also actively explore to reduce uncertainty. Seven- and
8-month-olds are more likely to look away from events that are
very predictable or unpredictable relative to moderately surpris-
ing events6 and 12-month-olds not only look longer at events that
violate their intuitive theories, but intervene to explore the viola-
tions (for example, dropping objects that appeared to violate grav-
ity; banging objects that appeared to violate solidity7).
Children’s sensitivity to the relationship between hypotheses
and evidence becomes increasingly sophisticated from toddlerhood
to middle childhood. Toddlers are able to integrate observed data
with their previous beliefs, and their inferences are not only sensi-
tive to the content of a sample, but also whether that sample was
drawn randomly or selectively8. When preschoolers observe events
that cannot be explained by a known cause, they posit unobserved
variables to explain the event9,10, and selectively explain and explore
events that violate their causal theories11,12. Preschoolers can also
use the base rate of events to distinguish more and less probable
hypotheses13,14, isolate variables in a causal system to distinguish
candidate causes11, and integrate the evidence they observe with the
testimony they hear from knowledgeable sources15.
However, it is less clear to what extent young children have an
explicit understanding of the relationship between the evidence
they observe and the knowledge they will gain. Some studies sug-
gest that at least some precursors to metacognition emerge early.
Preschoolers correctly distinguish objects they can and cannot
name16, spend more time considering response options given unin-
formative versus informative task instructions17, selectively with-
hold answers18–22, ask for help23 on items they struggle to remember
and show more pupillary dilation and give higher confidence rat-
ings to remembered items24,25, see also refs. 18,22,26–28. However, young
children may know when they are more or less certain about infor-
mation without using this knowledge to increase opportunities for
learning. In self-paced learning tasks, 5-, 6- and 7-year-olds are more
confident about correctly than incorrectly remembered items but
only 6- and 7-year-olds accurately anticipate which items they will
have trouble learning and dedicate more study time to these items29.
Consistent with this, school-aged children often struggle with meta-
cognitive tasks in the context of test-taking: children are overcon-
fident in their memory30,31, and choose randomly when given the
chance to restudy test items rather than choosing on the basis of
their previous performance32. Introspective self-reports of knowl-
edge also improve from early school age to adolescence33–37. Thus,
Foundations of intuitive power analyses in
children and adults
Madeline C. Pelz, Kelsey R. Allen, Joshua B. Tenenbaum and Laura E. Schulz  ✉
Decades of research indicate that some of the epistemic practices that support scientific enquiry emerge as part of intuitive
reasoning in early childhood. Here, we ask whether adults and young children can use intuitive statistical reasoning and meta-
cognitive strategies to estimate how much information they might need to solve different discrimination problems, suggesting
that they have some of the foundations for ‘intuitive power analyses’. Across five experiments, both adults (N = 290) and chil-
dren (N = 48, 6–8 years) were able to precisely represent the relative difficulty of discriminating populations and recognized
that larger samples were required for populations with greater overlap. Participants were sensitive to the cost of sampling, as
well as the perceptual nature of the stimuli. These findings indicate that both young children and adults metacognitively rep-
resent their own ability to make discriminations even in the absence of data, and can use this to guide efficient and effective
exploration.
NATURE HUMAN BEHAVIOUR | VOL 6 | NOVEMBER 2022 | 1557–1568 | www.nature.com/nathumbehav 1557
Content courtesy of Springer Nature, terms of use apply. Rights reserved