Labels shape infants' object representations

Full text

Labels shape infants'
object representations
Katherine E. Twomey
Gert Westermann
Lancaster University, UK

Infants form nonlinguistic
representations over time…
3mos: online
category learning
(Quinn, Eimas & Rosenkratnz,
1993, Perception)
…
pets > no pets
4mos: learned representations
affect image exploration
(Hurley & Oakes, 2015, J Cogn Dev)
…and in-the-moment

Newborns:
language experienced
in utero affects vowel
perception
(Moon, Lagercrantz & Kuhl, 2013,
Acta Paediatr)
8mos: online
probability
tracking for
word
segmentation
(Saffran, Aslin & Newport,
1996, Science)
Hallå!
Hello!
bi da ku pa do ti go la bu bi da ku…
Infants also form linguistic
representations over time…
…and in-the-moment
33% 33%
100%100% 100%100%

5-6mo: first learned words
(Bergelson & Swingley, 2012, PNAS; Delle Luche, Floccia, Granjon & Nazzi, 2016,
Infancy)
Linguistic / nonlinguistic
representations are linked (Sloutsky & Lo, 1999;
Dev Psych)
mouth
eye
bottle

Interactions occur across
timescales…
10mo: online
labeling affects
online nonlinguistic
processing
(Althaus & Westermann,
2016, JECP)
Geepee! Boota!
~36mo: learned
vocabulary structure
affects online category
generalization
(Perry & Samuelson, 2011; Front Psych)
This is my
blicket!
Which one’s
your blicket?
…and within timescales

Linguistic and nonlinguistic
representations interact within and
across timescales
But what’s the relationship between learned labels
(long timescale) and learned nonlinguistic (long
timescale) representations?
Novel (short timescale) labels affect online
attention (short timescale)
Learned labels (long timescale) affect online
attention (short timescale)
Online category and word learning (short
timescale) leads to long-term vocabulary
growth (long timescale)

Gliga, Volein & Csibra, 2010 (E2)
J. Cognitive Neurosci.
Trained 12-month-old infants for 3 min with 2
unfamiliar 3D objects, one with a novel label, one
without
Immediately afterwards, presented infants with
images of (a) labeled object, (b) unlabeled object,
(c) completely novel object in silence
Recorded EEG gamma band response (index of
object encoding)

Gliga, Volein & Csibra, 2010 (E2)
J. Cognitive Neurosci.
Results: Gamma band response stronger after
the labeled object relative to unlabeled and
novel object
Conclusion: labels modulate object
representations in infancy
But this label training is unlikely to result in
long-term word learning (Horst & Samuelson, 2008), so
whether learned language modulates
learned representations remains unclear

The current study
Infants’ looking times reflect their
what they have learned – i.e., their
representations
(Fantz, 1964, Science)
If label representations
modulate object
representations, we should
observe differences in looking
times to images of objects
with or without labels, even
when presented in silence
Train over time to ensure
we’re testing learned
representations

Participants
24 10-month-old infants
- or -
Stimuli

Session 1
5-min naturalistic play session with experimenter
Look! A tanzer!
Look at that!
Ask parents to repeat every day for a week (7 sessions)
•Equal time with each toy
•Equal reference to each toy
•Infant allowed to hold toys simultaneously
•Only use tanzer, for that object
Parents provided with instruction booklet

Session 2
Silent familiarization, word learning test
Look! A tanzer!
…16 x 10
second
trials
1 x 12s trial

Hypothesis
Learning a label modulates
learned object
representations, resulting in
differences in looking times
to those objects presented
in silence

Results – familiarization
●
●
●
●
●
●
●
●
3000
4000
5000
6000
2 4 6 8
Trial
Time (ms)
●
Labeled Unlabeled
Longer looking to labeled stimulus
(beta = 0.082, SE = 0.032,χ2(1) = 5.97, p = .014)

Results – word learning test
Fixation to target, 1.5s after label onset (Holmqvist et al. 2011, OUP)
Increase in target looking in 2s post-labeling from 2s pre-
labeling
t(22) = 2.00, p = .058 (cf. Mani & Plunkett, 2011, Psych. Sci.)
chance
"tanzer"
0.00
0.25
0.50
0.75
1.00
1 2 3 4 5 6 7 8 9 10 11 12
Seconds
Proportion target looking
Proportion target looking
Seconds

Individual differences?
Some - but not all - infants responded to the label
(indexed by change in proportion target looking from
2s prelabelling to 2s postlabelling)
0.6−
0.5−
0.4−
0.3−
0.2−
0.1−
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
a b c d e f g h i j k l m n o p q r s t u v w
Participant
Change in proportion target looking

Evidence in line with incremental accounts of word
learning
(McMurray, Horst & Samuelson, 2012, Psych Rev;
Yurovsky, Fricker, Yu & Smith, 2012, Psych B Rev)
Infants learned
something about the
object,
something about the label,
something about the
mapping
tanzer
These partial representations nonetheless affected
looking times during the silent familiarization phase

tanzer!
1) Labeling increases
attention to object
(Baldwin & Markman, 1989,
Child Dev)
Labels are represented
separately from objects
(Waxman & Markow, 1995, Lang L Dev;
Westermann & Mareschal, 2014, Philos Tr Soc B)
Infants implicitly activate
label when object is
encountered in silence; (Mani &
Plunkett 2010, Psych Sci)
2) Linguistic representations are associated with
nonlinguistic representations
1) What’s the
mechanism, and
2) what can this
tell us about
representations?

1) If infants integrate labels into object
representations, no label -> discrepancy
Novelty response: increase in looking to
previously labelled object
Labels are object features / attributes, like colour, shape
(Gliozzi, Mayor, Hu & Plunkett, 2009, Cognitive Sci ; Sloutsky & Lo, 1999, Dev Psych)
tanzer
≠
2) Linguistic representations are integrated into
nonlinguistic representations
1) What’s the
mechanism, and
2) what can this
tell us about
representations?

There’s always more to do!
•do 10mos implicitly activate
labels?
•what is the effect of labelling
on category representations?
Computational modelling
underway to unpick label
activation / novelty accounts
Overall, converging evidence (cf. Gliga et al., 2010)
that
learned labels modulate infants’ object
representations
(Shameless plug for Arthur Capelier-
Mourguy’s poster tomorrow!)

Questions?
Huge thanks to all the parents
and infants who took part and
to our funders
k.twomey@lancaster.ac.uk