Conference PaperPDF Available

Semantic Contamination of Visual Similarity Judgments


Abstract and Figures

The roles of semantic and perceptual information in cognition are of widespread interest to many researchers. However, disentangling their contributions is complicated by their overlap in real-world categories. For instance, attempts to calibrate visual similarity based on participant judgments are undermined by the possibility that semantic knowledge contaminates these judgments. This study investigated whether inverting stimuli attenuates semantic contamination of visual similarity judgments in adults and children. Participants viewed upright and inverted triads of familiar animals, and judged which of two test items looked most like the target. One test item belonged to the same category as the target, and one belonged to a different category. Test items’ visual similarity to the target either corresponded or conflicted with category membership. Across age groups, conflicting category membership reduced accuracy and slowed reaction times to a greater extent in upright than inverted triads. Therefore, inversion attenuates semantic contamination of visual similarity judgments.
Content may be subject to copyright.
Semantic Contamination of Visual Similarity Judgments
Layla Unger (
Carnegie Mellon University, Department of Psychology, 5000 Forbes Avenue
Pittsburgh, PA 15214 USA
Anna V. Fisher (
Carnegie Mellon University, Department of Psychology, 5000 Forbes Avenue
Pittsburgh, PA 15214 USA
The roles of semantic and perceptual information in cognition
are of widespread interest to many researchers. However,
disentangling their contributions is complicated by their
overlap in real-world categories. For instance, attempts to
calibrate visual similarity based on participant judgments are
undermined by the possibility that semantic knowledge
contaminates these judgments. This study investigated whether
inverting stimuli attenuates semantic contamination of visual
similarity judgments in adults and children. Participants
viewed upright and inverted triads of familiar animals, and
judged which of two test items looked most like the target. One
test item belonged to the same category as the target, and one
belonged to a different category. Test items’ visual similarity
to the target either corresponded or conflicted with category
membership. Across age groups, conflicting category
membership reduced accuracy and slowed reaction times to a
greater extent in upright than inverted triads. Therefore,
inversion attenuates semantic contamination of visual
similarity judgments.
Keywords: semantic knowledge; visual similarity
We perceive different things as related to each other in a
variety of potentially overlapping ways. For instance, entities
may be perceptually similar due to their shared perceptual
features, or belong to the same semantic, “taxonomic”
category. Many research endeavors and theoretical debates
have focused on questions surrounding the influence of these
relations on various facets of cognition, such categorization,
inductive inference, memory encoding, and visual search, as
well as the development and neural underpinnings of these
processes (e.g., Deák & Bauer, 1996; Gelman & Markman,
1986; Konkle, Brady, Alvarez, & Oliva, 2010). However,
perceptual and semantic relations commonly overlap
amongst real-world entities. Moreover, these relations may
interact during both learning and online processing of
perceptual input. This interplay between semantic and
perceptual relations severely complicates the study of
questions about their respective contribution to cognition.
The complication of interest in the present paper is the fact
that attempts to control for and manipulate perceptual
similarity of real-world items independent of the semantic
relations between them may be undermined by the
contamination of similarity judgments by semantic category
knowledge. Below we discuss this issue, and then present a
study designed to provide a possible solution to this problem.
Measuring Perceptual Similarity
Researchers have used multiple approaches to measure and
calibrate the perceptual similarity between stimuli, including
using their own intuition (e.g., Fisher, 2011), collecting
similarity judgments from adults (e.g., Deák & Bauer, 1996;
Gelman & Markman, 1986), and, in developmental studies,
collecting similarity judgments from children (e.g., Long, Lu,
Zhang, Li, & Deák, 2012; Sloutsky & Fisher, 2004). The aim
of these approaches is to assess visual similarity independent
of semantic relatedness. For example, to calibrate stimuli in
a match-to-sample task with triads consisting of a target, a
perceptual match, and a semantic match, researchers may ask
a separate sample of participants to judge the visual similarity
of the target to each match item on a likert scale (e.g., Deák
& Bauer, 1996; Gelman & Markman, 1986). Alternatively,
researchers may calibrate such triads by asking participants
to choose which match item looks most like the target in order
to obtain ratios of the similarity of the perceptual match to the
target versus the semantic category match to the target (e.g.,
Long et al., 2012; Sloutsky & Fisher, 2004).
The calibration of visual similarity based on participant
similarity judgments is common to studies in many areas,
such as memory, semantic knowledge, and semantic
development (e.g., Blaye, Bernard-Peyron, Paour, &
Bonthoux, 2006; Deák & Bauer, 1996; Gelman & Markman,
1986; Konkle et al., 2010). Intrinsic to this approach is the
assumption that people can judge visual similarity without
being influenced by the semantic knowledge.
However, this assumption may be unwarranted. Semantic
knowledge may instead influence perceptual similarity
judgments through any of multiple routes. Knowledge of
semantic relationships between items may influence
judgments of their similarity: 1) After perceptual similarity
has been independently evaluated (Pylyshyn, 1999), 2) By
feeding back into perceptual similarity evaluations (Lupyan,
Thompson-Schill, & Swingley, 2010), or 3) By influencing
the similarity of items perceptual representations during
prior learning (Goldstone, 1998; Goldstone, Lippa, &
Shiffrin, 2001; O’Reilly, Wyatte, Herd, Mingus, & Jilk,
The degree to which any of these routes truly characterizes
cognition is the subject of active research and debate (Chen
& Proctor, 2012; Lupyan et al., 2010). Although an in-depth
evaluation of this issue is beyond the scope of this paper, the
brief overview presented here highlights the many ways in
which semantic knowledge may contaminate perceptual
similarity judgments. The purpose of the present study is
therefore to assess the contamination of perceptual similarity
judgments by semantic category knowledge, and test whether
this contamination is attenuated by inversion.
The Present Study
The choice to test whether inversion attenuates the influence
of semantic category knowledge on perceptual similarity
judgments was motivated by numerous findings that rotation
away from a canonical orientation impedes the identification
of the category to which a familiar item belongs (e.g.,
Jolicoeur & Milliken, 1989; Lawson & Jolicoeur, 2003).
Specifically, increasing misorientation both slows and
increases errors for identifying an item’s category label. This
effect has been attributed to a process in which a perceived
item must be mentally normalized to its canonical orientation
before its category membership can be retrieved from
memory (Lawson & Jolicoeur, 2003). Consequently,
inversion may interfere with access to semantic category
knowledge, and therefore attenuate the influence of such
knowledge on perceptual similarity judgments.
In the present study, participants performed a match-to-
sample perceptual similarity judgment task in which they
were asked to choose which of two test items “looks most
like” a target item for triads of items that were presented in
both upright and inverted orientations on different trials. All
items were pictures of familiar animals, such as “dog” and
“pig”. One test item belonged to the same category as the
target (e.g., both were pigs), whereas the other test item
belonged to a different category (e.g., dog). The visual
similarity of the test items to the target was manipulated such
that it either corresponded or conflicted with the category
membership of the target (see Fig. 1).
We predicted that if semantic category knowledge
influences perceptual similarity judgements, participants
should choose the visual similarity match less accurately and
more slowly when category membership and visual similarity
were in conflict. Moreover, if inversion attenuates the
influence of category knowledge on perceptual similarity
judgments, less accurate and slower responses on conflict
versus no-conflict trials should manifest to a greater extent
when triads are upright than when they are inverted. To test
whether this predicted pattern manifests across ages to whom
the categories are familiar, we conducted this study with both
a kindergarten-age and an adult sample.
The total sample of 42 participants included 24 participants
in each of two age groups: Kindergarten (Mage = 5.45 years,
SD =0.43 years), and Adults. Kindergarten participants were
recruited from schools in a middle-class, metropolitan area in
a Northeastern US city, and Adults were recruited via
Amazon Mechanical Turk. Adults were compensated at a rate
of $5/hour for their participation.
Both kindergarten and adult participants completed a Visual
Similarity Judgment task, described below. Kindergarten
participants viewed this presentation on a laptop computer,
and made responses using a Cedrus RB-530 response box. To
help kindergarten participants distinguish between the
buttons they were instructed to use in the study (see
Procedure), these buttons were given different-colored
plastic covers. Adult participants viewed the presentation on
Qualtrics, an online survey platform, and made responses
using their personal keyboards. The Qualtrics version of the
presentation was designed to record keyboard response times.
Visual Similarity Judgment Task This task consisted of
triads of animal pictures presented on a computer screen that
consisted of a Target item (e.g., a white pig) on the top, a
Same Category test item (e.g., a black pig) on the bottom to
one side, and a Different Category test item (e.g., a white dog)
on the bottom to the other side. The pictures were photo-
realistic images manipulated in graphics editing software to
create two Semantic Conflict conditions: 1) No Conflict, in
which the Same Category test item was visually similar to the
target and the Different Category test item was visually
dissimilar to the target, and 2) Conflict, in which the
correspondence between category membership and visual
similarity was reversed (see Fig. 1). Visual similarity was
manipulated across several characteristics of the stimuli,
including color, shape, and internal features.
Both No Conflict and Conflict triads were presented in two
orientation conditions: Upright, and Inverted. The position of
the two types of test items on the left or the right of the bottom
of the screen was counterbalanced. Both, the Conflict
condition and the Orientation condition were manipulated
within participants.
The presentation included four practice trials, and 32
experiment trials. The practice trials consisted of a Conflict
and a No Conflict triad presented in upright and inverted
orientations, and the experiment trials consisted of eight
Conflict and eight No Conflict triads presented in upright and
inverted orientations. Each triad was presented once in
Figure 1. Examples of triads in each condition. A)
Conflict, Upright; B) No Conflict, Upright; C) Conflict,
Inverted; D) No Conflict, Inverted.
upright and inverted orientations. The order of the experiment
trials was pseudo-randomized such that different versions of
a given triad did not appear consecutively, and such that no
more than two triads in the same combination of visual
similarity and orientation conditions appeared consecutively.
Children Participants were tested individually in a quiet
space. To begin, participants were seated in front of the
laptop and button response box, and the first practice trial was
displayed. Participants were told that they were going to play
a game in which they decide which of two animals on the
bottom of the screen looks like the animal on the top. They
were further asked to use the buttons on the response box to
indicate which animal they chose. Participants were then
allowed to proceed through the practice and experiment trials
at their own pace. The button instructions were repeated on
subsequent trials if participants either failed to make a
response for several seconds, or started to press the buttons
quickly and randomly.
Adults Participants completed the task via Qualtrics. The
version adults completed was identical to the version children
completed, with the exception that participants were
instructed to use “z” and “m” keyboard keys rather than the
left and right buttons of a response box.
Results and Discussion
First, to ensure that participants understood that the purpose
of the task was to identify the similarity rather than the
category match, the accuracy with which participants in each
age group in each condition chose the similarity match was
compared to chance (i.e., .5). All contrasts revealed
significantly above chance performance (ps < .0001).
To test the prediction that conflict between semantic and
visual similarity would decrease accuracy and slow response
times (RTs) for Conflict versus No Conflict trials in the
Upright and not the Inverted condition, we analyzed the
effects of the Semantic Conflict and Orientation factors on
accuracy and RT using repeated measures ANOVAs.
Specifically, for each outcome measure, we calculated each
participant’s mean score for the four combinations of
conditions produced by our Semantic Conflict and
Orientation factors, and submitted these mean scores to
separate repeated measures ANOVAs for each age group.
Accuracy Analysis
For adult participants, this analysis revealed a main effect of
Semantic Conflict (F(1,23)=10.122, p=.004, ɳ2=.306), and a
main effect of Orientation (F(1,23)=6.457, p=.018, ɳ2=.219).
More importantly, both main effects were qualified by a
significant interaction (F(1,23)=15.826, p=.001, ɳ2=.408).
To explore this interaction, we conducted t-tests comparing
Upright versus Inverted RTs separately for the Conflict and
No Conflict conditions. In the Conflict condition, adults were
more accurate on Inverted (Maccuracy=89.96%) than on
Upright (Maccuracy=82.29%) trials (t(23)=3.680, p=.001,
Cohen’s d=1.54), whereas in the No Conflict condition, adult
accuracy did not significantly differ on Inverted and Upright
trials (t(23)=1, p=.328) (see Fig. 2).
Kindergarten participants exhibited similar patterns of
accuracy. The repeated measures ANOVA for this age group
also revealed a main effect of Semantic Conflict
(F(1,23)=6.4, p=.019, ɳ2=.218), though unlike adults, the
main effect of Orientation did not reach significance
(F(1,23)=1.15, p=.295). Moreover, like adults, the analysis
with this age group revealed a significant interaction between
Semantic Conflict and Orientation (F(1,23)=5.522, p=.028,
ɳ2=.194). T-tests comparing Upright and Inverted trials for
each Semantic Conflict condition revealed that Kindergarten
participants were marginally more accurate on Inverted
Maccuracy=84.90%) than on Upright (Maccuracy=77.08%) trials
in the Conflict condition (t(23)=2.01, p=.057, Cohen’s
d=.45), and numerically more accurate on Upright than
Inverted trials in the No Conflict condition (t(23)=1.772,
p=.09) (see Fig. 2).
RT Analysis
Prior to calculating each participant’s mean RT score, we
filtered RTs to remove inaccurate trials and trials on which
the participant responded either faster than 250 msec, or more
than three standard deviations more slowly than the average
RT for their age group.
For adults, this analysis revealed a main effect of Semantic
Conflict (F(1,23)=15.133,p=.001, ɳ2=.397), and no main
effect of Orientation (F(1,23)=1.116, p=.302). Critically, the
main effect of Semantic Conflict was qualified by a
significant interaction between this factor and Orientation
(F(1,23)=26.14,p=.000, ɳ2=.532). In the Conflict condition,
inverted trials yielded faster RTs than upright trials
(MUpright=1601ms, MInverted=1199ms, t(23)=4.08, p<.0001),
whereas in the No Conflict condition, inverted trials yielded
slower RTs (MUpright=912ms, MInverted=1195ms, t(23)=-
Figure 2. Accuracy and RT outcomes. Upright trials are
depicted in black, and Inverted trials are depicted in gray.
Error bars represent standard errors of the mean.
3.753,p=.001). These results are consistent with our
prediction that semantic category knowledge influences
perceptual similarity judgments such that participants are
slower to judge perceptual similarity when it is in conflict
with category membership, and that this influence is
attenuated by inversion.
The pattern of results for kindergarten participants was
similar to the pattern observed for adults. The repeated
measures ANOVA revealed a main effect of Semantic
Conflict (F(1,23)=6.096, p=.021, ɳ2=.210), and no main
effect of Orientation (F(1,23)=2.072, p=.164). As in adults,
the main effect of Semantic Conflict was qualified by a
significant interaction between this factor and Orientation
(F(1,23)=23.105, p<.0001, ɳ2=.501). Children’s responses
were also faster for inverted trials in the Conflict condition
(MUpright=2788ms, MInverted=2487ms, t(23)=2.153, p=.042),
and slower for inverted trials in the No Conflict condition
(MUpright=2092ms, MInverted=2685ms, t(23)=-4.376, p<.0001).
The pattern of results in this experiment is broadly
consistent with the prediction that decrements in accuracy
and RT of visual similarity judgments due to conflict between
category membership and visual similarity is attenuated by
inversion. Both adults and children were less accurate and
slower to respond on Upright than on Inverted trials in which
category membership and visual similarity conflicted (though
the effects on accuracy were marginally significant in
children). In contrast, inversion did not improve performance
in the No Conflict condition. Participants were instead
similarly accurate on both upright and inverted No Conflict
trials, and in fact slower on inverted than upright trials.
The observation of slower RTs in the inverted versus
upright trials in the No Conflict condition was not specifically
predicted in our semantic contamination hypothesis. This
finding indicates that, in the absence of semantic conflict,
inversion generally slows down responses in even non-
semantic tasks such as the visual similarity judgment task
used here. This possibility underscores the importance of the
finding that inversion speeds up responses in the presence of
semantic conflict.
General Discussion
Investigating the contributions of visual similarity and
semantic relatedness is the focus of a wide range of research
endeavors (Deák & Bauer, 1996; Gelman & Markman, 1986;
Konkle et al., 2010; Lupyan et al., 2010; Sloutsky & Fisher,
2004). A critical component of this research is calibrating the
degree to which the stimuli used in experiments are visually
similar or semantically related. With respect to calibrating
visual similarity, approaches taken to date in which
researchers intuit or ask participants to judge visual similarity
are undermined by the possibility that semantic knowledge
contaminates visual similarity judgments. The purpose of this
study was to examine this possibility and test whether
semantic contamination is attenuated by inverting stimuli.
Our findings show that both adults and children show an
effect of semantic conflict on visual similarity judgments that
lowers accuracy and slows response times, and that is
attenuated by inversion. Conversely, in the absence of
semantic conflict, inversion slowed response times. Taken
together, these findings suggest that semantic knowledge
contaminates visual similarity judgments, and that inversion
attenuates semantic contamination. Therefore, inverting
stimuli for which visual similarity judgments are elicited
provides a viable approach to calibrating stimuli for research
investigating the role of visual similarity and semantic
relatedness in various facets of cognition.
For example, recent cognitive neuroscience studies have
investigated the degree to which knowledge about semantic
relations is encoded as similar patterns of activity in brain
regions involved in visual processing versus only in regions
involved in subsequent amodal processing (Bruffaerts et al.,
2013; Weber, Thompson-Schill, Osherson, Haxby, &
Parsons, 2009). However, overlap between semantic
relatedness and perceptual similarity renders it difficult to
determine which of these sources is responsible for observed
brain activity pattern similarities. For instance, a study
conducted by Weber et al. (2009) found that the similarity of
brain activity patterns in visual cortex evoked by viewing a
set of animals correlated with behavioral judgments of their
semantic similarity, but the fact that behavioral judgments of
semantic similarity were in turn highly correlated with
judgments of visual similarity renders it difficult to determine
whether brain pattern similarity was related to semantic or
visual similarity.
The approach introduced here to collecting visual
similarity judgments that are relatively uncontaminated by
semantic knowledge provides a route for attenuating such
confounds. For example, the paradigm introduced in this
study could be used to calibrate both items that are
semantically but not visually similar, and items that are
visually but not semantically similar, for studies that aim to
disentangle the contributions of these forms of similarity to
various facets of cognition and/or brain activity. The finding
that this approach is similarly effective in both adult and child
samples renders it viable for studies of both adult and
developmental cognition. Therefore, the validation of the
inversion approach demonstrated by the present study has the
potential to support progress in a variety of lines of research.
This study demonstrated that conflict between visual
similarity and semantic category membership slows and
reduces accuracy of visual similarity judgments of upright,
but not inverted stimuli for both adults and children.
Therefore, this study suggests that semantic knowledge
contaminates visual similarity judgments in adults and
children, and that inversion attenuates this contamination.
This work was supported by a Graduate Training Grant
awarded to Carnegie Mellon University by the Department of
Education, Institute of Education Sciences (R305B040063)
and by the James S. McDonnell Foundation 21st Century
Science Initiative in Understanding Human Cognition
Upright Inverted
Scholar Award (220020401) to the second author. We thank
children, parents, teachers, and participating institutions for
making this project possible: the Children’s School at
Carnegie Mellon University, Sacred Heart Elementary
School, and Phipps Conservatory. Finally, we thank Anna
Vande Velde and Manon Sohn for their help collecting data.
Blaye, A., Bernard-Peyron, V., Paour, J.-L., & Bonthoux, F.
(2006). Categorical flexibility in children: Distinguishing
response flexibility from conceptual flexibility. European
Journal of Developmental Psychology, 3, 163-188.
Bruffaerts, R., Dupont, P., De Grauwe, S., Peeters, R., De
Deyne, S., Storms, G., & Vandenberghe, R. (2013). Right
fusiform response patterns reflect visual object identity
rather than semantic similarity. NeuroImage, 83, 87-97.
Chen, J., & Proctor, R. W. (2012). Influence of category
identity on letter matching: Conceptual penetration of
visual processing or response competition? Attention,
Perception, & Psychophysics, 74, 716-729.
Deák, G., & Bauer, P. (1996). The dynamics of preschoolers'
categorization choices. Child development, 67, 740-767.
Fisher, A. V. (2011). Processing of perceptual information is
more robust than processing of conceptual information in
preschool-age children: Evidence from costs of switching.
Cognition, 119, 253-264.
Gelman, S. A., & Markman, E. M. (1986). Categories and
induction in young children. Cognition, 23, 183-209.
Goldstone, R. L. (1998). Perceptual learning. Annual review
of psychology, 49, 585-612.
Goldstone, R. L., Lippa, Y., & Shiffrin, R. M. (2001).
Altering object representations through category learning.
Cognition, 78, 27-43.
Jolicoeur, P., & Milliken, B. (1989). Identification of
disoriented objects: effects of context of prior presentation.
Journal of Experimental Psychology: Learning, Memory,
and Cognition, 15, 200-210.
Konkle, T., Brady, T. F., Alvarez, G. A., & Oliva, A. (2010).
Conceptual distinctiveness supports detailed visual long-
term memory for real-world objects. Journal of
Experimental Psychology: General, 139, 558-578.
Lawson, R., & Jolicoeur, P. (2003). Recognition thresholds
for plane-rotated pictures of familiar objects. Acta
Psychologica, 112, 17-41.
Long, C., Lu, X., Zhang, L., Li, H., & Deák, G. O. (2012).
Category label effects on Chinese children’s inductive
inferences: Modulation by perceptual detail and category
specificity. Journal of experimental child psychology, 111,
Lupyan, G., Thompson-Schill, S. L., & Swingley, D. (2010).
Conceptual penetration of visual processing.
Psychological Science682-691.
O’Reilly, R. C., Wyatte, D., Herd, S., Mingus, B., & Jilk, D.
J. (2013). Recurrent processing during object recognition.
Frontiers in psychology, 4, 1-14.
Pylyshyn, Z. (1999). Is vision continuous with cognition?:
The case for cognitive impenetrability of visual perception.
Behavioral and brain sciences, 22, 341-365.
Sloutsky, V. M., & Fisher, A. V. (2004). Induction and
categorization in young children: a similarity-based model.
Journal of Experimental Psychology: General, 133, 166-
Weber, M., Thompson-Schill, S. L., Osherson, D., Haxby, J.,
& Parsons, L. (2009). Predicting judged similarity of
natural categories from their neural representations.
Neuropsychologia, 47, 859-868.
ResearchGate has not been able to resolve any citations for this publication.
Full-text available
This research explored the development of children's use of multiple conceptual organizations (thematic, taxonomic) in sorting sets of pictures. Experiment 1 revealed that between 5 and 9 years, two forms of categorical flexibility can be distinguished: response and conceptual flexibility. It appeared that children's multiple sorts do not necessarily reflect the use of different conceptual organizations. Such a lag was mainly due to a difficulty in accessing taxonomic representations, specifically in the younger age groups. Therefore, Experiment 2 investigated the development of taxonomic representations using an original approach requiring participants to decide whether new items could be included into an existing taxonomic sort. This approach showed that taxonomic representations were only gradually differentiated from thematic and perceptual ones over the 5 to 10 years period. The discussion raises new hypotheses about the interaction between developing executive control (specifically, increasing resistance to interference of irrelevant information) and increasing conceptual knowledge in accounting for the development of conceptual flexibility.
Full-text available
How does the brain learn to recognize objects visually, and perform this difficult feat robustly in the face of many sources of ambiguity and variability? We present a computational model based on the biology of the relevant visual pathways that learns to reliably recognize 100 different object categories in the face of naturally occurring variability in location, rotation, size, and lighting. The model exhibits robustness to highly ambiguous, partially occluded inputs. Both the unified, biologically plausible learning mechanism and the robustness to occlusion derive from the role that recurrent connectivity and recurrent processing mechanisms play in the model. Furthermore, this interaction of recurrent connectivity and learning predicts that high-level visual representations should be shaped by error signals from nearby, associated brain areas over the course of visual learning. Consistent with this prediction, we show how semantic knowledge about object categories changes the nature of their learned visual representations, as well as how this representational shift supports the mapping between perceptual and conceptual knowledge. Altogether, these findings support the potential importance of ongoing recurrent processing throughout the brain's visual system and suggest ways in which object recognition can be understood in terms of interactions within and between processes over time.
Full-text available
Participants performed same-different matching tasks, with physical-identity instructions, on letter pairs composed from the letters B, b, and p. The letters in a pair were presented simultaneously or sequentially, with the experiments differing in whether (1) the letters could appear in two or four positions, (2) two or five SOAs were used, (3) the font was Arial or one in which the two loops of the letter B were of the same size, and (4) p did or did not occur in same pairs. With sequential presentation, different RTs were longer when the letters had the same name (Bb; within-category pair) than when they did not (Bp; between-category pair), replicating a finding by Lupyan et al. (2010). However, unlike in their study, this category effect was also significant with simultaneous presentation, tending to be nonsignificantly smaller for RTs but larger for accuracy than that obtained with sequential presentation. A similar pattern was observed when we removed a bias to respond different whenever the letter p was detected in the experiments in which p did not appear in same pairs. The presence of a category effect with simultaneous presentation is predicted by a response competition account, but not by Lupyan et al.'s conceptual-penetration-of-visual-processing account.
Full-text available
Humans have a massive capacity to store detailed information in visual long-term memory. The present studies explored the fidelity of these visual long-term memory representations and examined how conceptual and perceptual features of object categories support this capacity. Observers viewed 2,800 object images with a different number of exemplars presented from each category. At test, observers indicated which of 2 exemplars they had previously studied. Memory performance was high and remained quite high (82% accuracy) with 16 exemplars from a category in memory, demonstrating a large memory capacity for object exemplars. However, memory performance decreased as more exemplars were held in memory, implying systematic categorical interference. Object categories with conceptually distinctive exemplars showed less interference in memory as the number of exemplars increased. Interference in memory was not predicted by the perceptual distinctiveness of exemplars from an object category, though these perceptual measures predicted visual search rates for an object target among exemplars. These data provide evidence that observers' capacity to remember visual information in long-term memory depends more on conceptual structure than perceptual distinctiveness.
One of the primary functions of natural kind terms (e.g., tiger, gold) is to support inductive inferences. People expect members of such categories to share important, unforeseen properties, such as internal organs and genetic structure. Moreover, inductions can be made without perceptual support: even when an object does not look much like other members of its category, and even when a property is unobservable. The present work addresses how expectations about natural kinds originate. Young children, with their usual reliance on perceptual appearances and only rudimentary scientific knowledge, might not induce new information within natural kind categories. To test this possibility, category membership was pitted against perceptual similarity in an induction task. For example, children had to decide whether a shark is more likely to breathe as a tropical fish does because both are fish, or as a dolphin does because they look alike. By at least age 4, children can use categories to support inductive inferences even when category membership conflicts with appearances. Moreover, these young children have partially separated out properties that support induction within a category (e.g., means of breathing) from those that are in fact determined by perceptual appearances (such as weight). Since we examined only natural kind categories, we do not know to what extent children have differentiated natural kinds from other sorts of categories. Children may start out assuming that categories named by language have the structure of natural kinds and with development refine these expectations.
We previously reported the neuropsychological consequences of a lesion confined to the middle and posterior part of the right fusiform gyrus (case JA) causing a partial loss of knowledge of visual attributes of concrete entities in the absence of category-selectivity (animate versus inanimate). We interpreted this in the context of a two-step model that distinguishes structural description knowledge from associative-semantic processing and implicated the lesioned area in the former process. To test this hypothesis in the intact brain, multi-voxel pattern analysis was used in a series of event-related fMRI studies in a total of 46 healthy subjects. We predicted that activity patterns in this region would be determined by the identity of rather than the conceptual similarity between concrete entities. In a prior behavioral experiment features were generated for each entity by more than 1000 subjects. Based on a hierarchical clustering analysis the entities were organized into 3 semantic clusters (musical instruments, vehicles, tools). Entities were presented as words or pictures. With foveal presentation of pictures, cosine similarity between fMRI response patterns in right fusiform cortex appeared to reflect both the identity of andthe semantic similarity between the entities. No such effects were found for words in this region. The effect of object identity was invariant for location, scaling, orientation axis and color (grayscale versus color). It also persisted for differentexemplars referring to a same concrete entity. The apparent semantic similarity effect however was not invariant. This study provides further support for a neurobiological distinction between structural description knowledge and processing of semantic relationships and confirms the role of right mid-posterior fusiform cortex in the former process, in accordance with previous lesion evidence.
Is processing of conceptual information as robust as processing of perceptual information early in development? Existing empirical evidence is insufficient to answer this question. To examine this issue, 3- to 5-year-old children were presented with a flexible categorization task, in which target items (e.g., an open red umbrella) shared category membership with one test item (e.g., a folded umbrella) and perceptual characteristics with another test item (e.g., a red mushroom). Participants were instructed to either categorize stimuli by the same dimension (i.e., perceptual similarity or category membership) in both phases of the task, or switch from categorizing by one dimension to categorizing by the other dimension. Results pointed to a strong asymmetry in switch costs: conceptual switch costs were higher than perceptual switch costs. These results suggest that processing of perceptual information remains more robust than processing of conceptual information at least until 5 years of age.
In traditional hierarchical models of information processing, visual representations feed into conceptual systems, but conceptual categories do not exert an influence on visual processing. We provide evidence, across four experiments, that conceptual information can in fact penetrate early visual processing, rather than merely biasing the output of perceptual systems. Participants performed physical-identity judgments on visually equidistant pairs of letter stimuli that were either in the same conceptual category (Bb) or in different categories (Bp). In the case of nonidentical letters, response times were longer when the stimuli were from the same conceptual category, but only when the letters were presented sequentially. The differences in effect size between simultaneous and sequential trials rules out a decision-level account. An additional experiment using animal silhouettes replicated the major effects found with letters. Thus, performance on an explicitly visual task was influenced by conceptual categories. This effect depended on processing time, immediately preceding experience, and stimulus typicality, which suggests that it was produced by the direct influence of category knowledge on perception, rather than by a postperceptual decision bias.
We report a combined behavioral and functional magnetic resonance imaging (fMRI) study of conceptual similarity among members of a natural category (mammals). The study examined the relationship between computed pairwise similarity of neural responses to viewed mammals (e.g. bear, camel, dolphin) and subjective pairwise similarity ratings of the same set of mammals, obtained from subjects after the scanning session. In each functional region of interest (fROI), measures of neural similarity were compared to behavioral ratings. fROIs were identified as clusters of voxels that discriminated intact versus scrambled images of mammals (no information about similarity was used to define fROIs). Neural similarity was well correlated with behavioral ratings in fROIs covering the lateral occipital complex in both hemispheres (with overlap of the fusiform and inferior temporal gyri on the right side). The latter fROIs showed greater hemodynamic response to intact versus scrambled images of mammals whereas the fROIs that failed to predict similarity showed the reverse pattern. The findings provide novel evidence that information about the fine structure of natural categories is coarsely coded in regions of the ventral visual pathway. Implications for the theory of inductive inference are discussed.