Dorsal and ventral streams: a framework for understanding aspects of the functional anatomy of language.

Abstract

Despite intensive work on language-brain relations, and a fairly impressive accumulation of knowledge over the last several decades, there has been little progress in developing large-scale models of the functional anatomy of language that integrate neuropsychological, neuroimaging, and psycholinguistic data. Drawing on relatively recent developments in the cortical organization of vision, and on data from a variety of sources, we propose a new framework for understanding aspects of the functional anatomy of language which moves towards remedying this situation. The framework posits that early cortical stages of speech perception involve auditory fields in the superior temporal gyrus bilaterally (although asymmetrically). This cortical processing system then diverges into two broad processing streams, a ventral stream, which is involved in mapping sound onto meaning, and a dorsal stream, which is involved in mapping sound onto articulatory-based representations. The ventral stream projects ventro-laterally toward inferior posterior temporal cortex (posterior middle temporal gyrus) which serves as an interface between sound-based representations of speech in the superior temporal gyrus (again bilaterally) and widely distributed conceptual representations. The dorsal stream projects dorso-posteriorly involving a region in the posterior Sylvian fissure at the parietal-temporal boundary (area Spt), and ultimately projecting to frontal regions. This network provides a mechanism for the development and maintenance of "parity" between auditory and motor representations of speech. Although the proposed dorsal stream represents a very tight connection between processes involved in speech perception and speech production, it does not appear to be a critical component of the speech perception process under normal (ecologically natural) listening conditions, that is, when speech input is mapped onto a conceptual representation. We also propose some degree of bi-directionality in both the dorsal and ventral pathways. We discuss some recent empirical tests of this framework that utilize a range of methods. We also show how damage to different components of this framework can account for the major symptom clusters of the fluent aphasias, and discuss some recent evidence concerning how sentence-level processing might be integrated into the framework.

Full text

Page 1

Dorsal and ventral streams: a framework for
understanding aspects of the functional
anatomy of language
Gregory Hickoka,*, David Poeppelb
aUniversity of California, Irvine, CA, USA
bUniversity of Maryland, College Park, MD, USA
Received 10 August 2001; revised 24 June 2002; accepted 23 October 2003
Abstract
Despite intensive work on language–brain relations, and a fairly impressive accumulation of
knowledge over the last several decades, there has been little progress in developing large-scale
models of the functional anatomy of language that integrate neuropsychological, neuroimaging, and
psycholinguistic data. Drawing on relatively recent developments in the cortical organization of
vision, and on data from a variety of sources, we propose a new framework for understanding aspects
of the functional anatomy of language which moves towards remedying this situation. The
framework posits that early cortical stages of speech perception involve auditory fields in the
superior temporal gyrus bilaterally (although asymmetrically). This cortical processing system then
diverges into two broad processing streams, a ventral stream, which is involved in mapping sound
onto meaning, and a dorsal stream, which is involved in mapping sound onto articulatory-based
representations. The ventral stream projects ventro-laterally toward inferior posterior temporal
cortex (posterior middle temporal gyrus) which serves as an interface between sound-based
representations of speech in the superior temporal gyrus (again bilaterally) and widely distributed
conceptual representations. The dorsal stream projects dorso-posteriorly involving a region in the
posterior Sylvian fissure at the parietal–temporal boundary (area Spt), and ultimately projecting to
frontal regions. This network provides a mechanism for the development and maintenance of
“parity” between auditory and motor representations of speech. Although the proposed dorsal stream
represents a very tight connection between processes involved in speech perception and speech
production, it does not appear to be a critical component of the speech perception process under
normal (ecologically natural) listening conditions, that is, when speech input is mapped onto a
conceptual representation. We also propose some degree of bi-directionality in both the dorsal and
ventral pathways. We discuss some recent empirical tests of this framework that utilize a range of
0022-2860/$ - see front matter q 2004 Elsevier B.V. All rights reserved.
doi:10.1016/j.cognition.2003.10.011
Cognition 92 (2004) 67–99
www.elsevier.com/locate/COGNIT
* Corresponding author. Department of Cognitive Sciences, University of California, Irvine, CA 92612, USA.
E-mail address: gshickok@uci.edu (G. Hickok).

Page 2

methods. We also show how damage to different components of this framework can account for the
major symptom clusters of the fluent aphasias, and discuss some recent evidence concerning how
sentence-level processing might be integrated into the framework.
q 2004 Elsevier B.V. All rights reserved.
Keywords: Dorsal and ventral streams; Functional anatomy; Language; Aphasia; Speech perception; Speech
production
1. Introduction and preliminaries
The functional anatomic framework for language which is presented in this paper is
based on a rather old insight in language research dating back at least to the 19th century
(e.g. Wernicke, 1874/1969), namely that sensory speech codes must minimally interface
with two systems: a conceptual system and a motor–articulatory system. The existence of
an interface with the conceptual system requires no motivation; such an interface is
required if we are to comprehend the meaning of the words we hear. The need for an
interface with the motor system may at first seem less obvious, but in fact, many areas of
language science either explicitly or implicitly posit an auditory–motor connection. The
simplest demonstration of this comes from development: infants must shape their
articulatory gestures in a way that matches the phonetic structure of the language they are
exposed to; yet the primary input to this motor learning task is acoustic. Therefore, there
must be some mechanism for using auditory input to shape motor output (Doupe & Kuhl,
1999). And there is good reason to believe that this auditory–motor interface system is
functional in adults as we will see below.
Recent work in the cortical organization of vision has also emphasized that sensory
input must interface both with conceptual systems (for object recognition) and with motor
systems (e.g. visually guided reaching/grasping) (Milner & Goodale, 1995). It has been
demonstrated empirically that these two interface systems comprise functionally and
anatomically differentiated (but probably interacting) processing streams in which a
ventral (occipital–temporal) stream supports object recognition/understanding, and a
dorsal (occipital–parietal) stream supports visuomotor integration functions (see Milner
& Goodale, 1995 for a review).
The point of the present paper is to show that by thinking of aspects of language
processing in terms of these two kinds of interfaces (sensory–conceptual and sensory–
motor), and by making use of what is known about the cortical organization of the visual
processing streams which also make use of a similar functional distinction, we can
advance our understanding of the cortical organization of language. Indeed, we will argue
that sensory representations of speech in auditory-related cortices (bilaterally) interface (i)
with conceptual representations via projections to portions of the temporal lobe (the
ventral stream), and (ii) with motor representation via projections to temporal–parietal
regions (the dorsal stream).
Before presenting the details of our framework, it is worthwhile considering two related
issues. One concerns the concept of linguistic specificity in the neural organization of
G. Hickok, D. Poeppel / Cognition 92 (2004) 67–9968

Page 3

language, and the other concerns the impact of task selection in brain mapping studies of
language.
It is common to find brain mapping studies of language which contrast the perception of
a linguistic stimulus, say auditorily presented syllables or sentences, with a non-linguistic
control, say, tone sequences, rotated speech, or time reversed speech. The goal of this
approach is to identify cortical fields which are speech- or language-specific in their
response properties, and therefore may represent dedicated cortical processing systems.
The identification of language-specific processing systems is an interesting and important
empirical enterprise, but is unlikely to yield, on its own, a complete understanding of the
neural organization of language. The reasoning behind this assertion is as follows.
Language processing systems can be viewed as a set of transformations over
representations (not necessarily in series), for example, mapping between an acoustic
input and a conceptual representation (as in comprehension), or between a conceptual
representation and a sequence of motor gestures (as in production). Early stages of this
mapping process on the input side – for example, cochlear, brain stem and thalamic
processing as well as at least early cortical auditory mapping – likely perform
transformations on the acoustic data that are relevant to linguistic as well as non-linguistic
auditory perception. Because these early processing stages are not uniquely involved in
language perception, they are often dismissed as being merely “auditory” areas and not
relevant to understanding language processing. But clearly each stage in this analytic
process interacts with other stages in important ways: the computations performed at one
level depend on the input received from other levels, and therefore each transformation
plays a role in the entire process of mapping sound onto meaning (or meaning onto motor
articulation). Viewed in this way, a complete understanding of the functional anatomy of
language will involve an understanding of each step in the translation between conceptual
representations and the sensory and motor periphery, independent of the linguistic
specificity of each of the computations. Of course, it may turn out to be important to
determine the degree of linguistic specificity of each of these steps, but this question need
not be answered to understand the relevant computations underlying the neural basis of
language (e.g. we need not understand the role of orofacial articulators in eating before we
understand their role in speech production). We argue that it is premature to dismiss areas
that activate during language tasks as “non-linguistic” simply because they also respond
vigorously to non-linguistic stimulation. Similarly, because we do not know how linguistic
categories (e.g. phonetic, phonemic, etc.) map onto these neural systems, we will use these
terms sparingly, and only in a very general sense. In some cases, we will resort to using
generic terms, such as “sub-lexical” or “sound-based representation of speech”, which
reflects our agnosticism on the issue of how the processing stages we propose map onto
traditional labels.
An important consequence of this view of language processes as a set of computations
or mappings between representations is that the neural systems involved in a given
language operation (task) will depend to some extent on what representation is being
mapped onto. For example, speech input which is mapped onto a conceptual
representation (as in comprehension tasks) will clearly involve a set of computations
which is non-identical to those involved in mapping that same input onto a motor–
articulatory representation (as in a repetition task). Of course, the mapping stages in these
G. Hickok, D. Poeppel / Cognition 92 (2004) 67–9969

Page 4

two tasks will be shared up to some point, but they must diverge in accordance with the
different requirements entailed by the endpoints of the mapping process. The upshot is that
the particular task which is employed to investigate the neural organization of language
(that is, the mapping operation the subject is asked to compute) determines which neural
circuit is predominantly activated. This point is obvious in extreme cases, such as the
example above, but the issue may also arise in more subtle cases: to what extent does
mapping an auditorily presented sentence onto a judgment of grammaticality differ from
mapping that input onto a representation of meaning? Considerations such as these will
play an important role in our argument below.
2. Overview of the framework
The framework we have proposed (Hickok & Poeppel, 2000) and further develop here
draws heavily on what is known about the functional anatomy of vision, and more recently
audition, particularly the distinction that has been made between dorsal and ventral
streams. Most of the discussion of dorsal and ventral streams in the literature centers on the
concept of “where” and “what” pathways (Ungerleider & Mishkin, 1982). The
fundamental distinction proposed by Ungerleider and Mishkin was that visual processing
could be coarsely divided into two processing streams, a ventral stream projecting to
inferior temporal areas which is involved in processing object identity (the “what”
pathway), and a dorsal stream projecting to parietal areas which is involved in processing
object location (the “where” pathway). In the last several years, however, there has been
mounting evidence suggesting that the concept “where” may be an insufficient
characterization of the dorsal stream (Milner & Goodale, 1995). Instead, it has been
proposed that the dorsal visual stream is particularly geared for visuo-motor integration, as
requiredin visually guided reachingororientingresponses.1According tothis view,dorsal
stream systems appear to compute coordinate transformations – for example, transform
representations in retino-centric coordinates to head-, and body-centered coordinates –
that allows visual information to interface with various motor-effector systems which act
on that visual input (Andersen, 1997; Rizzolatti, Fogassi, & Gallese, 1997).
In the auditory system, a dorsal–ventral partitioning has also been proposed
(Rauschecker, 1998). While there is general agreement regarding the role of the ventral
stream in auditory “what” processing, the functional role of the dorsal stream is debated.
Some groups argue for a dorsal “where” stream in the auditory system (Kaas & Hackett,
1999; Rauschecker, 1998; Romanski et al., 1999), whereas others have proposed that the
dorsal stream is more involved in tracking changes in the frequency spectra of the
auditory signal over time, a capacity which would make it particularly relevant for
speech perception (Belin & Zatorre, 2000). We have put forward a third hypothesis, that
the dorsal auditory stream is critical for auditory–motor integration (Hickok & Poeppel,
2000), similar to its role in the visual domain (see Wise et al., 2001 for a
similar proposal). This system, we suggest, serves both linguistic and non-linguistic
1Processing of spatial location would, of course, be a sub-process in a system geared toward visuo-motor
integration, makingthis view consistentwithpreviousobservationsof the involvement of dorsal streamstructures
in spatial processing.
G. Hickok, D. Poeppel / Cognition 92 (2004) 67–99 70

Page 5

(e.g. orienting responses, aspects of musical ability) processes. A more detailed
discussion of the necessity for an auditory–motor integration system for speech is taken
up in Section 5 below.
With this background, we turn to a brief outline of the framework (Fig. 1A,B), which
has been developed primarily in the context of single word processing. (We consider an
extension of this proposal into sentence-level processing in Section 4.) The model posits
that early cortical stages of speech perception involve auditory-responsive fields in
Fig. 1. (A) The proposed framework for the functional anatomy of language. See text for details. Adapted from
Hickok and Poeppel (2000). (B) General locations of the model components shown on a lateral view of the brain;
area boundaries are not to be interpreted as sharply as they are drawn. Note that the cortical territory associated
with a given function in the model is not hypothesized to be dedicated to that function, although there may be
subsystems with these fields which are functionally specialized. Delineation of frontal areas thought to support
articulatory-based speech codes comes from the general distribution of activated areas in functional imaging
studies of object naming and articulatory rehearsal processes (e.g. see Awh et al., 1996; Hickok, Buchsbaum,
Humphries, & Muftuler, 2003; Indefrey & Levelt, this volume). The stippled area (superior temporal sulcus)
represents a region which appears to support phoneme-level representations (see text).
G. Hickok, D. Poeppel / Cognition 92 (2004) 67–9971