BookPDF Available

Bottom-up and top-down processes in reading : influences of frequency and predictability on event-related potentials and eye movements

Authors:

Abstract

In reading, word frequency is commonly regarded as the major bottom-up determinant for the speed of lexical access. Moreover, language processing depends on top-down information, such as the predictability of a word from a previous context. Yet, however, the exact role of top-down predictions in visual word recognition is poorly understood: They may rapidly affect lexical processes, or alternatively, influence only late post-lexical stages. To add evidence about the nature of top-down processes and their relation to bottom-up information in the timeline of word recognition, we examined influences of frequency and predictability on event-related potentials (ERPs) in several sentence reading studies. The results were related to eye movements from natural reading as well as to models of word recognition. As a first and major finding, interactions of frequency and predictability on ERP amplitudes consistently revealed top-down influences on lexical levels of word processing (Chapters 2 and 4). Second, frequency and predictability mediated relations between N400 amplitudes and fixation durations, pointing to their sensitivity to a common stage of word recognition; further, larger N400 amplitudes entailed longer fixation durations on the next word, a result providing evidence for ongoing processing beyond a fixation (Chapter 3). Third, influences of presentation rate on ERP frequency and predictability effects demonstrated that the time available for word processing critically co-determines the course of bottom-up and top-down influences (Chapter 4). Fourth, at a near-normal reading speed, an early predictability effect suggested the rapid comparison of top-down hypotheses with the actual visual input (Chapter 5). The present results are compatible with interactive models of word recognition assuming that early lexical processes depend on the concerted impact of bottom-up and top-down information. We offered a framework that reconciles the findings on a timeline of word recognition taking into account influences of frequency, predictability, and presentation rate (Chapter 4). Wortfrequenz wird in der Leseforschung als wesentliche Bottom-up Determinante für die Geschwindigkeit des lexikalischen Zugriffs betrachtet. Darüber hinaus spielen Top-down Informationen, wie die kontextbasierte Wortvorhersagbarkeit, in der Sprachverarbeitung eine wichtige Rolle. Bislang ist die exakte Bedeutung von Top-down Vorhersagen in der visuellen Worterkennung jedoch unzureichend verstanden: Es herrscht Uneinigkeit darüber, ob ausschließlich späte post-lexikalische, oder auch frühe lexikalische Verarbeitungsstufen durch Vorhersagbarkeit beeinflusst werden. Um ein besseres Verständnis von Top-down Prozessen und deren Zusammenhänge mit Bottom-up Informationen in der Worterkennung zu gewährleisten, wurden in der vorliegenden Arbeit Einflüsse von Frequenz und Vorhersagbarkeit auf ereigniskorrelierte Potentiale (EKPs) untersucht. Die Ergebnisse aus mehreren Satzlesestudien wurden mit Blickbewegungen beim natürlichen Lesen sowie mit Modellen der Worterkennung in Beziehung gesetzt. Als primärer Befund zeigten sich in EKP Amplituden konsistent Interaktionen zwischen Frequenz und Vorhersagbarkeit. Die Ergebnisse deuten auf Top-down Einflüsse während lexikalischer Wortverarbeitungsstufen hin (Kapitel 2 und 4). Zweitens mediierten Frequenz und Vorhersagbarkeit Zusammenhänge zwischen N400 Amplituden und Fixationsdauern; die Modulation beider abhängigen Maße lässt auf eine gemeinsame Wortverarbeitungsstufe schließen. Desweiteren signalisierten längere Fixationsdauern nach erhöhten N400 Amplituden das Andauern der Wortverarbeitung über die Dauer einer Fixation hinaus (Kapitel 3). Drittens zeigten sich Einflüsse der Präsentationsrate auf Frequenz- und Vorhersagbarkeitseffekte in EKPs. Der Verlauf von Bottom-up und Top-down Prozessen wird demnach entscheidend durch die zur Wortverarbeitung verfügbaren Zeit mitbestimmt (Kapitel 4). Viertens deutete ein früher Vorhersagbarkeitseffekt bei einer leseähnlichen Präsentationsgeschwindigkeit auf den schnellen Abgleich von Top-down Vorhersagen mit dem tatsächlichen visuellen Input hin (Kapitel 5). Die Ergebnisse sind mit interaktiven Modellen der Worterkennung vereinbar, nach welchen Bottom-up und Top-down Informationen gemeinsam frühe lexikalische Verarbeitungsstufen beeinflussen. Unter Berücksichtigung der Effekte von Frequenz, Vorhersagbarkeit und Präsentationsgeschwindigkeit wird ein Modell vorgeschlagen, das die vorliegenden Befunde zusammenführt (Kapitel 4).
Michael Dambacher
Bottom-up and top-down processes
in reading
Influences of frequency and predictability
on event-related potentials and eye movements
Universität Potsdam
Potsdam Cognitive Science Series | 1
PotsdamCognitiveScienceSeries|1
PotsdamCognitiveScienceSeries|1
MichaelDambacher
Bottomupandtopdownprocesses
inreading
Influencesoffrequencyandpredictability
oneventrelatedpotentialsandeyemovements
UniversitätsverlagPotsdam
BibliografischeInformationderDeutschenNationalbibliothek
DieDeutscheNationalbibliothekverzeichnetdiesePublikationinder
DeutschenNationalbibliografie;detailliertebibliografischeDatensind
imInternetüberhttp://dnb.dnb.de/abrufbar.
UniversitätsverlagPotsdam2010
http://info.ub.unipotsdam.de/verlag.htm
UniversitätsverlagPotsdam,AmNeuenPalais10,14469Potsdam
Tel.:+49(0)3319774623/Fax:3474
EMail:verlag@unipotsdam.de
DieSchriftenreihePotsdamCognitiveScienceSerieswirdherausgegeben
vonJohannesHaack,Dr.ChristianeWotschackundMichaelDambacher
DasManuskriptisturheberrechtlichgeschützt.
Umschlagfotos:NathalieBélanger,LizSchotter,MichaelDambacher
Zugl.:Potsdam,Univ.,Diss.,2009
1streviewer:Prof.Dr.ReinholdKliegl
2ndreviewer:Prof.Dr.ArthurM.Jacobs
Dayoforaldefense:December18,2009
ISSN(print)21904545
ISSN(online)21904553
OnlineveröffentlichtaufdemPublikationsserverderUniversitätPotsdam
URLhttp://pub.ub.uni-potsdam.de/volltexte/2010/4202/
URNurn:nbn:de:kobv:517-opus-42024
http://nbn-resolving.de/urn:nbn:de:kobv:517-opus-42024
ZugleichgedruckterschienenimUniversitätsverlagPotsdam
ISBN9783869560595
Acknowledgments
I am very grateful to my advisors Reinhold Kliegl and Arthur M. Ja-
cobs who granted me the privilege of being part of two outstanding
labs at the University of Potsdam and the FU Berlin. In particular,
I thank Arthur M. Jacobs for being such a provident mentor for
almost 10 years. Since I was an undergraduate student, he kept on
opening doors and provided irreplaceable opportunities to gain
ground in the scientific community. Likewise, I thank Reinhold
Kliegl for his thrilling creativity and enthusiasm that generates
an extremely positive and productive atmosphere, and makes you
sure science is one of the most exciting things you can do. I also
thank my other colleagues and friends at the University of Pots-
dam and the FU Berlin; they always gave me the feeling of being
a substantial part of both groups. Many thanks to Ralf Engbert,
Eike Richter, Jochen Laubrock, and Antje Nuthmann who greatly
supported my first (and the following) steps in programming and
who shared their immense knowledge about eye movements. They
never hesitated to spend time on listening to questions and were
more often than not able to give valuable advice. Likewise, I am
grateful to Sascha Tamm, Lars Kuchinke, Markus Hofmann, Angela
Heine, and Markus Conrad for their helpfulness and for numerous
discussions about how to process, analyze, and understand EEG
data. I thank Anja Gendt, Katrin Göthe, Sarah Risse, and Daniel
Schad for sharing their intellectual skills on multiple occasions
and for always being there as a friend. Special thanks go to Mario
Braun who spent days and weeks to set up and maintain the lab
and who provided me with everything I needed to run experi-
ments. I am very grateful to the former (associated) members
of the Helmholtz Center for the Study of Mind and Brain Dynamics.
Especially, I wish to thank Olaf Dimigen, Konstantin Mergenthaler,
Martin Rolfs, Hans Trukenbrod, and Christiane Wotschack for their
support and friendship. I learned a lot in our enjoyable, frequent,
and sometimes endless scientific exchanges on various fields and
disciplines, as well as in our not so scientific conversations about
life or non-sense. I also thank the colleagues I got the opportunity
to collaborate with. In particular, I thank Werner Sommer for his
great EEG expertise as much as for his positive and caring attitude,
Kay-Michael Würzner for his irreplacable assistance concerning
stimulus materials, as well as Victor Kuperman for giving me a
flavor of how "easy" writing can be. Many thanks to Petra Köhler,
Elena Schmidt, and Nicole Stietzel who were more than helpful in
finding ways out of bureaucratic labyrinths. I am deeply indebted
to my parents for their unconditional support in every phase of
my life. Finally, and with all my heart, I thank my beloved wife
Carina, who did not only bear with great patience my physical and
oftentimes mental absence over the last months and years, but who
continuously encouraged me, built me up, and kept me grounded.
ii
Abstract
In reading, word frequency is commonly regarded as the major
bottom-up determinant for the speed of lexical access. Moreover,
language processing depends on top-down information, such as
the predictability of a word from a previous context. Yet, however,
the exact role of top-down predictions in visual word recognition
is poorly understood: They may rapidly affect lexical processes, or
alternatively, influence only late post-lexical stages.
To add evidence about the nature of top-down processes and
their relation to bottom-up information in the timeline of word
recognition, we examined influences of frequency and predictabil-
ity on event-related potentials (ERPs) in several sentence reading
studies. The results were related to eye movements from natural
reading as well as to models of word recognition.
As a first and major finding, interactions of frequency and pre-
dictability on ERP amplitudes consistently revealed top-down in-
fluences on lexical levels of word processing (Chapters 2 and 4).
Second, frequency and predictability mediated relations between
N400 amplitudes and fixation durations, pointing to their sensitiv-
ity to a common stage of word recognition; further, larger N400
amplitudes entailed longer fixation durations on the next word,
a result providing evidence for ongoing processing beyond a fix-
ation (
Chapter 3
). Third, influences of presentation rate on ERP
frequency and predictability effects demonstrated that the time
available for word processing critically co-determines the course of
bottom-up and top-down influences (
Chapter 4
). Fourth, at a near-
normal reading speed, an early predictability effect suggested the
rapid comparison of top-down hypotheses with the actual visual in-
put (
Chapter 5
). The present results are compatible with interactive
models of word recognition assuming that early lexical processes
depend on the concerted impact of bottom-up and top-down in-
formation. We offered a framework that reconciles the findings on
a timeline of word recognition taking into account influences of
frequency, predictability, and presentation rate (Chapter 4).
iii
Zusammenfassung
Wortfrequenz wird in der Leseforschung als wesentliche Bottom-
up Determinante für die Geschwindigkeit des lexikalischen Zu-
griffs betrachtet. Darüber hinaus spielen Top-down Informationen,
wie die kontextbasierte Wortvorhersagbarkeit, in der Sprachver-
arbeitung eine wichtige Rolle. Bislang ist die exakte Bedeutung
von Top-down Vorhersagen in der visuellen Worterkennung jedoch
unzureichend verstanden: Es herrscht Uneinigkeit darüber, ob
ausschließlich späte post-lexikalische, oder auch frühe lexikalische
Verarbeitungsstufen durch Vorhersagbarkeit beeinflusst werden.
Um ein besseres Verständnis von Top-down Prozessen und deren
Zusammenhänge mit Bottom-up Informationen in der Worterken-
nung zu gewährleisten, wurden in der vorliegenden Arbeit Ein-
flüsse von Frequenz und Vorhersagbarkeit auf ereigniskorrelierte
Potentiale (EKPs) untersucht. Die Ergebnisse aus mehreren Satzle-
sestudien wurden mit Blickbewegungen beim natürlichen Lesen
sowie mit Modellen der Worterkennung in Beziehung gesetzt.
Als primärer Befund zeigten sich in EKP Amplituden konsis-
tent Interaktionen zwischen Frequenz und Vorhersagbarkeit. Die
Ergebnisse deuten auf Top-down Einflüsse während lexikalischer
Wortverarbeitungsstufen hin (Kapitel 2 und 4). Zweitens medi-
ierten Frequenz und Vorhersagbarkeit Zusammenhänge zwischen
N400 Amplituden und Fixationsdauern; die Modulation beider ab-
hängigen Maße lässt auf eine gemeinsame Wortverarbeitungsstufe
schließen. Desweiteren signalisierten längere Fixationsdauern nach
erhöhten N400 Amplituden das Andauern der Wortverarbeitung
über die Dauer einer Fixation hinaus (Kapitel 3). Drittens zeigten
sich Einflüsse der Präsentationsrate auf Frequenz- und Vorhersag-
barkeitseffekte in EKPs. Der Verlauf von Bottom-up und Top-down
Prozessen wird demnach entscheidend durch die zur Wortverar-
beitung verfügbaren Zeit mitbestimmt (Kapitel 4). Viertens deutete
ein früher Vorhersagbarkeitseffekt bei einer leseähnlichen Präsenta-
tionsgeschwindigkeit auf den schnellen Abgleich von Top-down
Vorhersagen mit dem tatsächlichen visuellen Input hin (Kapitel 5).
v
Die Ergebnisse sind mit interaktiven Modellen der Worterkennung
vereinbar
, nach welchen Bottom-up und Top-down Informationen
gemeinsam frühe
lexikalische
Verarbeitungsstufen beeinflussen.
Unter Berücksichtigung der Effekte von Frequenz, Vorhersagbarkeit
und Präsentationsgeschwindigkeit wird ein Modell vorgeschlagen,
das die vorliegenden Befunde zusammenführt (Kapitel 4).
vi
Contents
1 Introduction 1
1.1 Tracking the timeline of word recognition . . . . . . . 2
1.1.1 Eye movements . . . . . . . . . . . . . . . . . . 3
1.1.2 Event-related potentials (ERPs) . . . . . . . . . 5
1.2
Bottom-up and top-down processes in word recogni-
tion ............................. 7
1.2.1 Bottom-up information: Word frequency . . . 8
1.2.2 Top-down information: Word predictability . 10
1.3
Towards a common timeline of bottom-up and top-
downprocesses ...................... 12
1.3.1
Top-down influences in word recognition:
Lexical or post-lexical? . . . . . . . . . . . . . . 12
1.3.2 Models of word recognition . . . . . . . . . . . 13
1.3.2.1 Serial approaches . . . . . . . . . . . 13
1.3.2.2 Parallel approaches . . . . . . . . . . 14
1.3.2.3 Other approaches . . . . . . . . . . . 15
1.3.3
Predictability and the anticipation of upcom-
ingevents ..................... 16
1.3.4
The interplay of bottom-up and top-down
information .................... 18
1.4 Overview of the present studies . . . . . . . . . . . . 20
1.4.1
Frequency and predictability effects on event-
related potentials during reading (Chapter 2) 20
1.4.2
Synchronizing timelines: Relations between
fixation durations and N400 amplitudes dur-
ing sentence reading (Chapter 3) . . . . . . . . 22
vii
Contents
1.4.3
The interplay of word frequency, predictabil-
ity, and SOA in sentence reading: Evidence
from event-related potentials (Chapter 4) . . . 23
1.4.4
Event-related potentials reveal rapid verifica-
tion of predicted visual input (Chapter 5) . . . 27
2 Frequency and predictability effects on event-related po-
tentials during reading 29
2.1 Introduction ........................ 31
2.1.1 Frequency and predictability in ERPs . . . . . 32
2.1.2 Present study . . . . . . . . . . . . . . . . . . . 35
2.2 Methods .......................... 37
2.2.1 Participants . . . . . . . . . . . . . . . . . . . . 37
2.2.2 Stimuli....................... 37
2.2.3 Procedure ..................... 39
2.2.4 Electrophysiological recording . . . . . . . . . 39
2.2.5 Analyses...................... 40
2.2.6 Plots ........................ 43
2.3 Results ........................... 44
2.3.1 P200 ........................ 44
2.3.2 N400 ........................ 49
2.3.3 Supplementary analyses . . . . . . . . . . . . . 52
2.3.4 Goodness of fit . . . . . . . . . . . . . . . . . . 53
2.4 Discussion ......................... 53
2.4.1 P200 ........................ 54
2.4.2 N400 ........................ 57
2.4.3
Frequency and predictability: Lexical and
post-lexical processes? . . . . . . . . . . . . . . 59
2.4.4 Conclusions . . . . . . . . . . . . . . . . . . . . 61
3 Synchronizing timelines: Relations between fixation dura-
tions and N400 amplitudes during sentence reading 63
3.1 Introduction ........................ 65
3.2 Methods .......................... 72
3.2.1 Stimuli....................... 72
viii
Contents
3.2.2 Eye movements . . . . . . . . . . . . . . . . . . 72
3.2.2.1 Participants . . . . . . . . . . . . . . . 72
3.2.2.2 Procedure . . . . . . . . . . . . . . . . 72
3.2.2.3 Recording and data processing . . . 73
3.2.3 ERPs ........................ 73
3.2.3.1 Participants . . . . . . . . . . . . . . . 73
3.2.3.2 Procedure . . . . . . . . . . . . . . . . 74
3.2.3.3 Recording and data processing . . . 74
3.2.4 Data reduction . . . . . . . . . . . . . . . . . . 74
3.3 Results ........................... 76
3.3.1 Fixation durations and N400 amplitudes . . . 76
3.3.2 Synchronizing the timelines . . . . . . . . . . . 79
3.3.3 Baseline path model . . . . . . . . . . . . . . . 81
3.3.4 Predictor path models . . . . . . . . . . . . . . 84
3.3.5 Modelt...................... 89
3.4 Discussion ......................... 90
4 The interplay of word frequency, predictability, and SOA in
sentence reading: Evidence from event-related potentials 99
4.1 Introduction........................101
4.1.1
Frequency: Bottom-up processes and lexical
access........................101
4.1.2
Predictability: Top-down processes and lexi-
calaccess......................102
4.1.3 Predictability and pre-activation . . . . . . . . 105
4.1.4 Present studies . . . . . . . . . . . . . . . . . . 106
4.2 Experiment1 .......................107
4.2.1 Methods......................108
4.2.1.1 Participants . . . . . . . . . . . . . . . 108
4.2.1.2 Materials . . . . . . . . . . . . . . . . 109
4.2.1.3 Procedure . . . . . . . . . . . . . . . . 111
4.2.1.4 Apparatus . . . . . . . . . . . . . . . 112
4.2.1.5 EEG recording . . . . . . . . . . . . . 112
4.2.1.6 Data processing and analyses . . . . 113
4.2.1.7 Reduction of subject sample . . . . . 114
ix
Contents
4.2.2 Results.......................115
4.2.2.1 Global analyses . . . . . . . . . . . . 115
4.2.2.2 Local analyses . . . . . . . . . . . . . 117
4.2.2.3
Additional analyses: Context effects
and reduction of subject sample . . . 120
4.2.3 Discussion.....................123
4.3 Experiment2 .......................127
4.3.1 Methods......................129
4.3.1.1 Participants . . . . . . . . . . . . . . . 129
4.3.1.2 Stimuli and procedure . . . . . . . . 129
4.3.1.3
Apparatus, EEG recording, data
processing, and analyses . . . . . . . 130
4.3.2 Results.......................130
4.3.2.1 Global analyses . . . . . . . . . . . . 131
4.3.2.2 Local analyses . . . . . . . . . . . . . 132
4.3.2.3
Comparison between the studies
(SOA effects) . . . . . . . . . . . . . . 135
4.3.3 Discussion.....................139
4.4 General discussion . . . . . . . . . . . . . . . . . . . . 142
4.4.1 Influences of presentation rate . . . . . . . . . 143
4.4.2 Integrative account . . . . . . . . . . . . . . . . 145
4.4.3 Further suggestions . . . . . . . . . . . . . . . 149
4.5 Conclusions ........................151
5 Event-related potentials reveal rapid verification of pre-
dicted visual input 153
5.1 Introduction........................155
5.2 Methods ..........................157
5.2.1 Participants . . . . . . . . . . . . . . . . . . . . 157
5.2.2 Materials......................158
5.2.3 Procedure .....................161
5.2.4
Electrophysiological recording and data pro-
cessing.......................161
5.3 Results ...........................162
5.4 Discussion .........................166
x
Contents
6 General summary and conclusions 173
6.1 General summary . . . . . . . . . . . . . . . . . . . . . 173
6.1.1
Bottom-up and top-down interactions in lexi-
calprocessing...................174
6.1.2 Rapid verification . . . . . . . . . . . . . . . . . 176
6.1.3 Reading rate matters . . . . . . . . . . . . . . . 177
6.1.4 Eye movements and ERPs . . . . . . . . . . . . 178
6.1.5 Models of word recognition . . . . . . . . . . . 180
6.2 Conclusions ........................181
References 183
Appendix 207
A. Potsdam Sentence Corpus 1 (PSC1) . . . . . . . . . . 207
B. Potsdam Sentence Corpus 3 (PSC3) . . . . . . . . . . 213
C. ANOVA Table (Chapter 4) . . . . . . . . . . . . . . . . 239
xi
List of Figures
2.1
Grand average ERPs for three frequency classes in
PSC1reading ....................... 45
2.2
Grand average ERPs for three predictability classes
inPSC1reading...................... 46
2.3 rmMRA predictor effects on P200 amplitudes . . . . 48
2.4 rmMRA predictor effects on N400 amplitudes . . . . 51
3.1
ERPs for three frequency and predictability classes
inPSC1reading...................... 68
3.2
Immediate relations between fixation durations and
N400amplitudes ..................... 77
3.3
Lagged relations between fixation durations and
N400amplitudes ..................... 78
3.4
Synchronizing the timelines of eye movements and
ERPs ............................ 80
3.5 Visualization of path analytic models . . . . . . . . . 85
3.6 Illustrations of immediacy, lag, and successor effects 87
4.1 PSC3 stimulus example . . . . . . . . . . . . . . . . . 109
4.2
Grand average ERPs for PSC3 target words in Exp.
1(SOA700).........................116
4.3
Frequency effects at posterior electrode clusters in
Exp.1............................119
4.4
Predictability effect at posterior electrode clusters in
Exp.1............................120
4.5 Effects at stimulus onset in Exp. 1 . . . . . . . . . . . 122
4.6
Grand average ERPs for PSC3 target words in Exp.
2(SOA280).........................131
xiii
List of Figures
4.7
Frequency effects at posterior electrode clusters in
Exp.2............................134
4.8
Predictability effects at posterior electrode clusters
inExp.2 ..........................135
4.9
Influence of SOA on frequency and predictability
effects............................137
4.10
Integrative account for the timelines of word recog-
nition............................148
5.1 Stimuli and procedure of PSC3 reading . . . . . . . . 159
5.2 ERPs to PSC3 targets and target-preceding words . . 163
5.3 Latencies of the first predictability effect . . . . . . . . 164
5.4 Early predictablity effects in grand average ERPs . . 166
xiv
List of Tables
2.1
Statistics of three frequency and predictability classes
inthePSC1......................... 41
2.2 PSC1 word statistics across word positions . . . . . . 42
2.3 Results of the rmMRA on P200 amplitudes . . . . . . 47
2.4 Results of rmMRAs on N400 amplitudes . . . . . . . 50
3.1 Statistics of word triplets in the PSC1 . . . . . . . . . 75
3.2
Path analytic models on fixation durations and N400
amplitudes......................... 82
3.2
Path analytic models on fixation durations and N400
amplitudes (continued) . . . . . . . . . . . . . . . . . . 83
3.3 Variance-covariance matrix for path analyses . . . . . 86
4.1 Statistics of PSC3 target words . . . . . . . . . . . . . 111
4.2
Global ANOVAs on ERPs to PSC3 target words in
Exp. 1 (SOA700) on N=20 data sets . . . . . . . . . 117
4.3
Global ANOVAs on ERPs to PSC3 target words in
Exp.2(SOA280)......................132
5.1 Statistics of PSC3 target words . . . . . . . . . . . . . 160
C
Global ANOVAs on ERPs to PSC3 target words in
Exp. 1 (SOA700) of Chapter 4 on N=32 data sets . . 239
xv
1 Introduction
Reading is an outstanding achievement of the human brain. The
ability to read has substantially formed our history and culture and
plays an essential role in our everyday lives. Skilled readers can
hardly prevent processing a written stimulus and, most often, they
grasp a word’s meaning in the fraction of a second. The under-
standing of the underlying mechanisms as well as the time course
of this seemingly effortless skill is the goal of psycholinguistic
research.
In general, two major sources of information contribute to lan-
guage comprehension. First, bottom-up processes transmit neural
codes of sensory input to increasingly complex levels. On success,
the appropriate word representation in the long-term memory is ac-
tivated and semantic information associated with a word becomes
available. Second, language-related knowledge and experiences
are expressed in top-down influences that guide the way words are
understood. They permit the integration of word meaning into a
wider context and hold the potential to bias expectations about
upcoming words.
Despite ample evidence for the relevance of bottom-up and top-
down processes, their joint role in the timeline of word recognition
is insufficiently understood. Clearly, bottom-up processes account
for the elaboration of sensory signals and therefore reflect opera-
tions giving rise to the retrieval of a word’s mental representation,
i.e., lexical access. The role of top-down processes, however, is
ambiguous. Those may be slow and only play a role for mental
operations after lexical access; alternatively, they may rapidly im-
pinge on early lexical processes and co-determine the course of
word identification.
1
1. Introduction
The present thesis investigates the relationship of bottom-up and
top-down processes in reading and aims at contributing to the
picture of their common role in the timeline of word recognition.
We will show that top-down information is rapidly available and
interacts with early levels of bottom-up processing.
1.1 Tracking the timeline of word
recognition
Apparently, our brains do not voluntarily disclose the mechanisms
underlying visual word recognition. Introspection alone does not
suffice to explicitly describe mental operations, let alone delineat-
ing temporal interdependencies. Thus, psycholinguistic research
resorts to experimental techniques and measures that shall bring
to light processes of word identification.
For several decades, reaction times and error rates served as
major tools for the investigation of word recognition. For instance,
stimulus properties affecting measures in the lexical decision task
(LDT
1
; e.g., Rubenstein, Garfield, & Millikan, 1970) were often
considered as (co-)determinants for lexical access. Without doubt,
behavioral evidence from this and numerous other paradigms has
substantially enriched and formed the current understanding of
visual word recognition.
Nevertheless, any inferences from behavioral data remain indi-
rect since they do not exclusively reflect the time necessary for
lexical access. For example, the LDT involves stages of decision as
well as preparation and execution of motor responses. As a related
issue, the exact temporal nature of influences often remains vague
since effects can be located on pre-lexical (i.e., prior to lexical access)
or post-lexical (i.e., after lexical access) levels. Thus, questions about
relevant processes and the time course of lexical access cannot be
fully answered on the basis of these measures.
1
In the LDT, participants are asked to indicate whether a stimulus is an existing
word or a non-sense letter string, i.e., a nonword. The LDT was widely under-
stood as pure measure of lexical access (but see Balota & Chumbley, 1984 for a
discussion).
2
1.1. Tracking the timeline of word recognition
To overcome this shortcoming, psycholinguists make use of sev-
eral additional sources of information. As one source, theories
of word recognition are implemented in computational models.
Simulations can be tested against human performance and there-
fore allow the validation of theoretical assumptions. Importantly,
advanced models incorporate hypotheses about dynamics in word
recognition, such that the time course can be tracked with high
accuracy
(
e.g., Grainger & Jacobs, 1996; McClelland & Rumelhart,
1981).
Another source of information comes from eye movements in
normal reading, reflecting visual word recognition in natural set-
tings. In particular, lexical processing is at least partly expressed in
fixation durations, which therefore permit inferences about under-
lying cognitive processes
(
e.g., Kliegl, Nuthmann, & Engbert, 2006;
Rayner, 1998).
As a third source, electrophysiological data provide insights
about cognitive processes with high temporal accuracy. Influences
of word properties and their interplay with mental operations can
be measured online, without requiring behavioral responses. Thus,
ERPs are an important tool to track word processing from the
initial uptake of sensory information up to the comprehension
and integration of meaning
(
e.g., Barber & Kutas, 2007; Kutas,
Van Petten, & Kluender, 2006).
Considering models, eye movements, and ERPs, the present work
takes advantage from all three sources. In accordance with calls
for multi-methodological approaches in psycholinguistic research
(
Barber & Kutas, 2007; Jacobs & Carr, 1995), we aim at gaining
insights into mechanisms of word recognition, that go beyond
evidence from single approaches. The following sections briefly de-
pict central aspects of eye movements and electroencephalographic
measures.
1.1.1 Eye movements
Maybe the most intuitive way to study the processing of written
language is to monitor the subjects’ gaze while they are reading a
3
1. Introduction
text. In reading, the eyes do not smoothly move across the lines but
perform sequences of rapid movements, known as saccades, and
intervals when they stand relatively still, i.e., fixations. Causes for
this behavior lie in the anatomical structure of the retina. Whereas
visual acuity is high in the fovea (i.e., the area of about 2
of visual
angle around the center of a fixation), it rapidly drops in parafoveal
(i.e., from 2
to 5
around the fixation) and peripheral (i.e., more
than 5
around the fixation) regions. Usually, saccades serve the
purpose to foveate visual information that is intended for detailed
analysis. For word recognition, a viewing location near the word
center grants optimal efficiency of stimulus processing
(
e.g., Jacobs,
Nuerk, Graf, Braun, & Nazir, 2008; Nazir, Jacobs, & O’Regan, 1998;
Nuthmann, Engbert, & Kliegl, 2007; O’Regan & Jacobs, 1992).
In alphabetic languages, typical reading saccades have durations
of 20 to 30 ms and their amplitudes encompass around seven letters;
however, a large proportion of saccades deviates from this average
score. Since vision is largely suppressed during saccades
(
Matin,
1974), information must be predominantly extracted in fixation
intervals. Skilled readers fixate around four to five words per
second, but also here, variability is very high, as fixation durations
range from 50 ms to more than 600 ms (Rayner, 1998, 2009).
Importantly, fixation-saccade sequences in reading are sensitive
to cognitive influences. For instance, increasing text difficulty en-
tails longer fixation durations as well as shorter saccades; similar
patterns are observed for dyslexic readers. Indeed, eye movements
permit distinguished inferences about processes of word recogni-
tion. There is a long tradition of investigating language-related
influences on various measures, such as fixation durations, saccade
amplitudes, backward regressions, word skippings, or landing site
distributions (for reviews see Rayner, 1998, 2009).
As a major advantage, eye movements disclose dynamics of word
processing across the course of sentences. For instance, fixation
durations are not only sensitive to properties of a fixated stimulus
(i.e., immediacy effects), but are also modulated by characteristics
of preceding (i.e., lag effects) and upcoming (i.e., successor effects)
4
1.1. Tracking the timeline of word recognition
words. They therefore unveil important information about the
interaction of word recognition and saccade planning
(
Kliegl et
al., 2006). Several sophisticated models deal with these insights
about the interplay of oculomotor and cognitive processes and are
able to reproduce dynamics of human reading behavior with high
accuracy on a quantitative level (e.g. SWIFT, Engbert, Nuthmann,
Richter, & Kliegl, 2005; EZ-Reader, Reichle, Pollatsek, & Rayner,
2006; Glenmore, Reilly & Radach, 2006).
Unquestionably, eye tracking is an irreplaceable tool for the inves-
tigation of visual word processing. Advantageously, participants
move their eyes normally and at their own pace; the data therefore
largely reflect natural reading behavior. However, eye movements
do not disclose information about processes between the moment
when the eyes initially fixate a word and the time when they leave
it. In particular, it is not clear, when during a fixation a specific
word is lexically accessed, or whether it is identified at all. Indeed,
words are often inspected longer when the previous stimulus was
difficult, a finding that may point to ongoing word processing
after the eyes have left a stimulus (Kliegl et al., 2006; see Chapter
3). Thus, despite their sensitivity to lexical processing, fixation
durations permit only indirect evidence about word recognition
itself. An approach to access this information more directly is the
use of event-related potentials.
1.1.2 Event-related potentials (ERPs)
Information processing in the brain relies on the communication
between neurons, expressed in the transmission of current flows.
When a large number of neighboring neurons fire synchronously,
brain-electrical activity is sufficiently strong to be captured by
electrodes applied to the surface of the scalp. The resulting elec-
troencephalogram (EEG) is the summation of all signals at an
electrode site. Ongoing neural activity manifests as a continuous
EEG stream that is characterized by positive- and negative-going
voltage changes (Berger, 1929).
5
1. Introduction
Compared to this "spontaneous" EEG, brain-electrical responses
to the processing of a specific stimulus are often too small to be
detected in the continuous signal. A common way to extract this
information is the calculation of event-related potentials (ERPs) by
averaging multiple EEG sequences, that are associated with the
same experimental class of stimuli. It is assumed that spontaneous
EEG activity is not phase-locked to stimulus processing and will
cancel out in the averaging procedure. Accordingly, the remaining
ERP course is believed to largely reflect brain potentials that are
related to the processing of experimentally relevant events (for
reviews see Kutas & Van Petten, 1994; Kutas et al., 2006).
A strategy to get insights about stimulus-related processes is the
comparison of ERPs from two or more experimental conditions.
Differences between the signals indicate that, on average, parts of
the brain responded differently to experimental conditions. Such
effects manifest in amplitude modulations or in temporal shifts
of ERP components. Thereby, the latency of an effect provides an
upper temporal bound for distinct processing, since it is possible
that variations earlier in the time course were not captured by
ERPs.
Over the past 30 years, ERPs have become an important tool
for psycholinguistic research. Their most obvious advantage is
the high temporal resolution that reflects ongoing neural activity
in the range of milliseconds. Unlike measures of reaction times,
error rates, or eye movements, ERPs allow the investigation of
word recognition while stimuli are being processed. As another
benefit, ERPs are independent from overt responses or do at least
not necessarily focus on them. In contrast, in classical psycholin-
guistic paradigms (e.g., LDT) speeded motor responses impose
additional tasks that may affect mechanisms and strategies of word
recognition.
Nevertheless, ERP research has to face some difficulties, that are
relevant for the study of reading. First, ERPs are sensitive to ocular
artifacts. Because the eyeball acts as a dipole, eye movements and
blinks induce changes in scalp voltages that supersede activity
6
1.2. Bottom-up and top-down processes in word recognition
related to word processing; further, (eye-)muscular activity can dis-
turb the EEG signal. Consequently, stimuli are usually presented
in a way that renders eye movements unnecessary, that is, one at a
time in the center of the screen (i.e., Rapid Serial Visual Presenta-
tion, RSVP). Second, prominent language-related ERP components
(e.g., N400 or P600) have long latencies and range up to one second
after stimulus onset. At a natural reading rate of four to five words
per second, ERPs to consecutive stimuli would overlap such that
signals could hardly be attributed to the processing of a unique
word. Therefore, words are often separated by unnaturally long
intervals. Given stimulus-wise presentation at rather slow rates,
ERP research of word recognition often uses setups that diverge
substantially from natural reading. Yet, the importance of ecologi-
cal validity for reading processes is unknown; parts of the present
work will address this issue (see Chapter 4).
Despite these shortcomings, ERPs essentially contribute to the un-
derstanding of word recognition. They provide information about
the time course of lexical processing with a resolution that is beyond
the precision of any behavioral measure. After all, electrophysi-
ology and eye movements appear as complementary approaches,
since ERPs hold the potential to uncover mental operations during
fixations, while eye movements reveal natural behavior in normal
reading.
1.2 Bottom-up and top-down processes in
word recognition
Considering empirical evidence from ERPs and eye movements,
the present work investigated the role of bottom-up and top-down
processes during reading, an issue that is discussed controversially
in psycholinguistic research. The following sections describe word
frequency and predictability as important representatives of these
processes.
7
1. Introduction
1.2.1 Bottom-up information: Word frequency
Reading is determined by bottom-up processing of visual informa-
tion. As the signal propagates along a hierarchy of increasingly
complex neuronal detectors, mental operations become more and
more elaborate. In particular, the left occipito-temporal cortex is
gradually sensitive to lexical information, ranging from individual
letters and bigrams to morphemes and, finally, entire words
(
e.g.,
Vinckier et al., 2007). Undoubtedly, word recognition substantially
depends on the visual processing of word characteristics; indeed,
there are more than 50 stimulus properties affecting performance
(cf., Graf, Nagler, & Jacobs, 2005).
Among these, probably the most important characteristic is the
frequency with which a word occurs in a language. Starting with
the observation that identification requires longer tachistoscopic
presentation times for low than for high frequency words
(
Howes
& Solomon, 1951), research over the last decades has consistently
yielded robust and sizeable frequency effects across numerous
tasks. For instance, in isolated word recognition (e.g., LDT or
naming), reaction times are shorter and accuracy is higher for
high than for low frequency words
(
Forster & Chambers, 1973;
Frederiksen & Kroll, 1976; Grainger, 1990; Rubenstein et al., 1970).
Analogously, during normal left-to-right reading, high frequency
words are fixated shorter and are skipped more often
(
e.g. Inhoff
& Rayner, 1986; Kliegl, Grabner, Rolfs, & Engbert, 2004; Kliegl et
al., 2006; Rayner & Duffy, 1986; Schilling, Rayner, & Chumbley,
1998). Furthermore, frequency effects in ERPs were reported on
the N400 component
2
, a negative deflection peaking at around
400 ms post-stimulus over centro-parietal scalp sites. Larger N400
amplitudes for low than for high frequency words pointed to
increased processing costs for less familiar stimuli
(
Rugg, 1990;
Van Petten & Kutas, 1990; Young & Rugg, 1992).
2
The N400 component is one of the best-studied ERP components in psycholinguis-
tic research. The N400 is predominantely modulated by contextual influences
(see next section) and is therefore often regarded as an index for the ease of
semantic integration into a context (Kutas & Hillyard, 1980, 1984).
8
1.2. Bottom-up and top-down processes in word recognition
Although the exact nature of frequency effects was debated
(e.g., Balota & Chumbley, 1984; see Monsell, 1990, for a review),
there is general agreement that the variable influences early pro-
cesses of word recognition
(
e.g., Hudson & Bergman, 1985; Forster,
1992; Forster & Chambers, 1973). This view is supported by ERPs
revealing frequency effects in time intervals well before 200 ms
(
Braun, Hutzler, Ziegler, Dambacher, & Jacobs, 2009; Hauk, Davis,
Ford, Pulvermüller, & Marslen-Wilson, 2006; Hauk & Pulvermüller,
2004; Penolazzi, Hauk, & Pulvermüller, 2007; Sereno, Brewer, &
O’Donnell, 2003; Sereno, Rayner, & Posner, 1998). In accordance
with influences on fixation durations of around 200 to
250 ms
, fre-
quency appears as a relevant factor for rapid bottom-up processing
of sensory information.
Given that essentially every task associated with lexical process-
ing is sensitive to word frequency, the variable is understood as a
core determinant for lexical access. Therefore, the latency of the
first word frequency effect in ERPs is often considered as an esti-
mate that word representations have been identified
3(
e.g., Braun
et al., 2009; Hauk & Pulvermüller, 2004; Sereno et al., 2003, 1998;
Sereno & Rayner, 2003). After all, the reliability of frequency effects
3
Certainly, this assumption is an oversimplification as a deterministic relationship
between lexical access and ERP frequency effects is rather unlikely. Nevertheless,
it is reasonable to assume a link between processes of word identification and
detectable influences of frequency in ERPs. For instance, while words of all
frequencies consist of a common set of sub-lexical features (e.g., letters), word
frequency describes a lexical characteristic of the entire word form. Frequency
effects in ERPs are therefore considered as neural responses to the activation of
unique word representations. This concept has been strongly inspired by models
of word recognition that - given the ubiquitous influence in tasks requiring
lexical access - have incorporated frequency as the critical determinant for lexical
access
(
Forster, 1976; McClelland & Rumelhart, 1981; Morton, 1969). As another
argument, eye movements in reading are widely believed to be optimized for
processes of word identification. Strong frequency effects on fixation durations
corroborate the idea of frequency as a valid predictor for the speed of lexical
access. On that note, the temporal similarity of fixation durations and early
ERP frequency effects provide compatible latencies for word identification
(
e.g.,
Sereno et al., 1998; Sereno & Rayner, 2003). Notably, despite these considerations,
unequivocal evidence for the moment of lexical access from empirical data is
lacking in every domain of psycholinguistic research. Therefore, the first word
frequency effect in ERPs must be understood as a proxy rather than as a direct
measure of the latency of lexical access.
9
1. Introduction
suggests a fundamental advantage for bottom-up processing of
familiar stimuli and a major advantage for the speed of lexical
access.
1.2.2 Top-down information: Word predictability
For a long time, the concept of visual perception was dominated
by the view that sensory processing relies first and foremost on
the hierarchical bottom-up flow of information. Recent findings,
however, have radically changed this uni-directional picture. Top-
down processes, as for instance attentional control or expectations
of upcoming sensory events, affect perception on virtually every
level
(
e.g., Bar, 2007; Carlsson, Petrovic, Skare, Petersson, & Ingvar,
2000; Churchland, Ramachandran, & Sejnowski, 1994; Corbetta &
Shulman, 2002; Engel, Fries, & Singer, 2001; Enns & Lleras, 2008;
Gilbert & Sigman, 2007; Kastner, Pinsk, De Weerd, Desimone, &
Ungerleider, 1999; Kveraga, Ghuman, & Bar, 2007; Mechelli, Price,
Friston, & Ishai, 2004; O’Connor, Fukui, Pinsk, & Kastner, 2002;
Somers, Dale, Seiffert, & Tootell, 1999; Williams et al., 2008).
Undoubtedly, top-down processes also play a critical role in
language comprehension. This becomes intuitively evident consid-
ering the processing of ambiguous words: Readers have a different
understanding of the word bank in a discourse about a river side
as compared to a financial institute. It is assumed that top-down
information relying on the interpretation of the context activates
the appropriate meaning of an ambiguous word
(
e.g., Sereno et
al., 2003; Simpson, 1994; Van Petten, 1995). As another example,
when readers are asked to complete a sentence fragment like The
lumberjack usually used his chainsaw to fell a..., most of them probably
fill in the word tree. Apparently, in a constraining context, the
interpretation of a sentence message affords specific predictions
that activate a mental word representation even in the absence of
visual information.
10
1.2. Bottom-up and top-down processes in word recognition
In fact, word predictability
4
from a previous context is an im-
portant factor for the efficiency of language processing. Similar
to word frequency, robust influences of predictability have been
observed across many tasks and in several measures. For instance,
in single-word recognition, reaction times as well as naming la-
tencies are shorter for words that are preceded by a supporting
compared to a non-supporting context
(
Kleiman, 1980; Fischler &
Bloom, 1979; Schuberth & Eimas, 1977; Stanovich & West, 1983;
West & Stanovich, 1982). During normal reading, eye movements
reveal shorter fixation durations and higher skipping rates for high
than for low predictability words
(
Ashby, Rayner, & Clifton, 2005;
Calvo & Meseguer, 2002; Duffy, Henderson, & Morris, 1989; Ehrlich
& Rayner, 1981; Kliegl et al., 2004, 2006; Rayner, Ashby, Pollatsek,
& Reichle, 2004; Rayner & Well, 1996). In ERPs, predictability has
strong effects on the N400 component; increasing amplitudes point
to augmenting processing difficulty as predictability decreases
(Kutas & Hillyard, 1980, 1984; for reviews see Kutas & Van Petten,
1994; Kutas et al., 2006; Barber & Kutas, 2007).
Thus, besides the undisputed role of bottom-up flow, word pro-
cessing appears to be strongly modulated by contextual top-down
information. The relation of the two streams, conceptualized as
influences of frequency and predictability
5
, is described in more
detail in the following sections.
4
Word predictability norms in experimental setups are usually assessed in a paper-
pencil cloze task. Analogous to the lumberjack example, subjects are presented
with a sentence fragment and are asked to guess the next word. Predictability is
then computed as the proportion of subjects predicting the correct word from
the prior context.
5
Note that a strict separation of word frequency and predictability into bottom-
up and top-down streams presumably is an oversimplification. For instance,
it is very likely that bottom-up processes theirselves involve feedback loops
reinforcing the signal; in turn, not all top-down processes are independent from
feedforward spreading of neural activation
(
e.g., Di Lollo, Enns, & Rensink,
2000; McClelland & Rumelhart, 1981). Further, during reading acquisition,
item-specific codes of a word form may be retained and affect recognition as
frequency-based top-down information
(
cf., Jacobs et al., 2008). After all, there
is a positive correlation of frequency and predictability in natural language
(
Kliegl et al., 2006). Arguably, however, the core influences of frequency and
predictability are expressed in bottom-up and top-down processes, respectively
(cf., this and the previous section). For the sake of simplicity, the present
terminology is confined to these main attributes of the variables.
11
1. Introduction
1.3 Towards a common timeline of
bottom-up and top-down processes
1.3.1 Top-down influences in word recognition:
Lexical or post-lexical?
In view of the obvious influence of word predictability on various
measures of language processing, there is little doubt about the
facilitative role of a supporting context. Yet, however, the exact
nature of top-down predictions in the course of word recognition
is unresolved. There are two controversial views that were both
supported by behavioral results. On the one hand, the anticipa-
tion of upcoming stimuli from a prior context was considered as
too slow to affect early stages of word identification. Contextual
top-down influences were therefore expected only on post-lexical
levels of semantic integration, while lexical access was regarded
as the result of bottom-up processes
(
e.g., Burgess, Tanenhaus, &
Seidenberg, 1989; Lucas, 1987; Onifer & Swinney, 1981; Swinney,
1979). On the other hand, it was assumed that the rapid interpreta-
tion of a context affords information that is relevant also on early
levels of word recognition. Accordingly, lexical processing should
depend on joint influences of bottom-up and top-down information
(e.g., Glucksberg, Kreuz, & Rho, 1986; Tabossi, 1988; Schvaneveldt,
Meyer, & Becker, 1976; Simpson, 1981; see Simpson, 1994 for a
review).
While the debate was not decisively resolved on the basis of
behavioral data, the high temporal resolution of ERPs holds the
potential to provide evidence about the role of top-down informa-
tion. In particular, the context-sensitive N400 component was often
considered as a pure reflection of post-lexical processes
(
Brown &
Hagoort, 1993; Holcomb, 1993; Misra & Holcomb, 2003), challeng-
ing the view of predictability effects on lexical levels. However, the
exact nature of the N400 is not yet understood, as several reports
pointed to lexical influences on the N400
(
Deacon, Dynowska, Rit-
ter, & Grose-Fifer, 2004; Deacon, Hewitt, Yang, & Nagata, 2000;
Wang & Yuan, 2008). Thus, the question about the time course of
top-down predictions in word recognition is still unresolved.
12
1.3. Towards a timeline of bottom-up and top-down processes
1.3.2 Models of word recognition
Given the central role of bottom-up and top-down information
in language processing, the opposing views concerning their rela-
tionship were also expressed in different models of visual word
recognition. The following paragraphs describe the most influential
representatives reflecting this controversy.
1.3.2.1 Serial approaches
According to serial search models, lexical access is exclusively
driven by bottom-up processing of visual input. In particular, word
frequency is considered as the primary agent for the speed of lexical
access
(
cf., Rubenstein et al., 1970). The most prominent approach
of this kind is the Bin Theory
(
Forster, 1976, 1992; Murray & Forster,
2004). In this account, word representations in the mental lexicon
are organized in sublists, called bins. Within each bin, lexical entries
are sorted in descending order according to their frequency. The
identification of a given letter string is performed in a sequential
comparison of the visual input with each entry, from top to bottom.
Due to the serial nature of the search, positive matches are found
faster for high than for low frequency words. Information of a
successful match is then used to activate semantic and syntactic
information of the word for later processing.
In contrast to word frequency, contextual top-down influences
are considered as slow factors in this framework. In line with
a modular view of word recognition, postulating the absence of
top-down effects on lexical phases of word processing
(
Fodor,
1983; Kintsch & Mross, 1985), word predictability is assumed to
act only on a post-lexical stage. Thus, high expectancy of words
facilitates the combination of linguistic information with contextual
and background knowledge and eases semantic integration into
a broader context, but has no role for lexical access
(
Murray &
Forster, 2004).
13
1. Introduction
1.3.2.2 Parallel approaches
Also parallel models have incorporated word frequency as the
major determinant for the speed of lexical access. In Morton’s
(
1969)
Logogen Model, the mental lexicon is described as a set of detectors
(i.e., logogens), each representing a word. Sensory input sharing
features with a logogen increases its activation level, until a virtual
threshold of lexical access is reached and a word is identified. The
threshold is lower and therefore accessed faster for high than for
low frequency words. Importantly, the Logogen Model assumes top-
down influences on early stages of word recognition. A supporting
context increases the activation level of an expected logogen, such
that less sensory information is necessary for lexical access of high
predictability words.
Another milestone in theories of visual word recognition is the
computational Interactive Activation Model (IAM; McClelland &
Rumelhart, 1981; Rumelhart & McClelland, 1982), providing quan-
titative estimates of dynamics in the mental lexicon. Similar to
Morton’s approach, words are represented as nodes that increase
their activation level with matching visual input. To account for fre-
quency effects, a higher resting level grants an activation advantage
for high over low frequency words. A core assumption of the IAM
is the interactive nature of perception. Top-down and bottom-up
processes work simultaneously, such that knowledge about words
interacts with the lower processing levels and co-determines lex-
ical access. In particular, the model comprises three hierarchical
levels for letter string processing: Feature detectors activate letter
units, which in turn project to whole word representations. Letter
and word levels are connected bi-directionally and send inhibitory
and excitatory signals; units within one level are subject to lat-
eral inhibition. Although the IAM primarily accounts for single
word recognition, a supporting sentence context was assumed to
increase the activation level of high predictability words, while low
predictability nodes are inhibited
(
Rumelhart & McClelland, 1982).
14
1.3. Towards a timeline of bottom-up and top-down processes
1.3.2.3 Other approaches
Of course, the landscape of theories on word recognition is much
richer than the approaches outlined above, as there are highly so-
phisticated models accounting for a variety of effects (for reviews
see e.g., Barber & Kutas, 2007; Jacobs & Grainger, 1994; Mon-
sell, 1990; Seidenberg, 2007). For instance, several localist models
build on the core principles of the IAM and successfully simu-
late phenomena of orthographic and phonological processing (e.g.,
DRC, Coltheart, Rastle, Perry, Langdon, & Ziegler, 2001; MROM,
Grainger & Jacobs, 1996; MROM-p, Jacobs, Rey, Ziegler, & Grainger,
1998). Further, Activation-Verification models incorporate serial as
well as parallel aspects, proposing an initial phase of parallel ac-
tivation of a set of candidates, which are subsequently compared
with the input in a serial manner
(
e.g., Becker, 1980; Schvaneveldt
& McDonald, 1981; Paap, Newsome, McDonald, & Schvaneveldt,
1982). As another case, the family of Connectionist Models expresses
lexical information as activation patterns that are distributed across
neuron-like units in a network, rather than as local representations
in a mental lexicon
(
e.g., Plaut, McClelland, Seidenberg, & Pat-
terson, 1996; Seidenberg & McClelland, 1989; Zorzi, Houghton,
& Butterworth, 1998). Especially interesting examples are Simple
Recurrent Networks implementing context-derived predictions as a
core mechanism in word processing (e.g., Elman, 1990, 2004).
Importantly, while some psycholinguistic models are rather de-
tached from neuroscience, recent approaches increasingly take
into account biological constraints and therefore follow calls for
physiological plausibility
(
e.g., Barber & Kutas, 2007; Dehaene,
Cohen, Sigman, & Vinckier, 2005; Grainger, 2008; Jacobs & Carr,
1995). As one case, some theories have incorporated the trans-
mission of visual information from the left and right hemifields
to contralateral hemispheres (e.g., SERIOL, Whitney, 2001; Split
Model, Shillcock, Ellison, & Monaghan, 2000). For instance, in
the SERIOL model, location gradients within the hemispheres en-
code the ordinal position of letters, which are serially processed
in oscillatory cycles
(
Whitney, 2008; Whitney & Cornelissen, 2008).
15
1. Introduction
As another case, models of word recognition take into account
temporal constraints of the availability of different information
types for stimulus processing. As a remarkable example, evidence
from various ERP studies yielded a timeline of orthographic and
phonological processing in the Bi-Modal Interactive Activation Model
(
Grainger & Holcomb, 2009, 2008). Finally, patterns of neural activa-
tion can serve as a touchstone for model assumptions. For instance,
MROM-based estimates of different levels of lexical activity in the
mental lexicon compatibly entailed graded neural responses in
ERPs (Braun et al., 2006).
In general, models of word recognition essentially contribute
to the understanding of language processing, since explicit as-
sumptions about mechanisms and determinants allow testable
predictions. In particular, the joint consideration of models and
neurophysiological data holds the potential to provide important
insights about language-related mental operations. Thereby, mod-
els will prospectively also be judged in terms of their anatomical
and functional plausibility.
Despite the large variety of models, the present work primarily
focused on the comparison of the serial Bin Theory
(
Forster, 1976,
1992; Murray & Forster, 2004) and the parallel IAM
(
McClelland &
Rumelhart, 1981; Rumelhart & McClelland, 1982) for two reasons.
First, the opposing implementations of bottom-up and top-down
processes permit strong and unambiguous predictions about the
time course of word recognition. Second, the core assumptions
of the two frameworks represent prototypes that have influenced
several other approaches (see above). Therefore, evidence about
the role of bottom-up and top-down information serves not only
as a validation of these two models, but generalizes as a proof of
concept to numerous other accounts.
1.3.3 Predictability and the anticipation of
upcoming events
While to date there is no conclusive agreement about the temporal
role of top-down influences in word recognition, recent findings
16
1.3. Towards a timeline of bottom-up and top-down processes