Content uploaded by Leen Catrysse
Author content
All content in this area was uploaded by Leen Catrysse on Jan 27, 2016
Content may be subject to copyright.
Frontline Learning Research Vol.4 No. 1 (2016) 1-16
ISSN 2295-3159
Mapping processing strategies in learning from expository text: an
exploratory eye tracking study followed by a cued recall
Catrysse Leena1, Gijbels Davida, Donche Vincenta, De Maeyer Svena,
Van den Bossche Pieta, Gommers Lucib
a University of Antwerp, Belgium
b University of St. Gallen, Switzerland
Article received 22 July / revised 12 December / accepted 12 December / available online 27 January
Abstract
This study starts from the observation that current empirical research on students’
processing strategies in higher education has mainly focused on the use of self-report
instruments to measure students’ general preferences towards processing strategies. In
contrast, there is a rather limited use of more direct and online observation techniques to
uncover differences in processing strategies at a task specific level. We based our study
on one of the most influential studies in the domain of Students’ Approaches to Learning
(SAL) (Marton, Dahlgren, Säljö, & Svensson, 1975). In our exploratory experiment we
used eye tracking followed by a cued recall to investigate how students use processing
strategies in learning from expository text. Nineteen university students participated in
the experiment. Results suggested that students in the deep condition did not look longer
at the essentials in the text compared with students in the surface condition, but that they
processed them in a more deep way. In our sample, students in the surface condition
looked longer at facts and details and also reported repeating these facts and details
more often. We suggest that the combination of eye tracking followed by a cued recall is a
promising tool to investigate students’ processing strategies since not all differences in
processing strategies are reflected in overt eye movement behaviour. The current
methodology allows researchers in the domain of SAL to complement and extend the
present knowledge base that has accumulated through years of research with self-report
questionnaires and interviews on students’ general preferences towards processing
strategies.
Keywords: Processing strategies; Expository text; Eye tracking; Cued recall; Higher
education
1 Corresponding author: Catrysse Leen, Faculty of Social Sciences, Department of Training and Education Sciences,
Research Group EduBROn. Gratiekapelstraat 10, 2000 Antwerpen, Belgium. E-mail: leen.catrysse@uantwerpen.be
DOI: http://dx.doi.org/10.14786/flr.v4i1.192
Catrysse(et(al(
(
(
2 | FLR
1. Introduction
Learning from text is one of the most essential skills in our modern society and the ability to
understand challenging texts is an important key to success in education and beyond (Mason, Tornatora, &
Pluchino, 2013; McNamara, 2004; Moss, Schunn, Schneider, McNamara, & Vanlehn, 2011). One of the
research traditions that is interested in how students learn from text is the domain of Student Approaches to
Learning (SAL) (Gijbels, Donche, Richardson, & Vermunt, 2014; Lonka, Olkinuora, & Mäkinen, 2004;
Richardson, 2000). Research in the SAL domain is founded on the seminal studies by Marton and his
colleagues in the 1970s in Sweden (Marton et al., 1975). They investigated how students went about reading
academic texts in experimental situations by conducting retrospective interviews (Marton et al., 1975;
Richardson, 2000). A distinction was made between deep processing strategies and surface processing
strategies, which has been influential in the later development of self-report questionnaires to quantify
individual differences in students’ processing strategies (Biggs, 1987; Entwistle & McCune, 2004).
Up till now, empirical studies in the SAL field have mainly been focused on the use of self-report
instruments such as interviews and questionnaires to uncover differences in students’ general preferences
towards processing strategies. Although these offline measures are claimed to be reliable and valid at this
general level, many authors argue that the results are poor indicators of the actual processing at a task
specific level (Perry & Winne, 2006; Samuelstuen & Braten, 2007; Veenman, 2005; Veenman, Bavelaar, De
Wolf, & Van Haaren, 2014). Recently, there has been a plea for the use of more direct and online
measurement tools when it comes to describe students’ processing strategies (Richardson, 2013). In the
present study we will therefore use eye tracking to map individual differences in cognitive processing
followed by a cued recall. Eye tracking provides a unique opportunity to study processing strategies in a
level of detail that no other measures can provide (Lai et al., 2013; van Gog & Jarodzka, 2013). In what
follows we will describe how different processing strategies can be manipulated in experimental designs by
the assessment demands, and how eye tracking followed by a cued recall can be useful to investigate
differences in processing strategies.
2. Uncovering differences in processing strategies
Processing strategies refer to cognitive activities a student applies whilst studying (Vermunt &
Vermetten, 2004). In general, two main types of processing strategies are described in the literature namely
deep and surface processing strategies (Gijbels et al., 2014). Research in the SAL domain showed that deep
processors try to comprehend what the author wants to say about a certain topic, try to understand the overall
meaning of the text, try to relate the message to a wider context and to prior knowledge, identify the main
ideas and adopt a critical angle to the conclusion. In contrast, surface processors direct their attention
towards learning the text itself, focus more on specific comparisons, focus on the parts of the text in
sequence, memorize details and definitions, remember introductory sentences and list points (Biggs & Tang,
2007; Entwistle & Ramsden, 1982; Marton et al., 1975; Richardson, 2000).
2.1. Processing strategies and task demands
In the 1960s, Rothkopf (1966) introduced the concept of mathemagenic activities, which refers to
activities that stimulate students to actively engage in learning. The use of adjunct questions in written texts
is one example of these mathemagenic activities. One possible type of an adjunct question is the inserted
post question, which is placed within the text and follows the text passage containing the information
needed. These questions result in a change in the processing strategy on subsequent text passages. They steer
students attention to a specific type of information in the text (Hamaker, 1986; Rothkopf, 1966).
Catrysse(et(al(
(
(
3 | FLR
Similarly, researchers in the SAL domain agree that one of the most salient contextual variables to
influence processing strategies is the assessment method (Baeten, Kyndt, Struyven, & Dochy, 2010; Gielen,
Dochy, & Dierick, 2003; Marton et al., 1975; Scouller, 1998; Scouller & Prosser, 1994; Segers, Nijhuis, &
Gijselaers, 2006). Research showed that how students learn is influenced by their initial preference for a
processing strategy (Baeten et al., 2010), but they can shift between deep and surface processing strategies
according to the assessment demands, also known as the backwash-effect of assessment (Baeten et al., 2010;
Gielen et al., 2003; Segers et al., 2006). In contrast to adjunct question research (Hamaker, 1986; Rothkopf,
1966), research in the SAL domain evaluated the effect of the assessment method at the end of a text or
study process, without inserting questions in the text or interrupting the study process.
In the experiments of Marton et al. (1975), students were asked to read three texts and to prepare for
answering some questions on the content after reading them. The questions they received after the first two
texts were the only indication on how to behave during reading the third text. Students in the deep condition
received questions at a deep level (e.g., making a summary statement), while students in the surface
condition received reproduction-oriented questions. After studying the third text, a semi-structured interview
was conducted to gather data on the effect of the experimental manipulation on the levels of processing. The
results of the interviews suggested that students tended to adapt the intended level of processing (Marton et
al., 1975; Richardson, 2000). This study was the first study in the SAL domain to confirm the possibility to
manipulate students’ levels of processing by appropriate questions or prompts. It shows that the level of
processing depends on the expected form of assessment (Richardson, 2000). Another study of Scouller and
Prosser (1994) suggested that the assessment method influences processing strategies. Their research showed
that multiple-choice questions led to more surface processing strategies. Also research of Scouller (1998)
investigated how students perceived two assessment methods namely multiple-choice examination and an
assignment essay and which processing strategies they used. The findings were in line with Scouller and
Prosser (1994), multiple-choice examination was perceived as assessing lower levels of intellectual abilities
and students indicated to engage in more surface processing strategies. An assignment essay was perceived
as testing higher-level intellectual abilities and students engaged in more deep processing strategies. A last
study of Segers et al. (2006) showed that students who perceive the demands on a deep level, to demonstrate
a thorough understanding and integration of knowledge, are more likely to employ deep processing
strategies. In contrast, students who perceive the demands of assessment on a surface level, to acquire
passive acquisition and reproduction of details, are expected to employ more surface processing strategies
such as rote learning and concentrating on facts and details.
2.2. Processing strategies and eye tracking
Online measures to map cognitive processing strategies include the think aloud method, observation
of behaviour and eye movement measurement (Schellings, 2011; Veenman, 2011). The think aloud method
provides a rich source of data, but it is intrusive and can alter the processing itself (Ericsson & Simon, 1993;
Veenman, 2005). The main limitation of the observation of behaviour is that it cannot detect covert cognitive
processes (Veenman, 2005). According to Hyönä and Lorch (2004) eye tracking is an attractive method for
studying cognitive processing strategies in comparison with other online measures because eye tracking
collects several indices of processing simultaneously and does not disrupt normal processing.
There are two theoretical assumptions that make the relation between eye movement and cognitive
processing clear: the immediacy assumption and the eye-mind assumption (Just & Carpenter, 1980). The
immediacy assumption states that information processing is not postponed and takes place when the
information is encountered. The eye-mind hypothesis explains that eye movements are closely linked to the
focus of attention as students process the information in the text. Therefore, eye movements can be used to
trace cognitive processing when learning from text (Hyönä, Lorch, & Rinck, 2003; Just & Carpenter, 1980).
In eye tracking research, the movement of the eyeball is recorded and these movements are related to a
stimulus. This allows us to investigate to what parts of the text a student allocates visual attention and for
how long (Holmqvist et al., 2011; van Gog & Jarodzka, 2013). A distinction is made between two main
Catrysse(et(al(
(
(
4 | FLR
measures namely fixations and saccades. During fixations the eye is almost completely still and information
can be extracted from the text. In contrast, during saccades the focus of visual attention is moved to another
location and the eye is rapidly moving between fixations, as a result students are not able to extract
information from text during saccades (Holmqvist et al., 2011; Lai et al., 2013; van Gog & Jarodzka, 2013).
Although eye tracking methodology seems a promising tool to investigate students’ processing
strategies, we could not find studies that examine eye movement behaviour that results from using different
cognitive processing strategies such as deep and surface processing. In another related research field, namely
research in reading comprehension, they already adopted the eye tracking methodology (Hyönä, Lorch, &
Kaakinen, 2002; Ponce & Mayer, 2014; Rayner, 1998). More specifically, the perspective driven text
comprehension framework states that the allocation of visual attention is influenced by the reading
perspective and this reading perspective shapes the cognitive processing in learning from text (Kaakinen &
Hyönä, 2005, 2007, 2010; Kaakinen, Hyönä, & Keenan, 2002). A reading perspective refers to the mental
frame from which the reader approaches a text and this perspective makes parts of the text more important to
the reader than others (Hyönä et al., 2003; Kaakinen & Hyönä, 2007). Kaakinen and Hyönä (2007) gave the
example that when you read a travel guide in order to find information about a specific country (e.g.,
Finland), you will approach the text with a specific reading perspective. This reading perspective is thus
content related. Alternatively, processing strategies correspond to the different aspects of the learning
material on which the learner focuses (Richardson, 2000). So students with different processing strategies
focus on the same content but search for other types of information (e.g., facts and details vs. essences)
(Schellings, van Hout-Wolters, & Vermunt, 1996). Research that investigates the influence of reading
perspective on eye movements showed that there is more time spent on relevant words or facts in the text
than on irrelevant words or facts (Kaakinen & Hyönä, 2007; Kaakinen et al., 2002). Next to that, relevant
words attracted more refixations than irrelevant words (Kaakinen & Hyönä, 2007). Research of Kaakinen
and Hyönä (2005) indicated that the extra time spent on relevant information is used to rehearse this
information in order to encode it to memory. Particularly relevant for research on learning from text is that
these refixations reflect purposeful and effortful strategic eye behaviour (Ariasi & Mason, 2010).
Eye tracking is an interesting method to investigate cognitive processes, but to reduce the amount of
inferences required by the researcher, eye movement data should be combined with other data such as verbal
reports (Hyönä, 2010; van Gog & Jarodzka, 2013). Recent studies have already applied the think aloud
method to obtain verbal reports on students’ processing strategies during reading and learning from text
(Dinsmore & Alexander, 2012, 2015). Concurrent reporting while learning from text can affect the eye
movement patterns, and therefore cued retrospective reporting offers a valuable alternative in combination
with eye tracking (van Gog & Jarodzka, 2013). Besides recording the eye movement, the eye tracking
software allows replaying the records of eye movements. Using this eye movement pattern as a memory cue,
it may help learners to recover how they encoded and interpreted elements in the text (Hyönä, 2010;
Penttinen, Anto, & Mikkilä-Erdmann, 2012; van Gog, Paas, & Van Merrienboer, 2005). Because of the
small delay after processing the text and the presentation of the memory cue, students are still able to report
on their cognitive processes (Veenman, 2005, 2011). For this reason we chose to use cued retrospective
reporting to triangulate with eye movement measures.
3. Present study
Our study aims to extend current research on processing strategies by using eye tracking
methodology followed by a cued recall to map differences in processing strategies. This more direct and
online way of measuring processing strategies allows to learn more about the actual processing behaviour of
students while learning from expository text.
Catrysse(et(al(
(
(
5 | FLR
As stated above processing strategies shape what information is looked for in a text and what
information is perceived as relevant (Kaakinen & Hyönä, 2005). Next to that, research using self-report
measures suggested that deep processors focus more on essences and surface processors focus more on
details and definitions (Biggs & Tang, 2007; Entwistle & Ramsden, 1982; Marton et al., 1975; Richardson,
2000). Based on findings from research on perspective driven text comprehension (Kaakinen & Hyönä,
2008) and the SAL domain (Lonka et al., 2004), we suggest the following hypotheses for students in the
deep condition (after receiving guiding questions at a deep level) and students in the surface condition (after
receiving reproduction-oriented questions):
a) Hypothesis 1: Students in the deep condition focus their attention longer on the essentials (e.g., key
phrases and words) in the text compared to students in the surface condition.
b) Hypothesis 2: Students in the deep condition, more often return back to essences compared to
students in the surface condition.
c) Hypothesis 3: Students in the surface condition focus their attention longer on facts and details (e.g.,
names) compared to students in the deep condition.
d) Hypothesis 4: Students in the surface condition, more often return back to facts and details compared
to students in the deep condition.
4. Method
4.1. Participants
Twenty-eight students (age range: 18-25) enrolled at the University of Antwerp (Belgium),
participated on a voluntary basis. Participants were randomly divided in either the deep condition (DC, N =
14) or the surface condition (SC, N = 14). Unfortunately, data of nine respondents could not be used due
equipment failure and problems with eye tracking calibration. Therefore, data of 19 students were considered
in the statistical analyses (Table 2). All participants had normal or corrected-to-normal vision and Dutch was
their native language.
Table 1
Participant characteristics
DC
SC
N
12
7
Gender
Male
5
5
Female
7
2
4.2. Materials
In order to test our hypotheses, we based our experimental design on the seminal studies by Marton
et al. (1975). In their experiments they induced either a deep or surface processing strategy by giving
students questions after they studied an academic text. In our experiment, students were asked to study a
series of three expository texts (± 800 words) on a topic they were not familiar with, namely research on
happiness. The texts were taken from the Dutch version of ‘The world book of Happiness’ (Bormans, 2010).
Catrysse(et(al(
(
(
6 | FLR
After processing each text they received a number of evaluation questions on the preceding text (Figure 1).
Students in the deep condition received questions at a deep level (e.g., give a summary of the text). In
contrast, students in the surface condition received reproduction-oriented questions (e.g., in which country
was the research discussed in the text conducted?). So in both conditions students processed the same
learning content, but received different questions. In the original study, Marton et al. (1975) interviewed and
tested the students after the third text and concluded that in the surface condition, students adopted more
surface processing strategies while students in the deep condition adopted more deep processing strategies.
Similarly, in our study we analysed the eye tracking data and cued recalls from the third text.
Figure 1. Experimental design.
4.3. Eye tracking
Eye movements were collected using the Tobii TX300 eye tracker (dark pupil tracking),
manufactured by Tobii Technology (Stockholm, Sweden). It is integrated into a 23-inch TFT monitor with a
maximum resolution of 1920 x 1080 pixels. The camera samples data at the rate of 300 Hz and registration
was binocular. Tobii TX300 does not require a head stabilization system and allows for more freedom of
head movement (37 x 17 cm). Gaze accuracy is 0.4° and gaze precision is 0.15°, as reported by the hardware
producer. The eye tracker latency is between 1.0 and 3.3 milliseconds. Data were recorded with Tobii-Studio
(3.2) software. Before starting the experiment, students were seated about 60 cm from the screen for the eye
tracking calibration. A five point calibration procedure was used in which students needed to track five red
calibration dots on a plain, grey background.
Areas of interest (AOI’s) define regions in the text that the researcher is interested in gathering data
about (Holmqvist et al., 2011). With regard to our hypotheses we are interested in key phrases and keywords
for the deep condition and in details and facts for the surface condition. Six volunteers (master students in
educational sciences) read the text in a pilot study to determine the key phrases and keywords. In total 15
deep AOI’s (e.g., a topic sentence with summary statements) and three surface AOI’s (e.g., name of a
country) were marked. There were only parts of the text defined as AOI’s, so not the whole text was covered
with AOI’s. The total size of the text was 1490 x 1087 pixels, the smallest AOI was 47 x 31 pixels and the
biggest AOI was 684 x 71 pixels. The complete text could be seen on the screen, so scrolling was not
needed.
In line with Hyönä et al. (2002) first pass fixation time, look back fixation time and total fixation
time were analysed at the level of AOI’s. An overview of the definitions is given in Table 2 (Holmqvist et
al., 2011; Hyönä et al., 2003). Students were able to process the text in a self-paced manner and therefore we
calculated relative duration measures. Next to that, AOI’s differed in size because they sometimes contained
phrases, while others consisted of only words. Therefore, AOI measures were normalized by calculating the
reading depth measure (Holmqvist et al., 2011; Holmqvist & Wartenberg, 2005; Holsanova, Holmqvist, &
Rahm, 2006). This reading depth measure is defined by the total time spent in an AOI per cm2 and is an
indication of how densely an AOI is processed. So for the three measures described in Table 2 we calculated
relative measures and reading depth measures.
Catrysse(et(al(
(
(
7 | FLR
Table 2
Overview of eye tracking measures and their definitions
Measure
Definition
First pass
fixation time
The time spent in an AOI when it was visited for the first time. A visit can consist of
more fixations. It reflects early processing and object recognition.
Look back
fixation time
Duration of all the regressions back to an AOI. It reflects delayed processing, for
example to integrate information.
Total fixation
time
The time spent in an AOI during the whole trial, it is the sum of the first pass fixation
time and the look back fixation time in that AOI.
The fixation indices were calculated for either the group of deep AOI’s or the group of surface
AOI’s. We used the Tobii fixation filter for fixation identification, which is an implementation of a
classification algorithm proposed by Olsson (2007). It uses a velocity threshold (35 pixels/window) and a
distance threshold (35 pixels). For all the measures, the means and standard deviations were calculated. To
compare students in both conditions, we used non-parametric tests due to the small sample sizes (van Gog et
al., 2005). Therefore the medians together with the first and third quartile were calculated as well. Relative
measures and reading depth measures for the eye movement measures were compared for students in both
conditions using Mann-Whitney U tests. We reported the exact two-tailed significance. Also in line with van
Gog et al. (2005), we used a less stringent significance level of 0.10 to avoid type II error and to increase
power.
4.4. Cued Recall
After the eye tracking experiment, a cued recall was conducted. After processing the third text, the
experimenter informed students that they would watch the replay of eye movements of the third text
together. The cued recall was conducted by using gaze videos produced by Tobii-Studio software (3.2). In
the cued recall, a video showed the text and a moving red dot representing the point of fixation. The bigger
the dot, the longer the fixation lasted. Students saw their gaze videos at the same speed they processed the
text. The interviewer instructed students to watch the video and to tell the interviewer what they were
thinking during processing the text. The interviewer also stated that she would occasionally stop the video
and ask questions about the reading process, such as ‘Here you fixated a lot, what where you doing?’ or
‘Here you are going back in the text, what were you doing?’.
Catrysse(et(al(
(
(
8 | FLR
Table 3
Coding scheme for the cued recall analysis
Strategy
Example
DC
SC
Surface processing
66
(65,3%)
45
(100%)
Rereading
I tried to understand that part so I was rereading it.
Skimming
Now I am reading it again and just scanning for important
words in the text.
Guessing meaning
word in context
That was GNP, I was wondering what the meaning of that
word was.
Rehearsing
Those countries, I was trying to remember them.
Connecting to prior
text
I realise that I go back a lot in the text and that is because I am
trying to link parts of the text.
Connecting to the
research task
I guess the first paragraph was going to give an overview
about the rest of the task, so I thought that was important.
Detecting mistakes
in the text
I was looking at the ‘n’ that was missing in that word.
Deep processing
35
(34,7%)
0 (0%)
Questioning
I was wondering what they meant with that phrase.
Paraphrasing
First, they name something and then you know a summation is
coming. Second, they talk about cross national comparisons,
…
Connecting to
personal experiences
You try to process the text critically and to take you own
findings and personal experience into account.
Interpreting and
elaborating
What I do most of the time is reading the text and then trying
to analyse what I just read. In this way I get a better picture of
what the text is about.
The cued recalls were transcribed from the audiotapes. Next to that, we linked comments of the cued
recalls to the part of the text that was discussed. The cued recalls were coded based on an initial set of ten
codes developed in a study of Dinsmore and Alexander (2015). Specifically, comments were coded as either
a surface or deep processing strategy (Table 3). After coding the interviews deductively, we added one extra
code in the surface processing category namely “detecting mistakes in the text”. Transcripts were coded with
the qualitative analysis software package Nvivo 10. Two judges (authors LC and LG) coded the cued recalls
and an inter-rater agreement of 73% was reached, which is considered as substantial. We compared the
number of coded utterances in each condition between the two categories (Table 3). We first analysed the
data on a general level and looked for differences between students in both conditions. We also analysed the
data at a more fine-grained level to see whether the reported strategies are linked to AOI’s and to examine
differences at the AOI level between groups.
5. Results
Table 4 shows the means and standard deviations. Standard deviations for the measures in the deep
condition are higher than in the surface condition. This may be an indication that students in the deep
condition differ more from each other. When we look at the cued recall results of students in the deep
Catrysse(et(al(
(
(
9 | FLR
condition, some students pointed out that they sometimes took a pause to integrate processed information
instead of looking back. This may also be an indication that there are two types of students in the deep
condition, on the one hand students who process information immediately and take a pause to integrate
information and on the other hand students who need to look back to parts in the text to integrate this
information and to encode it to memory.
“Sometimes I keep staring at the text, because I try to visualize it for myself” (R7, DC)
“I sometimes have the feeling that when I am staring at a word that I am not processing
that word but that I am just taking a moment to think about what I have read” (R3, DC)
“Sometimes I have the feeling that I am staring at something in the text, to process the
things I just read before” (R5, DC)
The most reported processing strategy in the cued recalls, is the surface processing strategy and more
specifically rereading. Students in both groups indicated that they reread parts of the text the most. Only
students in the deep condition reported deep and surface processing strategies. Students in the surface
condition only reported surface processing strategies. Deep processing strategies are reported on a more
general level and are not linked to certain phrases, paragraphs or AOI’s in the text.
“When you know you will need to answer questions after reading the text, you try to read
the text critically and I always try to take into account my personal experiences and
findings.” (R5, DC)
“I first think about what I read in the text, before I proceed with the next part. I try to
make a summary for myself of what I read in the previous parts.” (R3, DC)
Table 4
Means and standard deviations
Essentials
Facts and details
DC
SC
DC
SC
M
SD
M
SD
M
SD
M
SD
FPFT r
2.69
2.22
3.23
1.48
0.40
0.21
0.38
0.25
FPFT rd
45.53
23.77
57.50
25.19
80.60
24.49
79.02
51.30
LBFT r
12.65
3.63
11.22
2.69
1.09
0.49
1.94
0.77
LBFT rd
285.66
192.02
200.75
46.39
268.20
236.77
401.22
168.88
TFT r
15.35
3.01
14.46
2.85
1.50
0.57
2.32
0.84
TFT rd
331.19
189.72
285.25
43.45
348.80
234.62
480.24
185.68
FPFT = first pass fixation time; LBFT = look back fixation time; TFT = total fixation time; r =
relative measure; rd = reading depth measure.
We compared the total reading time of students in both conditions with a Mann-Whitney U test, but
no significant differences were found (U = 41, p = 0.97). So students in both conditions spent on average the
same amount of time on processing the text.
Catrysse(et(al(
(
(
10 | FLR
5.1. Essentials in the text
Table 5 shows the medians and quartiles for the essentials in the text for students in both conditions.
We conducted Mann-Whiney U tests on all these measures but no significant differences were found
between students in both groups.
Table 5
First quartile, median and third quartile for relative measures and reading depth measures.
DC
SC
Mann-Whiney U
Q1
Mdn
Q3
Q1
Mdn
Q3
U
p
FPFT r
1.53
1.92
2.86
2.13
2.57
4.43
27
0.227
FPFT rd
27.65
41.14
45.17
41.18
48.55
75.67
30
0.340
LBFT r
10.93
12.92
15.88
9.39
10.60
12.69
56
0.261
LBFT rd
173.39
215.14
354.49
170.57
178.85
215.59
53
0.385
TFT r
13.97
15.63
18.26
12.42
13.02
16.41
53
0.385
TFT rd
209.29
261.89
396.90
235.15
257.08
267.00
45
0.837
Results from the cued recalls indicate that both students in the deep and surface condition reread
essentials in the text. A reason for rereading is that they did not really understand essential parts of the text.
The motivation to better understand these essential parts in the text is only reported by students in the deep
condition. These results suggest that students in the deep condition reread these parts at a deeper level to get
a better understanding.
“I am rereading a lot, I read something fast and then I think whether I understood it and
no I did not, so then I go back again” (R3, DC)
“I was trying to understand that part better, so that is why I was rereading it over and
over again” (R2, DC)
Both groups indicated skimming the text after reading it for the first time to look back at
the essential parts of the text.
“What I often do when I finished reading, is rereading only the essential parts of the text”
(R4, DC)
“I am just scanning quickly to see if I missed important words in the text” (R16, SC)
A final finding from the cued recall results is that both groups guessed the meaning of keywords in
context, when they did not understand the word. Overall, cued recall results are in line with results from eye
tracking, in that no big differences are found between both groups when processing essential parts in the text.
“Here, that was a difficult word, elitist, I tried to understand the meaning in the text”
(R8, DC)
“Some keywords I do not know, I need to think about them or see the context to
understand them” (R19, SC)
Catrysse(et(al(
(
(
11 | FLR
5.2. Facts and details in the text
Table 6 shows the medians and quartiles for facts and details in the text for students in both
conditions. Students in the surface condition spent relatively more time on facts and details when they
looked back at them and also during the whole experiment. Next to that, these students read the facts and
details with more depth than students in deep condition when they look back at them and during the whole
experiment.
Table 6
First quartile, median and third quartile for relative measures and reading depth measures.
DC
SC
Mann-Whiney U
Q1
Mdn
Q3
Q1
Mdn
Q3
U
p
FPFT r
0.23
0.38
0.54
0.19
0.39
0.54
46
0.773
FPFT rd
66.34
78.06
102.04
38.77
82.39
112.36
44
0.902
LBFT r
0.74
1.04
1.41
1.74
2.28
2.41
17
0.036
LBFT rd
160.56
178.28
270.98
329.09
423.92
529.68
20
0.068
TFT r
1.01
1.47
1.95
2.23
2.62
2.86
15
0.022
TFT rd
241.65
276.45
351.52
440.90
479.24
611.01
18
0.045
Cued recall results showed that students in the surface condition repeated facts and details in the text,
while students in the deep condition did not. Other coding categories did not show a link with processing
facts and details in the text. Again we can see a clear link between the eye movement measures and the
results from the cued recalls.
“The names of those countries, I really tried to remember those” (R14, SC)
“Those four countries, I memorized them” (R19, SC)
“I tried to remember the name of the author, I thought that would be important” (R17,
SC)
6. Conclusion and discussion
This exploratory study aimed at extending current research on processing strategies during learning
from expository text. Research in the SAL domain is mostly based on students’ self-reports of processing
strategies at a general level in which the context of learning is not taken into account (Dinsmore &
Alexander, 2012; Gijbels et al., 2014). By looking at the actual processing behaviour of students while
learning from expository text, this study makes a first preliminary contribution to the field by using a more
direct and online measurement tool at a task specific level that takes the context explicitly into account. It is
the first experimental study to explore students’ cognitive processing strategies at a task specific level using
objective online measures. Most of the research using online measures is based on the think aloud method,
which can alter the processing itself (Veenman, 2005). By using eye tracking methodology followed by a
cued recall this problem is circumvented, in that this method does not demand students to manage cognitive
load of the task completion and self-reports of strategies at the same time (Samuelstuen & Braten, 2007).
Catrysse(et(al(
(
(
12 | FLR
In our study we manipulated the task demands to steer processing strategies. Results from the cued
recalls indicated that this manipulation was successful as students in the deep condition reported a
combination of surface and deep processing strategies, while students in the surface condition only reported
surface processing strategies. This is in line with previous research that showed that demands on a deep
level, to demonstrate a thorough understanding, lead to more deep processing strategies whereas demands on
a surface level, to acquire passive acquisition of facts and details, lead to more surface processing strategies
(Marton et al., 1975; Richardson, 2000; Scouller, 1998; Scouller & Prosser, 1994; Segers et al., 2006).
Results of the cued recalls indicated that students in both conditions processed facts and details and essential
parts in the text but they did it in a different way. These results are similar to results from think aloud studies
in which processing strategies were examined without manipulating task demands (Dinsmore & Alexander,
2012, 2015; Penttinen et al., 2012).
Based on the eye movement data, we cannot confirm the first and second hypothesis that stated that
students in the deep condition focus their attention longer on essentials in the text compared to students in
the surface condition and that they return more back to them. Both groups of students spent time on
processing the essentials in the text. Although we could not find differences between groups based on their
eye movement data, results from the cued recalls indicated that students in the deep condition reread the
essentials in the text to understand them better. This motivation to better understand these parts is related to a
deep way of processing (Biggs & Tang, 2007; Entwistle & Ramsden, 1982). Students in the surface
condition did not report this motivation. These descriptive findings indicate that students in our sample
processed the text in a different way but more substantive research is needed to further explore found
differences in overt eye movement behaviour. In contrast with research from the angle of perspective driven
text comprehension, these essential parts do not seem to be perceived as more relevant by students in the
deep condition (Kaakinen & Hyönä, 2005, 2007, 2008), they are just processed in a more deep way. Another
interesting finding from the cued recall results is that some students in the deep condition indicated that they
took a pause at some places in the text to integrate the processed information instead of actively looking
back. Other students in the deep condition reported actively looking back at these essential parts in the text.
Also the higher standard deviations for students in the deep condition may be an indication of these
differences. It is in line with other research that shows that building the necessary links to incorporate text
information to the developing memory representation can be achieved mentally or can result in overt
behaviour in which students actively reread essential parts (Hyönä et al., 2003; Kaakinen & Hyönä, 2008).
So, based on these preliminary findings, we suggest that some deep processors actively return back to
essentials to encode it to memory, while others take a pause to integrate the new information without looking
back to this information. Further research is needed to confirm these findings.
Regarding the third and fourth hypothesis, the results indicated that students in the surface condition
indeed looked longer at facts and details and returned more back to them. It seems that students in the
surface condition switch to strategic processing by paying more attention to relevant parts, namely facts and
details (Kaakinen & Hyönä, 2007). Research of Kaakinen and Hyönä (2005) showed that the extra time
spent on relevant information is used to rehearse this information in order to encode it to memory. Results
from both eye tracking and cued recalls indicate that facts and details are more repeated in order to encode
into memory in the surface condition (Kaakinen & Hyönä, 2007). Only students in the surface condition
reported repeating facts and details, while students in the deep condition did not report learning activities like
that.
Although our findings suggest that eye tracking followed by a cued recall is a fruitful way to
investigate processing strategies, we want to stress the preliminary nature of this study because of some
limitations. An important limitation of this study is the small sample size. Due to equipment failure or
problems with eye tracking calibration, the sample size decreased at the onset of this study. Because of this
smaller sample size we decided to use non-parametric tests and deepened the results obtained by a cued
recall. We also raised the significance level to increase power due to the smaller sample size (van Gog et al.,
2005). The findings from this study can serve as a baseline for further research in which larger samples can
be used to increase power without adjusting the significance level. Another limitation of this study is that we
Catrysse(et(al(
(
(
13 | FLR
used a between groups design. Reading times and online processing strategies can vary among adult readers
(Hyönä et al., 2002; Kaakinen & Hyönä, 2008). Therefore we suggest for further research to use a within
groups design in which students use both processing strategies to take this individual variability into account.
Another way to understand the significance of individual variability is to include control variables such as
reading ability, interest in the topic and prior knowledge about the topic (Fox, 2009; Mason et al., 2013). By
increasing the sample size and using a within subjects design, more complex statistical analysis can be
conducted to confirm our preliminary findings. In this way it will be possible to give more generalized
statements regarding processing strategies as measured by eye tracking. A last limitation is that students
needed to process the text on a computer screen to be able to use the eye tracking. By doing this it does not
reflect the natural setting in which students habitually process learning contents.
Despite the limitations, this study was able to show that eye tracking followed by a cued recall is a
promising tool to examine students’ processing strategies. An important finding from our study is that it is
valuable to combine eye tracking with a cued recall, because differences in processing strategies not always
lead to overt eye movement behaviour (Hyönä et al., 2003; Kaakinen & Hyönä, 2008). By using a cued
recall we were able to uncover differences in processing strategies that were not reflected in eye movement
behaviour. Based on our preliminary findings, the combination of eye tracking and a cued recall seems to be
a promising tool to further investigate cognitive processing strategies when learning from text. Students in
the deep condition do not seem to look longer at essentials and do not seem to return more back to them, but
processed them in a more deep way then students in the surface condition. Results suggest that students in
the surface condition looked longer at facts and details and did return more back to them. This first
exploratory eye tracking study in the SAL domain is an important illustration on how processing strategies
can be further examined beyond the use of self-report questionnaires. In our opinion it would be worthwhile
to use this innovative eye tracking methodology in multi-method designs to triangulate it with often used
self-report measures to look for convergent or divergent validity. In our study we steered students’
processing strategies by task demands. Although research indicated that it is possible to influence processing
strategies by manipulating this contextual variable (Baeten et al., 2010; Gielen et al., 2003; Marton et al.,
1975; Scouller, 1998; Scouller & Prosser, 1994; Segers et al., 2006), it would be interesting to combine it
with these self-report measures in order to examine a more natural way of processing behaviour. Next to
that, using multiple sources of data is important to develop a comprehensive understanding of how we can
adequately measure students’ processing strategies. Eye tracking methodology followed by a cued recall in
the SAL domain can also deepen the conceptual underpinnings on what constitutes deep and surface
processing of learning contents.
Keypoints
Eye tracking followed by a cued recall is a promising tool to uncover differences in students’
processing strategies while learning from expository text.
Students in the deep condition do not look longer at the essentials, but they process them in a
more deep way by trying to understand these parts better.
Students in the surface condition look longer at facts and details and try to rehearse these parts.
References
Ariasi, N., & Mason, L. (2010). Uncovering the effect of text structure in learning from a science text: An
eye-tracking study. Instructional Science, 39(5), 581-601. doi: 10.1007/s11251-010-9142-5
Baeten, M., Kyndt, E., Struyven, K., & Dochy, F. (2010). Using student-centred learning environments to
stimulate deep approaches to learning: Factors encouraging or discouraging their effectiveness.
Educational Research Review, 5(3), 243-260. doi: 10.1016/J.Edurev.2010.06.001
Catrysse(et(al(
(
(
14 | FLR
Biggs, J. (1987). Student Approaches to Learning and Studyin. Research Monograph. Melbourne: Australian
Council for Educational Research.
Biggs, J., & Tang, C. (2007). Teaching for Quality Learning at University: Open University Press /
McGraw-Hill Education.
Bormans, L. (2010). Geluk. The world book of Happiness. Tielt: Lannoo.
Dinsmore, D. L., & Alexander, P. A. (2012). A Critical Discussion of Deep and Surface Processing: What It
Means, How It Is Measured, the Role of Context, and Model Specification. Educational Psychology
Review, 24(4), 499-567. doi: 10.1007/s10648-012-9198-7
Dinsmore, D. L., & Alexander, P. A. (2015). A Multidimensional Investigation of Deep-level and Surface-
level Processing. The Journal of Experimental Education, 1-32. doi:
10.1080/00220973.2014.979126
Entwistle, N., & McCune, V. (2004). The conceptual bases of study strategy inventories. Educational
Psychology Review, 16(4), 325-345. doi: 10.1007/S1064800400030
Entwistle, N., & Ramsden, P. (1982). Understanding student learning. New York: Nichols Publishing
Company.
Ericsson, K. A., & Simon, H. A. (1993). Protocol Analysis. Verbal Reports as Data. Massachusetts:
Massachusetts Institute of Technology.
Fox, E. (2009). The Role of Reader Characteristics in Processing and Learning From Informational Text.
Review of Educational Research, 79(1), 197-261. doi: 10.3102/0034654308324654
Gielen, S., Dochy, F., & Dierick, S. (2003). Evaluating the consequential validity of new modes of
assessment: The influence of assessment on learning, including pre-, post- and true assessment
effects. . In M. Segers, F. Dochy & E. Cascallar (Eds.), Optimising new modes of assessment: In
search of qualities and standards (pp. 37-54). The Netherlands: Kluwer Academic Publishers.
Gijbels, D., Donche, V., Richardson, J. T. E., & Vermunt, J. D. (Eds.). (2014). Learning patterns in Higher
Education. Dimensions and research perspectives. . London: Routledge.
Hamaker, C. (1986). The effects of adjunct questions on prose learning. Review of Educational Research,
56(2), 212-242. doi: 10.3102/00346543056002212
Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., & van de Weijer, J. (2011). Eye
tracking : a comprehensive guide to methods and measures. Oxford ; New York: Oxford University
Press.
Holmqvist, K., & Wartenberg, C. (2005). The role of local design factors for newspaper reading behaviour -
an eye-tracking perspective. Lund University Cognitive Studies (Vol. 127). Lund University.
Holsanova, J., Holmqvist, K., & Rahm, H. (2006). Entry points and reading paths on newspaper spreads:
comparing a semiotic analysis with eye-tracking measurements. Visual communication, 5(1), 65-93.
doi: 10.1177/1470357206061005
Hyönä, J. (2010). The use of eye movements in the study of multimedia learning. Learning and Instruction,
20(2), 172-176. doi: 10.1016/j.learninstruc.2009.02.013
Hyönä, J., & Lorch, R. F. (2004). Effects of topic headings on text processing: evidence from adult readers’
eye fixation patterns. Learning and Instruction, 14(2), 131-152. doi:
10.1016/j.learninstruc.2004.01.001
Hyönä, J., Lorch, R. F., & Kaakinen, J. K. (2002). Individual differences in reading to summarize expository
text: Evidence from eye fixation patterns. Journal of Educational Psychology, 94(1), 44-55. doi:
10.1037//0022-0663.94.1.44
Hyönä, J., Lorch, R. F., & Rinck, M. (2003). Eye Movement Measures to Study Global Text Processing. In
J. Hyönä, R. Radach & H. Deubel (Eds.), The mind's eye: cognitive and applied aspects of eye
movement research. Amsterdam: Elsevier Science.
Just, M. A., & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension.
Pyschological Review, 87(4), 329-354. doi: 10.1037/0033-295X.87.4.329
Kaakinen, J. K., & Hyönä, J. (2005). Perspective Effects on Expository Text Comprehension: Evidence
From Think-Aloud Protocols, Eyetracking, and Recall. Discourse processes, 40(3), 239-257. doi:
10.1207/s15326950dp4003_4
Catrysse(et(al(
(
(
15 | FLR
Kaakinen, J. K., & Hyönä, J. (2007). Perspective effects in repeated reading: An eye movement study.
Memory & Cognition, 35(6), 1323-1336. doi: 10.3758/BF03193604
Kaakinen, J. K., & Hyönä, J. (2008). Perspective-Driven Text Comprehension. Applied Cognitive
Psychology, 22, 319-334. doi: 10.1002/acp.1412
Kaakinen, J. K., & Hyönä, J. (2010). Task effects on eye movements during reading. Journal of
Experimental Psychology: Learning, Memory and Cognition, 36(6), 1561-1566. doi:
10.1037/a0020693
Kaakinen, J. K., Hyönä, J., & Keenan, J. M. (2002). Perspective Effects on Online Text Processing.
Discourse processes, 33(2), 159-173. doi: 10.1207/S15326950DP3302_03
Lai, M.-L., Tsai, M.-J., Yang, F.-Y., Hsu, C.-Y., Liu, T.-C., Lee, S. W.-Y., . . . Tsai, C.-C. (2013). A review
of using eye-tracking technology in exploring learning from 2000 to 2012. Educational Research
Review, 10, 90-115. doi: 10.1016/j.edurev.2013.10.001
Lonka, K., Olkinuora, E., & Mäkinen, J. (2004). Aspects and Prospects of Measuring Studying and Learning
in Higher Education. Educational Psychology Review, 16(4), 301-323. doi: 10.1007/s10648–004–
0002–1
Marton, F., Dahlgren, L. O., Säljö, R., & Svensson, L. (1975). The Göteborg project on non-verbatim
learning. Göteborg: University of Göteborg.
Mason, L., Tornatora, M. C., & Pluchino, P. (2013). Do fourth graders integrate text and picture in
processing and learning from an illustrated science text? Evidence from eye-movement patterns.
Computers & Education, 60(1), 95-109. doi: 10.1016/j.compedu.2012.07.011
McNamara, D. S. (2004). SERT: Self-Explanation Reading Training. Discourse processes, 38(1), 1-30.
Moss, J., Schunn, C. D., Schneider, W., McNamara, D. S., & Vanlehn, K. (2011). The neural correlates of
strategic reading comprehension: cognitive control and discourse comprehension. Neuroimage,
58(2), 675-686. doi: 10.1016/j.neuroimage.2011.06.034
Olsson, P. (2007). Real-time and offline filters for eye tracking. KTH Royal Institute of Technology.
Penttinen, M., Anto, E., & Mikkilä-Erdmann, M. (2012). Conceptual Change, Text Comprehension and Eye
Movements During Reading. Research in Science Education, 43(4), 1407-1434. doi:
10.1007/s11165-012-9313-2
Perry, N. E., & Winne, P. H. (2006). Learning from learning Kits: gStudy Traces of Students' Self-Regulated
Engagements with Computerized Content. Educational Psychological Review, 18, 211-228. doi:
10.1007/s10648-006-9014-3
Ponce, H. R., & Mayer, R. E. (2014). An eye movement analysis of highlighting and graphic organizer study
aids for learning from expository text. Computers in Human Behavior, 41, 21-32. doi:
10.1016/j.chb.2014.09.010
Rayner, K. (1998). Eye Movements in Reading and Information Processing: 20 Years of Research.
Psychological Bulletin, 124(3), 372-422. doi: 10.1037//0033-2909.124.3.372
Richardson, J. T. E. (2000). Researching Student Learning. Buckingham: Open University Press and SRHE.
Richardson, J. T. E. (2013). Research issues in evaluating learning pattern development in higher education.
Studies in Educational Evaluation, 39(1), 66-70. doi: 10.1016/j.stueduc.2012.11.003
Rothkopf, E. Z. (1966). Learning from written instructive materials: An exploration of the control of
inspection behavior by test-like events. American Educational Research Journal, 3, 241-249. doi:
10.3102/00028312003004241
Samuelstuen, M. S., & Braten, I. (2007). Examining the validity of self-reports on scales measuring students'
strategic processing. Britisch Journal of Educactional Psychology, 77(Pt 2), 351-378. doi:
10.1348/000709906X106147
Schellings, G. L. M. (2011). Applying learning strategy questionnaires: problems and possibilities.
Metacognition and Learning, 6(2), 91-109. doi: 10.1007/s11409-011-9069-5
Schellings, G. L. M., van Hout-Wolters, B., & Vermunt, J. D. (1996). Individual Differences in Adapting to
Three Different Tasks of Selecting Information form Texts. Contemporary Educational Psychology,
21, 423-446. doi: 10.1006/ceps.1996.0029
Catrysse(et(al(
(
(
16 | FLR
Scouller, K. M. (1998). The influence of assessment method on students' learning approaches: Multiple
choice question examination versus assignment essay. Higher Education, 35, 453-472. doi:
10.1023/A:1003196224280
Scouller, K. M., & Prosser, M. (1994). Students' experiences in studying for multiple choice question
examinations. Studies in Higher Education, 19(3), 267-279. doi: 10.1080/03075079412331381870
Segers, M., Nijhuis, J., & Gijselaers, W. (2006). Redesigning a learning and assessment environment: the
influence on students' perceptions of assessment demands and their learning strategies. Studies in
Educational Evaluation, 32, 223-242. doi: 10.1016/j.stueduc.2006.08.004
van Gog, T., & Jarodzka, H. (2013). Eye Tracking as a Tool to Study and Enhance Cognitive and
Metacognitve Processes in Computer-Based Learning Environments. In R. Azevedo & V. A. W. M.
M. Aleven (Eds.), International Handbook of Metacognition and Learning Technologies. New York:
Springer.
van Gog, T., Paas, F., & Van Merrienboer, J. J. G. (2005). Uncovering expertise-related differences in
troubleshooting performance: combining eye movement and concurrent verbal protocol data.
Applied Cognitive Psychology, 19(2), 205-221. doi: 10.1002/acp.1112
Veenman, M. V. J. (2005). The assessment of Metacognitive Skills: What can be learned from multi-method
designs? In C. Artett & B. Moschner (Eds.), Lernstrategien und Metakognition. Implikationen für
Forschung und Praxis (pp. 77-99). Münster: Waxmann.
Veenman, M. V. J. (2011). Alternative assessment of strategy use with self-report instruments: a discussion.
Metacognition and Learning, 6(2), 205-211. doi: 10.1007/s11409-011-9080-x
Veenman, M. V. J., Bavelaar, L., De Wolf, L., & Van Haaren, M. G. P. (2014). The on-line assessment of
metacognitive skills in a computerized learning environment. Learning and Individual Differences,
29, 123-130. doi: 10.1016/j.lindif.2013.01.003
Vermunt, J. D., & Vermetten, Y. J. (2004). Patterns in student learning: Relationships between learning
strategies, conceptions of learning, and learning orientations. Educational Psychology Review, 16(4),
359-384. doi: 10.1007/S10648-004-0005-Y
