Content uploaded by James Dunn
Author content
All content in this area was uploaded by James Dunn on May 22, 2019
Content may be subject to copyright.
329
Feature Article
The Effects of Metacognition on
Reading Comprehension Tests in an
Intermediate Reading and Writing
Course
James D. Dunn
Tokai University
Reading comprehension skills are paramount to the future success of Japanese students.
Contextual clues and key word strategies help students take tests such as the TOEIC and
EIKEN, as well as develop critical thinking skills for use in their lives. e introduction of
reading comprehension strategies, in and of itself, helps students improve their scores, but
the question if students are fully utilizing these skills during a reading comprehension test
still remains. To determine whether students are better o with a “directed implementation”
versus a “non-directed implementation” of these skills is examined with sixty-four university
students of predominately Japanese ethnicity. e data clearly shows that students who are
involved in a metacognitive process while utilizing these skills score higher on a reading
comprehension test versus students that do not. ese results suggest that having students
actively utilize these skills, and showing examples of their metacognitive processes, positively
inuences test outcome.
英文読解力を高める事は日本の学生が、将来成功するために最も大切である。文
脈のヒントやキーワードを見極める力は、TOEICや英検といったテストにおいて学
生を助ける学習だけでなく、実生活において批判的思考力を伸ばすのに大いに
役立つ。また、テストの点数を上げるためには、それらの批判的思考力をふまえた
英語読解能力が必要である。しかしながら、学生のテストの点数が、手段の導入の
有無によってより有効かどうかを見極める必要がある。本研究結果より、英文読解
テストでメタ認知的アプローチをとった学生は、アプローチを行わなかった学生と比
べ、明らかに高い点数を記録したことが判明した。よって、これらのメタ認知的英文
読解プロセスを多く活用した方がテスト結果に良い影響があることが分かった。
330
Dunn
e landscape of learning English in Japan puts a great deal of weight on the
value of being able to score well on tests like the Test of English for International
Communication (TOEIC) and other, similar, standardized testing systems. All
of these tests have many types of assessments, but one common style of evaluation
among them is reading comprehension sections. ese sections oen represent a
substantial amount of the point total. One role of an educator, teaching reading
and writing in Japan, is to give students skills that will enable them to successfully
navigate these types of reading comprehension tests. Skills that help students
perform better on standardized tests are also applicable to other situations that
may arise in their future.
Background
Reading Strategies and Their Place in Japan
David Moore (2012) identies seven reading comprehension strategies that
eective educators encourage in their students. e seven strategies stated are
planning and monitoring, determining importance, asking questions, making
inferences, making connections, synthesizing, and visualizing. ese strategies
help students to improve their mastery of the reading process and aord gains
in reading comprehension and vocabulary acquisition (Short & Fitzsimmons,
2007).
It is widely agreed that expert reading is an interaction of four main areas: the
reader, the text, the uency of the reading, and the strategic aptitude of the reader
(Nuttal, 1996; Anderson, 1999, 2002; Rasinski, 2010). One major necessity
of reading comprehension is the realization that successful comprehension of
reading does not occur without concerted eort on the reader’s part (Weinstein
& Mayer, 1985; Weinstein & Underwood, 1985; Adams & Hamm, 1994;
Gettinger & Seibert, 2002; Baker, 2008). Furthermore, students must invoke the
strategies or risk losing the benets of the strategies taught to them (Alexander
& Jetton, 2000). is implies that simply teaching students the strategies to
employ when reading is not enough. Students must be made to employ strategies
in a clear and accountable way. One way to do this is to have the students show
their thought processes on the test paper. is practice is akin to a math teacher
331
e Eects of Metacognition, OnCUE Journal, 9(4), pages 329-343
who would require students to show their work on a math problem in order to
assist the formulas and algorithms becoming cemented in the student’s working
memory (Schoenfeld, 1992). Requiring English students to show their work
seems to be a logical extension of this teaching method.
In Japan, the Test of English for International Communication (TOEIC)
is one of the most popular exams for the evaluation of English communication
ability (Childs, 1995). e TOEIC tests English ability in grammar, reading,
and listening. Many Japanese companies use the TOEIC as a gauge for linguistic
competence for their employees—requiring their score either during the
company hiring process, or during their employment at a company for purposes
of procuring a position with the company overseas (Chapman, 2003; Powers,
Kim, & Weng, 2008). In addition to English reading skills being important to
nding employment, they are also essential for Japanese employees who deal
with correspondence from other countries. Higher TOEIC test scores can
inuence both the employability and future opportunities of students, as higher
scores can translate into better jobs and, according to a student of the author, in
some companies, bonuses for scoring over a certain threshold. All of these factors
have the possibility of inuencing pedagogy in the language learning classroom
that employs reading comprehension.
Within the TOEIC, there are three reading sections—incomplete sentences,
reading comprehension passages, and double reading questions—which require
answering questions about email correspondence (http:www.toeic.or.jp/).With
much of the TOEIC focused on reading comprehension, reading comprehension
skills could prove useful for students.
is need for reading comprehension skills, and the fact that the private
university at which this author teaches utilizes a TOEIC-style unied reading
comprehension test for both rst- and second-year students, led to the action
research described in this paper. Would skills and strategies be enough to help
students improve their reading comprehension scores, and would the students
need assistance in their metacognitive processes to fully utilize the strategies
introduced in class?
332
Dunn
Metacognition in Education
Metacognition, or “thinking about thinking” (Alexander & Jetton, 2000),
includes actively knowing about when, where, and how to implement strategies
for learning or problem solving, as well as engaging in the use of said strategies.
e “thinking about thinking” denition of metacogn ition is something of a gross
oversimplication. Metacognition can be thought of as two overlapping areas in
this regard: metacognitive knowledge and metacognitive processes (Dunlosky
& Metcalf, 2009; Jacobs & Paris, 1987; Nelson & Narens, 1990; Schwartz &
Bacon, 2008). Metacognitive knowledge can be broken down into three aspects
(Jacobs & Paris, 1987). e declarative aspect is what students know about their
ability to learn. e procedural aspect is what the students know about how
to implement strategies. e conditional aspect regulates the when and where
strategies are used. Metacognitive processes can be broken down into monitoring
and control (Nelson & Narens, 1990). e metacognitive process of monitoring
is represented by a student understanding where they are in a task or learning
situation and judging if the task is able to be completed. e metacognitive process
of control is in charge of making choices as to which strategy to implement and
when to terminate a task if it is deemed, by the monitoring process, as unable to
be completed.
ese ve factors are all in play in a student’s brain at the same time (Carroll,
2008) and are relevant to the student’s educational life. is action research set
out to determine if requiring students to perform metacognitive processes and
utilizing metacognitive knowledge during a test would result in better reading
comprehension scores. Since students need to consciously employ strategies in
order for them to be eective (Alexander & Jetton, 2000), metacognition on
the reading comprehension strategies taught before a test is required during the
taking of a test. While there have been studies on reading comprehension skills
and metacognition (Carrell, Pharis, & Liberto, 1989; Cubukcu, 2008; Takallou,
2011;Tregaskes & Daines, 1989), none of them, save for Cubukcu (2008) in a
dierent manner, have required proof of a student’s metacognitive processes on a
reading comprehension test.
While there are studies that suggest metacognition has a positive eect on
333
e Eects of Metacognition, OnCUE Journal, 9(4), pages 329-343
reading comprehension (Cubukcu, 2008; Takallou, 2011;Tregaskes & Daines,
1989), further studies needed to be conducted to gauge if explicit teaching of
metacognitive strategies inuences the use of reading comprehension strategies.
e underlying hypothesis of this paper is that students who are required to
show proof of their metacognitive processes, during the employment of reading
comprehension strategies will achieve higher test scores than students who are
not required to show their processes.
is study compared the reading comprehension test scores of students who
were required to show their metacognitive processes and those who were not
on a reading comprehension test. For the purposes of this study, metacognitive
processes refer to participants consciously thinking about the implementation
of the previously introduced reading strategies during a reading comprehension
test. e current pilot study was conducted in order to nd an answer to
the following question regarding the eect of students illustrating their
metacognitive processes on a reading comprehension test.
1. Is there any signicant dierence in reading comprehension test scores
between learners who are required to engage in metacognition by showing
their reading comprehension strategies and those who are not?
Methodology
Subjects
e participants in this pilot study consisted of 64 college students split between
a control group (n = 31) and a test group (n = 33) including 62 Japanese students,
one Chinese student, and one Korean student. All students were sophomores
at a private university in Japan, enrolled in the required Reading and Writing
Two (RW2) classes which had been streamed into the high-beginner to low-
intermediate levels of English as dened by the Common European Framework
of Reference for Languages (CFER). Each participant was a member of either the
science or engineering department, and all participants had completed six weeks
of reading comprehension strategy instruction focusing on contextual clues to
nd meaning, key word identication, making connections, and synthesizing
information from multiple areas of a reading passage on intermediate (CEFR
334
Dunn
A2 and B1 level) reading comprehension activities. e reading subjects that
the students were exposed to ranged from humorous stories and work emails to
general science topics and mechanical engineering. Each group attended class for
90 minutes twice a week, where they studied both reading comprehension and
process writing methods. Reading comprehension strategy instruction occurred
during one of their two classes a week, for the six-week period. As a result of reading
comprehension strategy instruction, all students were familiar with the reading
strategies prior to the taking of the test. e control group originally consisted of
33 members, but two participants did not take the reading comprehension test,
so the null scores were dropped for this pilot study.
Instructional Background
Of the seven reading comprehension strategies given by Moore (2012), this
study focuses on three—planning and monitoring, making connections, and
synthesizing—which were explicitly taught to the students in both the control
and test groups. ese three were chosen so as to give the students a broad range
of skills in a short amount of time that can assist in this aspect of a test like the
TOEIC. e planning and monitoring strateg y was chosen to give students a skill
to manage time during the test. Making connections and synthesizing strategy
was added to give students higher order thinking skills to assist in choosing
correct answers. Making connections as a reading comprehension skill is very
similar to the metacognitive strategy Cucukcu (2008) examined, i.e., searching
according to goals.
In both the control and test groups, reading comprehension test scores at
the start of the course were lower than expected. e control and test group
both scored an average of just over ten out of twenty-ve points on their rst
reading comprehension test, where the hope was that they would have an average
score that was in the passing range. is prompted the author to introduce
reading comprehension strategies into the classes to help bring students’ skills
up. From week four of the course, all participants were instructed in a variety of
reading comprehension strategies including, but not limited to the following:
developing a plan to read, identifying key words, identifying meaning from
335
e Eects of Metacognition, OnCUE Journal, 9(4), pages 329-343
context, making connections between the questions and reading passage, and
putting information from two or more locations in the text together to synthesis
new information. Making connections inside reading passages as well as between
the test questions and the reading passage is a skill that students can make use of
in their future professional lives. Similarly, being able to synthesize information
from multiple sections of a reading is an invaluable skill that translates beyond
the classroom into the students’ possible future employment areas. Planning and
monitoring was chosen to give students the ability to look at a text critically and
immediately start breaking down the questions and text to make sense of the
information presented.
Both groups were instructed on showing their metacognitive processes by
either drawing lines between the questions and reading passage or labeling the
area where the student felt the answer was with the numeral of the question
number. Since all of the participants were members of either the science or
engineering department, a small number of the reading exercises focused on
dierent aspects of science and engineering.
Reading Comprehension Test
e test of reading comprehension was taken from a reading comprehension
practice from the department of philosophy in the University of Venezia
(Gebhardt, 2012). is test was chosen for two primary reasons: one, it is freely
available and two, it contains reading passages of various topics and a variety of
question types. e test consisted of four reading passages of 100 to 150 words
each. e content of the four tests ranged widely with topics such as movies and
medicine. While the test was not designed for science and engineering students,
it did in a limited sense cover one science topic about medicine. e rst three
reading passages were paired with ve multiple-choice reading comprehension
questions each, and a fourth reading of 10 true/false questions. e maximum
score on the test was 25 points, and the time given to each group to complete
the test was 60 minutes. e students were not allowed to leave early if they
nished the test within the 60-minute window. It should be noted that this test
underwent no reliability testing and was used primarily for the two reasons stated
336
Dunn
above. Future research should employ standardized reading comprehension tests
that are more commonly recognized in the educational community.
Procedure
is study was conducted in week 10 of a 15-week course in a private Japanese
university. e control group (n = 31) and test group (n = 33) were both given
the same reading comprehension test. For the control group, the instruction was
to take the test normally. For the test group, in addition to taking the test and
checking their answers, they were required to show their thought processes by
connecting key words in the questions to the area in the reading passage where
they thought the answers were located. Out of four sets of reading comprehension
and questions, participants in the test group were required to show their thought
process on at least one. Students were not directed in how to show their thought
processes, but were encouraged to draw lines to utilize the making connections
strategy (Figure 1), as was practiced in the training sessions. Some participants in
the test group only showed where they found the answers by indicating the point
in the passage with a number that corresponded to the number of the question,
but this did demonstrate that they were making connections between the
questions and the locations of the answers in the text. All of the participants in
this study were required to answer all the questions to avoid articially lowering
the group’s mean score.
Figure 1. Example of drawing lines to utilize the making connections strategy.
337
e Eects of Metacognition, OnCUE Journal, 9(4), pages 329-343
Results
Data Analysis
An unpaired t-test was employed to analyze the two sets of test scores. A
statistical representation of the analyzed data is given in Table 1. Table 1
comprises of data that addresses the question— Is there any signicant dierence
in reading comprehension test scores between learners who are required to engage in
metacognition by showing their reading comprehension strategies and those who are
not?
As the data shows (Table 1), the test group displays a mean of 13.39 points
with a standard deviation of 1.97, and the control group displays a mean of
11.48 points with a standard deviation of 1.86. is data suggests that the two
groups diered signicantly in their average and mean test scores (t = 3.9855; p
= 0.0002).
Discussion
In previous reading quizzes, the dierence between two groups’ scores was not
statistically signicant (t = 1.7059; p = .2301) (Table 2), which is particularly
Table 1
Final Test Data Analysis
Mean SD n
Control Group 11.48 1.86 31
Test Group 13.39 1.97 33
p = 0.0002 d = 1.0
Table 2
Control vs. Test Score Comparison
Control Mean Test Mean
Quiz 1 10.40 10.64
Quiz 2 11.04 11.10
Study Test 11.48 13.39
338
Dunn
noteworthy when comparing them to the scores in this study. It bears noting that
the students in the test group were only required to show their metacognitive
processes on one of the four test items. Only one reading passage was required
due to time restraints of the reading comprehension test. Future experiments
could investigate the eect of requiring the students to show their metacognitive
processes on every test item.
As the results indicate (Table 1), the required metacognitive processes while
participants actively employing reading comprehension strategies resulted in
better overall scores. e t-test value of 3.9855 and standard error of dierence
of 0.479 are very promising for future studies. e p-value of 0.0002 is also
very exciting in that it shows a strong statistical signicance, even with such a
small sample size (n=64). e scores reect the direct and positive inuence
of engaging in metacognitive processes during reading comprehension strategy
implementation. In order to gauge the eect size of this study, a Cohen’s d was
determined (Table 1). e d value of 1.0 shows a very large eect size. e
combination of the large eect size and the high statistical signicance is exciting
in that it suggests this study could easily be scaled up to larger numbers with
similar results as those found in this action research.
e ability to increase the likelihood of nding correct answers on a reading
comprehension test by requiring metacognitive activity during the test makes
this study interesting. By requiring proof of reading strategies during the taking
of the test, scores increased signicantly. is also supports previous work on
metacognition and education where a positive impact was observed when
introducing metacognition to a class (Cubukcku, 2008). e hypothesis put
forth is supported but will need future studies to replicate the ndings. e
correlation between showing the participants’ strategy application on one part
of a four-part test also begs the question of what would happen if they were
required to show this implementation on all parts of a test. It is the belief of
this author that the dierence between the control group and test group would
widen even further.
339
e Eects of Metacognition, OnCUE Journal, 9(4), pages 329-343
Pedagogical Implications
As the data suggests, having students show their work on a test resulted in better
scores on reading comprehension tests. is pilot study suggests that requiring
the students to actively participate in thinking about implementation of reading
strategies, as well as showing proof of these thought processes, may be eective in
helping to improve test scores. It is interesting to note that in subsequent lessons
this author observed students in the test group continue to show their work,
albeit at a reduced rate, aer the study.
Metacognition has the potential to oer a wide range of benets for students.
One outcome an educator might expect would be better self-monitoring of
learning behavior in students and ease of strategy implementation when needed
on tests such as TOEIC. Metacognition also inuences other learning skills and
dispositions by aiding in planning, monitoring, and regulation of a student’s
cognitive processes which are relevant to both deductive and inductive thinking,
as well as critical thinking (Dimmitt & McCormick, 2012; Pintrich, 2002). In
regards to this author’s students, metacognition can provide benets past the
classroom. Science and engineering elds require a great deal of critical thinking.
Metacognition and the associated skills and dispositions not only help to pass a
reading comprehension test, but those same cognitive processes will help them
in their future endeavors. It has been shown in this action research that students
can benet from top-down instruction of skills and strategies, but it is perhaps
more important that the students are expected to use metacognitive processes to
fully utilize what they have learned.
Future Research
is study must be replicated using a larger pool of participants from a greater
range of skills. is author hopes to include other teachers and classrooms
in similar studies in the near future. Larger sample sizes and expanding the
requirement of showing the implementation of strategies to the entire test should
result in more denitive data that this style of metacognition has a positive
impact on scores. Standardization of the reading comprehension test and using
more sophisticated methods of data analysis would also benet the collection
340
Dunn
of data. As previously mentioned, students in the test group continued to show
their work in later quizzes, which is another avenue of study that needs to be
explored. e question of lasting benets of any method is incredibly important
for teachers and students alike. Further research into metacognition with other
styles of cognitive processes, such as Cubukcu (2008) examined, would be
interesting.
Conclusions
Ultimately, the best strategies for students are the ones that are actually employed
when needed. Giving students every possible way to do better in their lives, be
it on a TOEIC test or in their future jobs, is essential. By giving students the
opportunity to engage in metacognitive processes of “thinking about their
thinking” or actively thinking about what they know, teachers help them to build
skills that will stay with them beyond the walls of the classroom.
Acknowledgements
I would like to thank Simeon Flowers for his assistance in determining the
statistics for data contained in this paper.
References
Adams, D., & Hamm, M. (1994). New designs for teaching and learning:
Promoting active learning in tomorrow’s schools. San Francisco, CA: Jossey-
Bass.
Alexander, P. A., & Jetton, T. L. (2000). Learning from text: A
multidimensional and developmental perspective. In M. L. Kamil, P. B.
Mosenthal, P. D. Pearson, & R. Barr (Eds.), Handbook of reading research:
Vol. III (pp. 285-310). Mahwah, NJ: Lawrence Erlbaum Associates.
Allington, R. L. (2006). What really matters for struggling readers: Designing
research-based programs (2nd ed.). Boston, MA: Pearson.
Anderson, N. (1999). Exploring second language reading: Issues and strategies.
Boston, MA: Heinle & Heinle.
Anderson, N. J. (2002). e role of metacognition in second/foreign
341
e Eects of Metacognition, OnCUE Journal, 9(4), pages 329-343
language teaching and learning. ERIC Digest. Washington, DC: ERIC
Clearinghouse on Language and Linguistics. Retrieved from http://les.
eric.ed.gov/fulltext/ED463659.pdf
Baker, L. (2008). Metacognitive development in reading: Contributors and
consequences. In K. Mokhtari & R. Sheorey (Eds.), Reading strategies of
rst and second-language learners: See how they read (pp. 25-42). Norwood,
MA: Christopher-Gordon.
Carrell, P. L., Pharis, B. G., & Liberto, J. C. (1989). Metacognitive strategy
training for ESL reading. TESOL uarterly, 23(4), 647-678.
Carroll, M. (2008). Metacognition in the classroom. In J. Dunlosky & R.A.
Bjork (Eds.), Handbook of metamemory and memory (pp. 411-427). New
York, NY: Psychology Press.
Chapman, M. (2003). e role of the TOEIC in a major Japanese company.
Proceedings of the 2nd Annual JALT Pan-SIG Conference. Kyoto, Japan.
Retrieved from https://jalt.org/pansig/2003/HTML/Chapman.htm
Childs, M. (1995). Good and bad uses of TOEIC by Japanese companies. In
J. D. Brown & S. Yamashita (Eds.), Language testing in Japan (pp. 66-75).
Tokyo, Japan: JALT.
Cubukcu, F. (2008). Enhancing vocabulary development and reading
comprehension through metacognitive strategies. Issues in Education
Research, 18(1), 1-11.
Dimmitt, C., & McCormick, C. B. (2012). Metacognition in education. In
K. R. Harris, S. Graham, and T. Urdan (Eds.), APA educational psychology
handbook: Vol. 1. eories, constructs, and critical issues (pp. 157–187). doi:
10.1037/13273-007
Dunlosky, J., & Metcalf, M. (2009). Metacognition. ousand Oaks, CA: Sage.
Gebhardt, F. (2012). Reading Comprehension Practice. (B1 Idoneita).
Department of Philosophy. University of Venezia, Italy. Retrieved
from: http://lettere2.unive.it/lingue/lingua_ING/B1%20reading%20
comprehension.pdf
Gettinger, M., & Seibert, J. K. (2002). Contribution of study skills to academic
competence. School Psychology Review, 31(3), 350-365.
342
Dunn
Jacobs, J. E., & Paris, S. G. (1987). Children’s metacognition about reading:
Issues in denition, measurement, and instruction. Educational Psychologist,
22, 255-278. doi: 10.1207/s15326985ep2203&4_4
Moore, D. (2012). Reading comprehension strategies. Cengage Publishing.
Retrieved from http://ngl.cengage.com/assets/downloads/edge_
pro0000000030/am_moore_read_comp_strat.pdf
Nelson, T. O., & Narens, L. (1990). A theoretical framework and new ndings.
Psychology of Learning and Motivation, 26, 125-173. doi: 10.1016/S0079-
7421(08)60053-5
Nuttal, C. (1996). Teaching reading skills in a foreign language (2nd ed.).
Oxford, England: Heinemann English Language Teaching.
Pintrich, P. R. (2002). e role of metacognitive knowledge in learning,
teaching, and assessing. eory Into Practice, 41(4), 219-225. doi: 10.1207/
s15430421tip4104_3
Powers, D., Kim, H. J., & Weng, V. (2008). e redesigned TOEIC (Listening
and Reading) Test: Relations to test-taker perceptions of prociency in
English. ETS Research Report, No. RR-08-56. Princeton, NJ: ETS
Rasinski, T. V. (2010). e uent reader (2nd ed.). Boston, MA: Pearson.
Schoenfeld, A. H. (1992). Learning to think mathematically: Problem solving,
metacognition, and sense making in mathematics. Handbook for research
on mathematics teaching and learning (pp. 334-370). New York, NY:
Macmillan.
Short, D. J., & Fitzsimmons, S. (2007). Double the work: Challenges and
solutions to acquiring language and academic literacy for adolescent
English language learners: A report to Carnegie Corporation of New York.
Washington, DC: Alliance for Excellent Education.
Schwartz, B. L., & Bacon, E. (2008). Metacognitive neuroscience. In J.
Dunlosky & R. A. Bjork (Eds.), Handbook of metamemory and memory (pp.
355-371). New York, NY: Psychology Press.
Takallou, F. (2011). e eect of metacognitive strategy instruction on
EFL learners’ reading comprehension performance and metacognitive
awareness. Asian EFL Journal, 13(1), 275-320. Retrieved from http://
343
e Eects of Metacognition, OnCUE Journal, 9(4), pages 329-343
www.ibrarian.net/navon/paper/e_Asian_EFL_Journal_Quarterly.
pdf?paperid=19213415#page=275
Tregaskes, M. R., & Daines, D. (1989). Eects of metacognitive strategies on
reading comprehension. Literacy Research and Instruction, 29(1), 52-60.
Weinstein, C. E., & Mayer, R. F. (1985). e teaching of learning strategies. In
M. C. Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 315-
329). New York, NY: Macmillan.
Weinstein, C. E., & Underwood, V. (1985). Learning strategies. In J. Segal &
S. Chipman (Eds.), inking and learning skills (pp. 241-258). London,
England: Lawrence Erlbaum Associates.
Author bio
James Dunn is a full time teacher at Tokai University. His research interests are in
Cognitive Linguistics and Critical inking skills. His educational goal is to help
students understand that they are capable of more than they might expect om
themseles. james.d.dunn@outlook.com
Received: February 4, 2015
Accepted: December 18, 2015
