Content uploaded by Magda Kitano
Author content
All content in this area was uploaded by Magda Kitano on Jun 28, 2019
Content may be subject to copyright.
VERB Vol. 8 Issue 1, Summer 2019
29
Poster Presentation Synopsis
Motivating Low-Achievement Learners to Prepare for
Weekly Vocabulary Tests:
The Come-Back Vocabulary Test
Magda Kitano magda@shonan.bunkyo.ac.jp
Katsuhiro Chiba ch1ba@shonan.bunkyo.ac.jp
Theoretical Framework
While 6,000 to 9,000 words is a recommended goal for second language learners
of English, high-frequency vocabulary is particularly important. As these words make up
a large percentage of both spoken and written text, a lack in fluency of high-frequency
words is a serious obstacle to comprehension and language production (Nation, 2013).
However, while first-year university students in Japan know a range of only 1,700 to
2,300 words after six years of English instruction (Barrow, Nakanishi, & Ishino, 1999),
studies such as Browne and Culligan (2008) have found that there are large gaps in their
knowledge of the most frequent words. Combining this problem with the limited class
time provided in Japanese universities, Browne (Wadden, Browne, & Nation, 2019)
recommended the usage of out-of-class time for vocabulary study and that instructors set
regular vocabulary-learning goals such as a weekly list of words to learn. Robb (2019)
also indicated that out-of-school study in all aspects of English learning is necessary under
the Japanese university system in order for students to reach their TOEIC goals.
Compounding the problem is a general lack of motivation among Japanese
students to put in time for study. Robb (2019, p. 116) noted that while there are a few
highly motivated students in Japanese universities, for the “vast majority” a teacher
should expect them to try to get away with as little work as possible. Snyder (2019, p.
137) wrote that although students lacking motivation to learn English is a stereotype, such
students are in fact a “frequent source of frustration” for university instructors in Japan,
with students “at best . . . dutiful but barely participatory; at worst, they are indifferent to
homework and exams, often tardy, or absent altogether.” Teachers of remedial learners
face even more challenges. Kiyota (2009) noted a motivational gap between remedial and
regular learners in Japanese university English classes. He concluded that different
teaching approaches are necessary for these two types of learners. This situation is a
particular challenge for vocabulary learning, which requires multiple retrieval events and
spaced repetitions for successful acquisition (Nation, 2013).
The current study was an attempt to motivate near-remedial students to spend time
VERB Vol. 8 Issue 1, Summer 2019
30
outside of class preparing for weekly vocabulary tests by personalizing the testing system.
If successful, it could be a stepping stone to better study habits across the curriculum for
students, and high-frequency words learned in the process would benefit students by
making English more accessible.
Sample
The subjects were first-year students at a mid-level university in Kanagawa, Japan,
all majoring in international studies. At the beginning of the semester all incoming first-
year students took a test of the 1,000 most frequent words of the New General Service
List (NGSL; Browne, Culligan, & Phillips, 2013), were instructed on the importance of
high-frequency words to their English abilities, and were assigned the task of identifying
unknown words in the first 1,000 of the NGSL and learning those for a test at the end of
the semester.
Required English classes were divided into six levels for first-year students as
determined by scores on the Computerized Assessment System for English
Communication (CASEC), with two classes of students at each level. The subjects of this
study were those of the two lowest-level classes, with CASEC scores ranging from 219
to 405 (M = 341.68, SD = 43.88) for class A, and 122 to 315 (M = 260.06, SD = 59.82)
for class B. These scores are equivalent to CEFR levels B1 and B2 (“Data & Information,”
n.d.). Of the total of 44 students in these classes, weekly test score data were analyzed for
only those whose CASEC scores were available (N = 36).
Procedure
In order to aid students of these lowest-level classes in reaching the program-wide
goal of mastering the first 1,000 words of the NGSL during the spring (first) semester of
the 2018 school year, students were assigned weekly English-Japanese bilingual lists of
ten words to learn. These words were chosen from those unknown by many in the initial
test, and also included commonly unknown words from previous years. Weekly tests
required students to write the target words from cues of the Japanese equivalents.
When the students received the results of the first weekly test, they were told that
all words that were marked wrong on this test would appear on the following week’s test
along with the new words for that week. Subsequent weekly tests were personalized, with
a test of the new words of the week common to all students on one side of the paper, and
each student’s “come-back” words from all previous tests on the other. Once a word was
correctly supplied by the student, it no longer reappeared on future come-back tests.
Grades for student assessment were determined as the percentage of correct responses on
the entirety of the test (both sides of the paper).
VERB Vol. 8 Issue 1, Summer 2019
31
This system was made feasible by utilizing the software FileMaker Pro (version
16) to create a database that kept track of test items, students, and test results. After
indicating accurate or mistaken in the database for each word on the tests, subsequent
personalized tests could be printed automatically. Personalized test marking also became
feasible because the student’s current test answer key could be viewed on-screen.
Six weekly lists were introduced and tested. For the two weeks following, only
come-back words were tested. Students with no come-back words returning to them were
rewarded with full scores for those weeks. Finally, in an end-of-semester questionnaire,
students were asked their impressions of this testing method and whether they thought
this program should be continued in future classes. Of the 44 students, 41 were present
on the questionnaire day.
Preliminary Results
Looking at the weekly test scores of new target words only (i.e., not including
students’ come-back words), scores rose over the six weeks (Figures 1 and 2). In paired-
samples t-tests comparing performance on the first and last weekly tests, significant
improvement was seen. In class A, the first test had a mean of 7.05 (SD = 4.31), and the
final test a mean of 8.92 (SD = 1.71), t(18) = 2.50, p = .022, d = .57. In class B, the first
test had a mean of 6.05 (SD = 3.60), and the final test a mean of 8.88 (SD = 2.21), t(16)
= 3.11, p = .007, d = .95.
Illustrating how these results differ from the usual situation in classes at this level,
Figure 3 shows test scores from seven weeks of a 2017 class, also the lowest of six levels.
Students in this class were similarly tested on ten words weekly from the first 1,000 words
of the NGSL, but without the come-back method being used. While high scores are seen
throughout the period, many students peak and then return to zero, indicating sporadic
test preparation. In comparison, Figures 1 and 2 with the come-back method show very
few zero or low marks once the program started up, while most marks cluster at the top.
As to improvement between the first and last tests, a paired-samples t-test of the 2017
data did not indicate significant improvement (week one M = 5.47 (SD = 4.40), week
seven M = 6.26 (SD = 3.80), t(18) = 0.67, p = .508, d = .19.)
Questionnaire answers were in general positive about the method, with 37
students answering that they thought the come-back system should be used in future
classes, and two answering that it should not (2 abstentions). In a free-answer question,
29 comments were positive, some glad that the system pushed them to study when they
normally would not have, and some appreciating the chance to try again to get words right
that they had missed. Five comments were negative, some disliking how long their come-
back word lists grew.
VERB Vol. 8 Issue 1, Summer 2019
32
Figure 1. Class A student scores for weekly vocabulary tests. Scores include only the list
of ten new words assigned for the week. One half point was given for misspelled words.
Figure 2. Class B student scores for weekly vocabulary tests. Scores include only the list
of ten new words assigned for the week. One half point was given for misspelled words.
Preliminary Conclusions
In a group of students who were not in the habit of completing homework
assignments or preparing for vocabulary tests, the come-back method was seen to
encourage most of the students to study before tests, with a goal of getting as many correct
as possible. The prospect of an increase in the amount of words one must study seemed
to be a key aspect of this system, and success was rewarded immediately by correctly-
answered words being taken off students’ come-back lists.
0
1
2
3
4
5
6
7
8
9
10
123456
Correctly Answered Items
Week
0
1
2
3
4
5
6
7
8
9
10
123456
Correctly Answered Items
Week
VERB Vol. 8 Issue 1, Summer 2019
33
Figure 3. Student scores for weekly vocabulary tests, 2017 class of second year students.
In addition to ten weekly vocabulary words, tests included two bonus point questions
concerning textbook content, resulting in a total of 12 possible points.
Future Directions
This study only looked at the impact of the come-back method on weekly
vocabulary scores; it did not investigate whether words were being acquired. Future
research on this teaching practice ought to involve a control group given weekly lists and
tests without the come-back aspect, and would need to test for target word knowledge
retained over time.
References
Barrow, J., Nakanishi, Y., & Ishino, H. (1999). Assessing Japanese college students’
vocabulary knowledge with a self-checking familiarity survey. System, 27, 223-
247.
Browne, C., & Culligan, B. (2008). Combining technology and IRT testing to build
student knowledge of high frequency vocabulary. The JALT CALL Journal, 4(2),
3-16.
Browne, C., Culligan, B., & Phillips, J. (2013). The New General Service List. Retrieved
from http://www.newgeneralservicelist.org
Data & information. (n.d.). Computerized Assessment System for English
Communication. Retrieved from https://global.casec.com/data/
Kiyota, Y. (2009). Motivation of remedial EFL learners (A case study of Japanese college
EFL learners).
リ メ デ ィ ア ル 教 育 研 究
[The Japan Association for
Developmental Education] 4(2): 173-179.
0
2
4
6
8
10
12
1234567
Correctly Answered Items
Week
VERB Vol. 8 Issue 1, Summer 2019
34
Nation, I. S. P. (2013). Learning vocabulary in another language. Cambridge: Cambridge
University Press.
Robb, T. N. (2019). Homework in the Japanese university classroom. In P. Wadden & C.
C. Hale (Eds.), Teaching English at Japanese universities (pp. 115-121). New
York, NY: Routledge.
Snyder, B. (2019). Creating engagement and motivation in the Japanese university
language classroom. In P. Wadden & C. C. Hale (Eds.), Teaching English at
Japanese universities (pp. 137-143). New York, NY: Routledge.
Wadden, P., Browne, C., & Nation, P. (2019). Teaching and learning vocabulary in the
Japanese university. In P. Wadden & C. C. Hale (Eds.), Teaching English at
Japanese universities (pp. 84-96). New York, NY: Routledge.
