Conference PaperPDF Available

Research Syntheses in L2 Vocabulary Research: A Scoping Review

Authors:
Ali H. Al-Hoorie at Royal Commission for Jubail and Yanbu
Ali H. Al-Hoorie
Joseph Vitta at Waseda University
Joseph Vitta
Christopher Nicklin at The University of Tokyo
Christopher Nicklin
Download full-text PDF
Read full-text
Download citation
Content uploaded by Ali H. Al-Hoorie
Author content
All content in this area was uploaded by Ali H. Al-Hoorie on Nov 06, 2022
Content may be subject to copyright.
Research Syntheses in L2
Vocabulary Research:
A Scoping Review
Ali H. Al-Hoorie, Saudi TESOL Association
Joseph P. Vitta, Kyushu University
Christopher Nicklin, Rikkyo University
JALT Vocabulary SIG's 2022 Symposium on Vocabulary & Learning
Tokyo International University - Kawagoe, Japan
October 29, 2022
Background
–Vocabulary Research has gone from a niche to a mainstay of SLA research
–L2 Research Syntheses has likewise grown in popularity from Norris & Ortega
(2000) and Ross (1998)
–The time is now ripe to look back at what synthesizers of L2 vocabulary
research has been interested in and what they have found.
Scoping Review?
–Driven by RQs
–Designed to cast a wider net than systematic reviews
–Uncovers patterns and trends for future research
Flow of this talk
–Report Pool Creation Process
–Results by RQ
–Concluding Thoughts
Report Pool Creation (31 WoS research synthesis reports)
Research Question 1
What types of vocabulary-based research syntheses were conducted between
January 1990 through May 2022?
A. Meta-analyses vs. Systematic reviews
B. For meta-analyses: Correlational vs. Experimental
C. For systematic reviews: What type (e.g., methodological syntheses)?
Research Question 2
A. What are the descriptive characteristics of these reports (e.g., size of report
pool, aggregate sample size)?
B. To what extent did meta-analyses in the report pool employ inferential
testing? Which tests were employed?
RQ2A
It was also observed via post-hoc analysis that time and the frequency of reports were positively associated (ρ = .74).
RQ2B
“Of the 25 meta-analyses, 17 featured Q-testing as the inferential assessment
testing the significance of its reported aggregated effect size(s) and/or moderator
analyses. Of the eight meta-analyses not using Q-testing completely, five featured
parametric testing where procedures were stated to treat each effect size as a
‘case.’ Three reports had no inferential assessment with only Nicklin and Vitta
(2021) offering a justification as to why. In the case of Nicklin and Vitta, Q-testing
was avoided to capture the full heterogeneity of studies’ effect sizes using a simple
approach to ascertain effect sizes for a series of power analyses.”
Research Question 3
What areas have L2 vocabulary research syntheses investigated?
(bottom-up coding)
(categories only created if 4+ reports were coded as being in a given category)
Area
k
=
Notes
Technology and Vocabulary
Use
9 (reports)
MALL, CALL, digital games
(glossing excluded)
Vocabulary as a Predictor of
Reading
4
1 of the 4 (Zhang & Zhang,
2020) also focused on
listening
Glossing
5
NA
Uncategorized
17
Trends are present among
these reports however (e.g.,
Webb and colleagues’ work)
Research Question 4
A. When reviewing meta-analyses in the pool, which areas emerge as being
conclusively addressed? and which areas emerge as requiring further
investigations?
B. What are apparent gaps in the existing body of L2 vocabulary systematic
reviews?
RQ4A.
●Technology enhances the acquisition of vocabulary
●Glossing enhances the acquisition of vocabulary
●Vocabulary predicts reading proficiency (and listening proficiency [see Zhang &
Zhang, 2020])
**There is now a need to ask second and third generation questions (Zanna &
Fazio, 1982)
Potential application of 2nd/3rd generation RQs
“What such research projects would look like are modeled using examples from the report pool.
Consider glossing as an example with Abraham’s (2008) finding that glossing had strong yet
nonsignificant effects for more proficient learners. A second-generation research question-
governed project could address the conditions under which this finding holds true. To illustrate
the potential of a third-generation research question consider Zhang and Zhang (2020), who
presented a moderator analysis detailing that vocabulary knowledge when measured via
meaning recall had a significantly stronger association with reading than when such vocabulary
knowledge was measured via form recall or meaning recognition. To explain the ‘how’ of this
finding, researchers could design mixed-methods studies where qualitative data helps to explain
what meaning recall is testing to give it its significantly stronger association.”
RQ4B
–Difficult to answer with the findings we observed.
**More methods work is required.
**The curious finding of Choi and Zhang (2021) warrants more discussion and
research: They reviewed multivariate models where vocabulary and grammar
predicted reading and concluded that it was inclusive as to which construct
provided the stronger association. (17 studies / no inferential testing)
Concluding Thoughts (open questions)
Why do synthesizers of vocabulary research favor experimental RQs and
hypotheses to correlation ones?
*Does this pattern exist among primary vocabulary research?
Are vocabulary researchers relatively less engaged with the ongoing L2 methods
reform movement? (only 4 of 31 SRs were methods focus)
Will 2nd and 3rd generation RQs be addressed by future L2 vocabulary
researchers?
Thank you!!
References
●Abraham, L. B. (2008). Computer-mediated glosses in second language reading comprehension and vocabulary learning: A meta-
analysis. Computer Assisted Language Learning, 21(3), 199–226. https://doi.org/10.1080/09588220802090246
●Choi, Y., & Zhang, D. B. (2021). The relative role of vocabulary and grammatical knowledge in L2 reading comprehension: a systematic
review of literature. International Review of Applied Linguistics in Language Teaching, 59(1), 1–30. https://doi.org/10.1515/iral-2017-
0033
●Elgort, I. (2018). Technology-mediated second language vocabulary development: A review of trends in research methodology. CALICO
Journal, 35(1), 1–29. https://doi.org/10.1558/cj.34554
●Lin, J. J., & Lin, H. (2019). Mobile-assisted ESL/EFL vocabulary learning: A systematic review and meta-analysis. Computer Assisted
Language Learning, 32(8), 878–919. https://doi.org/10.1080/09588221.2018.1541359
●Nicklin, C., & Vitta, J. P. (2021). Effect-driven sample sizes in second language instructed vocabulary acquisition research. The Modern
Language Journal, 105(1), 218–236. https://doi.org/10.1111/modl.12692
●O’Connor, D., Green, S., & Higgins, J. P. (2008). Defining the review question and developing criteria for including studies. In J. P. Higgins
& S. Green (Eds.), Cochrane handbook for systematic reviews of interventions (pp. 81–94). Wiley-Blackwell.
●Yang, X., Kuo, L. J., Eslami, Z. R., & Moody, S. M. (2021). Theoretical trends of research on technology and L2 vocabulary learning: A
systematic review. Journal of Computers in Education, 8(4), 465–483. https://doi.org/10.1007/s40692-021-00187-8
●Zanna, M. P., Fazio, R. H. (1982). The attitude-behavior relation: Moving toward a third generation of research. In M. P. Zanna, E. T.
Higgins, & C. P. Herman (Eds.), Consistency in social behavior: The Ontario symposium: Volume 2 (pp. 283–301). Lawrence Erlbaum.
●Zhang, S., Zhang, X. (2020). The relationship between vocabulary knowledge and L2 reading/listening comprehension: A meta-analysis.
Language Teaching Research, 26(4), 696–725. https://doi.org/10.1177/1362168820913998
ResearchGate has not been able to resolve any citations for this publication.
The relative role of vocabulary and grammatical knowledge in L2 reading comprehension: a systematic review of literature
Article
Full-text available
  • Jun 2021
The purpose of this systematic review is to synthesize the findings in the literature on the relative contribution of two types of linguistic knowledge (i. e., vocabulary and grammatical knowledge) to L2 reading comprehension; and to discuss the possible factors that might have led to the complexity and incongruity of the research findings. Nineteen studies were identified for this review through a systematic process of selection, and were analyzed in terms of the relationship of vocabulary and grammatical knowledge with L2 reading comprehension and their relative significance. The result of this review showed that there was no clear evidence supporting which type of linguistic knowledge is more contributive than the other; and the inconclusive findings might be attributed to the variations in the design of the studies. Two factors emerged, including participants' language proficiency and construct definition, operatio-nalization, and measurement. Some considerations for future research are suggested. Reading comprehension is a constructive process whereby the reader creates a mental representation of the meaning of a text while multiple factors-text, reader, and activity-mutually interact (RAND Reading Study Group 2002). In this process, the reader first accesses word meanings, integrates pieces of syntactic information, and builds local and global coherence (i. e., the text base model; Van Dijk and Kintsch 1983); this is followed by connecting and integrating information from the text with the reader's relevant prior knowledge, which allows deeper comprehension (i. e., the situation model; Van Dijk and
View
View
Theoretical trends of research on technology and L2 vocabulary learning: A systematic review
Article
  • May 2021
Vocabulary development is critical for second language (L2) learners in both English as a Second Language (ESL) and English as a Foreign Language (EFL) contexts. Recently, a large body of research has been dedicated to how computer-assisted language learning (CALL) and mobile-assisted language learning (MALL) can facilitate vocabulary knowledge in L2 learners in both EFL and ESL settings. A number of reviews on this topic have been conducted, however, little attention has been given to learners in PreK-12. Also missing from the existing research is an in-depth examination of the theories underlying vocabulary learning within technological programs. However, understanding theoretical foundations of vocabulary learning is critical for both researchers and educators who seek to improve vocabulary development in L2 learners. The current study aims to close these gaps by investigating research on the use of technology for L2 vocabulary learning for learners in PreK-12 between 2011 and 2020. Using systematic review procedures, a total of 80 articles were identified for analysis. Results showed information/cognitive theories were most frequently and explicitly referenced, followed by social theories of learning. Consistent with previous research syntheses on CALL and MALL, many studies did not articulate an explicit theoretical framework used in their research. These findings suggest that research on technology-mediated vocabulary learning for Prek-12 L2 learners should be conducted from more diverse and explicit theoretical perspectives.
View
The relationship between vocabulary knowledge and L2 reading/listening comprehension: A meta-analysis
Article
  • Mar 2020
  • LANG TEACH RES
This study set out to investigate the relationship between L2 vocabulary knowledge (VK) and second-language (L2) reading/listening comprehension. More than 100 individual studies were included in this meta-analysis, which generated 276 effect sizes from a sample of almost 21,000 learners. The current meta-analysis had several major findings. First, the overall correlation between VK and L2 reading comprehension was .57 ( p < .01) and that between VK and L2 listening was .56 ( p < .01). If the attenuation effect due to reliability of measures was taken into consideration, the ‘true’ correlation between VK and L2 reading/listening comprehension may likely fall within the range of .56–.67, accounting for 31%–45% variance in L2 comprehension. Second, all three mastery levels of form–meaning knowledge (meaning recognition, meaning recall, form recall) had moderate to high correlations with L2 reading and L2 listening. However, meaning recall knowledge had the strongest correlation with L2 reading comprehension and form recall had the strongest correlation with L2 listening comprehension, suggesting that different mastery levels of VK may contribute differently to L2 comprehension in different modalities. Third, both word association knowledge and morphological awareness (two aspects of vocabulary depth knowledge) had significant correlations with L2 reading and L2 listening. Fourth, the modality of VK measure was found to have a significant moderating effect on the correlation between VK and L2 text comprehension: orthographical VK measures had stronger correlations with L2 reading comprehension as compared to auditory VK measures. Auditory VK measures, however, were better predictors of L2 listening comprehension. Fifth, studies with a shorter script distance between L1 and L2 yielded higher correlations between VK and L2 reading. Sixth, the number of items in vocabulary depth measures had a positive predictive power on the correlation between VK and L2 comprehension. Finally, correlations between VK and L2 reading/listening comprehension was found to be associated with two types of publication factors: year-of-publication and publication type. Implications of the findings were discussed.
View
Mobile-assisted ESL/EFL vocabulary learning: a systematic review and meta-analysis
Article
  • Jan 2019
Mobile-assisted language learning (MALL) has been a popular research area in recent years in the SLA field, as it has been considered as being able to facilitate language learning by offering an authentic, socially connective, contextually sensitive, and personalized mobile-mediated language learning environment. To investigate the benefits and possible limitations of mobile-assisted L2 vocabulary retention, many researchers continuously conducted relevant experiments worldwide. Among the current mobile-assisted L2 vocabulary learning research, short message services (SMS), multimedia message service (MMS), and mobile applications (apps) are the predominant approaches supported by the situated learning theory, cognitive mechanisms of L2 word retention, behaviorism, and social constructivism. Although language educators have been focusing on this topic, the effectiveness of mobile L2 vocaubulary learning still remained inconclusive. Besides, to date, little research synthesis has been conducted to investigate the effect of mobile-assisted L2 vocabulary learning. The purpose of this meta-analysis is to systematically synthethisize findings from (quaisi-) experimental studies carried out between 2005 and 2018 to examine whether there is a connection between the use of mobile technologies and L2 word retention. The results of 33 primary eligible studies showed a positive and large effect of mobile-assisted L2 word learning interventions. Additionally, SMS/MMS mode was found to be more beneficial for L2 word retention than mobile application mode; however, it is more prudent to interpret this finging as tentative because the number of target words and intervention time were unequal in the two modes. Moreover, the research settings, treatment durations, and task-afforded autonomy are three variables that tend to significantly moderate the effect of mobile-assisted L2 word retention. However, the above results should be viewed as suggestive rather than definitive with a view to the research flaws found in the primary studies and the small sample size of certain categories of moderators when compared.
View
Technology-Mediated Second Language Vocabulary Development: A Review of Trends in Research Methodology
Article
  • Aug 2017
Technology-mediated vocabulary development (TMVD) in a second language (L2) covers a wide range of instructional and learning treatments, contexts and technologies, and is situated in a broader field of second language vocabulary learning. Vocabulary knowledge is a complex, multidimensional construct that has been interpreted and categorized in second language research in many different ways. This review identifies methodological practices in research into L2 TMVD and provides a synthesis of learning and instructional approaches, aspects of vocabulary knowledge being investigated and measures of vocabulary development used in this research. A sample of 82 primary studies from 2010 to 2017 was selected and coded for target methodological features. The results of the review show that TMVD research has achieved a certain degree of methodological maturity, while a number of issues have also been identified. These issues include: the practice of comparing technology-mediated instruction with ‘traditional’ instruction without technology; insufficient reporting of participants’ L2 proficiency; and a dearth of treatments targeting fluency development. The review also showed a preference for evaluating the knowledge of individual vocabulary items rather than the development of the L2 lexicon, and insufficient use of online and implicit measures of vocabulary knowledge. Recommendations for future TMVD research are provided.
View
Computer-mediated glosses in second language reading comprehension and vocabulary learning: A meta-analysis
Article
  • Jul 2008
Language learners have unprecedented opportunities for developing second language literacy skills and intercultural understanding by reading authentic texts on the Internet and in multimedia computer-assisted language learning environments. This article presents findings from a meta-analysis of 11 studies of computer-mediated glosses in second language reading comprehension and incidental vocabulary learning. Computer-mediated glosses had an overall medium effect on second language reading comprehension and a large effect on incidental vocabulary learning. Mean effect sizes varied from medium to large depending upon the level of instruction, text type, and assessment tasks. Drawing upon findings of this meta-analysis and the overall characteristics of gloss studies, we propose recommendations for future research, including replications and the systematic study of reading variables and learner individual differences in multimedia learning environments with authentic texts.
View