Figure - uploaded by Suzanne Graham
Content may be subject to copyright.
Source publication
Assessment is a central challenge within classroom-based early language learning, where there is a need to employ assessment methods which, as well as being valid and reliable for a range of learners, protect rather than diminish motivation. The motivational properties of digital or serious games within language learning are increasingly recognised...
Context in source publication
Context 1
... overwhelmingly positive responses to Q1_4 were also echoed in the responses to Q1_5, which asked learners what they disliked about the game: almost 60% of learners of the 1057 1 learners who gave a response to this question said 'nothing'. However, in contrast to the learners who liked the game because it was challenging (see Table 4), a good proportion considered the difficulty of the game as something that they disliked (13.2%, see Table 5). It should be noted, however, that of the 141 learners commenting that the game was difficult, only seven had a mean positivity score of 2 or below on the closed questionnaire items, indicating that the vast majority of those who thought the game was hard were still positive about it overall. ...Citations
... YLLs have specific characteristics that distinguish them from older and adult learners. Compared to older/adult learners, YLLs have a shorter attention span and a slower processing speed (Bailey et al., 2014;McKay, 2006), resulting in the need to develop tests and instruments that are concrete rather than abstract, that reflect the lived experiences of YLLs and that are of appropriate length so as to sustain learners' motivation (Courtney & Graham, 2019;McKay, 2006). Additionally, YLLs may be linguistically diverse; for example, in 2023 in England, 22 % of children were believed to use a language other than English at home (DfE, 2024). ...
... Test length represents another crucial aspect to balance when designing research instruments for YLLs. Whilst shorter tests are more suitable for this population of learners (Courtney & Graham, 2019;McKay, 2006), a large number of items may be needed to achieve satisfactory reliability and validity. This is particularly the case when using a multiple-choice design, as this item format is likely to introduce considerable noise in the data due to guessing. ...
With mandatory second language learning in primary education becoming the norm worldwide, research investigating young language learners’ (YLLs) linguistic development has increased. However, designing language tests appropriate for YLLs poses unique challenges due to population characteristics and variability in national and institutional contexts. In this article, we present a new vocabulary test designed to track the rate of progression in receptive vocabulary size of primary school children learning French, German or Spanish in England.
Test content was selected after an analysis of programmes of study commonly used in primary schools in England. The test required two validation phases using Rasch analysis. The initial tests were administered to 1662 students from Year 3 (7–8 years old) to Year 5 (9–10 years old). All tests showed poor person reliability, which was driven by a mismatch between item difficulties and participant abilities. Various actions were taken in relation to vocabulary identification, test format and length, and sampling procedures, and the revised tests were re-administered to 2202 students from Year 3 to Year 6 the following year. As a result, person reliability considerably improved, and all test versions showed good fit to the Rasch model.
Drawing on the lessons learnt, we discuss some of the key population- and context-related challenges of designing robust language tests for beginner YLLs learning a language other than English in input-poor, instructed contexts. Further, we provide recommendations on suitable approaches for test-item identification, test format and length, and data analysis.
... In addition, language teaching is a situated and sociocultural practice where learning emerges within a culture by learners' interacting and communicating with others. Many language researchers have documented the application of game-based learning in listening and speaking teaching (Hwang et al., 2016), vocabulary instructions (Zou et al., 2021), and assessment (Courtney & Graham, 2019;Zakaria & Hashim, 2020). The most prominent research on the benefits of game-based language learning has focused mostly on technology's potential to increase motivation (Eltahir et al., 2021;Hung et al., 2018). ...
Language's complex and dynamic nature requires language assessment to be conducted on spontaneous speech in authentic contexts. Assessment in task-based language teaching has the potential to achieve this goal and would benefit from a game-based virtual environment. However, conducting assessments in game-based and task-based teaching remains challenging. Based on the data collected in a workshop for 15 pre-service and in-service teachers, this chapter reports difficulties in leveraging assessment in game-based and task-based language teaching from teachers' perspectives. Results highlight the need to adopt a stealth assessment approach. This chapter proposes a constructive framework for stealth assessment in language teaching. Specific technological tools such as automated speech recognition, eye-tracking, and artificial intelligence are discussed, with a focus on their application in stealth assessment for game-based and task-based language teaching.
... People's prior experience of playing games in their daily lives can lead to better engagement when introduced in educational contexts (Eyupoglu & Nietfeld, 2019). This positive attitude towards games allows increased data collection (due to more time spent on task) and increased overall accuracy of the inferences (due to lower test anxiety and higher engagement) (Courtney & Graham, 2019;Mavridis & Tsiatsos, 2017;Mulligan et al., 2018;Sundre & Wise, 2003;Wise, 2006). The integration of player's background knowledge and prior experiences can create challenges for data collection as player in-game decisions are blended between prior and current understandings, thus creating noisier data (Basu et al., 2020). ...
Game‐based assessment (GBA), a specific application of games for learning, has been recognized as an alternative form of assessment. While there is a substantive body of literature that supports the educational benefits of GBA, limited work investigates the validity and generalizability of such systems. In this paper, we describe applications of learning analytics methods to provide evidence for psychometric qualities of a digital GBA called Shadowspect, particularly to what extent Shadowspect is a robust assessment tool for middle school students' spatial reasoning skills. Our findings indicate that Shadowspect is a valid assessment for spatial reasoning skills, and it has comparable precision for both male and female students. In addition, students' enjoyment of the game is positively related to their overall competency as measured by the game regardless of the level of their existing spatial reasoning skills.
Practitioner notes
What is already known about this topic: Digital games can be a powerful context to support and assess student learning.
Games as assessments need to meet certain psychometric qualities such as validity and generalizability.
Learning analytics provide useful ways to establish assessment models for educational games, as well as to investigate their psychometric qualities.
What this paper adds: How a digital game can be coupled with learning analytics practices to assess spatial reasoning skills.
How to evaluate psychometric qualities of game‐based assessment using learning analytics techniques.
Investigation of validity and generalizability of game‐based assessment for spatial reasoning skills and the interplay of the game‐based assessment with enjoyment.
Implications for practice and/or policy: Game‐based assessments that incorporate learning analytics can be used as an alternative to pencil‐and‐paper tests to measure cognitive skills such as spatial reasoning.
More training and assessment of spatial reasoning embedded in games can motivate students who might not be on the STEM tracks, thus broadening participation in STEM.
Game‐based learning and assessment researchers should consider possible factors that affect how certain populations of students enjoy educational games, so it does not further marginalize specific student populations.
... On the other hand, based on the research on 3437 young language learners of English, Spanish, German, Italian and French, Suzanne Graham found that there is little relationship with population characteristics and learning performance (2019). [29] So, Population differences could be ignored here. The learning materials provided by the project mainly included videos of lecture series, teaching cases, and tutorials of application software according to the curriculum standards of training. ...
Engagement is an important factor influencing the effectiveness and quality of online learning programs and how satisfied online learners are with the online learning experience. Therefore, when developing online learning programs, ways to increase engagement should be one of the top priorities. To determine the parameters that increase engagement of independent online learners, this paper selects three types of quantitative data by which to assess online learning engagement: video playing options, video lecture viewing time, and concurrent learning behaviors. This data, collected from real-time observation and analysis of authentic online learning of 14,000 adult learners, was used to run a real-time growth algorithm that determined a few key parameters that increase independent-learning engagement. A few primary results of this study are: (1) real-time, dynamically-calculated data representing general or individual engagement parameters of independent learning can help both teachers and learners be aware of, recognize, and adjust learning status accurately and effectively; (2) the algorithm can identify optimal parameters for online learning by analyzing numerical values of engagement, rules, and characteristics of online learning that are difficult to be observed directly; and (3) it is necessary to dynamically analyze learners’ engagement based on their learning processes and behaviors in response to different video lectures to know the effectiveness and feasibility of such materials and to support the design, production, and modification of optimal learning materials.
Knjiga pruža sveobuhvatan pregled suvremenih znanstvenih spoznaja o vrednovanju jezične kompetencije učenika rane školske dobi (od šest do jedanaest godina). Polazeći od razvojnih obilježja djece srednjega djetinjstva, sustavno se analiziraju temeljna načela poučavanja stranoga jezika u ranoj školskoj dobi te različite vrste vrednovanja s posebnim naglaskom na razvojno primjeren pristup. Središnji dio knjige posvećen je vrednovanju četiriju temeljnih jezičnih vještina – slušanja, govorenja, čitanja i pisanja – pri čemu se svako poglavlje temelji na relevantnome teorijskom okviru te nudi konkretne tehnike i primjere zadataka prilagođene mlađim učenicima. Posebna se pažnja posvećuje emocionalnoj sigurnosti učenika, važnosti formativnoga vrednovanja, uporabi digitalnih alata i poticanju samovrednovanja i vršnjačkoga vrednovanja. Knjiga također obrađuje teme povratne informacije i ispravljanja pogrešaka. Namijenjena znanstvenicima i nastavnicima visokih učilišta koji obrazuju buduće učitelje koji će predavati strani jezik od prvoga do četvrtoga razreda osnovne škole, studentima koji se obrazuju za učitelje i konačno samim učiteljima, knjiga služi kao teorijska podloga za oblikovanje pravednoga, poticajnoga i učinkovitoga sustava vrednovanja u nastavi stranoga jezika u nižim razredima osnovne škole.
This mixed-methods study investigates the impact of digital game-based learning (DGBL) on enjoyment, ideal L2 self, and intrinsic motivation among Chinese English as a Foreign Language (EFL) learners. Seventy participants were divided into DGBL and control groups. The DGBL group engaged in Duolingo activities, while the control group received traditional EFL instruction. Data collection included pre-post self-determination theory (SDT) questionnaires, experience sampling method (ESM) to assess real-time enjoyment, and stimulated recall interviews. Quantitative analysis using paired-samples t-tests, one-way ANCOVAs, and multilevel modeling revealed that DGBL significantly enhanced enjoyment and ideal L2 self-perception, with pre-existing autonomy and ideal L2 self predicting greater enjoyment during gameplay. Qualitative findings highlighted increased engagement, perceived learning gains, and a sense of autonomy fostered by DGBL. Importantly, this study demonstrates that integrating DGBL into EFL classrooms can significantly boost enjoyment and cultivate a positive self-concept as language learners among Chinese students. These findings have practical implications for educators, suggesting that incorporating well-designed game-based activities can create a more motivating and effective learning environment, addressing the specific challenges faced by Chinese EFL learners.
This study aims to provide a systematic review of existing research on the use of digital game-based learning (DGBL) technology for foreign language (English) in preschool and elementary schools. Digital game-based technology enables learning to become more interactive and inspirational, thereby enhancing language acquisition. The authors present a DGBL classification schema with four main components, design principles, language content, pedagogical factors, and feedback mechanisms, as guidelines for designing and implementing DGBL in foreign language learning. This survey classifies and filters 110 published articles from January 2010 to April 2022 based on the proposed DGBL schema adhering to the insertion and omission criteria, from which 50 studies focus on preschool and primary school students between the ages of 2 and 10 years. The findings indicate that DGBL influences motivation, creativity, and problem-solving ability. However, the synthesized literature also reveals challenges and issues in designing and developing DGBL systems based on novel technologies. The findings are fundamental for exploring the game classification schema for designing and implementing game-based learning to augment the trends of DBGL in foreign language learning and identify the possible issues and solutions for DGBL to lessen health concerns and technology challenges. The authors recommend emerging technologies and software engineering processes as possible solutions for continuity in DGBL research by enlightening the significance of evolving technologies such as artificial intelligence, blockchain, deep learning, AR/VR, and DevOps.
Despite the growing interest in utilizing commercial off-the-shelf (COTS) games for instructional and assessment purposes there is a lack of research evidence regard- ing COTS games for these applications. This chapter considers the application of COTS games for instruction and assessment and provides preliminary evidence com- paring COTS game scores to traditional multiple-choice assessments. In a series of four studies, we collected data and compared results from the performance in a COTS game to scores on a traditional multiple-choice assessment written for the purposes of each study. Each assessment was written to evaluate the same content presented in the game for each respective study. Three of the four studies demonstrated a significant correlation between the COTS game and the traditional multiple choice assessment scores. The non-significant value in Study 4 was likely due to a small sample size
(n < 100). The results of these studies support our hypothesis and demonstrate that COTS games may be a useful educational tool for training or assessment purposes. We recommend that future research focuses on specific applications of COTS games to explore further opportunities for utilizing COTS in education and assessment.
Given the exponential growth in the popularity of early foreign language programs, coupled with an emphasis of evidence-based instruction, assessing young learners’ (YLs) foreign language abilities has moved to center stage. This article canvasses how the field of assessing young learners of foreign languages has evolved over the past two decades. The review offers insights into how and why the field has developed, how constructs have been defined and operationalized, what language proficiency frameworks have been used, why children were assessed, what aspects of their foreign language proficiency have been assessed, who was involved in the assessment, and how the results have been used. By surveying trends in foreign language (FL) and content-based language learning programs involving children between the ages of 3 and 14, the article highlights research into assessment of and for learning, and critically discusses areas such as large-scale assessments and proficiency examinations, comparative and experimental studies, the impact of assessment, teachers’ beliefs and assessment practices, young learners’ test-taking strategies, age-appropriate tasks, alternative and technology-mediated assessment, as well as game-based assessments. The final section of the article highlights where more research is needed, thus outlining potential future directions for the field.
