Journal Of Computer Assisted Learning

Published by Wiley
Online ISSN: 1365-2729
Discipline: Education
Learn more about this page
Aims and scope

The Journal of Computer Assisted Learning is an international peer-reviewed journal which covers the whole range of uses of information and communication technology to support learning and knowledge exchange. In other words, to be considered for publication, the research submitted to the journal must include the measurement of actual learning. Though research dealing with learners’ judgements of their own learning, teacher judgements of student learning, effects of interventions on motivation, and so forth may be of interest to the scientific community at large, JCAL has chosen to use the measurement of actual learning as a primary, but not sole, inclusion principle. In this way, the journal aims to provide a medium for communication among researchers as well as a channel linking researchers, practitioners, and policy makers.



Recent publications
The professional knowledge of pre‐service teachers is highly important for effective and successful teaching. In recent years, many research groups have been engaged in developing simulated classroom environments to capture especially the pedagogical knowledge (PK) of pre‐service teachers, neglecting the content‐related facets of professional knowledge such as pedagogical content knowledge (PCK). In the present study, we describe the development of a simulated classroom environment—the Simulated Classroom Biology (SCRBio)—and provide evidence regarding its validity to assess pre‐service biology teachers' action‐oriented PCK in the area of evolution. This study examined the evidence supporting the validity of using the SCRBio to investigate action‐oriented PCK of pre‐service biology teachers. The (1) evidence based on test content (expert ratings) and the (2) evidence based on relation to other variables (known‐groups comparison) was obtained. We tested the SCRBio with N = 76 German pre‐service biology teachers. Our results show the successfully operationalized PCK in the SCRBio through explicit allocation of specific misconceptions to each virtual student's answer and the valid measurement of pre‐service biology teachers' action‐oriented PCK. This results in a validated simulated classroom environment for pre‐service but also in‐service teachers. In the future, the SCRBio will be developed from an assessment instrument to a training tool to simulate explicit teaching situations. This allows to complement the predominantly theoretical components of university‐based teacher education with practice‐based simulated classroom environments. The professional knowledge of pre‐service teachers is a prerequisite for quality teaching in the future, as well as student achievement. Simulated classroom environments are already being used, primarily to examine and train the pedagogical knowledge of pre‐service teachers. The action‐oriented character of simulated classroom environments can bridge the theory‐practice gap of university‐based education programs. Description of an innovative simulated classroom environment—the Simulated Classroom Biology. Ability to capture several domains of professional knowledge in an action‐oriented setting. Validation of the Simulated Classroom Biology using various validity aspects. Implications of study findings for practitioners. Description of validation steps for developing simulated classroom environments. A validated simulated classroom environment is provided to capture various knowledge domains of professional knowledge in an actionable setting. The Simulated Classroom Biology can be adapted to new subjects and teaching topics by changing the content.
The theoretical model of this study
Model with standardized path coefficients (***p < 0.001)
Background Self‐regulated learning (SRL) ability is the key determinant of the success of full‐time online learning. Thus, exploring the influencing factors of SRL and their influencing mechanisms is necessary to improve this ability among K‐12 students. Objectives The purpose of this study was to investigate the influence mechanism of teacher autonomy support on students' online SRL by examining the structural relationship among teacher autonomy support, parental autonomy support, students' self‐efficacy, and students' online SRL. Methods We use structural equation modelling and effect analysis to analyse the collected data from 961 Chinese K‐12 students who engaged in full‐time online learning in their homes during the coronavirus disease 2019 (COVID‐19) outbreak. Results and Conclusions Parental autonomy support and students' self‐efficacy play crucial independent mediating roles in the influence of teacher autonomy support on students' online SRL. Parental autonomy support and students' self‐efficacy have a chain mediating effect on the influence of teacher autonomy support on students' online SRL. Implications On the basis of the results, we suggest that in order to develop students' online SRL ability, it is important for teacher to improve parental autonomy support and students' self‐efficacy. In addition, base on the chain mediating effect, to improve students' online SRL, teacher autonomy support needs focus on parental autonomy support, and then parental autonomy support needs focus on improving students' self‐efficacy.
Background Learners in Massive Open Online Courses (MOOCs) are presented with great autonomy over their learning process. Learners must engage in self‐regulated learning (SRL) to handle this autonomy. It is assumed that learners' SRL, through monitoring and control, influences learners' behaviour within the MOOC environment (e.g., watching videos). The exact relationship between SRL and learner behaviour has however not been investigated. Objectives We explored whether differences in SRL are related to differences in learner behaviour in a MOOC. As insight in this relationship could improve our understanding of the influence of SRL on behaviour, could help explain the variety in online learner behaviour, and could be useful for the development of successful SRL support for learners. Methods MOOC learners were grouped based on their self‐reported SRL. Next, we used process mining to create process models of learners' activities. These process models were compared between groups of learners. Results and conclusions Four clusters emerged: average regulators, help seekers, self‐regulators, and weak regulators. Learners in all clusters closely followed the designed course structure. However, the process models also showed differences which could be linked to differences in the SRL scores between clusters. Takeaways The study shows that SRL may explain part of the variability in online learner behaviour. Implications for the design of SRL interventions include the necessity to integrate support for weak regulators in the course structure.
Illustration showing the difference between Bloom's original taxonomy and the revised version. Own elaboration based on Diagram 1.1, Wilson, Leslie O. 2001
Technology Acceptance Model (TAM)
Word clouds per dimension
Analysis of emotions based on the NRC dictionary by dimension
Analysis of feelings (positive and negative) based on the NRC dictionary per question
Background The development of students' digital skills is essential to access the labour market and interact with society, being especially important in disadvantaged socioeconomic contexts affected by the digital divide. Objectives The investigation has two objectives. On the one hand, to study the integration of ICT in disadvantaged contexts and, on the other, to know the critical use that is made of them in schools as a source of educational change. Methods In this research, eight international experts in education are interviewed on what they consider to be the best way of implementing new technologies in disadvantaged contexts to achieve educational improvement. A mixed methodology has been used, qualitative through semi‐structured interviews and quantitative through a text mining analysis of the interviews' transcripts with the R programming language. Results and Conclusions The results show that the teaching staff is the key element. The curriculum, objectives and school director's help are also highlighted, as well as the collaboration of the students who have to be provided with the necessary digital resources. At the state level, greater funding and support from universities are needed to support research and the quality of teacher training.
Lay Description What is already known about this topic The opportunities for using ICT in schools are developing rapidly, and with them, the demands on teachers' skills (TPACK; e.g., Becker et al., 2020; Eickelmann & Gerick, 2020; Redecker, 2017). Many studies measuring TPACK largely use measurement instruments that required students to self‐assess their TPACK (e.g., Handal et al., 2013; Kaplon‐Schilis & Lyublinskaya, 2019; Lee & Tsai, 2010; Willermark, 2017). Research on TPACK emphasizes that self‐testing TPACK via self‐assessment should be complemented by performance tests in order to better understand the domain‐specific nature of TPACK and its complexity (e.g., Akyuz, 2018; Graham et al., 2009; Voogt et al., 2013). What this paper adds The research provides information on the extent to which TPACK changes triggered by an intervention can be captured using self‐assessment scales and a self‐developed performance test. The data shows how consistent students' self‐assessments are compared to their demonstrated performance. Furthermore, we critically discuss to what extent scales developed to measure TPACK in scientific studies can also be applied to self‐directed learning. Implications for practice Prospective teachers must acquire TPACK competences for the implementation of contemporary teaching, which will enable them to produce digital teaching and learning materials in the future, for example. This TPACK is to be trained through self‐organized project work in the pedagogical Makerspace within the framework of university subject teaching. Self‐assessment instruments for TPACK can be more easily used in the field to accompany projects so that students can receive feedback on knowledge, learning process, and learning outcome. However, our study shows that existing self‐assessment instruments can be used mainly in self‐directed learning processes when learning gains on the complexity levels ‘remembering’ and ‘understanding’ are to be captured, even though feedback especially in the area of ‘applying’ TPACK is of great importance for teaching practice. Combining self‐assessments (based on work samples) and performance tests is therefore a promising approach, as the complementary use of performance tests provides insights into preservice teachers' abilities to ‘apply’ TPACK. The moderate correlation between self‐assessment and performance in our study also shows that self‐assessment is a skill that needs to be learned. Therefore, the use of self‐assessments for self‐regulated learning should be more accompanied by learning opportunities for learning self‐assessment (Bol et al., 2012).
The association of the internet epistemic beliefs with related outcomes in the reviewed studies
The experimental procedure
A screenshot for the individual note taking in multiple tabs (G1)
A screenshot for the collaborative note taking in multiple tabs (G2)
A screenshot for the individual notetaking in one tab (G3)
Background Processing and comprehending information from multiple sources have been a primary means of learning and are essential 21st‐century skills to construct knowledge for a deeper understanding. Objectives This study examined students' individual differences in search strategies and internet epistemic beliefs as well as the effect of distinct note‐taking formats on students' online multiple‐document reading about measurement scales and statistical graphs, considering students' gender and prior knowledge. Methods Participants were 124 Taiwanese university students (79% female) taking the Educational Statistics course in three different classes. I used a quasi‐experimental design to test the effect of different note‐taking formats on online multiple document reading. Three classes of students were randomly assigned to one of the experimental conditions with sourcing cues: the individual note taking with multiple tabs, the collaborative note taking with multiple tabs, or the individual note taking in a single tab. Results and conclusion The study showed that the uncertainty belief about Internet‐based information and note‐taking formats predicted students' online multiple document reading. A stronger belief that the internet information is uncertain and may change with time positively predict online multiple document reading comprehension. Moreover, individuals taking notes in a single tab outperformed those taking notes individually in separate tabs. Major takeaways from the study Implications for instruction were made to foster students' uncertainty belief and teach them how to construct pre‐outlined tables with sourcing cues to collect the contextual information and reflect on the source information to streamline learners' online multiple document reading comprehension.
Flow‐chart for the algorithms' decision process of providing feedback in the personalized refutation condition
Example of an item presentation in the Misconceptions about Multimedia Learning Questionnaire (used both as pretest and posttest). There was always a true‐false statement above, with the certainty rating beneath it
Graphic overview of the experimental procedure
Improvement from pretest to posttest (conceptual change) scores as a function of the experimental condition
Graphic mediation model with the hypothesis contrast as independent variable, ratings of guilt‐shame as mediator, and conceptual change scores as dependent variable
Lay Description What is already known about this topic? Teachers hold misconceptions about multimedia learning (e.g., learning materials should be adapted to students' individual learning styles, such as visualizers or verbalizers). Refutation texts, naming a commonly held misconception, disproving it and introducing a scientific explanation, are a common means to reduce misconceptions. Personalization fosters learning by drawing the learner's attention toward the discrepancy between their own beliefs and the learning material, further creating an impasse experience. Said impasse experience may trigger teachers' conceptual change, as, for teachers' conceptual change, a certain degree of discomfort is required. Yet, anger, caused by lessoning teachers on their topic may cause repulse and hamper learning. What this paper adds? With a computer algorithm, we can efficiently personalize refutation texts by automatically matching them to teachers' answers in a pre‐test. Such a personalized refutation instruction may especially foster conceptual change. Within a randomized experiment, the personalized refutation instruction worked best compared to common refutation texts and expository texts. Feelings of guilt and shame moderated the effect of a personalized refutation, as teachers felt more addressed in their misconceptions and thus experienced the required impasse experience. Feelings of anger did not play an important role within our experiment. The implications of study findings for practitioners Computer algorithms enable efficient personalization of instruction to better deal with heterogeneous groups of learners (e.g., with big differences in prior knowledge or experience, such as in the case of in‐service teachers). Refutation texts work better for teachers when they are personalized. Common refutation texts do not work better than expository texts. An advantage of digital instruction is the use of algorithms to efficiently personalize instructions even for larger groups.
Screenshot of a participant with a head‐mounted display (left) and with a flat screen device (right)
Screenshots of a virtual patient VR content
Screenshots of a mural town VR content
Interaction effect between perspective‐taking and immersion levels on situational empathy for the first viewing
Background Immersive VR is still rarely used as an intervention for meeting the affective end goals of student learning despite its positive impact on affection. Also, studies regarding the use of immersive VR as an intervention for affective achievement in broader educational contexts are still lacking. Objectives: This study aimed to examine the effect of immersive VR and perspective‐taking on presence and empathy. Methods A total of 148 pre‐service teachers participated in experiments, using either a head‐mounted display or a flat screen device to view two VR videos with different perspective‐taking affordances. This study used a mixed design with one between‐subject variable of immersion level and one within‐subject variable of perspective‐taking to explore how immersive VR experiences influenced participants’ perceived level of presence and empathy. Results and Conclusions The results showed that the level of immersion affects perceived presence, but it was the type of perspective‐taking that affects empathetic reactions. We also found an interaction effect between immersion levels and perspective‐taking. The direct embodiment in VR combined with high immersion produced stronger empathy than with low immersion, while the perspective of an observer was better in evoking empathy when experienced with low immersion. Implications This study gives a guidance on how to take advantage of this new technology in educational settings, and apply it to instructional activities to enhance students’ empathy. In addition, it could serve as a reference when developing or introducing educational contents with respect to the types of contents that are more effective in educational settings.
Background Guidance has showed positive effects on promoting inquiry‐based learning in science education. While an increasing number of studies focus on the design of guidance in simulation‐based inquiry learning due to recent technology developments, how different designs of a same type of guidance affect learning remains a question. Objectives This study reviews the (quasi‐)experimental research on the learning effects of differently designed guidance in simulation‐based inquiry learning in the past decade (2011–2020). The investigation is guided by two questions: how differently designed guidance affects inquiry‐based learning in simulated environments in terms of learning process and learning outcomes, and what role technology plays in the design of guidance. In particular, we report on studies that compares the effects of a same type (process constraints/prompts/heuristics/scaffolds/metacognitive supports/direct presentation of information) of guidance with different instructional designs, and select a total of 28 peer‐reviewed journal articles. The results indicate no equivocal tendency of effectiveness towards a specific design. Instead, each type of guidance has related factors that may influence its effectiveness. Three major factors that related to optimization of inquiry learning guidance in simulation‐based environments are identified: the learner factor, the pedagogical factor, and the technological factor.
Screenshots of the introductory video
Screenshot of the photosynthesis lab. Copyright (2018) by ExploreLearning. Reprinted with permission
Performance on conceptual knowledge as a function of low and high domain‐specific self‐concept and experimental condition
Background Engaging students in computer‐assisted guided inquiry learning has great potential to scaffold their scientific understanding: Students are expected to improve their scientific problem‐solving skills, and at the same time gain a deep conceptual understanding of the subject‐matter. Additional generative activities such as creating video explanations subsequent to inquiry learning activities can also further deepen students' knowledge. Objectives In this experiment, we therefore compared the effectiveness of computer‐based direct instruction (to mimic traditional classroom teaching) versus computer‐based inquiry learning. Methods University students (N = 118) either received video‐based direct instruction (direct instruction), including the demonstration of a virtual experiment, or conducted the virtual experiment themselves supported by prompts (inquiry learning). A third group of students additionally generated a video explanation as consolidation activity subsequent to conducting the virtual experiment (inquiry learning + generative activity). Results and Conclusions Contrarily to our hypotheses, the direct instruction condition outperformed the inquiry learning conditions. There were no significant differences between the inquiry learning and the inquiry learning + generative activity condition. Moderation analyses revealed that the effectiveness of direct instruction predominantly held true for students with low levels of domain‐specific self‐concept. All in all, our present study contributes to a better understanding of effects of direct instruction versus guided inquiry learning in computer‐based science education settings. Importantly, our findings show that the effectiveness of instructional approaches may depend on students' domain‐specific self‐concept as a motivational prerequisite. As such it is up for further research in science education to identify motivating instructional strategies to enhance students' learning.
Background Digital‐first assessments leverage the affordances of technology in all elements of the assessment process: from design and development to score reporting and evaluation to create test taker‐centric assessments. Objectives The goal of this paper is to describe the engineering, machine learning, and psychometric processes and technologies of a test security framework (part of a larger ecosystem; Burstein et al., 2021) that can be used to create systems that protect the integrity of test scores. Methods We use the Duolingo English Test to exemplify the processes and technologies that are presented. This includes methods for actively detecting and deterring malicious behaviour (e.g., a custom desktop app). It also includes methods for passively detecting and deterring malicious behaviour (e.g., a large item bank created through automatic generation methods). We describe the certification process that each test administration undergoes, which includes both automated and human review. Additionally, we describe our quality assurance dashboard which leverages psychometric data mining techniques to monitor test quality and inform decisions about item pool maintenance. Results and Conclusions As assessment developers transition to online delivery and to a design approach that places the test taker at the centre, it becomes increasingly important to take advantage of the tools and methodological advances in different fields (e.g., engineering, machine learning, psychometrics). These tools and methods are essential to maintaining the security of assessments so that the score reliability is sustained and the interpretations and uses of test scores remain valid.
Illustration of pilot study research design
Screenshots from the virtual reality safety simulation: The left panel shows the introduction phase; the middle panel shows the trainees view of the pier; the right panel shows a scenario where an accident occurred because the trainee failed to warn a crewmember of danger
Illustration of main study research design
Lay Description What is already known about this topic Recent meta‐analyses have found a small effect size benefit to using immersive virtual reality (IVR) learning interventions compared to than less‐immersive learning approaches. Recent reviews call for more IVR‐based research that is integrated within actual learning and training interventions. Furthermore, as with most other fields, it is common to use Western, Educated, Industrialized, Rich, and Democratic (WEIRD) samples for IVR educational and training research purposes. What this paper adds Evidence that an IVR training intervention can be successfully used in an international maritime training programme with employees from an understudied non‐Western population on UN's list of the 50 least developed countries. The finding that IVR‐based safety training on the topic of dynamic risk assessment during a mooring operation resulted in significantly higher levels of enjoyment, motivation, perceived learning and behavioural change intentions and significantly lower extraneous cognitive load compared to personal trainer instruction. The finding that IVR safety simulation with embedded reflection performed just as well as when an extra post‐simulation exercise was added to the training intervention. Implications for practice and/or policy Organizations can benefit from using IVR‐based safety training methods to increase engagement in dynamic risk assessment. IVR technology can be used broadly with employees from developing countries who have a low level of technology literacy. IVR simulations are effective when they are designed based on state of the art instructional design research and are integrated appropriately into a more general training framework.
Screenshot from a dynamic gaze display. The gaze location is shown in red with a two‐second trail. Circles denote fixations (i.e. moments when the eye is relatively still and takes in information) with the size of the fixation reflecting its duration. Lines denote jumps between fixations (saccades). The text was translated from Dutch
Two screenshots from the instruction video. (a) Taken from the first half of the instruction video. On this slide, an explanation is given about the differences between a brook valley and a ridge, since they are visually quite similar. (b) Taken from the worked example. On this slide, an example is given on how to choose the right direction after knowing your location. Text translated from Dutch
Example task from the post‐test. Text translated from Dutch. This map had the highest average score (M = 0.96), the correct answer to this map is ‘Right’
Violin plots with embedded boxplots for variables bias (before and after the review phase) and absolute accuracy (before and after review phase)
Background Performance monitoring plays a key role in self‐regulated learning, but is difficult, especially for complex visual tasks such as navigational map reading. Gaze displays (i.e. visualizations of participants' eye movements during a task) might serve as feedback to improve students' performance monitoring. Objectives We hypothesized that participants who review their performance based on screen recordings that also display their gaze would have a higher monitoring accuracy and increase in post‐test performance and would remember more executed actions than participants who review based on a screen recording only (i.e. control condition). Methods Sixty‐four higher education students were randomly assigned to a gaze‐display or control condition. After watching an instruction video, they practiced five navigational map‐reading tasks and then reviewed their performance while thinking aloud, either prompted by a screen recording with gaze display or a screen recording only. Before and after reviewing, participants estimated the number of correctly solved tasks and finally made a five‐item post‐test. Results and conclusions Analyses with frequentist and Bayesian statistics showed that gaze displays did not improve monitoring accuracy (i.e. estimated minus actual performance), post‐test performance, or the number of reported actions. It is concluded that scanpath gaze displays do not provide useful cues to improve monitoring accuracy in this task. Takeaways Gaze displays are a promising tool for education, but scanpath gaze displays did not help to enhance monitoring accuracy in a navigational map‐reading task.
Flow diagram of literature search and study inclusion criteria
Forest plot of the 39 studies (PAEs) included in the meta‐analysis
Forest plot of the 22 studies (NAEs) included in the meta‐analysis
Funnel plot of the 39 studies (PAEs) included in the meta‐analysis
Funnel plot of the 22 studies (NAEs) included in the meta‐analysis
Background Researchers debate whether game-based learning (GBL) affects students' achievement emotions, as past studies yielded mixed results. Objectives This study determines the overall effect of GBL on students' positive achievement emotion and their negative achievement emotions (PAEs, NAEs), along with their moderators. Methods This meta-analysis examined true or quasi-experimental studies, reported in English or Chinese, and with students (ranging from primary schools to universities) who did not report physiological and/or psychological illnesses. Results Compared to traditional instruction, GBL had a positive effect on students' PAEs (g =0.526, k = 39, 95% CI = [0.319, 0.733]) and a negative effect on NAEs (g = −0.517, k = 22, 95% CI = [−0.709, −0.324]). GBL's positive effects on students' PAEs were (a) largest in middle school and otherwise generally larger in higher grades, and (b) largest for competition GBLs (compared to cooperative or individual GBLs). GBL's negative effects on students' NAEs were (a) larger in collectivist cultures than individualistic ones, (b) progressively larger in primary school, middle school, and university, (c) larger for non-digital games than digital games, (d) smallest for individual games, (e) progressively larger in mathematics, social sciences, language, science, and highest in engineering and technology, and (f) smallest for extremely short or extremely long intervention durations (very short: up to 45 min; very long: over 2 months). Take Away GBL often increases students' PAEs and decreases their NAEs, though their effect sizes differ across cultural values, grade level, game type, academic subject, or intervention duration. Hence, educators should consider these aspects when designing / selecting education games for their students and instructional contexts.
The competition field
Some pictures from the study
Behaviour categories and number of occurrences of behaviours for all groups
Differences in behavioural patterns between the lower and higher‐achievement groups
Background Educational robotics (ER) is a means of teaching technology and engineering to students that offers an active learning environment by encouraging them to create meaningful and unique products. ER also gives students the opportunity to work collaboratively. Objectives In this study, elementary school students' behavioral patterns were explored while they were working on a collaborative robotics project. Additionally, behavioral patterns of higher‐achievement and lower‐achievement groups were compared. Methods The participants of the study were a total of 18 students (aged 10–12), including 17 males and 1 female. A problem‐based robotics competition was designed within the scope of the study. The students were asked to design and program a robot based on certain rules in groups within four weeks for this competition. All robot design processes and the competition were video‐recorded. Quantitative content analysis and lag‐sequential analysis methods were used to analyze students' behaviors. Results and Conclusions The results showed that ER improves collaborative learning, and behaviors in the contributing and planning categories were the dominant behaviors during the robotics project development. Grouping skills of higher‐achievement groups were better. Based on the significant collaborative behavioral patterns that emerged from the study, implications were discussed in terms of theoretical insights and collaborative educational robotics practices.
Background Learning to code is increasingly embedded in secondary and higher education curricula, where solving programming exercises plays an important role in the learning process and in formative and summative assessment. Unfortunately, students admit that copying code from each other is a common practice and teachers indicate they rarely use plagiarism detection tools. Objectives We want to lower the barrier for teachers to detect plagiarism by introducing a new source code plagiarism detection tool (Dolos) that is powered by state‐of‐the art similarity detection algorithms, offers interactive visualizations, and uses generic parser models to support a broad range of programming languages. Methods Dolos is compared with state‐of‐the‐art plagiarism detection tools in a benchmark based on a standardized dataset. We describe our experience with integrating Dolos in a programming course with a strong focus on online learning and the impact of transitioning to remote assessment during the COVID‐19 pandemic. Results and Conclusions Dolos outperforms other plagiarism detection tools in detecting potential cases of plagiarism and is a valuable tool for preventing and detecting plagiarism in online learning environments. It is available under the permissive MIT open‐source license at https://dolos.ugent.be. Implications Dolos lowers barriers for teachers to discover, prove and prevent plagiarism in programming courses. This helps to enable a shift towards open and online learning and assessment environments, and opens up interesting avenues for more effective learning and better assessment.
Lay Description What is already known about this topic The e‐books have begun to be utilized in schools and homes all around the world for the last two decades. The literature suggested that the e‐book use support students' learning performance. There is little knowledge about the effects of the interactive e‐books based on effective educational theories. What this paper adds We investigated the impacts of the e‐book technology and type of feedback on the learning, motivation, and cognitive load of students within a multimedia‐learning environment Lecture videos, question solution videos, and interactive problem‐solving applications were designed and imported into the e‐book. Video feedback applications were designed and imported into the e‐book Implications for practice and/or policy The students generally believe that video feedback is more beneficial and more useful for their understanding of the topic The results suggest that students believe that interactive e‐book activities are valuable for them. The findings will contribute to understanding the cognitive and motivational influences of incorporating the interactive e‐book and video feedback into lessons, individually or together. What are the 1 or 2 major takeaways from the study? According to these results, it can be said that the interactive e‐book, when combined with video feedback, is an effective way both supporting learning achievement and also fostering students' intrinsic motivation. In addition, such a combination can contribute to the creation of one's own knowledge by reducing the load on the limited cognitive capacity of the individual. The implications of the educational practice are discussed, and the direction of future studies in this field are also addressed.
PRISMA flowchart for search protocol
Forest plots showing the distribution of effect sizes and 95% CI for language gains
Forest plots showing the distribution of effect sizes and 95% CI for motivation
Funnel plot for language gains
Funnel plot for motivation
Background As a recently emerging innovative technology, augmented reality (AR) has become a popular tool for language learning. However, to date, very few meta‐analytical studies has been conducted on AR in this field to understand its effectiveness on language learning. Objectives This meta‐analysis was conducted to systematically synthesize the findings from primary studies published between 2008 and 2020 to establish the effects of AR on language learning gains and students' motivation. Methods The 21 studies met all the inclusion criteria were included in the meta‐analysis to extract effect size statistics. The robust variance estimation (RVE) technique using the “Robumeta” R‐package was adopted to estimate the pooled effect size. Given the heterogeneity of the effect sizes, a mixed‐effects meta‐regression model was estimated to examine any association between the effectiveness of AR technologies and moderator variables. Results and conclusion The pooled effect‐size estimate was 0.93 for language gains and 0.42 for motivation, which indicates that AR applications have a large effect on learners' language gains and a small to medium effect on learners' motivation. The moderator analysis results suggested that learners' educational levels and intervention durations are significant moderating factors that impact the effect of AR on learners' motivation. In particular, elementary school students in this meta‐analysis study experienced a large measurable effect in terms of both language gains and motivation. Additionally, exposure to AR applications for up to 1 week is especially effective for enhancing learners' motivation. Major takeaways The findings of this meta‐analysis study demonstrated how AR applications can be utilized in language teaching and learning contexts, and how language educators could adopt AR technologies in classrooms to promote learners' language gains and motivation.
Three steps of peer feedback activity
An example of peer feedback guidelines
An example of peer feedback exchanges on MosoTeach
Average length of PF and CF across the three motivation groups. CF, corrective feedback; PF, positive feedback
Proportions of CF and PF across the three motivation groups. CF, corrective feedback; PF, positive feedback
Lay Description What is already known about this topic Previous research has recognized mobile‐assisted peer feedback as an effective way of mobile collaborative language learning. Previous studies on mobile‐assisted oral peer feedback have examined different dimensions of feedback in relation to their effects on speaking performance. Previous research is less clear about how learners' motivational characteristics may influence peer feedback performance. What this paper adds Situated in a mobile learning context, this study investigated how English as a Foreign Language (EFL) students with different motivational levels participated in mobile‐assisted peer feedback provision, and examined the characteristics of the feedback they provided. Results revealed that EFL students with higher motivation were more likely to provide feedback on time using more words. Some unique characteristics of mobile‐assisted peer feedback among students with different motivational levels were also observed. Implications This study demonstrates the importance of considering learners' motivational levels in teachers' design and implementation of peer feedback with technology. It is crucial for teachers to provide training and scaffolding to students in their construction of peer feedback with mobile technologies. Teachers are suggested to encourage the students to provide detailed positive comments for their peers, which may increase students' confidence and reduce their anxiety.
Background Online game communities are inhabited by millions of players daily with relevant learning opportunities and dynamics. However, little efforts have been done for exploring their potential. Objectives The goal of this study is to explore if and how social learning processes are occurring in online game communities. Methods n = 480 adult players completed a questionnaire including demographics, game community habits, and the game community of inquiry scale. Results were analysed by referring to social learning theory's processes and factors. Results and Conclusions Online game communities provide relevant instances of social learning, with an emphasis on soft skills and application to off‐game contexts. Moreover, the more a community is inclusive and supporting, the more individuals learn and benefit from it from a cognitive perspective. Major takeaways This study provides a wide overview of the processes through which players observe, filter, and reproduce behaviours from their game communities of reference. Moreover, findings point at which malleable factors should be considered for facilitate this social learning dynamic, which are active participation, community receptiveness, and being aware of the learning process itself.
Backgroud uring the COVID‐19 pandemic, online learning has played an increasingly crucial role in the educational system. Academic dishonesty (AD) in online learning is a challenging problem that represents a complex psychological and social phenomenon for learners. However, there is a lack of comprehensive and systematic reviews of AD in online learning environments. Objectives This study presents a systematic study of AD in online learning environments to delineate its trends and uncover potential areas for further research. Methods We conducted this review based on various sources of evidence‐based research and followed the guidelines of the PRISMA statement and procedure for selection. After the exclusion criteria were employed, 59 eligible articles were selected and then analysed in a descriptive overview. Two frameworks were identified in the structured content analysis to analyse these articles. One was the framework of Gilbert's Behaviour Engineering Model (BEM), and the other was the types of interventions for online AD, where 36 articles were analysed. Results and Conclusions The descriptive results showed that most studies used quantitative methods and focused on students. The analysis results of influencing factors under the BEM framework showed that the category of environment support and tools accounts for the largest proportion. And the types of interventions for online AD we classified include individual AD & high technological complexity, individual AD & low technological complexity, collective AD & high technological complexity, and collective AD & low technological complexity. These findings provide a comprehensive understanding and guidance of AD in the online environment for relevant managers, designers and developers.
A screenshot of an eye movement modelling example depicting the mastermind task. The blue circle represents the gaze location of the model, in this case inspecting the feedback of the third code‐breaking attempt. On top the three rows with the flower codes along with the corresponding feedback on the right are displayed. Underneath the codes the answer selection pane with the different possible flower types are displayed
Background Eye movement modelling examples (EMME) are demonstrations in which learners' not only see a model's (e.g., a teacher's) task performance on a computer screen (as in regular video examples) but also the model's eye movements (represented as moving coloured dots overlaid on the screen). Thereby EMME help guide learners' attention towards the relevant information and can model cognitive strategies which are otherwise unobservable for learners. Objectives This study investigated whether EMME can help to learn deductive reasoning strategies and how the presence/absence of a teacher's verbal explanation affects learning from EMME. Methods Secondary education students (N = 137) were randomly assigned to study video examples under one of four conditions in a 2 (EMME: yes/no) x 2 (verbal explanations: yes/no) between‐subjects design. Results and Conclusions Results revealed only a beneficial effect of the presence of verbal explanations on performance on the practice problems, but no pretest‐to‐posttest learning gains. Implications Seeing the teacher's eye movements does not appear to enhance learning of deductive reasoning. The presence/absence of the teacher's verbal explanation does not seem to affect learning deductive reasoning.
Lay Description What is already known about this topic? Learning design (LD) is the pedagogic process used in teaching/learning that leads to the creation and sequencing of learning activities and the environment in which it occurs. Learning analytics (LA) is the measurement, collection, analysis & reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs. There are multiple studies on the alignment of LA and LD but research shows that there is still room for improvement. What this paper adds? To achieve better alignment between LD and LA. We address this aim by proposing a framework, where we connect the LA indicators with the activity outcomes from the LD. To demonstrate how learning events/objectives and learning activities are associated with LA indicators and how an indicator is formed/created by (several) LA metrics. We address this aim in our review. This article also aims to assist the LA research community in the identification of commonly used concepts and terminologies; what to measure, and how to measure. Implications for practice and/or policy This article can help course designers, teachers, students, and educational researchers to get a better understanding on the application of LA. This study can further help LA researchers to connect their research with LD.
Background Data‐driven educational technology solutions have the potential to support teachers in different tasks, such as the designing and orchestration of collaborative learning activities. When designing, such solutions can improve teacher understanding of how learning designs impact student learning and behaviour; and guide them to refine and redesign future learning designs. When orchestrating educational scenarios, data‐driven solutions can support teacher awareness of learner participation and progress and enhance real time classroom management. Objectives The use of learning analytics (LA) can be considered a suitable approach to tackle both problems. However, it is unclear if the same LA indicators are able to satisfactorily support both the designing and orchestration of activities. This study aims to investigate the use of the same LA indicators for supporting multiple teacher tasks, that is, design, redesign and orchestration, as a gap in the existing literature that requires further exploration. Methods In this study, first we refer to the previous work to study the use of different LA to support both tasks. Then we analyse the nature of the two tasks focusing on a case study that uses the same collaborative learning tool with LA to support both tasks. Implications The study findings led to derive design considerations on LA support for teachers’ design and orchestrating tasks.
Illustration of the Annoto hyper‐video interface. Source: Taken from https://www.annoto.net/
Background Hyper‐video technology allows reflection on learning materials by writing personal notes and by interactions with lecturers and peers through shared posts and replies. While research shows that integrating hyper‐videos in educational systems can promote the learning processes and outcomes, an open question remains regarding its actual utilization patterns by students and teachers. Objectives The study aimed to reveal the extent to which students utilize different functions of hyper‐videos in real‐life learning contexts, and to examine the role of instructional design on hyper‐video utilization. Method In a mixed‐method design, the study characterized active and passive interactions in hyper‐videos in higher education courses, and amongst active interactions, compared shared versus private annotations. Additionally, the study compared voluntary and mandatory instructional design for promoting hyper‐video interactions. In particular, we used learning analytics to explore hyper‐video patterns in 25 undergraduate and graduate courses. Results and Discussion A log analysis revealed that students wrote most of the posts, while lecturers mainly provided replies to students' posts. Private annotations were more prevalent than shared writing, pointing to the unfulfilled potential of hyper‐video annotations to benefit from collaborative learning. A qualitative analysis on instructional design was consistent with quantitative findings, revealing important differences between the courses with mandatory versus voluntary hyper‐video integration in pedagogical design, and in cognitive presence and teaching presence, based on the community of inquiry framework. Major takeaways Findings point to an unfulfilled potential of hyper‐video technology to promote active and collaborative learning. This highlights the role of design and instructive guidance when integrating technology into educational systems.
Proposed structural research framework
Background Blended online courses, which combine synchronous and asynchronous online activities, have expanded rapidly in higher education. How to enhance student engagement in such courses is unclear, although it is recognized that student engagement is malleable through instructional strategies. Objectives Given the above, this study aims to examine the influence of categories of strategies on student engagement in blended online courses. Methods A conceptual framework of instructional strategies indicated as fostering student engagement in the relevant literature was first presented, divided in eight categories (structure, pace, relevance, active, choice, relationships, explanations, guide). Then a research framework linking the categories of strategies to student engagement dimensions (emotional‐cognitive, social, behavioral) was built and tested in blended online courses. Data collected in various disciplines and university levels at four universities (n = 482) were examined using partial least squares structural equation modeling. Results and Conclusions The structural model examination confirmed the combined effects of categories of instructional strategies on student engagement in such courses in all disciplines. Particularly, this study revealed that 1) establishing trusting relationships, 2) demonstrating the relevance of activities, content, and resources, and 3) maintaining a sustained course pace significantly impacted student engagement in blended online courses in all disciplines. Takeaways This study draws upon the blended learning literature to bring together key instructional strategies that foster student engagement while highlighting empirical quantitative evidence of their effects on student engagement in blended online courses. Detailed measures of categories of instructional strategies and student engagement dimensions also provide reliable instruments for future research.
The detection workflow for moving from a set of student submissions to a set of submissions suspected of possessing plagiarized content
An example TurnItIn report. An example of output from TurnItIn using an essay deliberately constructed to contain plagiarized content
An example MOSS report. An example of output from MOSS using a code segment deliberately constructed to contain plagiarized content
The categorization workflow for moving from a set of suspicious submissions to one of four potential next steps
The intervention workflow for intervening with a student suspected of misconduct and closing out the case
Background Plagiarism is a very serious offence in academic institutions. Yet there is some reluctance to address plagiarism by educators as its enforcement can require a significant time commitment if not handled wisely. Handling plagiarism at scale has the potential to exacerbate this problem. Objectives This article explores the challenges educators face when it comes to enforcing plagiarism from within the course environment, presents a solution that addresses these challenges, and provides guidelines for preventing and detecting plagiarism in our at‐scale courses. Methods A workflow is shared which was created from our experience of handling plagiarism within our large online computer science graduate level program. We present an empirical study of this workflow that shows the overall prevalence of misconduct. Results and Conclusions Results demonstrate this workflow being effectively and efficiently applied to address plagiarism across four courses and three semesters by a single individual working half time. We have observed that warnings on low‐stakes assignments can be very effective at deterring future misconduct. We have also observed that cases come to a quicker resolution when students readily acknowledge their own misconduct after being promptly notified. Takeaways from the study Plagiarism should not only be managed through institution wide policies, but also through effective strategies implemented in the course environment. This is particularly important when handling plagiarism at scale. Addressing plagiarism starts in the course and therefore effective strategies for handling it there are needed.
PRISMA flow chart of events (Liberati et al., 2009)
Background The Covid‐19 pandemic disrupted higher education in many ways, such as the move to Emergency Remote Online Teaching and Learning (EROTL), often including a move to online assessments and examinations. With evidence of increased academic dishonesty in unproctored online assessment, institutions sought ways to ensure academic and institutional integrity and reputation. In doing this, many institutions selected and implemented online proctoring solutions. Objectives This article maps considerations of online proctoring solutions in the nexus between ensuring academic and institutional integrity and reputation, and addressing stakeholder concerns regarding invasive surveillance and the impacts on student privacy. Methods The study involved a PRISMA‐informed systematic review of three digital libraries, namely Clarivate's Web of Science, Elsevier's Scopus, and Springer's SpringerLink, for peer‐reviewed journal articles and conference proceedings. After screening, a final corpus of 27 articles was analysed. Results and Conclusions The findings include evidence that, in the midst of the Covid‐19 pandemic, higher education institutions were largely influenced by cost, usability and efficiency in choosing online proctoring solutions to ensure academic and institutional integrity. Student privacy was either considered in terms of data protection and transparency, or not at all. This article aims to provide valuable insight into the criteria used to select online proctoring solutions to ensure academic and institutional integrity in online examination environments. Student privacy appears not to have the consideration it warrants.
Background As a result of the COVID‐19 pandemic, many teachers found themselves making a rapid and often challenging shift from in‐person classroom teaching to teaching in an online environment. As teachers continue to learn about working in this new environment, research in cognitive and learning sciences, specifically findings from cognitive load theory and related areas, can provide meaningful strategies for teaching in this ‘new normal’. Objectives This paper describes 12 tips derived from contemporary research in educational psychology, focusing particularly on empirically supported strategies that teachers may apply in their online classroom to ensure that learning is optimized. Implications for Practice These strategies are generalizable across age groups and learning areas, and are categorized into one of two themes: approaches to optimize the design of online learning materials, and instructional strategies to support student learning. A discussion follows, outlining how teachers may apply these strategies in different contexts, with a brief overview of emerging efforts that aim to bridge cognitive load theory and self‐regulated learning research.
Flowchart of the included studies describing AD and TA in online learning. AD, academic dishonesty; TA, trustworthy assessment
Number of studies based on the subject keywords
Descriptive statistical analysis of literature collections based on (a) publication year (2017–2021), (b) the subject keywords to review. AH, arts and humanities; BMA, business, management and accounting; Che, chemistry; CS, computer science; Ene, energy; Eng, engineering; ES, environmental science; med, medicine; Psy, psychology; SS, social sciences
Background Academic dishonesty (AD) and trustworthy assessment (TA) are fundamental issues in the context of an online assessment. However, little systematic work currently exists on how researchers have explored AD and TA issues in online assessment practice. Objectives Hence, this research aimed at investigating the latest findings regarding AD forms, factors affecting AD and TA, and solutions to reduce AD and increase TA to maintain the quality of online assessment. Methods We reviewed 52 articles in Scopus and Web of Science databases from January 2017 to April 2021 using the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses model as a guideline to perform a systematic literature review that included three stages, namely planning, conducting, and reporting. Results and conclusions Our review found that there were different forms of AD among students in online learning namely plagiarism, cheating, collusion, and using jockeys. Individual factors such as being lazy to learn, lack of ability, and poor awareness as well as situational factors including the influence of friends, the pressure of the courses, and ease of access to information were strongly associated with AD. A technology‐based approach such as using plagiarism‐checking software, multi‐artificial intelligence (AI) in a learning management system, computer adaptive tests, and online proctoring as well as pedagogical‐based approaches, such as implementing a research ethics course programme, and a re‐design assessment form such as oral‐based and dynamic assessment to reduce cheating behaviour and also sociocultural and sociotechnical adjustment related to the online assessment are reported to reduce AD and increase TA. Implications Educators should adjust the design of online learning and assessment methods as soon as possible. The identified gaps point towards unexplored study on AI, machine learning, learning analytics tools, and related issues of AD and TA in K12 education could motivated future work in the field.
Background Artificial intelligence (AI) has gained increasing popularity in human society, and it is important to educate people about this emerging technology. Many countries have adopted school curricula to incorporate AI into their classrooms. However, developing tools for discovering AI concepts remains challenging. There are few studies on AI education tools, particularly in Thailand. Objectives This study designs AIThaiGen, a web‐based learning platform for junior high school students that introduces AI concepts. It can communicate with remote hardware stations, allowing students to test their AI models in real‐world scenarios. Methods A total of 106 students in 7th and 8th grade in Thailand participated, and a single‐group pre‐test‐post‐test research design was employed in this study. Pre‐post‐tests on the basic concepts of AI and the students' attitude questionnaire on AIThaiGen were used to collect and analyse data. Results and Conclusions The results show that there is a significant improvement (p < 0.001) in the pre‐post‐tests on the basic concepts of AI, and the overall result of the students' attitude questionnaire on AIThaiGen is X¯=3.88$$ \overline{X}=3.88 $$, indicating positive outlooks. Furthermore, notable student projects are showcased, highlighting their ability to initiate new ideas for solving real problems after studying with AIThaiGen.
The FA cycle and the five phases (permission to use from Gulikers & Baartman, 2017)
Introduction During the COVID‐19 lockdown in 2020, teachers had to shift their teaching and assessment to online. Formative assessment (FA) can help teachers to engage, guide and monitor students' (online) learning. However, more knowledge is needed of how teachers could use the full FA process online. Methods In this study data from 50 secondary school teachers, who taught different grade levels and subjects and joined a FA learning network that started before and continued during the lockdown, were collected. The study investigates how they used online FA practices differently than face‐to‐face FA, what challenges and opportunities they experienced in online FA and what lessons they learned and intended to keep. This mixed methods study used data from a questionnaire, interviews and webinars that were segmented, coded and analysed. Results Results showed that many teachers implemented new FA strategies and adopted, more often than in their face‐to‐face practice, all the five phases of the FA process in an aligned matter in online FA. Teachers indicated opportunities in stimulating student engagement and guiding and monitoring student learning more at an individual level in the online FA process, but also experienced challenges, mainly in lack of interaction online. Discussion The sudden and necessary shift to online FA, due to the COVID‐19 lockdown, challenged teachers to more fundamentally reconsider their assessment practices and assumptions. Teachers intended to make use of these learned lessons to improve their future (blended) FA practice.
Screenshots and structure of the unit Sports Club
Flow diagram of students' cluster membership across units. Box sizes are proportional to sample sizes within clusters; line thickness represents the sample size of students in boxes connected by the line; colours indicate students' cluster membership in the first unit
Background Computer‐based assessment allows for the monitoring of reader behaviour. The identification of patterns in this behaviour can provide insights that may be useful in informing educational interventions. Objectives Our study aims to explore what different patterns of reading activity exist, and investigates their interpretation and consistency across different task sets (units), countries, and languages. Three patterns were expected: on‐task, exploring and disengaged. Methods Using log data from the PISA 2012 digital reading assessment (9226 students from seven countries), we conducted hierarchical cluster analyses with typical process indicators of digital reading assessments. We identified different patterns and explored whether they remained consistent across different units. To validate the interpretation of the identified patterns, we examined their relationship to performance and student characteristics (gender, socio‐economic status, print reading skills). Results and Conclusions The results indicate a small number of transnational clusters, with unit‐specific differences. Cluster interpretation is supported by associations with student characteristics—for example, students with low print reading skills were more likely to show a disengaged pattern than proficient readers. Exploring behaviour tended to be exhibited only once across the three units: It occurred in the first unit for proficient readers and in later units for less skilled readers. Major Takeaways Behavioural patterns can be identified in digital reading tasks that may prove useful for educational monitoring and intervention. Although task situations are designed to evoke certain behaviours, the interpretation of observed behavioural patterns requires validation based on task requirements, assessment context and relationships to other available information.
Attitudes towards digital proctoring—general
Attitudes towards digital proctoring—test‐taking
Attitudes towards digital proctoring—efficacy and efficiency
Attitudes towards digital proctoring—technical difficulties
Attitudes towards digital proctoring—ethical concerns
Background It is important for institutions of higher education to maintain academic integrity, both for students and the institutions themselves. Proctoring is one way of accomplishing this, and with the increasing popularity of online courses—along with the sudden shift to online education sparked by the COVID‐19 pandemic—digital proctoring has seen an increase in use. However, there are privacy and bias concerns related to digital proctoring, so it is important to critically examine its role in higher education—when it should and should not be used, and how it is perceived among those who interact with it. Objectives In this paper, we: examine the features of and concerns about digital proctoring; analyse the results of a survey regarding student and teaching assistant (TA) attitudes towards digital proctoring; and present alternatives to digital proctoring and a framework for evaluating the need for a digital proctoring tool. Methods We surveyed students and TAs in an online graduate computer science program, asking them to provide their agreement or disagreement with 20 statements related to digital proctoring. For each response option on each statement, we calculated overall percentages as well as percentages broken out by demographics. We compared these percentages to develop a picture of student and TA perceptions. Results and Conclusions Students and TAs alike are generally tolerant of digital proctoring software and perceive some benefits to using it, including adding integrity to course grades and value to degree programs. However, they have some concerns in the areas of privacy, equity, and technical difficulties. Takeaways Digital proctoring software should be used only when necessary, with thought devoted to its impact on students and TAs and any concerns they may have. There exist alternative methods for maintaining academic integrity in a course. The framework we have presented can help with determining the need for digital proctoring.
The 9‐week instruction of writing metacognitive strategies and writing skills in both flipped and non‐flipped classes
The processes of implementing the writing metacognitive strategies and writing skills in the flipped class
The entire instructional processes of both flipped and non‐flipped classes
Background The transition from teacher‐centred towards student‐centred approaches in English language teaching, accompanied by developments in educational technology, has attracted researchers' attention to reverse teaching or flipped class. The related literature asserts that this mode of instruction might influence instructors' and students' participation in the class. Objectives To shed more light on the role of flipped class in affecting language learning strategies and skills, this study applied a sequential explanatory mixed‐methods approach to explore the impact of flipped class on English as a foreign language (EFL) learners' writing metacognitive strategies and writing skills. Methods Forty‐six intermediate EFL learners with an age range of 21–27, studying at a private language institute, were randomly divided into two groups of 23 learners. The two groups were randomly assigned to a flipped class, which received writing metacognitive strategies and writing skills through video clips before the class time, and a non‐flipped class, which received writing metacognitive strategies and writing skills in a conventional way during the class time. Results and Conclusions The results revealed that both flipped and non‐flipped classes improved the EFL learners' writing metacognitive strategies, writing performance, writing content, and writing organisation. Furthermore, one‐way ANCOVA analyses indicated that the flipped class outperformed the non‐flipped class in writing metacognitive strategies, writing performance, writing content, and writing organisation. Thematic analyses, which were used to help analyse individual semi‐structured interviews, uncovered a number of categories and themes, signifying the flipped learners' positive perceptions towards the flipped class, such as enhancing motivation, self‐confidence, and writing collaboration. Implications Pedagogical implications were discussed for flipping writing metacognitive strategies and writing skills.
Flow chart of the inclusion and exclusion of studies, following the PRISMA statement
Forest plot
Regression of Hedges' g on year
Funnel plot of standard error by Hedges' g, distribution of all 84 included studies
Funnel plot of standard error by Hedges' g, distribution of 84 observed and 21 imputed studies (black circles)
Background Blended learning programs in Kindergarten through Grade 12 (K‐12) classrooms are growing in popularity; however, previous studies assessing their effects have yielded inconsistent results. Further, their effects have not been completely quantitatively synthesized and evaluated. Objectives The purpose of this study is to synthesize the overall effects of blended learning on K‐12 student performance, distinguish the most effective domains of learning outcomes, and examine the moderators of the overall effects. Methods For the purpose, this study conducted a meta‐analysis of 84 studies published between 2000 and 2020, and involved 30,377 K‐12 students. Results and Conclusions Results revealed that blended learning can significantly improve K‐12 students' overall performance [g = 0.65, p < 0.001, 95% CI = (0.54–0.77)], particularly in the cognitive domain [g = 0.74, p < 0.001, 95% CI = (0.61–0.88)). The testing of moderators indicates that the factors moderating the impact of blended learning on student performance in these studies included group activities, educational level, subject, knowledge type, instructor, sample size, intervention duration and region. Implications The results indicate that blended learning is an effective way to improve K‐12 students' performance compared to traditional face‐to‐face (F2F) learning. Additionally, these findings highlight valuable recommendations for future research and practices related to effective blended learning approaches in K‐12 settings.
Background Video lectures which include the instructor's presence are becoming increasingly popular. Presenting a real human does, however, entail higher financial and time costs in making videos, and one innovative approach to reduce costs has been to generate a virtual speaking instructor. Objectives The current study examined whether the use of a virtual instructor in video lectures would facilitate learning as well as a human instructor, and whether manipulating the virtual instructor's characteristics (i.e., voice and appearance) might optimize the effectiveness of the virtual instructor. Methods Our study set four conditions. In the control condition, students watched a human instructor. In the experiment conditions, students watched one of (a) a virtual instructor which used the human instructor's voice and an AI image, (b) a virtual instructor which spoke in an AI voice with an AI image made to speak using text‐to‐speech and lip synthesis techniques, or (c) a virtual instructor with used an AI voice and an AI likable‐image of an instructor. Results and Conclusions The AI likable instructor condition had a significant positive effect on students' learning performance and motivation, without decreasing the attention students paid to the learning materials. Implications Our findings suggest that instructional video designers can make use of AI voices and AI images of likable humans as instructors to motivate students and enhance their learning performance.
Excerpt from the game on a topic of focus during the intervention. Intervention gameplay was in Norwegian
Background The development and promotion of educational games are still outpacing knowledge of these games' effects, raising calls for evidence of benefits and challenges. Studies suggest that students and teachers like games, but the payoff of the investment in terms of increased motivation and achievement remains unclear. Objectives This study investigates the pure effect of a marketing simulation game on motivation, perceived learning and achievement, above and beyond regular student‐active instruction. Methods We applied a randomized, controlled experiment in a marketing course in upper‐secondary schools (Nclasses = 22; Nstudents = 433) comparing a collaborative–competitive marketing simulation game with regular, case‐based, student‐active instruction on three groups of outcome measures: motivation, perceived ability, and achievement. Additionally, students and teachers provided quantitative and qualitative feedback on game experiences. Results and Conclusions We showcase the importance of a robust study design with valid compound instruments. Moreover, investigations of the game implementation and experiences reveal insights about intervention timing, differential negative consequences by gender and need for reflection opportunities. We find no clear evidence of positive or negative effects of the game, despite students' and teachers' satisfaction. Implications Beyond the effect evaluation, we offer recommendations to researchers and developers of educational games about scaffolding, timing and teacher competence building.
Research model
Head mount display and controller
Virtual reality‐based job‐training start screen and instruments
Learners participating in virtual reality‐based job training
The moderating effect of presence on the relationship between virtual reality media characteristics and flow
Background This study aims to establish the mechanism by which virtual reality (VR) media characteristics affect learning transfer in a VR‐based job‐training environment. The two main determinants of learning transfer are the technology and the learner, but not many studies have considered both factors simultaneously. Objectives This study aims to ascertain the mechanism by which VR media characteristics affect learning transfer with flow as a mediator, and to explore whether presence moderates the effect of VR media characteristics on flow. Methods A survey of semiconductor facilities engineers who received VR‐based job training was conducted. A total of 106 responses were used in the statistical analysis. A path analysis was conducted to determine the promotional mechanism that connects VR media characteristics, flow, and learning transfer. Results and Conclusions The results were as follows. First, VR media characteristics had a positive impact on learning transfer. Second, flow acted as a mediator in the relationship between VR media characteristics and learning transfer. Third, presence played a moderating role in the relationship between VR media characteristics and flow. Key takeaways This study is significant as it identifies elements to consider in relation to both the technological and learner aspects of VR while designing a VR‐based job‐training programme. A corporation should consider VR media characteristics from a technological perspective, and flow and presence from a learner's perspective.
Dimensions of technology‐enhanced learning
Selection process
The active and authentic learning continuum
Clusters on the active and authentic learning continuum. The paper numbers and years of publication correspond with Table 2
Background Since about 2010 e‐learning has been embedded in educational practice and has become, surely due to the Covid‐19 pandemic, increasingly important. Objectives Although much has been written about e‐learning, little is known about crucial didactic and pedagogical design principles for e‐learning. This review tried to fill that gap. Methods Based on a systematic literature review, 42 studies (out of 1857 unique hits) were included that address e‐learning design in higher education. Open and axial coding was used for analysis. Results and conclusions There were two continuums distinguished as important for e‐learning: (1) the active learning continuum and (2) the authentic learning continuum. Those continuums appear to be useful to give a visual representation of included studies through an active and authentic learning continuum. This resulted in four clusters with (slightly) different properties. These properties vary from a relatively low to a high level of authenticity, and from teacher to student centred. Analysis also revealed four crucial aspects for e‐learning design: (1) content scaffolding, (2) process scaffolding, (3) peer‐to‐peer learning, and (4) formative strategies. In general, most of the e‐learning approaches demand an educational design that facilitates authentic learning and self‐regulation. Takeaways To help practitioners in realizing e‐learning design, this paper will provide some concrete suggestions and tips for e‐learning design. Furthermore, this research shows that more well‐founded research is necessary to gain more insight in didactic and pedagogical design principles for e‐learning.
Background Digital game‐based learning (DGBL) has been shown to be an effective strategy for promoting learning motivation. However, a lack of attention has been paid to investigating the effects of individual differences on elementary students' reading motivation and in‐gaming achievements in DGBL. Objectives This study developed an educational mobile game to facilitate learners' reading activities, and investigated how prior knowledge and gaming experience affected learners' reading motivation, gaming motivation, perceptions and in‐game achievements. Methods A total of 53 third to fifth grade elementary school students participated in this study, and were divided into the high prior knowledge (HPK)/low prior knowledge (LPK) and the high gaming experience (HGE)/low gaming experience (LGE) groups. Results and Conclusions The HPK learners had higher reading motivation, perceptions, gaming and social achievements than the LPK learners, whereas they had similar gaming motivation and reading achievements. In addition, the HGE learners had lower reading motivation, reading and gaming achievements than the LGE learners, whereas they had similar gaming motivation, perceptions and social achievements. Moreover, correlational results showed that similarities and differences were observed between the HPK/LPK and HGE/LGE learners. Implications The findings of this study suggest that learners' prior knowledge and gaming experience play influential roles in reading motivation, gaming motivation, perceptions and in‐game achievements. A framework is proposed for researchers or game designers to develop personalized DGBL systems to accommodate the HPK/LPK and HGE/LGE learners' individual needs or preferences in DGBL.
AI cognitive architecture research and development over years. Source: Kotseruba and Tsotsos (2020) – A review of 40 years of cognitive architecture research (p. 4)
Map of current AGI project location across the world. Source: Fitzgerald et al. (2020). Nationality map of AGI projects active in 2020 (p. 29)
Ethics, risk and policy issues from AGI projects. Source: Baum (2017). A survey of AGI projecs for ethics, risk, and policy (pp. 19–27)
Background In fact, most schools around the world are not well equipped to have discussions and keep current on the expansion of artificial intelligence (AI) in many aspects of society and economy. They either ignore this conversation, or simply criticize technology, but these resistances are not stopping wide spread of various types of AI projects in schools, mainly driven by corporations, and fueled by incentives that might not match well with long term educational objectives of student success, diversity, equity, and inclusivity. Objectives The overall purpose is to address the rising gap between the ultrafast development of AI and the meticulous technological application of education, and to suggest the important bridge of building technological leadership in teacher preparation to get ready for the grow of AI in education. Methods The contextual review examines the field of AI development in education through three lenses: conceptual context, practice context, and research context. Results and Conclusions The paper provides educators and policy makers an overall background of the phenomenon of AI in education. The study has revealed there is an urgent need for research and development in teacher preparation as well as in the philosophy of technology in education to bridge the gap between AI and education.
Prisma Flow diagram
Quality appraisal diagram
Background Due to recent lockdown conditions, which restricted opportunities for face‐to‐face contact and the ability to be physically in schools, the need for novel, safe ways to train pre‐service teachers emerged even more pressingly. Whilst virtual simulation has received some attention in pedagogy and its benefits have been demonstrated in many disciplines, there appears to be less synthesized evidence on the use of physical and/or mixed‐reality simulation utilized in teacher training. Objectives The goal of this systematic scoping review was to summarize and synthesize the literature on the use of physical and/or mixed‐reality simulation in pre‐service teacher training. Methods A systematic scoping literature review combined with a textual narrative synthesis was undertaken. Ten reference databases were searched in May 2020: Academic search premier, CINAHL, Education Research Complete, Humanities International Complete, Psychology and Behavioural Sciences Collection, PsycInfo, Teacher Reference Center, Science Direct, Web of Science and Scopus. Results and Conclusions Following inclusion/exclusion criteria assessment and screening, 13 articles were included for appraisal and synthesis. Seven papers examined physical simulations, while the remainder examined mixed‐reality simulations. The evidence from this review suggests that simulation, including physical and mixed‐reality types, could be used as a tool to increase confidence, self‐efficacy, classroom management skills and communication. Implications In comparison to other fields (e.g., nursing, medicine and aviation) simulation in education appears to be in its infancy—more large‐scale research is needed. At the same time, this review indicates that mixed‐reality simulation in particular has the potential for contributing to teacher education, because it offers the potential for learning in various contexts when compared to traditional didactic teaching practices.
Quizzes show in Shaking‐On
Individual score shows in Shaking‐On
The screenshot of Kahoot!
Research framework
Lay Description What is already known about this topic The adoption of technology in learning pedagogies should enable students to interact with their technological devices in a fun way and there needs to be interested during the learning process. Different designs of e‐learning platforms have different functions for students to interact that promote in them different psychological outcomes, by shaping their learning behaviours and enhancing their motivation and engagement. What this paper adds The present study adapted an app, named Shaking‐On, which requires students to shake their mobile devices to send their answers to multiple‐choice questions to the teacher, students will then learn from their performance. To understand how this approach can stimulate participants' emotions in achieving learning goals, the present study compared Shaking‐On to Kahoot! by checking participants' gameplay anxiety, learning interest, perceived learning value, and learning achievement when learning the Taiwanese language. Implications for practice and/or policy The research on the application of CATML to education is increasing, but there has been less research in the field of Taiwanese languages. The studies related to digital game‐based learning, there are few studies that compare games with different operation movements. Therefore, this study can be used as a reference for the application of cognitive and affective theories of multimedia learning to study the field of learning other native and disappearing languages.
PhET computer learning tools of electric charges and fields
Background Previous work has identified that the benefits of learning with videogames and learning from simulations. However, recent meta‐analytic work has also identified that little research directly compares learning with videogames and learning with simulations. Objectives This study examines two learning technologies and their corresponding pedagogical approaches and compares them for learning the Science, Technology, Engineering and Mathematics topic of electric charges. Methods Participants were randomly assigned to either an intervention using a computer simulation for inquiry‐based learning or a computer videogame for game‐based learning. Their learning gains, self‐reported emotional state and experienced cognitive load were recorded. Results We found that both learning environments improved conceptual learning, and there were no statistically significant differences between the two conditions. Participants did perceive the game‐based environment to be more engaging as well as more frustrating. We also found that cognitive load did not predict learning—however, different types of cognitive load correlated with different emotions. Overall, participants in both conditions were engaged and perceived understanding of the topic, yet they also experienced both confusion and task‐unrelated thoughts. Takeaways When learning with simulations and videogames, educators need to align intended learning outcomes with pedagogical approaches enabled by technology. In addition, a balance between principles of multimedia learning to reduce or prevent extraneous processing, and scaffolding to reduce negative effects of learning with technology, need to be considered.
The integrated model of the leaderboard's effects on learning
Moderated mediation models
The leaderboard interface
Flowchart of the experimental procedure
Moderated mediation model. N = 156. *p < 0.05, **p < 0.01. The control variable was omitted in the model. All variables in the model were standardized. The coefficients outside or inside the brackets were specific effects when the outcome variable was quiz performance or test performance
Background As one of the gamification elements, leaderboard, especially absolute leaderboard, is widely used in educational gamification systems. However, empirical studies on the optimal use condition of the leaderboard and underlying influence mechanisms are deficient. Objectives This study explored which difficulty was more conducive to learning performance in leaderboard context, and when and how it played a role. Methods To address these questions, this study conducted a 2 (dominant goal orientation: learning/performance) × 2 (difficulty: high/low) between‐subjects design. Seventy‐eight dominant learning‐oriented and 78 dominant performance‐oriented participants were recruited and randomly assigned to the high or low difficulty group respectively. Results and Conclusions Participants in the low difficulty group experienced more positive emotions, less negative emotions, and higher learning motivation than those in the high difficulty group, but the effect of difficulty on performance was not significant. Moreover, goal orientation did not moderate the effects of difficulty, dominant learning‐oriented and performance‐oriented learners were equally affected by difficulty. Further mediating analysis showed that negative emotions and learning motivation rather than positive emotions mediated the relationship between difficulty and learning performance. Implications These results confirmed the positive effect of low difficulty in leaderboard context, as well as the mediating roles of emotions and motivation involved in the relationship between difficulty and learning performance. These findings enlighten us that it is necessary to equip leaderboards in educational gamification with achievable difficulty.
PRISMA flow diagram of the search process
Funnel plot of standard error by Hedges' g effect sizes for SRL activities
Funnel plot of standard error by Hedges' g effect sizes for learning outcomes
Funnel plot of standard error by Hedges' g effect sizes for transfer tasks
Background It has been assumed that prompting students to plan, monitor and evaluate their learning process could stimulate strategy use and thereby improve learning outcomes. Objectives This study aimed to examine the effects of metacognitive prompts on students' self‐regulated learning (SRL) and learning outcomes in the context of computer‐based learning environments (CBLEs). Methods To achieve this, the current study took a meta‐analytic approach to critically evaluate evidence for the effectiveness of metacognitive prompts and identify potential moderators of the effects. Results and conclusions With random‐effects models, the results showed that metacognitive prompts significantly enhanced SRL activities (g = 0.50, 95% confidence interval [0.37, 0.63]) and learning outcomes (g = 0.40, 95% confidence interval [0.31, 0.49]) relative to the control conditions. Furthermore, moderator analyses revealed that the effects varied as a function of three prompts features: feedback, specificity and adaptability. Implications Developing task‐specific, individual‐adaptive prompts and feedback should be a design principle in CBLEs, such that the prompt effect could be retained, sustainably enhanced and transferred to novel situations.
Lay Description What is already known about this topic Screen‐sharing technologies are becoming commonplace in many contemporary educational settings. The technologies represent integrated systems that enable functions such as mirroring, video streaming, online polls, real‐time editing, screen‐capturing and learner analytics. Screen‐sharing is indirectly examined in several studies with findings indicating improvements in group awareness, emotional engagement, learner confidence to share ideas and teachers' visualization of concepts. Existing findings are largely anecdotal and there are very few studies that have focused directly on screen‐sharing affordances. What this paper adds An empirical study directly examining teachers' first‐hand perspectives of using screen‐sharing technologies. Benefits relate to greater task focus, deeper thinking, improvements to teaching practice and ease of sharing content. A minority of teachers experienced issues such as technical problems, inappropriate sharing and distraction. Mobility was recognized as a key affordance though remained underutilized by many teachers. Implications for practice and/or policy Professional learning accompanying the use of screen‐sharing technologies can support teachers to take advantage of all features of screen sharing such as enhanced teacher mobility. Institutions need to provide requisite technical and professional support for teachers to effectively and reliably leverage screen‐sharing in the classroom. There is a need to explore teacher's and learner's screen‐sharing uses in situ to more fully understand issues and possibilities.
Confidence (retrospective confidence judgment [RCJ]) distributions of all samples separated for time and correct and incorrect answers. Sample 1a (N = 33), Sample 1b (N = 36), Sample 2a, 2b, 2c (N = 51). Straight lines indicate mean RCJ scores of the distribution. RCJs were measured on a 5‐point scale in Samples 1a and b and on a 6‐point scale in Samples 2a–2c
Background Explorative online information search activities are self‐regulated learning processes that require monitoring in the form of accurate metacognitive judgments about one’s own knowledge. People have to judge what they know, but also understand what they do not know. Previous research has explored those two aspects in relation to each other mostly by employing measures of metacognitive calibration and points toward a potential detrimental effect of using the internet to answer knowledge test items. Objectives This research aimed to disentangle those two aspects of knowledge and to describe a false certainty effect (FaCE), indicating an increase in confidence in the correctness of incorrect responses in a knowledge test after online information search. We also aimed to further explore the conditions under which this effect occurs. Methods In two studies participants’ knowledge and on‐item confidence ratings were measured before and after a short online information search activity. In Study 1, we analyzed data of two samples with and without knowledge gain. In Study 2, we manipulated the familiarity of knowledge test items and whether there was an information search activity. Results Study 1 showed that false certainty occurred even when a search activity did not lead to any learning success. This FaCE, however, was stronger when there was actual knowledge gain. In Study 2, we found that the effect was caused by the information search activity itself and not by the pre‐post methodology of identical knowledge‐test items. Implications Our results demonstrate the emergence of increased confidence in the correctness of incorrect responses in knowledge tests after online information search activities. Educators and researchers should be aware of this undesired false certainty effect of computer‐assisted learning on individuals’ knowledge monitoring.
A social construction of knowledge model, adapted from Gunnawardena et al. (1997)’s five phases of knowledge co‐construction
Diagram of the social network of the first online discussion. Red circles indicate students commenting on others' posts, while blue squares indicate students receiving comments from others. N = 106
Background Online discussion is one of the commonly used tools to enhance students' interactions and engagements in online courses, but it is not clear how social presence in online discussions impacts students' learning and what kinds of interactions we should encourage. Social network analysis provides a new methodology to investigate how social interactions in online discussions influence learning. Objectives The study aims to identify students' online discussion interaction patterns and investigate how they are related to students' learning performance in an asynchronous course. Methods Students who enrolled in an asynchronous undergraduate course were invited to participate in this study. Participants' online discussion participation data were collected from six online discussions from an asynchronous online course. Data were analysed using social network analysis methods as well as correlation and regression analysis. Results and Conclusions We found that statistically significant positive correlations existed between learning performance and out‐degree and closeness, respectively, which provided evidence to support that social interactions in online discussions have positive correlations with learning. We also found that out‐closeness contributes to students' learning performance. The results of the study imply that knowledge construction occurs mainly in the centre of the outgoing network. By interpreting this study's results with the social construction of knowledge model, we conclude that knowledge construction mainly happens in reading and commenting on others' posts by internalizing and integrating external useful information from others. Implications While numerous studies have emphasized the importance of interactions in online discussions or collaborative learning, there are limited studies on what kind of interactions we should encourage and what role students play in knowledge construction. The results of this study provide evidence that knowledge construction happens when students are reading and commenting on others' posts, which includes a hidden process of integrating external useful information into their own understanding. Practical strategies are provided to encourage students to be expansive and actively reach out to other students.
Journal metrics
95 days
Submission to first decision
Acceptance rate
$3,650 / £2,450 / €3,050
3.761 (2021)
Journal Impact Factor™
7 (2021)
Top-cited authors
Greg Heiberger
  • South Dakota State University
Reynol Junco
  • Harvard University
Eric Loken
  • Pennsylvania State University
Alex H. Johnstone
  • University of Glasgow
Ching Sing Chai
  • The Chinese University of Hong Kong