Content uploaded by Zhihong Xu
Author content
All content in this area was uploaded by Zhihong Xu on Aug 15, 2019
Content may be subject to copyright.
A preview of the PDF is not available
The effectiveness of intelligent tutoring systems on K-12 students' reading comprehension: A meta-analysis
Abstract and Figures
Abstract:
This meta-analysis examined the effectiveness of improving reading comprehension for students in K-12 classrooms using intelligent tutoring systems (ITSs), a computer-based learning environment that provides customizable and immediate feedback to the learner. Nineteen studies from 13 publications incorporating approximately 10 000 students were included in the final analysis; using robust variance estimation to account for statistical dependencies, the 19 studies yielded 88 effect size estimates. The meta-analysis indicated that the overall random effect size of ITSs on reading comprehension was 0.60 (using a mix of standardized and researcher-designed measures) with a 95% confidence interval 0.36 to 0.85 (p < 0.001). This review confirms previous studies comparing ITSs to human tutoring: ITSs produced a small effect size when compared to human tutoring (0.20, 0.02-0.38, p = 0.036, n = 21). All comparisons to human tutoring used standardized measures. This review also found that ITSs produced a larger effect size on reading comprehension when compared to traditional instruction (0.86) for mixed measures and (0.26) for standardized measures. These findings may be of interest to practitioners and policy makers seeking to improve reading comprehension using consistent and accessible ITSs. Recommendations for researchers include conducting studies to understand the difference between traditional and updated versions of ITSs and employing valid and reliable standardized tests and researcher-designed measures.
Figures - uploaded by Zhihong Xu
Author content
All figure content in this area was uploaded by Zhihong Xu
Content may be subject to copyright.
... Then, they revealed that ITSs were not significantly more effective when compared with small-group or individual human instruction, whereas they were more effective than large-group human instruction. Xu et al. (2019) conducted a meta-analysis with 88 individual effect sizes to synthesize empirical evidence regarding the effect of ITSs on K-12 students' reading comprehension. They found that when compared to traditional instruction, ITSs had a large effect size on learning measured with researcher-designed and standardized instruments; on the other hand, they had a small effect when compared to human tutoring. ...
... adult and peer) conversations. Likewise, meta-analysis studies on ITSs (Ma et al., 2014;Xu et al., 2019) indicated the lack of effect of this type of conversational agent when compared to small-group or individual human instruction or tutoring. Despite the latest advancements in chatbot technology that can perform various natural language tasks, Kasneci et al. (2023) underscore that "they can only serve as assistive tools to human learners and educators and cannot replace the teacher" (p. ...
This meta-analysis aimed to comprehensively review empirical studies on the effect of chatbots on learning and quantitatively synthesize their findings to produce an overall effect size. Searching several databases yielded 28 eligible reports with 31 individual effect sizes. The results revealed a significant and medium effect (g = .48) of chatbots on learning. Further analyses indicated four significant moderators: the type of instruction in the comparison group, experimental duration, chatbot type, and chatbot tasks. The highest effect sizes emerged when the comparison group had no specific support, the experiment lasted only one session, and chatbots were task-focused and took care of frequently asked questions. These results suggest that chatbots can be more effective in certain cases within their overall contribution area to learning.
... A review study by VanLehn (2011) suggested that intelligent tutoring systems are almost on par with human tutors if the ITS is step-based, i.e., the interaction between tutor and tutee has more granularity than answer-based tutoring systems. In a K-12 setting, i.e., from primary and lower-and upper secondary school, the outcomes and effects of ITS use have been studied concerning reading comprehension (McCarthy et al., 2018;Xu et al., 2019), mathematics (Steenbergen-Hu & Cooper, 2013;Walkington & Bernacki, 2018), and chemistry (McLaren et al., 2011). According to Zawacki-Richter and colleagues (2019), studies on the use of intelligent tutoring systems in higher education cover teaching course content such as computer science, mathematics, or writing and reading comprehension, facilitating collaboration between students, providing automated feedback, and curating learning materials based on students' needs. ...
This chapter maps currently ongoing debates on the use of artificial intelligence (AI) in schools and universities to highlight the main enthusiasms and concerns these debates have sparked both amongst the scholars of critical data studies, media and communication, and educational sciences. Our analysis is framed around a wider notion of the “school” in order to cover thematic debates about changes in learning, teaching, teacher-student relationships, and administrative matters. Hence, the first part of the chapter covers recent progress made in student-facing AI solutions, developments in teachers’ AI tools and perceptions of implementing AI in educational settings as well as how AI is increasingly used in education management. In the second part, we balance the progress discourse of AI with the equally, if not more, important concerns that the implementation of different AI solutions raises in educational settings.
... They can be used as supplements to traditional approaches to education or as standalone applications for self-study. They can be used in any educational context with learners of any age (e.g., Xu et al., 2019). They leverage human obsession with digital technology to provide encapsulated learning experiences (Mohamed & Lamia, 2018). ...
Artificial intelligence (AI) is changing many aspects of education and is gradually being introduced to language education. This article reviews the literature to examine main trends and common findings in relation to AI technologies and applications for second and foreign language learning and teaching. With special reference to computer-assisted language learning (CALL), the article explores natural language processing (NLP), data-driven learning (DDL), automated writing evaluation (AWE), computerized dynamic assessment (CDA), intelligent tutoring systems (ITSs), automatic speech recognition (ASR), and chatbots. It contributes to discussions on understanding and using AI-supported language learning and teaching. It suggests that AI will be continuously integrated into language education and AI technologies and applications will have a profound impact on language learning and teaching. Language educators need to ensure that AI is effectively used to support language learning and teaching in AI-powered contexts. More rigorous research on AI-supported language learning and teaching is recommended to maximise second and foreign language learning and teaching with AI.
... These systems aim to emulate the expertise of human one-to-one tutoring, offer personalized learning experiences that optimize the quality of learning for users [4], and strive to cater to students' social needs [5]. AI-based systems possess autonomy, adaptability, and interactivity as their defining features [6]. AI techniques can gather and assess learners' behavioral and psychological information and establish links with knowledge networks [7]. ...
Although there has been some progress, the integration of artificial intelligence into higher education remains far from sufficient. The demand for teachers will persist for some time; however, with the introduction of AI-based robots into classrooms, the role of teachers has reduced to a minimum. The purpose of the current study was to evaluate Chinese higher education students’ intentions to adopt AI-based robots for educational purposes. Based on the TAM3 model, the current study proposes 14 hypotheses to evaluate students’ intention to adopt AI-based robots in education. The students’ data were collected and analyzed using PLS-SEM. The study findings revealed that 12 hypotheses were accepted and two were rejected. The results indicate that students are willing to accept AI-based robots in their education. However, the findings revealed an insignificant influence of job relevance and robot anxiety on perceived usefulness and ease of use, respectively. The findings of this study will provide insight into university administrations regarding the significance of AI-based robots in education. Moreover, the findings will help robot developers, policy makers, and university administrators design and implement AI-based robots that fulfill contemporary education needs.
I-LOVE-RARY Approach(ILA) was implemented through a reading manual to improve phonetics, phonemic awareness, vocabulary, comprehension, and fluency to feasibly realized a reading instructional guide. With an itemized progression point system from prior reading skills to post reading performances, ILA affected significantly in the improvement of reading skills among experimental participants through the sixe of the respondents did not differentiate largely. Phonetic word and fluency word count per minute among respondents both prior and post between control and experimental groups were lower than 138-word count per minute. Vocabulary, comprehension, and fluency were attributed to law of exercise. ILA has significant effect on reading skills, influenced with local family’s exposure to Cebuano with blended local TV, online commercials, and lack of exposure.
This research aims to determine whether or not it is possible to use artificial intelligence (AI) in English for speakers of other languages (ESOL) courses. The purpose of the research is to analyze and review previous research pertinent to artificial intelligence in EFL/ESL instruction to present a comprehensive picture of the current degree of artificial intelligence in EFL/ESL instruction. The utilisation of intelligent teaching systems, self-regulated learning, virtual reality, immersive virtual environment, and natural language processing in teaching English as a foreign language classroom. The study adopted the questionnaire as a tool for data collection then data was analyzed and discussed to reach the results. The results showed that the ethical responsibility for making the most effective use of AI in the classroom now falls on both educators and students themselves. The article also draws the conclusion that artificial intelligence (AI) has a positive impact on the field of English language teaching (ELT) and learning; however, it needs to be better integrated into educational settings. Teachers and students need to be more aware of the new applications and tools that have flooded the field of AI in recent years. This conclusion was reached in the context of the article.
Background
Learning to thinking critically about health information and choices can protect people from unnecessary suffering, harm, and resource waste. Earlier work revealed that children can learn these skills, but printing costs and curricula compatibility remain important barriers to school implementation. We aimed to develop a set of digital learning resources for students to think critically about health that were suitable for use in Kenyan, Rwandan, and Ugandan secondary schools.
Methods
We conducted work in two phases collaborating with teachers, students, schools, and national curriculum development offices using a human-centered design approach. First, we conducted context analyses and an overview of teaching strategies, prioritised content and collected examples. Next, we developed lessons and guidance iteratively, informed by data from user-testing, individual and group interviews, and school pilots.
Results
Final resources include online lesson plans, teachers’ guide, and extra resources, with lesson plans in two modes, for use in a classroom equipped with a blackboard/flip-chart and a projector. The resources are accessible offline for use when electricity or Internet is lacking. Teachers preferred the projector mode, as it provided structure and a focal point for class attention. Feedback was largely positive, with teachers and students appreciating the learning and experiencing it as relevant. Four main challenges included time to teach lessons; incorrect comprehension; identifying suitable examples; and technical, logistical, and behavioural challenges with a student-computer mode that we piloted. We resolved challenges by simplifying and combining lessons; increasing opportunities for review and assessment; developing teacher training materials, creating a searchable set of examples; and deactivating the student-computer mode.
Conclusion
Using a human-centred design approach, we created digital resources for teaching secondary school students to think critically about health actions and for training teachers. Be smart about your health resources are open access and can be translated or adapted to other settings.
The effects of graphene nanoplatelet (GNP) and organoclay montmorillonite (OMMT) on the curing, relaxation, and the mechanical properties of acrylonitrile‐butadiene rubber/polyvinyl chloride (NBR/PVC) nanocomposites were investigated. Scorch and cure times of nanocomposites reinforced with GNP increased, while these parameters for nanocomposites reinforced with OMMT decreased. Maximum torque and cure rate increased after adding the nanoparticles. In the relaxation curves, with increasing nanoparticle and acrylonitrile (ACN) content, both the slopes of short‐ and long‐term regions, raised. Cured compounds have the significantly higher initial and final modulus, the greater elastic slope, and lower viscous slope than the uncured ones. The results of the relaxation test showed that the relaxation percentage in the uncured samples was much higher than the cured ones. Also, with the addition of both GNP and OMMT, this parameter had an upward trend. DGS/DGL parameter increased with the addition of nano‐reinforcements in the uncured state while decreasing in the cured state. For example, the blend containing N3 reinforced via GNP showed an 11% increase in the uncured state, whereas 14% reduction in the cured state. Tensile strength, elastic modulus, and hardness were improved by incorporating nanoparticles, while the effect of GNP was more intense. The similar trend observed in samples with NBR type of higher ACN content. © 2019 Wiley Periodicals, Inc. J. Appl. Polym. Sci. 2019, 136, 47550.
Learning to write well can be a significant contributor to English language proficiency. The large population of English language learners (ELLs) has highlighted the need for educational technology as an important support for the different approaches to writing instruction for ELLs. In this meta-analysis, we examined the effectiveness of educational technology applications on the writing quality of adult ELLs. Twenty-one independent studies from 16 publications were included in this review. Our findings confirmed that technology applications produce a large effect size (.93 and 1.28, fixed effect model and random effect model respectively) as compared to non-technology instructional methods. We conducted sub-group analyses with six substantive and methodological factors: type of technology, genre of writing, program duration, program intensity, measures outcome, and research design, and followed the analyses with research and pedagogical implications.
The Component Model of Reading expanded upon the Simple View of Reading by adding an ecological and psychological component. Elements of the ecological component include teacher knowledge, information provided in textbooks, and teacher instructional practices. In this study, the authors examined the extent of teacher knowledge about text structure, the extent to which textbooks focused on text structure related skills and strategies as well as the percentage each skill and strategy was covered in lessons and teacher instructional practices. Such analysis shows that although text structure interventions may have positive effects on student reading comprehension, there are multiple elements of the ecological component that may be counteracting the benefits of the intervention. First, teachers have a limited knowledge of the five common text structures. Second, textbooks systematically minimize text structure instruction and only cover comprehension skills and strategies sporadically throughout a year-long curriculum. Third, teacher learning of text structures and change in practice was moderated by these ecological factors including textbook scheduled instruction and administrator support.
Human one-to-one tutoring has been shown to be a very effective form of instruction. Three contrasting hypotheses, a tutor-centered one, a student-centered one, and an interactive one could all potentially explain the effectiveness of tutoring. To test these hypotheses, analyses focused not only on the effectiveness of the tutors' moves, but also on the effectiveness of the students' construction on learning, as well as their interaction. The interaction hypothesis is further tested in the second study by manipulating the kind of tutoring tactics tutors were permitted to use. In order to promote a more interactive style of dialogue, rather than a didactic style, tutors were suppressed from giving explanations and feedback. Instead, tutors were encouraged to prompt the students. Surprisingly, students learned just as effectively even when tutors were suppressed from giving explanations and feedback. Their learning in the interactive style of tutoring is attributed to construction from deeper and a greater amount of scaffolding episodes, as well as their greater effort to take control of their own learning by reading more. What they learned from reading was limited, however, by their reading abilities.
Reading comprehension in the content areas is a challenge for many middle grade students. Text structure-based instruction has yielded positive outcomes in reading comprehension at all grade levels in small and large studies. The text structure strategy delivered via the web, called Intelligent Tutoring System for the Text Structure Strategy (ITSS), has proven successful in large-scale studies at 4th and 5th grades and a smaller study at 7th grade. Text structure-based instruction focuses on selection and encoding of strategic memory. This strategic memory proves to be an effective springboard for many comprehension-based activities such as summarizing, inferring, elaborating, and applying. This was the first large-scale randomized controlled efficacy study on the web-based delivery of the text structure strategy to 7th-grade students. 108 classrooms from rural and suburban schools were randomly assigned to ITSS or control and pretests and posttests were administered at the beginning and end of the school year. Multilevel data analyses were conducted on standardized and researcher designed measures of reading comprehension. Results showed that ITSS classrooms outperformed the control classrooms on all measures with the highest effects reported for number of ideas included in the main idea. Results have practical implications for classroom practices.
In this meta-analysis of 45 studies involving students in Grades 2–12, the authors present evidence on the effects of text structure instruction on the expository reading comprehension of students. The meta-analysis was deigned to answer 2 sets of questions. The first set of questions examined the effectiveness of text structure instruction on proximal measures of comprehension, including examination of potential moderators and effectiveness for students with or at-risk for disabilities. The second set of questions examined the effectiveness on transfer measures of the effectiveness of the intervention across temporal contexts (maintenance), near-contexts (untaught text structures), and far-contexts (general reading comprehension). Overall, the results indicated that text structure instruction improves expository reading comprehension, but the effects were tempered when text structure instruction was compared with stronger comparison groups. The findings also identified 2 moderators that led to increased effect sizes (teaching more text structures and including writing in the instruction). Text structure instruction was also found to be effective across all 3 levels of transfer, although the effects for far-transfer are small and lack consistency. Recommendations include conducting more research to understand the nuances of potential interactions between various instructional approaches and student populations.
This review describes a meta-analysis of findings from 50 controlled evaluations of intelligent computer tutoring systems. The median effect of intelligent tutoring in the 50 evaluations was to raise test scores 0.66 standard deviations over conventional levels, or from the 50th to the 75th percentile. However, the amount of improvement found in an evaluation depended to a great extent on whether improvement was measured on locally developed or standardized tests, suggesting that alignment of test and instructional objectives is a critical determinant of evaluation results. The review also describes findings from two groups of evaluations that did not meet all of the selection requirements for the meta-analysis: six evaluations with nonconventional control groups and four with flawed implementations of intelligent tutoring systems. Intelligent tutoring effects in these evaluations were small, suggesting that evaluation results are also affected by the nature of control treatments and the adequacy of program implementations.
A 7-month study of 178 students in grades 1-4 at two Blue Ribbon schools compared two daily 20-minute treatments. Eighty-eight students used the 2000-2001 version of Project LISTEN's Reading Tutor (www.cs.cmu.edu/similar to listen) in 10-computer labs, averaging 19 hours over the course of the year. The Reading Tutor served as a computerized implementation of the National Reading Panel's recommended guided oral reading instruction (NRP, 2000). The Reading Tutor listened to students read aloud, giving spoken and graphical help when it noticed them click for help, make a mistake, or get stuck. Students using the Reading Tutor averaged significantly higher gains across measures of reading ability, especially those involving word level skills (word identification, blending words, and spelling) than their matched classmates who spent that time doing Sustained Silent Reading (SSR) in their classrooms. Additionally, these students trended toward higher gains in fluency and reacting comprehension. Overall, use of the Reading Tutor resulted in the types of improvement that would be expected from guided oral reading, but with the benefit of scalability, a problem for human-guided oral reading practice.