No file available
To read the file of this research,
you can request a copy directly from the authors.
Can Large Language Models Provide Feedback to Students? A Case Study on ChatGPT
Preprints and early-stage research may not have been peer reviewed yet.
Abstract
Educational feedback has been widely acknowledged as an effective approach to improving student learning. However, scaling effective practices can be laborious and costly, which motivated researchers to work on automated feedback systems (AFS). Inspired by the recent advancements in the pre-trained language models (e.g., ChatGPT), we posit that such models might advance the existing knowledge of textual feedback generation in AFS because of their capability to offer natural-sounding and detailed responses. Therefore, we aimed to investigate the feasibility of using ChatGPT to provide students with feedback to help them learn better. Specifically, we first examined the readability of ChatGPT-generated feedback. Then, we measured the agreement between ChatGPT and the instructor when assessing students' assignments according to the marking rubric. Finally, we used a well-known theoretical feedback framework to further investigate the effectiveness of the feedback generated by ChatGPT. Our results show that i) ChatGPT is capable of generating more detailed feedback that fluently and coherently summarizes students' performance than human instructors; ii) ChatGPT achieved high agreement with the instructor when assessing the topic of students' assignments; and iii) ChatGPT could provide feedback on the process of students completing the task, which benefits students developing learning skills.
... These systems draw upon vast linguistic databases to generate responses that simulate human-like engagement with student submissions. AI-generated feedback has ignited significant discussions and debates within academic circles (Baidoo-Anu & Ansah, 2023;Cope & Kalantzis, 2023b;Dai et al., 2023;Otaki, 2023) with the potential to reshape the way in which we teach and learn (Božić, 2023). ...
... Lane et al. (2016) echoed these beliefs, pointing to the enormous possibilities for AI in areas such as collaborative, immersive, affective, and exploratory learning. Clearly, there is an increased demand for developing AI technologies that can be utilised by educators for more effective and inclusive support of learners' educational needs (McGrath et al., 2023), and two areas of interest are assessment and formative feedback (Dai et al., 2023;Tzirides, Zapata, Bolger, et al., 2024;Tzirides, Zapata, Kastania, et al., 2024;Zapata et al., 2024). ...
... For example, Mizumoto and Eguchi's (2023) study, which analysed 12,100 English essays authored by individuals from 11 distinct linguistic backgrounds between 2006 and 2007, revealed that GenAI significantly reduced the time required for grading, ensured consistency in scoring, and was able to provide immediate scores and feedback. Dai et al. (2023) have reported similar results. This work compared ChatGPT-based reviews and instructor feedback on 85 openended student-written assignments in a postgraduate university programme. ...
... Dai et al. [45] conducted a study to determine the practicality of using ChatGPT as an assessment tool for providing feedback on student's assignments. A dataset was taken from a postgraduate data science course from an Australian university, in which students were tasked with proposing a data science project based on a business scenario [45]. ...
... Dai et al. [45] conducted a study to determine the practicality of using ChatGPT as an assessment tool for providing feedback on student's assignments. A dataset was taken from a postgraduate data science course from an Australian university, in which students were tasked with proposing a data science project based on a business scenario [45]. The feedback to the assessments was given by instructors and ChatGPT through a series of prompts based on the following rubric points: the clarity of the project goals, the relevance of the topic to data science, details on the business benefits, the creativity of the topic and the overall clarity of the proposed solution [45]. ...
... A dataset was taken from a postgraduate data science course from an Australian university, in which students were tasked with proposing a data science project based on a business scenario [45]. The feedback to the assessments was given by instructors and ChatGPT through a series of prompts based on the following rubric points: the clarity of the project goals, the relevance of the topic to data science, details on the business benefits, the creativity of the topic and the overall clarity of the proposed solution [45]. The feedback on each criterion received from ChatGPT and instructors was further graded by three experts on a five-point scale, determining fluency and coherency [45]. ...
With the emergence of artificial intelligence (AI), machine-learning (ML), and chatbot technologies, the field of education has been transformed drastically. The latest advancements in AI chatbots (such as ChatGPT) have proven to offer several benefits for students and educators. However, these benefits also come with inherent challenges, that can impede students’ learning and create hurdles for educators. The study aims to explore the benefits and challenges of AI chatbots in educational settings, with the goal of identifying how they can address existing barriers to learning. The paper begins by outlining the historical evolution of chatbots along with key elements that encompass the architecture of an AI chatbot. The paper then delves into the challenges and limitations associated with the integration of AI chatbots into education. The research findings from this narrative review reveal several benefits of using AI chatbots in education. AI chatbots like ChatGPT can function as virtual tutoring assistants, fostering an adaptive learning environment by aiding students with various learning activities, such as learning programming languages and foreign languages, understanding complex concepts, assisting with research activities, and providing real-time feedback. Educators can leverage such chatbots to create course content, generate assessments, evaluate student performance, and utilize them for data analysis and research. However, this technology presents significant challenges concerning data security and privacy. Additionally, ethical concerns regarding academic integrity and reliance on technology are some of the key challenges. Ultimately, AI chatbots offer endless opportunities by fostering a dynamic and interactive learning environment. However, to help students and teachers maximize the potential of this robust technology, it is essential to understand the risks, benefits, and ethical use of AI chatbots in education.
... In their study, Dai et al. (2023b) also compared the readability and nature of the feedback generated by ChatGPT with the feedback provided by a human instructor. They found that the bot-generated feedback was more detailed, more consistent and more process-oriented, more prose-like and more positive as well as "significantly more readable than instructor feedback (p < 0.001, examined by paired t-test)" (Dai et al., 2023b, p. 4). ...
... For example, Adamopoulou and Moussiades (2020) noted that "human-chatbot communication has noticeable differences in the content and quality in comparison to the human-human discussion" (p. 1), whereas Dai et al. (2023b) argued that this disadvantage has been resolved to a certain extent, as "specific training allows ChatGPT to generate more natural-sounding and context-specific responses" (p. 1). Moreover, the user interface is "simple and intuitive" (Essel et al., 2022, p. 2), making it easy to use (ibid., p. 12) and lowering the threshold for interaction even further. ...
... This can lead to inaccurate feedback and recommendations for specific learning contexts. For instance, Dai et al. (2023b) remarked that "ChatGPT could not offer a reliable assessment of student performance compared to the instructor" (p. 5). ...
Rapid developments in the field of Artificial Intelligence (AI), particularly since the public release of ChatGPT-3 in November 2022 (and of more recent versions), offer several opportunities for enhancing learning and teaching processes. The present paper concentrates on one crucial factor contributing to learning success, which is feedback. After suggesting a definition of chatbot feedback, the chapter outlines various advantages and discusses prevailing limitations. Moreover, the chapter provides recommendations for utilizing chatbots for feedback purposes in educational settings. Altogether, it is argued that chatbots can serve as feedback assistants for teachers and learners and can play a supportive role in the development of learners’ and teachers’ feedback literacy.
... As such, the present study accepts the first null hypothesis that there is no difference between the two groups' performance in the English structural and semantic pretest. The finding disagree with those found by Baskara and Mukarto (2023), Dai et al. (2023), Koraishi (2023), Meniado, 2023, andXiao andZhi (2023). ...
... The finding led to the rejection of the second hypothesis that there is no different between the performance of both groups in the English structural and semantic posttest. The finding is similar to those introduced by Baskara and Mukarto (2023), Dai et al. (2023), Koraishi (2023), Xiao and Zhi (2023). Table 3 also shows that both groups' pretest performance is not statistically different. ...
... The result is based on the p value with is less than 0.05. The result is compatible with those findings reported by Baskara and Mukarto (2023), Dai et al. (2023), Koraishi (2023), Meniado, 2023 andXiao andZhi (2023) . Table 4 introduces the results of independent samples t-test. ...
... Since ChatGPT's introduction to the public in 2022, the academic community has engaged in extensive discourse regarding its potential implications for various stakeholders, including educators, students, and policymakers (Aydin & Karaarslan, 2023; Baidoo-Anu & Owusu Ansah, 2023; Zhai, 2023). Particularly, the AI-generated feedback or GenAI reviews has ignited significant discussions and debates within these circles (Baidoo-Anu & Owusu Ansah, 2023; Dai et al., 2023;Otaki, 2023) as it can use NLP to analyze work and provide feedback based on predefined criteria (Clarizia et al., 2018;Wongvorachan & Bulut, 2022). This development aligns with Hattie and Timperley's (2007) model, offering a means to solicit continuous personalized feedback more easily for students (Otaki, 2023;Wongvorachan & Bulut, 2022). ...
... Studies have showcased that generative models trained on a dataset of human-graded essays could efficiently grade essays written by students (; Kim, Park, & Lee, 2019 as cited in Baidoo-Anu & Owusu Ansah, 2023; Otaki, 2023;Tzirides et al., 2023a) with a certain degree of accuracy (Mizumoto & Eguchi, 2023). Research suggests that the GenAI reviews are more critical (Tubino & Adachi, 2022;Tzirides et al., 2023a), readable (Dai et al., 2023;Tzirides et al., 2023a), personalized (Brynjolfsson et al., 2023;Cotton et al., 2023;Farrokhnia et al., 2023;Norreton & Schörling, 2023;Zhai, 2023), objective (Tzirides et al., 2023a), consistent (Baidoo-Anu & Owusu Ansah, 2023; Dai et al., 2023;Mizumoto & Eguchi, 2023;Tubino & Adachi, 2022;Tzirides et al., 2023a;Zapata et al., 2023), logistical effective (Dai et al., 2023;Mizumoto & Eguchi, 2023;Norreton & Schörling, 2023;Tzirides et al., 2023a;Zapata et al., 2023), convenient (Mizumoto & Eguchi, 2023;Tzirides et al., 2023a), quicker (Baidoo-Anu & Owusu Ansah, 2023; Cotton et al., 2023;Mizumoto & Eguchi, 2023;Norreton & Schörling, 2023;Tubino & Adachi, 2022;Zapata et al., 2023), and ubiquitous (Cotton et al., 2023;Dai et al., 2023;Tzirides et al., 2023a) as compared to peer feedback. In a study conducted by Seo et al. (2021), it was evident that instructors unanimously recognized the value of immediate feedback facilitated by AI in aiding student learning. ...
... Studies have showcased that generative models trained on a dataset of human-graded essays could efficiently grade essays written by students (; Kim, Park, & Lee, 2019 as cited in Baidoo-Anu & Owusu Ansah, 2023; Otaki, 2023;Tzirides et al., 2023a) with a certain degree of accuracy (Mizumoto & Eguchi, 2023). Research suggests that the GenAI reviews are more critical (Tubino & Adachi, 2022;Tzirides et al., 2023a), readable (Dai et al., 2023;Tzirides et al., 2023a), personalized (Brynjolfsson et al., 2023;Cotton et al., 2023;Farrokhnia et al., 2023;Norreton & Schörling, 2023;Zhai, 2023), objective (Tzirides et al., 2023a), consistent (Baidoo-Anu & Owusu Ansah, 2023; Dai et al., 2023;Mizumoto & Eguchi, 2023;Tubino & Adachi, 2022;Tzirides et al., 2023a;Zapata et al., 2023), logistical effective (Dai et al., 2023;Mizumoto & Eguchi, 2023;Norreton & Schörling, 2023;Tzirides et al., 2023a;Zapata et al., 2023), convenient (Mizumoto & Eguchi, 2023;Tzirides et al., 2023a), quicker (Baidoo-Anu & Owusu Ansah, 2023; Cotton et al., 2023;Mizumoto & Eguchi, 2023;Norreton & Schörling, 2023;Tubino & Adachi, 2022;Zapata et al., 2023), and ubiquitous (Cotton et al., 2023;Dai et al., 2023;Tzirides et al., 2023a) as compared to peer feedback. In a study conducted by Seo et al. (2021), it was evident that instructors unanimously recognized the value of immediate feedback facilitated by AI in aiding student learning. ...
This paper explores the integration of generative artificial intelligence (AI) in education to enhance feedback processes and improve learning experiences. The main goal of the study is to investigate the potential of generative AI for feedback, specifically in complementing peer feedback practices among graduate students enrolled at a US-based university during the 2023 academic term. Drawing on existing literature, the study examines the application of generative AI and its implications for feedback mechanisms. Employing an exploratory research design, the study gathers both quantitative and qualitative data through post-course surveys to address key research questions regarding the quality, usefulness, and actionability of peer and AI reviews, as well as their respective advantages and disadvantages. Findings indicate that peer reviews were consistently perceived slightly higher across all three dimensions compared to AI reviews, with thematic analysis revealing the unique strengths and limitations of each review type. This research underscores the importance of integrating human expertise with AI technology in feedback mechanisms, offering practical insights for educators, instructional designers, and policymakers seeking to enhance feedback experiences through emerging digital technologies.
... Nevertheless, educators and AIED researchers have good reason to explore the use of AI in education. In the face of pandemic learning loss and the looming expiration of pandemic relief funds [24,55], AI tools are touted as a relatively inexpensive way to meet learners where they are rather than providing the same lessons or assignments to students with different background knowledge or learning needs [13,18,59]. AI tools also have the potential to make teachers' jobs easier; for example, by providing feedback and support to students outside of teachers' working hours or handling administrative and other non-instruction responsibilities, such as lesson plan development 7 and grading 8 [13]. ...
... if something comes out on their own accord instead of me blatantly calling the whatever-you-call-it-police, I'll have more time to cultivate that discussion appropriately, " E4; also E5, E10, E15, E16, E18, E23). Other educators reported changing their teaching practices to account for AI, asking "if a computer can do the task that you assigned, is it the most meaningful task?" (E21; also E3, 18 https://www.turnitin.com/products/originality/ E13, E17). ...
Education technologies (edtech) are increasingly incorporating new features built on large language models (LLMs), with the goals of enriching the processes of teaching and learning and ultimately improving learning outcomes. However, the potential downstream impacts of LLM-based edtech remain understudied. Prior attempts to map the risks of LLMs have not been tailored to education specifically, even though it is a unique domain in many respects: from its population (students are often children, who can be especially impacted by technology) to its goals (providing the correct answer may be less important for learners than understanding how to arrive at an answer) to its implications for higher-order skills that generalize across contexts (e.g., critical thinking and collaboration). We conducted semi-structured interviews with six edtech providers representing leaders in the K-12 space, as well as a diverse group of 23 educators with varying levels of experience with LLM-based edtech. Through a thematic analysis, we explored how each group is anticipating, observing, and accounting for potential harms from LLMs in education. We find that, while edtech providers focus primarily on mitigating technical harms, i.e., those that can be measured based solely on LLM outputs themselves, educators are more concerned about harms that result from the broader impacts of LLMs, i.e., those that require observation of interactions between students, educators, school systems, and edtech to measure. Overall, we (1) develop an education-specific overview of potential harms from LLMs, (2) highlight gaps between conceptions of harm by edtech providers and those by educators, and (3) make recommendations to facilitate the centering of educators in the design and development of edtech tools.
... Son et al. (2023) assert that through repeated practice and correction, these tools assess learners' writing and give precise feedback, enabling them to develop and improve their writing abilities. This feedback has been proven to help students identify and correct their errors more efficiently compared to traditional feedback methods (Cao & Zhong, 2023;Dai et al., 2023). Additionally, Loyola and Helan (2024) add that AI tools can adapt to the proficiency level of learners, providing more relevant and challenging tasks as students progress in their learning. ...
... Some students expressed concerns that relying heavily on ChatGPT for corrections might lead to a passive learning attitude. This aligns with the observations by Dai et al. (2023), who stressed the importance of maintaining a balance between AI assistance and traditional teaching methods to foster active learning and critical engagement with feedback. ...
Introduction: The integration of AI in educational settings offers significant potential for enhancing learning experiences, particularly in Content and Language Integrated Learning (CLIL) contexts. AI tools, such as ChatGPT, provide personalized feedback on writing, addressing issues like unclear content, grammatical errors, or poor vocabulary. This study examines students' perceptions of AI-assisted feedback in a business CLIL course and evaluates the actual improvements in their writing based on the feedback provided by AI. Methodology: University students (n=205) participated in a 15-week Data Description writing course, using ChatGPT to receive specific criteria-based feedback on weekly compositions. Students revised their drafts based on this feedback before their submission. A survey (n=192) assessed their experiences and the perceived impact on writing skills and task efficiency. Additionally, a sample (n=336) of the writing compositions was coded and analyzed to evaluate linguistic enhancement. Results: Results indicate that students found AI feedback beneficial for improving writing skills and appreciated its immediacy and specificity. However, concerns were noted about the complexity and relevance of the feedback. Discussions: Despite these issues, students responded positively, showing significant improvement in content accuracy and linguistic proficiency. Conclusions: The study highlights the potential of AI tools and the need for refining AI feedback mechanisms.
... 2 Related Work 2.1 Writing with AI AI-powered writing tools now span a spectrum of feedback provision, from surface-level grammar and spelling corrections such as Grammarly 1 to broader adjustments to structure (Weber et al., 2024;Meyer et al., 2024;Han et al., 2024;Yang et al., 2024), language (Wambsganss et al., 2022; 1 www.grammarly.com Meyer et al., 2024;Han et al., 2024), and adherence to writing requirements (Dai et al., 2023;Han et al., 2024). These tools have shifted from simple rulebased systems (Ding and Zou, 2024;Ware, 2011) to sophisticated AI-driven assistants that analyze and generate writing feedback at multiple levels. ...
... Some studies have been conducted to highlight the significance of AI in learning lexical items and language structure (Wang & Guo, 2023), in giving feedback and invaluable data (Dai et al., 2023;Rudolph et al., 2023), promoting learners' motivation (Ali et al., 2023), in creating fine texts (Gao et al., 2023), in developing learners' writing skill (Mahapatra, 2021;Yan, 2023) AI-generated feedback with human tutors, finding no significant difference in outcomes, suggesting that a blended approach might be most effective. However, the study does not address the nuances of integrating such feedback seamlessly into traditional teaching methods. ...
The rise of artificial intelligence (AI) chatbots has significantly transformed the educational landscape, offering numerous opportunities for innovation and change. The current study assessed the comparative effects of ChatGPT-based instruction and Microsoft Copilot in helping Iranian English-as-a-foreign language (EFL) learners identify and realize interactional metadiscourse markers (IMMs) in argumentative writing and exploring their attitudes towards these two chatbots. Grounded in the theoretical framework of IM, this study followed a convergent parallel design. The study involved 90 male and female language learners randomly assigned to three groups: ChatGPT-based group (n = 30), Microsoft Copilot group (n = 30), and a control group (n = 30). The experimental groups were provided with 10 prompts per session for the implementation of IMMs, resulting in 60 prompts across six sessions. Instruction included initial training on using the respective AI tools, followed by practice sessions focusing on identifying and using IMMs in writing. The control group received conventional instruction, which involved identifying IMMs in reading passages with the guidance of the instructor. Interview questions were designed to elicit perspectives from learners on their experiences with ChatGPT and Microsoft Copilot. The responses from the interview data concerning learners' perceptions were analyzed through thematic analysis. Results showed that the Microsoft Copilot group demonstrated superior performance compared to the other two groups in identifying IMMs in the posttest. However, a one-way analysis of covariance (ANCOVA) showed that the difference between the ChatGPT-based group and the control group was not statistically significant. Additionally, responses to semi-structured interviews indicated that all learners had a positive perception of Microsoft Copilot for employing IMMs in argumentative writing. This study contributes to the field by providing empirical evidence on the effectiveness of specific AI-driven chatbots in enhancing critical writing skills, specifically through the lens of IMMs.
... Therefore, it is of immense importance that students are provided with clear feedback, a comprehensive explanation of the scoring process, and the rationale for the feedback provided for the students to understand their strengths and the areas that they need to further work on (Obilor, 2019). To further support this argument, Lin et al. (2023) encourage every teacher to utilize rubrics in providing feedback to their students for their clear evaluation and criteria which support students with clear expectations of what their teachers want as well as providing clear outlines for the areas that need improvement. ...
The current study aims to investigate Kurdish EFL students' views of the assessment process conducted at EFL departments of public universities in the Kurdistan Region of Iraq (KRI). Due to the fact that assessment is the core factor for students' learning, involvement, and evaluation as the only gauge for their progress and development, much attention needs to be given to the assessment process. This study specifically aims at studying the perceptions of the Kurdish EFL students of the criteria including design, administration, purpose, effectiveness and washback, scoring and grading, and feedback of testing and assessment process. Hence, for the purpose of data collection, a questionnaire was administered to 116 students of semesters 3,5, and 7 at the English language departments of some public universities in the KRI during the academic year 2024-2025. Cronbach Alpha was used to analyze the reliability of the items of the questionnaire along with SPSS (version 25) to analyze the mean values of the items and ANOVA was utilized to compare the mean values across the six criteria. Findings indicate significant challenges in the alignment and execution of testing and assessment processes in higher education. While testing and assessment items align with course objectives, they often fail to adequately measure critical thinking and comprehensive language skills. Procedural issues, including unclear instructions, unfair scoring and grading practices, and overemphasis on grading rather than fostering students' progress and engagement, have badly affected the effectiveness of assessments. Additionally, environmental factors such as cheating, unsupportive classroom dynamics, and poor seating quality negatively impact students' performance. A lack of constructive feedback further hinders the development of students' overall skills and learning outcomes. The findings further highlight the need for a holistic approach to assessment that emphasizes student growth, fair evaluation, and the integration of diverse language competencies.
... They include automated writing evaluation tools (e.g. Pigai) that provide feedback, scores and suggestions on student writing (Dai et al., 2023;Huang et al., 2022). Other tools diagnose students' strengths, weaknesses and knowledge gaps (Liu et al., 2017) or identify those at risk of failure (Luckin et al., 2022). ...
This study explored the perspectives of English instructors from Thai higher education institutions, with a focus on teachers' familiarity with generative artificial intelligence (GenAI) and its potential impact on teachers' professional roles and responsibilities. The results suggested that GenAI tools may allow English instructors to transition from traditional teachers to facilitators by using the tools to assist with both routine writing tasks and high-level academic work. Meanwhile, it was found that instructors worried about possible over-reliance on GenAI. The participants emphasised that human instructors were still needed, although their roles needed to evolve. Significant gaps were identified in the competencies related to professional development, curriculum design, teacher training programmes, ethics, and responsibility. The findings may support the professional growth of current and future English instructors and facilitate the incorporation of GenAI in teaching practice. The findings also underscore the necessity of comprehensive GenAI training for pre-service teachers, the development of robust guidelines to navigate ethical challenges, and the examination of the impact of GenAI tools on student engagement and learning outcomes.
... LLMs have garnered significant attention in the field of natural language processing (NLP) because of the ability to generate human-like text. This feature makes LLMs a promising technology in educational settings (Finnie-Ansley et al., 2022;Dai et al., 2023;Pardos & Bhandari, 2023), where the provision of personalized feedback is integral to scaffold learning effectively (Jackson & Graesser, 2007;Hull & du Boulay, 2015). ...
Addressing the challenge of generating personalized feedback for programming assignments is demanding due to several factors, like the complexity of code syntax or different ways to correctly solve a task. In this experimental study, we automated the process of feedback generation by employing OpenAI’s GPT-3.5 model to generate personalized hints for students solving programming assignments on an automated assessment platform. Students rated the usefulness of GPT-generated hints positively. The experimental group (with GPT hints enabled) relied less on the platform's regular feedback but performed better in terms of percentage of successful submissions across consecutive attempts for tasks, where GPT hints were enabled. For tasks where the GPT feedback was made unavailable, the experimental group needed significantly less time to solve assignments. Furthermore, when GPT hints were unavailable, students in the experimental condition were initially less likely to solve the assignment correctly. This suggests potential over-reliance on GPT- generated feedback. However, students in the experimental condition were able to correct reasonably rapidly, reaching the same percentage correct after seven submission attempts. The availability of GPT hints did not significantly impact students' affective state.
... LLMs and Education: Prior research indicates that LLMs can serve as effective tools for generating educational content [12] and highquality and effective feedback [2,5,13,17,18]. Sarsa et al. [26] show that OpenAI Codex -a commercial large-language model specialized for code generation -can be used for constructing novel and sensible programming exercises as well as their answers and explanation for educational purposes. Ochieng [20] shows LLMs can be used for crafting substantial in-context questions that foster guided learning. ...
The integration of LLM-generated feedback into educational settings has shown promise in enhancing student learning outcomes. This paper presents a novel LLM-driven system that provides targeted feedback for conceptual designs in a Database Systems course. The system converts student-created entity-relationship diagrams (ERDs) into JSON format, allows the student to prune the diagram by isolating a relationship, extracts relevant requirements for the selected relationship, and utilizes a large language model (LLM) to generate detailed feedback. Additionally, the system creates a tailored set of questions and answers to further aid student understanding. Our pilot implementation in a Database System course demonstrates effective feedback generation that helped the students improve their design skills.
... Teacher-facing systems are tailored to enhance pedagogical methodologies or alleviate teaching burdens. Key inclusions are AWE tools like Criterion and Pigai that offer insights into students' writing and render scores (Dai et al., 2023;Huang et al., 2022). Moreover, diagnostic tools pinpoint students' strengths, areas of improvement, and knowledge voids (Liu et al., 2017), while also identifying those teetering on the edge of academic failure (Luckin et al., 2022). ...
This study explored the impact of ChatGPT on classroom teaching and lesson preparation through the experiences of 12 English language teachers at a Hong Kong university. The primary research method, qualitative interviews, provided critical insights into the intersection of artificial intelligence (AI) and pedagogy. Four themes emerged: increased reliance on ChatGPT for lesson planning due to its convenience; the transformation of teaching methodologies, with ChatGPT becoming an important tool; the challenges presented by ChatGPT (inappropriate content; the risk of becoming over-reliant on it); and the new opportunities that ChatGPT offers for differentiated instruction and customised assessment. While ChatGPT significantly reshaped teaching practice, it was clear that systematic training was needed to allow them to leverage its benefits fully. Addressing challenges such as adjusting content to student abilities and balancing AI with other resources is vital to the optimisation of AI in pedagogy. This study underscores the need for continuous professional development, balanced resource management and curriculum revisions to incorporate AI in the teaching of English; encourages the exploration of innovative AI-enhanced pedagogies; contributes to the understanding of how AI is reshaping pedagogical practice; and offers practical recommendations for educators navigating the integration of AI into their teaching processes.
... In the context of evaluating students' assignments, Dai et al. 's study [23] explores the use of ChatGPT, as a pre-trained language model, in providing feedback to students. Recognizing the effectiveness of feedback in enhancing learning, the researchers aim to leverage ChatGPT's capabilities to generate natural-sounding and detailed responses. ...
The evaluation of student essay corrections has become a focal point in understanding the evolving role of Artificial Intelligence (AI) in education. This study aims to assess the accuracy, efficiency, and cost-effectiveness of ChatGPT's essay correction compared to human correction, with a primary focus on identifying and rectifying grammatical errors, spelling, sentence structure, punctuation, coherence, relevance, essay structure, and clarity. The research involves collecting essays from 100 randomly selected university students, covering diverse themes, with anonymity maintained and no prior corrections by humans or AI. An analysis sheet, outlining linguistic and informational elements for evaluation, serves as a benchmark for assessing the quality of corrections made by ChatGPT and humans. The study reveals that ChatGPT excels in fundamental language mechanics, demonstrating superior performance in areas like grammar, spelling, sentence structure, relevance, and supporting evidence. However, thematic consistency remains an area where human evaluators outperform the AI. The findings emphasize the potential for a balanced approach, leveraging both human and AI strengths, for a comprehensive and effective essay correction process.
... In this vein, building on Reeves and Nass (1996), even though LLMs are clearly non-human, people tend to ascribe them human characteristics (e.g., trust) based on similar cues (e.g., expertise). There is evidence that LLM-and instructor-feedback align (Dai et al., 2023). However, LLMs can be biased by their developers, the training data, and/or any learning that occurs during the LLM's lifecycle (i.e., aspects determining the LLM's competence). ...
Feedback is an integral part of learning in higher education and is increasingly being provided to students via modern technologies like Large Language Models (LLMs). But students’ perception of feedback from LLMs vs. feedback from educators remains unclear even though it is an important facet of feedback effectiveness. Further, feedback effectiveness can be negatively influenced by various factors; For example, (not) knowing certain characteristics about the feedback provider may bias a student’s reaction to the feedback process. To assess perceptions of LLM feedback and mitigate the negative effects of possible biases, this study investigated the potential of providing provider-information about feedback providers. In a 2×2 between-subjects design with the factors feedback provider (LLM vs. educator) and provider-information (yes vs. no), 169 German students evaluated feedback message and provider perceptions. Path analyses showed that the LLM was perceived as more trustworthy than an educator and that the provision of provider-information led to improved perceptions of the feedback. Furthermore, the effect of the provider and the feedback on perceived trustworthiness and fairness changed when provider-information was provided. Overall, our study highlights the importance of further research on feedback processes that include LLMs due to their influential nature and suggests practical recommendations for designing digital feedback processes.
... The quality of feedback provided by ChatGPT has been questioned by Dai et al. (2023) who concluded that the tool, while good at providing task and process type feedback, was relatively poor at providing feedback at the self-regulatory level (Hattie & Timperley, 2007). The role of educators in checking the quality of feedback and supporting students' ability to evaluate the accuracy and quality of feedback is crucial (Laato et al., 2023). ...
In this chapter we explore how generative artificial intelligence (AI) such as ChatGPT can be used
to support students’ development of assessment and feedback skills through a principled approach. Whether students should use AI or not is a redundant question given the AI skills and attributes that students now need to be successful in their higher education and increasingly integrated working lives. We argue that those institutions that have prioritised and properly resourced AI literacy training will see the greatest gains in assessment outcomes for students. Training will need to centre ethics, bias, and fairness, and how to use the data generated by AI to inform our understanding of human and AI integrated learning. AI developments provide a significant opportunity to rethink assessment design to ensure an evidence-informed approach that enables a focus on the knowledge and skills we value: What is the key knowledge and what are the skills that students will need to take forward into their futures?
... Early research on C-LLMs in education points both to their constructive and worryingly disruptive potentials. Addressing the constructive potentials first: ChatGPT (v.3.5) has been shown to offer 'more detailed feedback that fluently and coherently summarizes students' performance than human instructors', demonstrating 'high agreement with the instructor when assessing the topic of students' assignments' (Dai et al. 2023). It has been demonstrated that GPTs can be used constructively to support literacy development, from young children's storytelling (Li and Xu 2023) to academic writing (Buruk 2023;Liu et al. 2023). ...
The launch of ChatGPT in November 2022 precipitated panic among some educators while prompting qualified enthusiasm from others. ChatGPT is an example of Generative AI, a combination of technologies that will deliver computer-generated text, images, and other digitized media. This paper examines the implications for education of one generative AI technology, chatbots responding from large language models (C-LLM). It reports on an application of a C-LLM to AI review and assessment of complex student work. In a concluding discussion, the paper explores the intrinsic limits of generative AI, bound as it is to language corpora and their textual representation through binary notation. Within these limits, we suggest the range of emerging and potential applications of Generative AI in education.
... [3] developed BART based Insta-Reviewer for automatically generating instant textual feedback on students' project reports, using state-of-the-art natural language processing (NLP) models and found that that feedback generated by Insta-Reviewer on real students' project reports can achieve near-human performance. [7] investigate the feasibility of using ChatGPT to provide students with feedback to help them learn better and found that ChatGPT can generate more detailed feedback that fluently and coherently summarizes students' performance than human instructors. [8] developed AcaWriter a web-based writing analytics tool that provides automated formative feedback on the rhetorical moves in students' texts. ...
... This also extends to the idea of using GenAI outputs for feedback, and the results of empirical studies suggest that automated writing evaluation from GenAI tools may have a place in English as a New Language (ENL) instruction (Escalante et al., 2023), and that GPT tools can be used for automated essay scoring with L2 writers (Mizumoto & Eguchi, 2023). Dai et al. (2023) was found to provide feedback that was more detailed than that of an instructor, yet showed high levels of agreement with the marking of the instructor. On the other hand, these systems are at an early stage in language assessment (Chiu et al., 2023), and questions remain over automated markings' ethics, legality, cost, and privacy (Kumar, 2023). ...
The rapid advancement of Generative Artificial Intelligence (GenAI) presents both opportunities and challenges for English for Academic Purposes (EAP) instruction. This paper proposes an adaptation of the AI Assessment Scale (AIAS) specifically tailored for EAP contexts, termed the EAP-AIAS. This framework aims to provide a structured approach for integrating GenAI tools into EAP assessment practices while maintaining academic integrity and supporting language development. The EAP-AIAS consists of five levels, ranging from "No AI" to "Full AI", each delineating appropriate GenAI usage in EAP tasks. We discuss the rationale behind this adaptation, considering the unique needs of language learners and the dual focus of EAP on language proficiency and academic acculturation. This paper explores potential applications of the EAP-AIAS across various EAP assessment types, including writing tasks, presentations, and research projects. By offering a flexible framework, the EAP-AIAS seeks to empower EAP practitioners seeking to deal with the complexities of GenAI integration in education and prepare students for an AI-enhanced academic and professional future. This adaptation represents a step towards addressing the pressing need for ethical and pedagogically sound AI integration in language education.
... GenAI applications may also be used to develop assessments and provide feedback. It has been suggested that teachers can use tools like ChatGPT to generate prompts for open-ended questions and develop rubrics (Baidoo-Anu & Owusu Ansah, 2023), and for scoring student work (Swiecki et al., 2022), providing feedback (Crawford et al., 2023;Dai et al., 2023), or wholly automating the marking process (Mizumoto & Eguchi, 2023;Ramesh & Sanampudi, 2022). However, critics of these approaches point to privacy and data risks (Nguyen et al., 2023) and the shifting of assessment responsibility to the developers of GenAI tools rather than the educator (Swiecki et al., 2022). ...
The rise of Artificial Intelligence (AI) and Generative Artificial Intelligence (GenAI) in higher education necessitates assessment reform. This study addresses a critical gap by exploring student and academic staff experiences with AI and GenAI tools, focusing on their familiarity and comfort with current and potential future applications in learning and assessment. An online survey collected data from 35 academic staff and 282 students across two universities in Vietnam and one in Singapore, examining GenAI familiarity, perceptions of its use in assessment marking and feedback, knowledge checking and participation, and experiences of GenAI text detection. Descriptive statistics and reflexive thematic analysis revealed a generally low familiarity with GenAI among both groups. GenAI feedback was viewed negatively; however, it was viewed more positively when combined with instructor feedback. Academic staff were more accepting of GenAI text detection tools and grade adjustments based on detection results compared to students. Qualitative analysis identified three themes: unclear understanding of text detection tools, variability in experiences with GenAI detectors, and mixed feelings about GenAI's future impact on educational assessment. These findings have major implications regarding the development of policies and practices for GenAI-enabled assessment and feedback in higher education.
... The paper suggest that ChatGPT can be a good feedback tool in large-size writing classes with proper student training. In another study by Dai et al. (2023), the study investigated the feasibility of using ChatGPT to provide students with feedback to help them learn better. The results show that ChatGPT is capable of providing feedback on the process of students completing the task, which benefits students developing learning skills. ...
This study aims to investigate the utilization of AI-assisted feedback by EFL college learners in their English writing. Specifically, the research sought to explore how writing assisted by ChatGPT improves students’ writing skills in comparison to peer feedback. Additionally, the study aimed to understand the learners’ perceptions regarding the use of ChatGPT in editing English writing. Participants were tasked with submitting their presentation scripts before their speaking exams. They received an instructional session on how to use the tool effectively and were told useful prompts for using ChatGPT. The collected writings were analyzed holistically by two experienced EFL instructors, and analytically by using Grammarly (2022), a free online grammar and spelling checker, to identify characteristics. The analysis revealed that there was a significant difference in holistic scores including content and organization. The experimental group who used the AI tool had significantly reduced grammatical and lexical errors. However, no significant difference between the groups was found in the word counts and use of vocabulary types. Additionally, reflections from participants indicates a positive attitude toward the use of AI-assisted feedback in English writing but there were some concerns about reliability and over-reliance. The study suggests pedagogical implications for the effective integration of AI-assistance in English writing based on these findings.
... Early research into using generative AI for AES and feedback provision have shown positive results. In a preprint by Dai et al. (2023), researchers used ChatGPT to formulate feedback on 102 learner essays and compared the feedback to the instructors. They found that ChatGPT wrote more detailed and easier to read feedback than the teacher did, but struggled to achieve a high level of agreement with the teacher's feedback on certain writing points. ...
... Another study that examined the texts generated by ChatGPT focused on the differences between feedback texts given to students through evaluation by ChatGPT and evaluation by instructors in a data science course (Dai et al., 2023). Additionally, Cooper (2023) examined ChatGPT's responses to science education-related questions and suggested that ChatGPT could be used in designing science lesson materials, preparing exam questions, and creating evaluation rubrics. ...
The interaction between humans and technologies has a long historical lineage. Presently, generative artificial intelligence tools such as ChatGPT are being employed for various purposes, especially in education and across diverse domains. These artificial intelligence tools carry significant potential to induce profound transformations in the overarching mission and vision of higher education institutions, thereby exerting a considerable influence on higher education institutions dedicated to training teaching professionals. In fact, tools influence and guide human activities through the genesis or development of “utilization schemes”. This study aims to explore mathematics teacher candidates' instrumentation process of ChatGPT for developing lesson plans with particular attention to the utilization purposes and utilization schemes. The research was designed as a case study conducted during the spring semester of the 2022-2023 academic year. The participants included ten mathematics teacher candidates. The study lasted for eight weeks. The data collection tools used in the study included journals filled out by the participants at least once a week when they interacted with artificial intelligence, screenshots of the conversations between the participants and ChatGPT, and the lesson plans prepared by the participants. The collected data was analyzed within the framework of the instrumental approach. The content analysis results revealed that during the lesson planning process, teacher candidates used ChatGPT in six different utilization types associated with six distinct utilization purposes.
... Wang et al. (2023), for example, have praised the timeliness of AI feedback while warning of inherent cultural biases in the AI evaluation process. Dai et al. (2023) emphasize the need to establish an effective feedback model by which to evaluate the efficacy of AI generated feedback. Researchers Buşe and Căbulea (2023) have serious reservations about AI's impact on creative thinking, human interaction, and technology dependence, while Cardon et al. (2023) argue that because AI-assisted writing is here to stay, instructors will have to greatly change how and what they teach. ...
This study explores the feasibility of using AI technology, specifically ChatGPT-3, to provide reliable, meaningful, and personalized feedback. Specifically, the study explores the benefits and limitations of using AI-based feedback in language learning; the pedagogical frameworks that underpin the effective use of AI-based feedback; the reliability of ChatGPT-3’s feedback; and the potential implications of AI integration in language instruction. A review of existing literature identifies key themes and findings related to AI-based teaching practices. The study found that social cognitive theory (SCT) supports the potential use of AI chatbots in the learning process as AI can provide students with instant guidance and support that fosters personalized, independent learning experiences. Similarly, Krashen’s second language acquisition theory (SLA) was found to support the hypothesis that AI use can enhance student learning by creating meaningful interaction in the target language wherein learners engage in genuine communication rather than focusing solely on linguistic form. To determine the reliability of AI-generated feedback, an analysis was performed on student writing. First, two rubrics were created by ChatGPT-3; AI then graded the papers, and the results were compared with human graded results using the same rubrics. The study concludes that e-Learning arning certainly has great potential; besides providing timely, personalized learning support, AI feedback can increase student motivation and foster learning independence. Not surprisingly, though, several caveats exist. It was found that ChatGPT-3 is prone to error and hallucination in providing student feedback, especially when presented with longer texts. To avoid this, rubrics must be carefully constructed, and teacher oversight is still very much required. This study will help educators transition to the new era of AI-assisted e-Learning by helping them make informed decisions about how to provide useful AI feedback that is underpinned by sound pedagogical principles.
... In the context of feedback, AI-powered ChatGPT introduces what is referred to as AIgenerated feedback (Farrokhnia et al., 2023). While the literature suggests that ChatGPT has the potential to facilitate feedback practices (Dai et al., 2023;Katz et al., 2023), this literature is very limited and mostly not empirical leading us to realize that our current comprehension of its capabilities in this regard is quite restricted. Therefore, we lack a comprehensive understanding of how ChatGPT can effectively support feedback practices and to what degree it can improve the timeliness, impact, and personalization of feedback, which remains notably limited at this time. ...
Peer feedback is introduced as an effective learning strategy, especially in large-size classes where teachers face high workloads. However, for complex tasks such as writing an argumentative essay, without support peers may not provide high-quality feedback since it requires a high level of cognitive processing, critical thinking skills, and a deep understanding of the subject. With the promising developments in Artificial Intelligence (AI), particularly after the emergence of ChatGPT, there is a global argument that whether AI tools can be seen as a new source of feedback or not for complex tasks. The answer to this question is not completely clear yet as there are limited studies and our understanding remains constrained. In this study, we used ChatGPT as a source of feedback for students’ argumentative essay writing tasks and we compared the quality of ChatGPT-generated feedback with peer feedback. The participant pool consisted of 74 graduate students from a Dutch university. The study unfolded in two phases: firstly, students’ essay data were collected as they composed essays on one of the given topics; subsequently, peer feedback and ChatGPT-generated feedback data were collected through engaging peers in a feedback process and using ChatGPT as a feedback source. Two coding schemes including coding schemes for essay analysis and coding schemes for feedback analysis were used to measure the quality of essays and feedback. Then, a MANOVA analysis was employed to determine any distinctions between the feedback generated by peers and ChatGPT. Additionally, Spearman’s correlation was utilized to explore potential links between the essay quality and the feedback generated by peers and ChatGPT. The results showed a significant difference between feedback generated by ChatGPT and peers. While ChatGPT provided more descriptive feedback including information about how the essay is written, peers provided feedback including information about identification of the problem in the essay. The overarching look at the results suggests a potential complementary role for ChatGPT and students in the feedback process. Regarding the relationship between the quality of essays and the quality of the feedback provided by ChatGPT and peers, we found no overall significant relationship. These findings imply that the quality of the essays does not impact both ChatGPT and peer feedback quality. The implications of this study are valuable, shedding light on the prospective use of ChatGPT as a feedback source, particularly for complex tasks like argumentative essay writing. We discussed the findings and delved into the implications for future research and practical applications in educational contexts.
Large language models (LLMs) have gained considerable attention for their excellent natural language processing capabilities. Nonetheless, these LLMs present many challenges, particularly in the realm of trustworthiness. Therefore, ensuring the trustworthiness of LLMs emerges as an important topic. This chapter presents the TrustLLM framework (Sun et al., Trustllm: Trustworthiness in large language models. International Conference on Machine Learning (2024)), a comprehensive study of trustworthiness in LLMs, including principles for different dimensions of trustworthiness, established benchmark, evaluation, and analysis of trustworthiness for mainstream LLMs. Specifically, we first introduce a set of principles for trustworthy LLMs that span eight dimensions. Based on these principles, we further establish a benchmark across six dimensions including truthfulness, safety, fairness, robustness, privacy, and machine ethics. Based on the evaluation of 16 mainstream LLMs in TrustLLM (Sun et al., Trustllm: Trustworthiness in large language models. International Conference on Machine Learning (2024)), consisting of over 30 datasets, this chapter summarizes the main findings.
This study aims to provide a comprehensive framework for examining the voices of students regarding the use of ChatGPT as an AWE tool, and what challenges and conveniences are associated with using ChatGPT. This narrative inquiry research explores the diverse perspectives of students in the context of ChatGPT in Automated Writing Evaluation. The findings of this study explained that students' perspectives on the integration of AI technologies in higher education highlight the need to ethically integrate AI technologies into education, taking into account responsible practices. Therefore, it is suggested that the integration of ChatGPT in the students' writing process should be accompanied by careful consideration of the ethical ramifications and potential impact on students’ academic progress.
This study investigates what Artificial Intelligence (AI) and the generative pre-trained transformer (ChatGPT) software offer to engineering education. First, our literature review shows key challenges and considerations linked to infrastructure and resource requirements, digital inequality, data privacy, copyright and some complex ethical issues that come with this pedagogical transformation. However, the article next contextualizes AI and ChatGPT within foundational learning theories such as Constructivist Learning Theory, Personalized Learning Theory, Cognitive Load Theory, and Vygotsky's Zone of Proximal Development, showing how AI and ChatGPT can shift educational practices towards more individualized, dynamic, and active learning experiences. By bringing together practical considerations demanding attention when introducing AI and ChatGPT to engineering course, and the learning theories the digital tools can promisingly enact, this study significantly contributes to ongoing discourse on AI and ChatGPT in engineering education. Different stakeholders, including educators, students, institutions, and industry partners contemplating AI and ChatGPT introduction are likely to be weighing up options. Readers considering whether pedagogical benefits are worth the considerable care needed to introduce these tools in their own engineering faculty should find this article helpful.
This case study explores ChatGPT’s practical use in analysing textual content from an educational forum. It examines three texts from a sample of 26, written by first‑year university students during a guided English as a foreign language activity. This highlights generative AI’s potential in teaching. ChatGPT is used to create an evaluative rubric and conduct linguistic analysis. It shows logical cohesion, output consistency, and an ability to identify both global and specific textual features. With few documented case studies on AI in education, this contribution presents a practical experience to foster experimentation and results.
The article aims to explore the potential of generative artificial intelligence (AI) for assessing written work and providing feedback on it. The goal of this research is to determine the possibilities and limitations of generative AI when used for evaluating students’ written production and providing feedback. To accomplish the aim, a systematic review of twenty-two original studies was conducted. The selected studies were carried out in both Russian and international contexts, with results published between 2022 and 2025. It was found that the criteria-based assessments made by generative models align with those of instructors, and that generative AI surpasses human evaluators in its ability to assess language and argumentation. However, the reliability of this evaluation is negatively affected by the instability of sequential assessments, the hallucinations of generative models, and their limited ability to account for contextual nuances. Despite the detailisation and constructive nature of feedback from generative AI, it is often insufficiently specific and overly verbose, which can hinder student comprehension. Feedback from generative models primarily targets local deficiencies, while human evaluators pay attention to global issues, such as the incomplete alignment of content with the assigned topic. Unlike instructors, generative AI provides template-based feedback, avoiding indirect phrasing and leading questions contributing to the development of self-regulation skills. Nevertheless, these shortcomings can be addressed through subsequent queries to the generative model. It was also found that students are open to receiving feedback from generative AI; however, they prefer to receive it from instructors and peers. The results are discussed in the context of using generative models for evaluating written work and formulating feedback by foreign language instructors. The conclusion emphasises the necessity of a critical approach to using generative models in the assessment of written work and the importance of training instructors for effective interaction with these technologies.
This study focuses on both quantitative and qualitative assessments of automatic grammatical error identification, correction, and explanation for learners of Chinese using four large language models (LLMs) (namely, BART CGEC, GPT 4.0, Bard, and Claude 2) from linguistic and educational viewpoints. It was found that general-purpose chat LLMs like GPT 4.0, Bard, and Claude 2 outperformed those specifically designed for Chinese grammatical error correction such as BART CGEC. In particular, Claude 2 excelled in precision and recall for error correction, achieving nearly 95% accuracy with a modified prompt, while GPT 4.0 and Bard lagged behind with around 87.5% precision and 80% recall, and 68.97% precision and 60.6% recall, respectively. Although Claude 2 achieved approximately 66% accuracy in error identification and error explanation, its high precision and recall in error correction made it a strong candidate for an intelligent Chinese grammar checker. Our study suggests the significance of prompt engineering in using LLMs effectively, leading to an 8% improvement in error correction precision for both GPT 4.0 and Claude 2 and over 15% recall improvement in GPT 4.0. Prompt engineering plays a crucial role in optimizing AI tool performance, paving the way for their integration into language learning processes. It is anticipated that LLMs will dramatically revolutionize the outlook of language learning in the near future.
Generative artificial intelligence provides both challenges and opportunities for higher education. Few studies to date have accounted for student experiences of purposeful use of generative AI. This article reports on a mixed methods study of two university classes using ChatGPT to generate feedback on written assignments. Students’ attitudes were collected through a survey, lab reports, and in-class discussions. The analyses show that students experienced their role as feedback receiver qualitatively different in the AI feedback situation compared to teacher- and peer feedback, because they themselves had to assume all the responsibility for the critical judgment of prompts and replies. Students felt that asking ChatGPT for feedback was more frustrating but emotionally easier than asking peers or teachers, which points to important differences in the dynamics of sociality and interaction between feedback receivers and human vs. AI feedback givers.
Purpose:
Self-testing has been proven to significantly improve not only simple learning outcomes, but also higher-order skills such as clinical reasoning in medical students. Previous studies have shown that self-testing was especially beneficial when it was presented with feedback, which leaves the question whether an immediate and personalized feedback further encourages this effect. Therefore, we hypothesised that individual feedback has a greater effect on learning outcomes, compared to generic feedback.
Materials and methods:
In a randomised cross-over trial, German medical students were invited to voluntarily answer daily key-feature questions via an App. For half of the items they received a generalised feedback by an expert, while the feedback on the other half was generated immediately through ChatGPT. After the intervention, the students participated in a mandatory exit exam.
Results:
Those participants who used the app more frequently experienced a better learning outcome compared to those who did not use it frequently, even though this finding was only examined in a correlative nature. The individual ChatGPT generated feedback did not show a greater effect on exit exam scores compared to the expert comment (51.8 ± 22.0% vs. 55.8 ± 22.8%; p = 0.06).
Conclusion:
This study proves the concept of providing personalised feedback on medical questions. Despite the promising results, improved prompting and further development of the application seems necessary to strengthen the possible impact of the personalised feedback. Our study closes a research gap and holds great potential for further use not only in medicine but also in other academic fields.
General Audience Summary
Generative chatbots are artificial intelligence (AI) programs designed to have natural conversations with users. Since the release of ChatGPT (GPT stands for Generative Pre-trained Transformer), generative chatbots have become widely available. Generative chatbots are especially powerful because they are built on computer neural networks and trained on vast amounts of data. In addition, the text they produce can closely resemble expert knowledge and writing on nearly any topic. Consequently, individuals across industries and governments are interested in the potential for generative chatbots to support cognition in experts and nonexperts. This article reviews the history of chatbots, compares human expertise and artificial expertise, and then describes how individuals attain expertise through observing models, completing scaffolded tasks, and engaging in deliberate practice. Afterward, the article discusses how generative chatbots have been used—and could be used in the future—to support cognition (i.e., thoughts and mental processes) for users with varied amounts of domain knowledge (i.e., experts, novices, and laypersons). Research on potential approaches to leveraging generative chatbots to support cognition by these users is primarily drawn from three applied domains: education, medicine, and law. Research with the current generation of generative chatbots like ChatGPT is new and rapidly progressing, but research thus far suggests that (a) the roles that generative chatbots take on to support thinking vary depending on how much knowledge a user has on a particular topic and (b) generative chatbots show promise in supporting experts’ cognition and the training of novices who might be future experts in their fields, but (c) laypersons’ lack of prior knowledge currently limits generative chatbots’ ability to support their thinking and their agency in engaging with unfamiliar domains more broadly.
The unprecedented advancements of Generative Artificial Intelligence (GenAI) tools have generated controversies surrounding their potential for transforming educational practices, and democratising knowledge sharing, while acknowledging risks to current educational practices. In order to understand the potential and risks of GenAI tools on educational practices, recent research point to the need to develop student and teacher AI literacy skills and to investigate the impact of GenAI integration in learning and assessment practices. In addressing the call for more research in the field, this study reports on the integration of GenAI tools in an academic assessment in a postgraduate course for pre-service language teachers at an Australian university. Data were collected from students’ reflections of their adoption of GenAI tools for planning their assessment and from qualitative surveys. Thematic analysis was employed to identify students’ perceived challenges and benefits in adopting GenAI for essay writing. Pre-service teachers recognised benefits in adopting GenAI for planning and generating ideas for academic writing but recognised the importance of mitigating risks created by inaccuracies or biases in content. The findings confirm the importance of transparency in the integration of GenAI and developing student awareness of and training in ethical use of GenAI for higher education.
Technological advancements in education offer innovative tools that significantly impact the teaching and learning processes. Among these innovations, artificial intelligence (AI)-supported tools such as AutoGPT promise revolutionary changes in the field of English Language Teaching (ELT). This study aims to investigate the integration of AutoGPT into feedback processes in ELT at a private school in Konya. The research seeks to explore the effects of AutoGPT on feedback mechanisms in ELT and to examine teachers’ perceptions and experiences regarding the use of this AI tool. Conducted from a basic qualitative research design, this study involved semi-structured interviews with English teachers who had at least two years of teaching experience and had used AutoGPT for feedback purposes. The interviews aim to uncover teachers' views on the effectiveness of AutoGPT and the challenges encountered. The data were analyzed using thematic analysis with MAXQDA 24 software, identifying key themes related to the advantages, limitations, and practical applications of AutoGPT in ELT. The findings reveal that teachers consider AutoGPT a valuable tool for providing quick and comprehensive feedback on student writing. It was highlighted that AutoGPT effectively addresses students' difficulties in understanding concepts, alleviates teachers' workload, and offers objective evaluations to save time. However, concerns about the excessive use of technology potentially reducing students' sense of responsibility were also expressed. This study indicates that experienced teachers are necessary for the effective use of AutoGPT in ELT, and in this context, the development of comprehensive AI training programs for teachers is proposed.
This research analyzed the efficacy of ChatGPT as a tool for the correction and provision of feedback on primary school students' short essays written in both the English and Greek languages. The accuracy and qualitative aspects of ChatGPT-generated corrections and feedback were compared to that of educators. For the essays written in English, it was found that ChatGPT outperformed the educators both in terms of quantity and quality. It detected more mistakes, provided more detailed feedback, its focus was similar to that of educators, its orientation was more balanced, and it was more positive although more academic/formal in terms of style/tone. For the essays written in Greek, ChatGPT did not perform as well as educators did. Although it provided more detailed feedback and detected roughly the same number of mistakes, it incorrectly flagged as mistakes correctly written words and/or phrases. Moreover, compared to educators, it focused less on language mechanics and delivered less balanced feedback in terms of orientation. In terms of style/tone, there were no significant differences. When comparing ChatGPT's performance in English and Greek short essays, it was found that it performed better in the former language in both the quantitative and qualitative parameters that were examined. The implications of the above findings are also discussed.
This chapter uses the micro-level network model as a prompting framework for the development of AI applications that can intervene in individual learning processes.
This chapter introduces a complex hierarchical network framework of learning drawing from developmental psychology, computational biology, instructional design, cognitive science, complexity, and sociocultural theory. As a tri-level network of entities and relationships (i.e. nodes and edges of a network), the model includes a causal mechanism of learning with strong domain generality that is extended to perspectives on learning processes in micro, meso, and macro level clusters. The model’s edges represent influence pathways for aggregation and dissipation of the energy and information of learning within and across the levels by individuals, expert groups, and interdisciplinary communities.
One-on-one tutoring is widely acknowledged as an effective instructional method, conditioned on qualified tutors. However, the high demand for qualified tutors remains a challenge, often necessitating the training of novice tutors (i.e., trainees) to ensure effective tutoring. Research suggests that providing timely explanatory feedback can facilitate the training process for trainees. However, it presents challenges due to the time-consuming nature of assessing trainee performance by human experts. Inspired by the recent advancements of large language models (LLMs), our study employed the GPT-4 model to build an explanatory feedback system. This system identifies trainees’ responses in binary form (i.e., correct/incorrect) and automatically provides template-based feedback with responses appropriately rephrased by the GPT-4 model. We conducted our study using the responses of 383 trainees from three training lessons ( Giving Effective Praise , Reacting to Errors , and Determining What Students Know ). Our findings indicate that: 1) using a few-shot approach, the GPT-4 model effectively identifies correct/incorrect trainees’ responses from three training lessons with an average F1 score of 0.84 and AUC score of 0.85; and 2) using the few-shot approach, the GPT-4 model adeptly rephrases incorrect trainees’ responses into desired responses, achieving performance comparable to that of human experts.
Zusammenfassung
Eine zentrale Herausforderung der Unterrichtswissenschaft bzw. der unterrichtsbezogenen Lehr-Lern-Forschung stellt ohne Frage die gewinnbringende Nutzung Künstlicher Intelligenz dar. Obwohl Künstliche Intelligenz (KI) in den letzten Jahren und Jahrzehnten zunehmend Einzug in unseren Alltag gehalten hat (z. B. im Rahmen der Sprachsteuerung von Geräten), markierte die Veröffentlichung von chatGPT einen Meilenstein. ChatGPT unterstützt den Menschen nicht nur in einem eng begrenzten Anwendungsbereich, sondern eröffnet in vielen Bereichen völlig neue Möglichkeiten, die aktuell allenfalls oberflächlich erforscht sind. So kann chatGPT Aufgaben zur Leistungsbewertung generieren (und lösen), Schülervorstellungen erkennen und Strategien zum Umgang damit vorschlagen oder etwa Lehrkräfte bei der Erstellung von Unterrichtsmaterialien und der Planung von Unterrichtsreihen unterstützen. Bisher ist jedoch noch überwiegend unklar, wie dieses Potenzial entsprechender KI gewinnbringend für den Unterricht bzw. das unterrichtliche Lehren und Lernen genutzt werden kann und wo mögliche Herausforderungen liegen. Insbesondere ist bisher auch offen, welches Potenzial der Einsatz von KI für die Unterrichtsforschung hat und wo hier im Sinne wissenschaftlicher Lauterkeit (ethische) Grenzen zu setzen sind. Dieser Beitrag beleuchtet Potenzial und Herausforderungen, die der Einsatz von KI in Unterricht und Unterrichtsforschung mit sich bringt und leitet Fragen ab, denen sich die Unterrichtsforschung widmen sollte, um eine wissenschaftliche Basis für den Einsatz von KI zu schaffen.
The emergence of Large Language Models (LLMs) has marked a significant change in education. The appearance of these LLMs and their associated chatbots has yielded several advantages for both students and educators, including their use as teaching assistants for content creation or summarisation. This paper aims to evaluate the capacity of LLMs chatbots to provide feedback on student exercises in a university programming course. The complexity of the programming topic in this study (concurrency) makes the need for feedback to students even more important. The authors conducted an assessment of exercises submitted by students. Then, ChatGPT (from OpenAI) and Bard (from Google) were employed to evaluate each exercise, looking for typical concurrency errors, such as starvation, deadlocks, or race conditions. Compared to the ground-truth evaluations performed by expert teachers, it is possible to conclude that none of these two tools can accurately assess the exercises despite the generally positive reception of LLMs within the educational sector. All attempts result in an accuracy rate of 50%, meaning that both tools have limitations in their ability to evaluate these particular exercises effectively, specifically finding typical concurrency errors.
The evaluation of student essay corrections has become a focal point in understanding the evolving role of Artificial Intelligence (AI) in education. This study aims to assess the accuracy, efficiency, and cost-effectiveness of ChatGPT's essay correction compared to human correction, with a primary focus on identifying and rectifying grammatical errors, spelling, sentence structure, punctuation, coherence, relevance, essay structure, and clarity. The research involves collecting essays from 100 randomly selected university students, covering diverse themes, with anonymity maintained and no prior corrections by humans or AI. An analysis sheet, outlining linguistic and informational elements for evaluation, serves as a benchmark for assessing the quality of corrections made by ChatGPT and humans. The study reveals that ChatGPT excels in fundamental language mechanics, demonstrating superior performance in areas like grammar, spelling, sentence structure, relevance, and supporting evidence. However, thematic consistency remains an area where human evaluators outperform the AI. The findings emphasize the potential for a balanced approach, leveraging both human and AI strengths, for a comprehensive and effective essay correction process.
Literature review articles are essential to summarize the related work in the selected field. However, covering all related studies takes too much time and effort. This study questions how Artificial Intelligence can be used in this process. We used ChatGPT to create a literature review article to show the stage of the OpenAI ChatGPT artificial intelligence application. As the subject, the applications of Digital Twin in the health field were chosen. Abstracts of the last three years (2020, 2021 and 2022) papers were obtained from the keyword "Digital twin in healthcare" search results on Google Scholar and paraphrased by ChatGPT. Later on, we asked ChatGPT questions. The results are promising; however, the paraphrased parts had significant matches when checked with the Ithenticate tool. This article is the first attempt to show the compilation and expression of knowledge will be accelerated with the help of artificial intelligence. We are still at the beginning of such advances. The future academic publishing process will require less human effort, which in turn will allow academics to focus on their studies. In future studies, we will monitor citations to this study to evaluate the academic validity of the content produced by the ChatGPT.
Good explanations are essential to efficiently learning introductory programming concepts. To provide high-quality explanations at scale, numerous systems automate the process by tracing the execution of code, defining terms, giving hints, and providing error-specific feedback. However, these approaches often require manual effort to configure and only explain a single aspect of a given code segment. Large language models (LLMs) are also changing how students interact with code. For example, Github's Copilot can generate code for programmers, leading researchers to raise concerns about cheating. Instead, our work focuses on LLMs’ potential to support learning by explaining numerous aspects of a given code snippet. This poster features a systematic analysis of the diverse natural language explanations that GPT-3 can generate automatically for a given code snippet. We present a subset of three use cases from our evolving design space of AI Explanations of Code.
Feedback is a crucial element of a student's learning process. It enables students to identify weaknesses and improve self-regulation. However, studies show this to be an area of great dissatisfaction in higher education. With ever-growing course participation numbers, delivering effective feedback is becoming an in-creasingly challenging task. The efficacy of feedback will depend on four levels of feedback; namely, feedback about the self, task, process or self-regulation. Hence, this paper explores the use of automated content analysis to exam-ine feedback provided by instructors for feedback practices measured on self, task, process, and self-regulation levels. For this purpose, four binary XGBoost classifiers were trained and evaluated, one for each level of feedback. The re-sults indicate effective classification performance on self, task, and process levels with accuracy values of 0.87, 0.82, and 0.69, respectively. Additionally, inter-language transferability of feedback features is measured using cross-language classification performance and feature importance analysis. Findings indicate a low generalizability of features between English and Portuguese feedback spaces.
Currently, there is little agreement as to how Natural Language Generation (NLG) systems should be evaluated, with a particularly high degree of variation in the way that human evaluation is carried out. This paper provides an overview of how (mostly intrinsic) human evaluation is currently conducted and presents a set of best practices, grounded in the literature. These best practices are also linked to the stages that researchers go through when conducting an evaluation research (planning stage; execution and release stage), and the specific steps in these stages. With this paper, we hope to contribute to the quality and consistency of human evaluations in NLG.
The learning analytics community has matured significantly over the past few years as a middle space where technology and pedagogy combine to support learning experiences. To continue to grow and connect these perspectives, research needs to move beyond the level of basic support actions. This means exploring the use of data to prove richer forms of actions, such as personalized feedback, or hybrid approaches where instructors interpret the outputs of algorithms and select an appropriate course of action. This paper proposes the following three contributions to connect data extracted from the learning experience with such personalized student support actions: 1) a student–instructor centred conceptual model connecting a representation of the student information with a basic set of rules created by instructors to deploy Personalized Learning Support Actions (PLSAs); 2) a software architecture based on this model with six categories of functional blocks to deploy the PLSAs; and 3) a description of the implementation of this architecture as an open-source platform to promote the adoption and exploration of this area.
There is little debate regarding the importance of student feedback for improving the learning process. However, there remain significant workload barriers for instructors that impede their capacity to provide timely and meaningful feedback. The increasing role technology is playing in the education space may provide novel solutions to this impediment. As students interact with the various learning technologies in their course of study, they create digital traces that can be captured and analysed. These digital traces form the new kind of data that are frequently used in learning analytics to develop actionable recommendations that can support student learning. This paper explores the use of such analytics to address the challenges impeding the capacity of instructors to provide personalised feedback at scale. The case study reported in the paper showed how the approach was associated with a positive impact on student perception of feedback quality and on academic achievement. The study was conducted with first year undergraduate engineering students enrolled in a computer systems course with a blended learning design across three consecutive years (N2013 = 290, N2014 = 316 and N2015 = 415).
Programming environments intentionally designed to support novices have become increasingly popular, and growing research supports their efficacy. While these environments offer features to engage students and reduce the burden of syntax errors, they currently offer little support to students who get stuck and need expert assistance. Intelligent Tutoring Systems (ITSs) are computer systems designed to play this role, helping and guiding students to achieve better learning outcomes. We present iSnap, an extension to the Snap programming environment which adds some key features of ITSs, including detailed logging and automatically generated hints. We share results from a pilot study of iSnap, indicating that students are generally willing to use hints and that hints can create positive outcomes. We also highlight some key challenges encountered in the pilot study and discuss their implications for future work.
The purpose of this study was to investigate the effectiveness of an online automated evaluation and feedback system that assessed students' word processing assignments prepared with Microsoft Office Word. The participants of the study were 119 undergraduate teacher education students, 86 of whom were female and 32 were male, enrolled in different sections of Computer-I course taught at one of the major public universities in Istanbul, Turkey. A total of 52 and 67 participants were assigned to the control and experimental group, respectively. No statistically significant difference was found between the experimental and control group students’ post-tests performance, self-efficacy perception and technology acceptance scores after the implementation in which the experimental group students used the online automated evaluation and feedback system to get feedback on their assignments, and the control group students didn’t receive any feedback. However, the interview results showed that the experimental group students had positive experiences with the system such as contributions to their learning performance, high perceptions, easy use of the system and saving time for the assignments.
Self-regulated learning (SRL) is a pivot upon which students’ achievement turns. We explain how feedback is inherent in and a prime determiner of processes that constitute SRL, and review areas of research that elaborate contemporary models of how feedback functions in learning. Specifically, we begin by synthesizing a model of self-regulation based on contemporary educational and psychological literatures. Then we use that model as a structure for analyzing the cognitive processes involved in self-regulation, and for interpreting and integrating findings from disparate research traditions. We propose an elaborated model of SRL that can embrace these research findings and that spotlights the cognitive operation of monitoring as the hub of self-regulated cognitive engagement. The model is then used to reexamine (a) recent research on how feedback affects cognitive engagement with tasks and (b) the relation among forms of engagement and achievement. We conclude with a proposal that research on feedback and research on self-regulated learning should be tightly coupled, and that the facets of our model should be explicitly addressed in future research in both areas.
Student feedback is a contentious and confusing issue throughout higher education institutions. This paper develops and analyses two models of feedback: the first is based on the origins of the term in the disciplines of engineering and biology. It positions teachers as the drivers of feedback. The second draws on ideas of sustainable assessment. This positions learners as having a key role in driving learning, and thus generating and soliciting their own feedback. It suggests that the second model equips students beyond the immediate task and does not lead to false expectations that courses cannot deliver. It identifies the importance of curriculum design in creating opportunities for students to develop the capabilities to operate as judges of their own learning.
The research on formative assessment and feedback is reinterpreted to show how these processes can help students take control of their own learning, i.e. become self-regulated learners. This refor-mulation is used to identify seven principles of good feedback practice that support self-regulation. A key argument is that students are already assessing their own work and generating their own feedback, and that higher education should build on this ability. The research underpinning each feedback principle is presented, and some examples of easy-to-implement feedback strategies are briefly described. This shift in focus, whereby students are seen as having a proactive rather than a reactive role in generating and using feedback, has profound implications for the way in which teachers organise assessments and support learning.
Feedback is an effective way to assist students in achieving learning goals. The conceptualisation of feedback is gradually moving from feedback as information to feedback as a learner-centred process. To demonstrate feedback effectiveness, feedback as a learner-centred process should be designed to provide quality feedback content and promote student learning outcomes on the subsequent task. However, it remains unclear how instructors adopt the learner-centred feedback framework for feedback provision in the teaching practice. Thus, our study made use of a comprehensive learner-centred feedback framework to analyse feedback content and identify the characteristics of feedback content among student groups with different performance changes. Specifically, we collected the instructors' feedback on two consecutive assignments offered by an introductory to data science course at the postgraduate level. On the basis of the first assignment, we used the status of student grade changes (i.e., students whose performance increased and those whose performance did not increase on the second assignment) as the proxy of the student learning outcomes. Then, we engineered and extracted features from the feedback content on the first assignment using a learner-centred feedback framework and further examined the differences of these features between different groups of student learning outcomes. Lastly, we used the features to predict student learning outcomes by using widely-used machine learning models and provided the interpretation of predicted results by using the SHapley Additive exPlanations (SHAP) framework. We found that 1) most features from the feedback content presented significant differences between the groups of student learning outcomes, 2) the gradient boost tree model could effectively predict student learning outcomes, and 3) SHAP could transparently interpret the feature importance on predictions.
Due to recent conceptual shifts towards learner-centred feedback, there is a potential gap between research and practice. Indeed, few models or studies have sought to identify or evaluate which semantic messages, or feedback components, teachers should include in learner-centred feedback comments. Instead, teacher practices are likely to be primarily shaped by ‘old paradigm’ conceptualisations of feedback. In response, the current study develops a taxonomy of learner-centred feedback components based on a rapid systematic review of the literature. The face, content and construct validity of the taxonomy are then established through an empirical study with teachers and students at two Australian universities. The outcome of this study is a conceptual model featuring eight learner-centred feedback components. This model will help teachers design effective feedback processes and support learners to make sense of and use feedback information to improve their future work and learning strategies.
Among all the learning resources within MOOCs such as video lectures and homework, the discussion forum stood out as a valuable platform for students’ learning through knowledge exchange. However, peer interactions on MOOC discussion forums are scarce. The lack of interactions among MOOC learners can yield negative effects on students’ learning, causing low participation and high dropout rate. This research aims to examine the extent to which the deep-learning-based natural language generation (NLG) models can offer responses similar to human-generated responses to the learners in MOOC forums. Specifically, under the framework of social support theory, this study has examined the use of state-of-the-art deep learning models recurrent neural network (RNN) and generative pretrained transformer 2 (GPT-2) to provide students with informational, emotional, and community support with NLG on discussion forums. We first trained an RNN and GPT-2 model with 13,850 entries of post-reply pairs. Quantitative evaluation on model performance was then conducted with word perplexity, readability, and coherence. The results showed that GPT-2 outperformed RNN on all measures. We then qualitatively compared the dimensions of support provided by humans and GPT-2, and the results suggested that the GPT-2 model can comparably provide emotional and community support to human learners with contextual replies. We further surveyed participants to find out if the collected data would align with our findings. The results showed GPT-2 model could provide supportive and contextual replies to a similar extent compared to humans.
With an increase in technology to mediate learning and a shift to more student-centred approaches, open-ended online assignment tasks are becoming more common in higher education. Open-ended tasks offer opportunities for students to develop their own interpretations of the requirements, and online technologies offer greater flexibility and afford new types of interactions with teachers and other students. This paper presents a study of students' task interpretation and self-set goals in the context of five open-ended online assignment tasks. The findings presented in this paper demonstrate the importance of a high-quality task understanding for goal setting and suggest practical implications for task design and support.
This paper presents a novel framework called error case frames for correcting preposition errors. They are case frames specially designed for describing and correcting preposition errors. Their most distinct advantage is that they can correct errors with feedback messages explaining why the preposition is erroneous. This paper proposes a method for automatically generating them by comparing learner and native corpora. Experiments show (i) automatically generated error case frames achieve a performance comparable to conventional methods; (ii) error case frames are intuitively interpretable and manually modifiable to improve them; (iii) feedback messages provided by error case frames are effective in language learning assistance. Considering these advantages and the fact that it has been difficult to provide feedback messages by automatically generated rules, error case frames will likely be one of the major approaches for preposition error correction.
Personalized tutoring feedback is a powerful method that expert human tutors apply when helping students to optimize their learning. Thus, research on tutoring feedback strategies tailoring feedback according to important factors of the learning process has been recognized as a promising issue in the field of computer-based adaptive educational technologies. Our paper seeks to contribute to this area of research by addressing the following aspects: First, to investigate how students' gender, prior knowledge, and motivational characteristics relate to learning outcomes (knowledge gain and changes in motivation). Second, to investigate the impact of these student characteristics on how tutoring feedback strategies varying in content (procedural vs. conceptual) and specificity (concise hints vs. elaborated explanations) of tutoring feedback messages affect students' learning and motivation. Third, to explore the influence of the feedback parameters and student characteristics on students' immediate post-feedback behaviour (skipping vs. trying to accomplish a task, and failing vs. succeeding in providing a correct answer). To address these issues, detailed log-file analyses of an experimental study have been conducted. In this study, 124 sixth and seventh graders have been exposed to various tutoring feedback strategies while working on multi-trial error correction tasks in the domain of fraction arithmetic. The web-based intelligent learning environment ActiveMath was used to present the fraction tasks and trace students' progress and activities. The results reveal that gender is an important factor for feedback efficiency: Male students achieve significantly lower knowledge gains than female students under all tutoring feedback conditions (particularly, under feedback strategies starting with a conceptual hint). Moreover, perceived competence declines from pre- to post-test significantly more for boys than for girls. Yet, the decline in perceived competence is not accompanied by a decline in intrinsic motivation, which, instead, increases significantly from pre- to post-test. With regard to the post-feedback behaviour, the results indicate that students skip further attempts more frequently after conceptual than after procedural feedback messages.
Feedback is one of the most powerful influences on learning and achievement, but this impact can be either positive or negative. Its power is frequently mentioned in articles about learning and teaching, but surprisingly few recent studies have systematically investigated its meaning. This article provides a conceptual analysis of feedback and reviews the evidence related to its impact on learning and achievement. This evidence shows that although feedback is among the major influences, the type of feedback and the way it is given can be differentially effective. A model of feedback is then proposed that identifies the particular properties and circumstances that make it effective, and some typically thorny issues are discussed, including the timing of feedback and the effects of positive and negative feedback. Finally, this analysis is used to suggest ways in which feedback can be used to enhance its effectiveness in classrooms.
Written feedback on professional behaviors is an important part of medical training, but little attention has been paid to the quality of written feedback and its expected impact on learning. A large body of research on feedback suggests that feedback is most beneficial when it is specific, clear, and behavioral. Analysis of feedback comments may reveal opportunities to improve the value of feedback.
Using a directed content analysis, the authors coded and analyzed feedback phrases collected as part of a pilot of a developmental multisource feedback program. The authors coded feedback on various dimensions, including valence (positive or negative) and whether feedback was directed at the level of the self or behavioral performance.
Most feedback comments were positive, self-oriented, and lacked actionable information that would make them useful to learners.
Comments often lack effective feedback characteristics. Opportunities exist to improve the quality of comments provided in multisource feedback.
Insta-reviewer: A data-driven approach for generating instant feedback on students' project reports
- Q Jia
- M Young
- Y Xiao
- J Cui
- C Liu
- P Rashid
- E Gehringer
Q. Jia, M. Young, Y. Xiao, J. Cui, C. Liu, P. Rashid, and E. Gehringer,
"Insta-reviewer: A data-driven approach for generating instant feedback
on students' project reports." Proceedings of the 15th International
Conference on Educational Data Mining, p. 5-16, 2022.
Measuring inconsistency in written feedback: A case study in politeness
- W Dai
- Y.-S Tsai
- Y Fan
- D Gašević
- G Chen
W. Dai, Y.-S. Tsai, Y. Fan, D. Gašević, and G. Chen, "Measuring
inconsistency in written feedback: A case study in politeness," in Proceedings of the 23rd International Conference of Artificial Intelligence
in Education, 2022, pp. 560-566.