No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
EVALUATING AN AI-BASED ADAPTIVE LEARNING SYSTEM: GOALS, METHODS AND INITIAL RESULTS
Abstract
The aim of this paper is to describe the evaluation process and findings of an AI-based Adaptive Learning System for the Computer Science discipline at two different German universities and discuss an array of methods in regard to assessing such a system. The primary objectives have been twofold: firstly, to examine the reception of selected learning elements, which were conceptually outlined based on relevant literature, among the student body; and secondly, to investigate the efficacy of individualized adaptive learning paths. These paths were generated by employing a variety of algorithms to analyze students learning style tendencies, with a particular emphasis on adaptive navigational techniques. The used algorithms encompassed a modified version of a literature based adaptive mechanism, an Ant-Colony-Algorithm and a Genetic Algorithm, alongside a lecturer-recommended learning path for a non-adaptive comparison. While the system suggested suitable learning paths based on student data, it never forced the individuals to give up their self-directed learning. The evaluation criteria revolved around the evolution of student motivation, interest levels, and knowledge acquisition during the time they spent working in the system. The evaluation sought to facilitate comparative analyses and assess algorithmic fitness for proficient learning path generation. The methods included both quantitative and qualitative approaches to gather data, seeking to strike a balance between being student-friendly and scientifically informative. They ranged from Likert Scale self-assessments to screen and video observations with retrospective interviews. Since the purpose of adaptive learning systems is intertwined with personalized learning it seems imperative to already take the preferences and opinions of students into account while the system is still in development. This complexity underscores the challenge of evaluating such systems, as significant constraints on student choice - though simplifying evaluation - directly oppose the ethos of individualized, self-directed learning. Initial findings suggest that the underlying theoretical considerations on sequencing and structuring of learning elements are confirmed, coupled with providing adequate flexibility to meet diverse learning needs. Cross-site evaluation of the literature-based learning elements indicated a high comprehensibility and positive student ratings. While significant positive trends were observed regarding knowledge acquisition, they cannot be definitively attributed to a specific method of learning path generation. Motivation and interest analyses show no significant differences among learning path types, albeit heavily limited by sample size. Similarly, emotion measurements, though limited, hint at positive impacts from HASKI system use. Despite limitations, early indications suggest student acceptance and potential effectiveness of learning paths, highlighting the need for larger sample sizes for validation and expansion. Ensuring alignment with student needs and user-friendly design are crucial considerations.
... Simultaneously, the system itself should autonomously analyze the emotional states of students without requiring direct intervention from teachers. The subsequent section outlines the methodology employed for emotional state measurement in the initial prototype of the HASKI-System [27] and draws conclusions. ...
The assessment of affective states in online learning often relies on various devices, such as sensors, which indicate physiological reactions of a person's emotional state. It is important to note that not every university can afford the sensors and devices required for this purpose. Furthermore, not every student may be willing to monitor their emotional state using sensors on a personal computer during online learning. This article focuses on the potential benefits of detecting affective states through self-reported survey responses. It aims to explore ways to improve student‘s learning by identifying and responding to specific emotional states during online learning. This paper presents the identification of emotions using a survey, administered to engineering students during an e-learning course in the adaptive
learning system (ALS). In addition, the authors developed a model of Bayesian Network (BN) to analyze the current emotional state of students in e-learning and propose supportive messages to students in order to improve their emotional state.
This systematic review of research on adaptive learning used a strategic search process to synthesize research on adaptive learning based on publication trends, instructional context, research methodology components, research focus, adaptive strategies, and technologies. A total of 61 articles on adaptive learning were analyzed to describe the current state of research and identify gaps in the literature. Descriptive characteristics were recorded, including publication patterns, instructional context, and research methodology components. The count of adaptive learning articles published fluctuated across the decade and peaked in 2015. During this time, the largest concentration of adaptive learning articles appeared in Computers and Education. The majority of the studies occurred in higher education in Taiwan and the United States, with the highest concentration in the computer science discipline. The research focus, adaptive strategies, and adaptive technologies used in these studies were also reviewed. The research was aligned with various instructional design phases, with more studies examining design and development, and implementation and evaluation. For examining adaptive strategies, the authors examined both adaptive sources based on learner model and adaptive targets based on content and instructional model. Learning style was the most observed learner characteristic, while adaptive feedback and adaptive navigation were the most investigated adaptive targets. This study has implications for adaptive learning designers and future researchers regarding the gaps in adaptive learning research. Future studies might focus on the increasing availability and capacities of adaptive learning as a learning technology to assist individual learning and personalized growth.
Mixed methods research is an approach that combines both quantitative and qualitative methods into a single study in order to provide a broader and more complete vision of a problem. Mixed methodologies are employed when both comparative analysis and the development of aspects of the study need to be undertaken comprehensively and in depth. The use of mixed methods turns possible to overcome the limitations of quantitative and qualitative methodologies, allowing the researcher to get rich information that could not be obtained using each method alone. However, the number of published scientific studies addressing the use of mixed methods is limited, and most of them focus on describing a single implementation approach without giving a global and comparative overview of the various approaches. In this sense, this study tries to synthesize and describe each of the mixed methods approach, also providing indications about the advantages and limitations of each of these approaches. In total, ten mixed methods approaches are identified that can be grouped into four major groups: (i) sequential design; (ii) concurrent design; (iii) multiphase design; and (iv) multilevel design. It was also possible to conclude that although each mixed method design presents specific advantages and limitations, sequential approaches are easier to adopt since they facilitate the integration process of both studies but generally lead to higher implementation time. On the other hand, concurrent design typically leads to greater difficulties in the integration of both studies, but they speed up the development process, because quantitative and qualitative can be executed in parallel. Finally, multiphase and multilevel designs are emergent approaches that are used in more complex studies in which it becomes necessary to consider several dimensions of analysis.
There is much enthusiasm in higher education about the benefits of adaptive learning and using big data to investigate learning processes to make data-informed educational decisions. The benefits of adaptive learning to achieve personalized learning are obvious. Yet, there lacks evidence-based research to understand how data such as user behavior patterns can be used to design effective adaptive learning systems. The purpose of this study, therefore, is to investigate what behavior patterns learners with different characteristics demonstrate when they interact with an adaptive learning environment. Incoming 1st-year students in a pharmacy professional degree program engaged in an adaptive learning intervention that aimed to provide remedial instruction to better prepare these professional students before they began their formal degree program. We analyzed the participants’ behavior patterns through the usage data to understand how they used the adaptive system based upon their needs and interests. Using both statistical analyses and data visualization techniques, this study found: (1) apart from learners’ cognitive ability, it is important to consider affective factors such as motivation in adaptive learning, (2) lack of alignment among various components in an adaptive system can impact how learners accessed the system and, more importantly, their performance, and (3) visualizations can reveal interesting findings that can be missed otherwise. Such research should provide much needed empirical evidences and useful insights about how the analytics can inform the effective designs of adaptive learning.
The Index of Learning Styles© (ILS) is an instrument designed to assess preferences on the four dimensions of the Felder-Silverman learning style model. The Web-based version of the ILS is taken hundreds of thousands of times per year and has been used in a number of published studies, some of which include data reflecting on the reliability and validity of the instrument. This paper seeks to provide the first comprehensive examination of the ILS, including answers to several questions: (1) What are the dimensions and underlying assumptions of the model upon which the ILS is based? (2) How should the ILS be used and what misuses should be avoided? (3) What research studies have been conducted using the ILS and what conclusions regarding its reliability and validity may be inferred from the data?
Qualitative content analysis: theoretical foundation, basic procedures and software solution Mayring, Philipp Erstveröffentlichung / Primary Publication Monographie / monograph Empfohlene Zitierung / Suggested Citation: Mayring, Philipp : Qualitative content analysis: theoretical foundation, basic procedures and software solution. Klagenfurt, 2014. URN: http://nbn-resolving.de/urn:nbn:de:0168-ssoar-395173 Nutzungsbedingungen: Dieser Text wird unter einer CC BY-NC-ND Lizenz (Namensnennung-Nicht-kommerziell-Keine Bearbeitung) zur Verfügung gestellt. Nähere Auskünfte zu den CC-Lizenzen finden Sie hier: http://creativecommons.org/licenses/ Terms of use: This document is made available under a CC BY-NC-ND Licence (Attribution Non Comercial-NoDerivatives). For more Information see: http://creativecommons.org/licenses/
An Intelligent Tutoring System (ITS) is critical in education because it provides one to one personalized teaching assistance to learners as they educate how to solve problems through guidance and prompt feedback. ITS is one application of Artificial Intelligence (AI) in education. It provides a smart learning environment for students without intervention from the teacher. ITS’s primary goal is to support and help learners obtain domain-specific intellectual knowledge in a practical and produc- tive manner through the use of different computing technologies. This paper presents a comprehensive survey for previous research on ITS that utilize various techniques of AI and Machine Learning (ML). It gives an overview of ITS, its architecture, and some existing ITS examples. In addition, it highlights and summarizes the current research efforts and obstacles to ITS using AI, as well as some future opportunities. This study shows the importance of AI and ML in ITS development. It is noticed that researchers focus more on Reinforcement Learning (RL), Artificial Neural Networks (ANN), clustering, Bayesian Network (BN) and Fuzzy Logic (FL) approaches.
Index Terms—Artificial Intelligence, Machine Learning, Intel- ligent Tutoring System, ITS, Bayesian network, Reinforcement Learning, Artificial Neural Networks.
Learning at scale (LAS) systems like Massive Open Online Classes (MOOCs) have hugely expanded access to high quality educational materials however, such material are frequently time and resource expensive to create. In this work we propose a new approach for automatically and adaptively sequencing practice activities for a particular learner and explore its application for foreign language learning. We evaluate our system through simulation and are in the process of running an experiment. Our simulation results suggest that such an approach may be significantly better than an expert system when there is high variability in the rate of learning among the students and if mastering prerequisites before advancing is important, and is likely to be no worse than an expert system if our generated curriculum approximately describes the necessary structure of learning in students.
This unique and ground-breaking book is the result of 15 years research and synthesises over 800 meta-analyses on the influences on achievement in school-aged students. It builds a story about the power of teachers, feedback, and a model of learning and understanding. The research involves many millions of students and represents the largest ever evidence based research into what actually works in schools to improve learning. Areas covered include the influence of the student, home, school, curricula, teacher, and teaching strategies. A model of teaching and learning is developed based on the notion of visible teaching and visible learning. A major message is that what works best for students is similar to what works best for teachers - an attention to setting challenging learning intentions, being clear about what success means, and an attention to learning strategies for developing conceptual understanding about what teachers and students know and understand. Although the current evidence based fad has turned into a debate about test scores, this book is about using evidence to build and defend a model of teaching and learning. A major contribution is a fascinating benchmark/dashboard for comparing many innovations in teaching and schools.
This article is a review of experiments comparing the effectiveness of human tutoring, computer tutoring, and no tutoring. “No tutoring” refers to instruction that teaches the same content without tutoring. The computer tutoring systems were divided by their granularity of the user interface interaction into answer-based, step-based, and substep-based tutoring systems. Most intelligent tutoring systems have step-based or substep-based granularities of interaction, whereas most other tutoring systems (often called CAI, CBT, or CAL systems) have answer-based user interfaces. It is widely believed as the granularity of tutoring decreases, the effectiveness increases. In particular, when compared to No tutoring, the effect sizes of answer-based tutoring systems, intelligent tutoring systems, and adult human tutors are believed to be d = 0.3, 1.0, and 2.0 respectively. This review did not confirm these beliefs. Instead, it found that the effect size of human tutoring was much lower: d = 0.79. Moreover, the effect size of intelligent tutoring systems was 0.76, so they are nearly as effective as human tutoring.
In the past decade, a number of adaptive hypermedia learning systems have been developed. However, most of these systems tailor presentation content and navigational support solely according to students’ prior knowledge. On the other hand, previous research suggested that cognitive styles significantly affect student learning because they refer to how learners process and organize information. To this end, the study presented in this paper developed an adaptive hypermedia learning system tailored to students’ cognitive styles, with an emphasis on Pask’s Holist–Serialist dimension. How students react to this adaptive hypermedia learning system, including both learning performance and perceptions, was examined in this study. Forty-four undergraduate and postgraduate students participated in the study. The findings indicated that, in general, adapting to cognitive styles improves student learning. The results also showed that the adaptive hypermedia learning system have more effects on students’ perceptions than performance. The implications of these results for the design of adaptive hypermedia learning systems are discussed.
The development of learner models takes an active part in upcoming adaptive learning environments. The purpose of learner models is to drive personalization based on learner and learning characteristics that are considered as important for the learning process, such as cognitive, affective and behavioral variables. Despite the huge amount of theoretical propositions of learner characteristics considered as relevant for learner models, practical payoffs are rather sparse. This study aims to overview the empirical research on the mere value of learner models in the development of adaptive learning environments. The results show that a lot of high-quality studies are situated in a rather shattered research field, building few bridges from theory to practice. We conclude with the call for a theory or framework integrating current and past research results that is able to guide theory-based and systematic empirical research having concrete hypotheses on the merits of learner characteristics in adaptive learning environments.
- W Stangl
W. Stangl, Adaptivität, 2024. Retrieved from https://lexikon.stangl.eu/982/adaptivitaet.
Personalization in OELEs: Developing a Data-Driven Framework to Model and Scaffold SRL Processes
- A Munshi
- G Biswas
A. Munshi and G. Biswas, "Personalization in OELEs: Developing a Data-Driven Framework to
Model and Scaffold SRL Processes" in Proceedings of the International Conference on Artificial
Intelligence in Education (AIED 2019), pp. 354-358, 2019.