Article

Predicting Student Performance from LMS Data: A Comparison of 17 Blended Courses Using Moodle LMS

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

With the adoption of Learning Management Systems (LMSs) in educational institutions, a lot of data has become available describing students’ online behavior. Many researchers have used these data to predict student performance. This has led to a rather diverse set of findings, possibly related to the diversity in courses and predictor variables extracted from the LMS, which makes it hard to draw general conclusions about the mechanisms underlying student performance. We first provide an overview of the theoretical arguments used in learning analytics research and the typical predictors that have been used in recent studies. We then analyze 17 blended courses with 4,989 students in a single institution using Moodle LMS, in which we predict student performance from LMS predictor variables as used in the literature and from in-between assessment grades, using both multi-level and standard regressions. Our analyses show that the results of predictive modeling, notwithstanding the fact that they are collected within a single institution, strongly vary across courses. Thus, the portability of the prediction models across courses is low. In addition, we show that for the purpose of early intervention or when in-between assessment grades are taken into account, LMS data are of little (additional) value. We outline the implications of our findings and emphasize the need to include more specific theoretical argumentation and additional data sources other than just the LMS data.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... The deficiencies of LMS data in predicting academic success has been acknowledged in past studies (Conijn et al., 2016). For instance, LMS data cannot address course differences and complexity. ...
... Consequently, Francis et al. (2020) stated that structural disadvantage does impact student outcomes, particularly for disadvantaged students and this impact cannot be identified in LMS data. Scholars recommend the use of alternative data sources (Conijn et al., 2016;Francis et al., 2020;Tempelaar et al., 2015) and machine learning approach in determining students' academic success (Rinc on-Flores et al., 2020). In response to these calls, this paper opts for socio-demographic data and machine learning approach to determine students' academic success. ...
... The present work unveils catchy theoretical implications. A burgeoning stream of research highlighted several limitations of self-reported survey data and LMS data in predicting students' academic success (Conijn et al., 2016;Rinc on-Flores et al., 2020;Tempelaar et al., 2020;Tempelaar et al., 2015). ...
Article
Full-text available
Purpose Academic success and failure are relevant lifelines for economic success in the knowledge-based economy. The purpose of this paper is to predict the propensity of students’ academic performance using early detection indicators (i.e. age, gender, high school exam scores, region, CGPA) to allow for timely and efficient remediation. Design/methodology/approach A machine learning approach was used to develop a model based on secondary data obtained from students’ information system in a Nigerian university. Findings Results revealed that age is not a predictor for academic success (high CGPA); female students are 1.2 times more likely to have high CGPA compared to their male counterparts; students with high JAMB scores are more likely to achieve academic success, high CGPA and vice versa; students from affluent and developed regions are more likely to achieve academic success, high CGPA and vice versa; and students in Years 3 and 4 are more likely to achieve academic success, high CGPA. Originality/value This predictive model serves as a classifier and useful strategy to mitigate failure, promote success and better manage resources in tertiary institutions.
... To date, most learning analytics studies have focused on LMS data alone, with no learner-specific data (Conijn et al., 2017). Learning analytics studies have also largely ignored theory (Conijn et al., 2017;. ...
... To date, most learning analytics studies have focused on LMS data alone, with no learner-specific data (Conijn et al., 2017). Learning analytics studies have also largely ignored theory (Conijn et al., 2017;. Ellis, Han and Pardo found that results were far more predictive when the research was theory-based and included self-report data. ...
... With learning analytics, researchers can now obtain much more objective data, in much larger quantities. Predictors of grades found to date have included, among others, early access to course sites (Conijn et al., 2017;Jo, Kim & Yoon, 2015); lack of lastminute assignment submission (You, 2015); and lack of cramming, consistency of access, and pacing (Asarta and Schmidt, 2013). Each of these is discussed briefly below. ...
Thesis
For this exploratory study, 109 adult students (73.4% female) completed an online survey with measures of time management behavior and college wellbeing during the fall semester. Students were in 10 courses at a continuing education school within a large northeastern U.S. university. On a follow-up survey, 87 reported their grades and answered additional questions about time use and management. Factor analysis of selfreport measures identified a three-factor structure for time management: Satisfaction with Time Use, Monitoring and Evaluating, and Planning and Prioritizing. Satisfaction with Time Use best predicted college wellbeing on the College Student Subjective Wellbeing Scale (CSSWQ, Renshaw & Boligno, 2016). Number of time management tools used negatively predicted grades and course completion, and the Mechanics dimension of the Time Management Behavior Scale (Macan, Shahani, Dipboye & Phillips, 1990) positively predicted grades and course completion. Each student’s activity on the course learning management system (LMS) was collected, de-identified, and used to show study times of day. Study times of day did not emerge as significant predictors. Some differences between first-and second-generation college students were seen: first-gen students worked more hours per week, on average, than their peers, and fewer of them got at least seven hours of sleep per night. Still, their grades and course completion rates were similar to their peers’. Satisfaction with Time Use was a better predictor of grades and course completion than Mechanics for first-generation students. Directions for future research are identified.
... The leitmotif of those studies was to develop a predictive model of student success that would be independent of a particular learning context and thus could be scaled across higher education institutions (Lauría, Moody, Jayaprakash, Jonnalagadda, & Baron, 2013). Later studies pointed to and demonstrated the drawbacks of focusing on models that do not consider the specificities of the course design and disciplinary context (Conijn, Snijders, Kleingeld, & Matzat, 2017;Jovanović, Mirriahi, Gašević, Dawson, & Pardo, 2019;Rienties, Toetenel, & Bryan, 2015). For example, studies that applied predictive modelling across multiple courses, including courses from different discipline and with different instructional design (e.g., Finnegan, Morris, & Lee 2008;Gašević, Dawson, Rogers, & Gašević, 2016), tended to produce inconsistent and even conflicting findings when porting models from one course to the next (see Sect. 2.1). ...
... The study reported in this paper aimed to further investigate the role of instructional conditions in the prediction of academic success. In particular, we wanted to empirically investigate if the conclusions from earlier studies (e.g., Finnegan et al., 2008;Gašević et al., 2016;Conijn et al., 2017) hold in a situation where courses are based on the same pedagogical underpinnings and belong to the same discipline. To that end, the current study leverages trace data from a multitude of blended medical courses, all based on problem-based learning, and examines the predictive power of several indicators of students' engagement with the online component of the courses. ...
... Sociocultural approaches recognise the role of social context and individual differences in the ways this context is internalised (Nolen & Ward, 2008). By grounding their work in contemporary learning theories, several recent studies demonstrated the weaknesses of general (context-agnostic) predictive models and the relevance of considering learning settings (Conijn et al., 2017;Finnegan, Morris, & Lee, 2009;Gašević et al., 2016;Joksimović, Gašević, Loughin, Kovanović, & Hatala, 2015;Jovanović, Mirriahi, et al., 2019;Kizilcec, Reich, Yeomans, Dann, Brunskill, Lopez et al., 2020). For instance, Finnegan et al. (2009) examined the association between several indicators of student online learning behaviour and academic achievement across 22 courses in 3 broad academic disciplines (English and Communication; Social Sciences; and Math, Science, and Technology). ...
Article
Full-text available
Predictive modelling of academic success and retention has been a key research theme in Learning Analytics. While the initial work on predictive modelling was focused on the development of general predictive models, portable across different learning settings, later studies demonstrated the drawbacks of not considering the specificities of course design and disciplinary context. This study builds on the methods and findings of related earlier studies to further explore factors predictive of learners' academic success in blended learning. In doing so, it differentiates itself by (i) relying on a larger and homogeneous course sample (15 courses, 50 course offerings in total), and (ii) considering both internal and external conditions as factors affecting the learning process. We apply mixed effect linear regression models, to examine: i) to what extent indicators of students' online learning behaviour can explain the variability in the final grades, and ii) to what extent that variability is attributable to the course and students' internal conditions, not captured by the logged data. Having examined different types of behaviour indicators (e.g., indicators of the overall activity level, those indicative of regularity of study, etc), we found little difference, if any, in their predictive power. Our results further indicate that a low proportion of variance is explained by the behaviour-based indicators, while a significant portion of variability stems from the learners' internal conditions. Hence, when variability in external conditions is largely controlled for (the same institution, discipline, and nominal pedagogical model), students’ internal state is the key predictor of their course performance.
... To early predict exam failures or course dropout, many research efforts have been devoted to analyzing data about learners and their learning contexts through machine learning techniques (e.g., [3][4][5][6]). A common approach entails predicting the per-student success rate of an exam well before the end of course by means of classification techniques [7]. ...
... In the context of student performance prediction, recent findings [3] have shown the high variability of classifier performance according to the learning context, the analyzed features, and the considered algorithms. The main research works conducted in this field have mainly addressed the following research questions: ...
... Even the presence of unlabeled data could contribute to the improvement of predictors' reliability [27]. In [3], the authors analyze 17 blended learning courses using the Moodle LMS. The prediction models achieve high recall values (i.e., they identify most of the at-risk students) but low precision values (i.e., the number of false positives is fairly high). ...
Article
Full-text available
The Learning Analytics community has recently paid particular attention to early predict learners’ performance. An established approach entails training classification models from past learner-related data in order to predict the exam success rate of a student well before the end of the course. Early predictions allow teachers to put in place targeted actions, e.g., supporting at-risk students to avoid exam failures or course dropouts. Although several machine learning and data mining solutions have been proposed to learn accurate predictors from past data, the interpretability and explainability of the best performing models is often limited. Therefore, in most cases, the reasons behind classifiers’ decisions remain unclear. This paper proposes an Explainable Learning Analytics solution to analyze learner-generated data acquired by our technical university, which relies on a blended learning model. It adopts classification techniques to early predict the success rate of about 5000 students who were enrolled in the first year courses of our university. It proposes to apply associative classifiers at different time points and to explore the characteristics of the models that led to assign pass or fail success rates. Thanks to their inherent interpretability, associative models can be manually explored by domain experts with the twofold aim at validating classifier outcomes through local rule-based explanations and identifying at-risk/successful student profiles by interpreting the global rule-based model. The results of an in-depth empirical evaluation demonstrate that associative models (i) perform as good as the best performing classification models, and (ii) give relevant insights into the per-student success rate assignments.
... Based on the literature review in 2013 [8], we consider two main groups: "Student Modeling" and "Decision Support Systems" in terms of EDM. Some widely used methods are regression and classification for predicting [9,38], but other methods have also been used such as clustering and feature selection for exploring patterns or emphasizing the interesting features [10,24]. ...
... Related studies usually exploit potential factors from the university's data [4] to build a prediction model such as GPA or student's performance. Various Machine Learning algorithms are used to solve these problems including Decision Tree, Random Forest, Regression, and Neural Network [9,34,42]. Some other techniques based on Recommendation System (e.g., Collaborative Filtering and Matrix Factorization) also found a lot of successes [17,[27][28][29]36]. ...
... Some other techniques based on Recommendation System (e.g., Collaborative Filtering and Matrix Factorization) also found a lot of successes [17,[27][28][29]36]. Furthermore, some studies focus on different types of predictor variables rather than the explicit ratings such as ages, sex, online time, and response efficiency in improving the accuracy [9,13]. However, the This article is part of the topical collection "SoftwareTechnology and Its Enabling Computing Platforms" guest edited byLam-Son Lê and Michel Toulouse. ...
Article
Full-text available
Student data play an important role in evaluating the effectiveness of educational programs in the universities. All data are aggregated to calculate the education criteria by year, region, or organization. Remarkably, recent studies showed the data impacts when making exploration to predict student performance objectives. Many methods in terms of data mining were proposed to be suitable to extract useful information in regards to data characteristics. However, the reconciliation between applied methods and data characteristics still exists some challenges. Our paper will demonstrate the analysis of this relationship for a specific dataset in practice. The paper describes a distributed framework based on Spark for extracting information from raw data. Then, we integrate machine learning techniques to train the prediction model. The experiments results are analyzed through different scenarios to show the harmony between the influencing factors and applied techniques.
... For the model to have predictive utility when teachers intervene, it is essential that it can be applied to groups other than the initial one [36]. The portability of predictive models of academic performance has been extensively studied in the analyses of data related to the use of learning platforms [37][38][39], but there are few studies in which the results of previous evaluations are taken as factors. The goal of this study was to analyze the portability of predictive academic performance models based on evaluation results. ...
... Studies by Widyahastuti and Tjhin [59] and Thakar, Mehta and Manisha [60], analyzing academic publications between 2011 and 2016, and 2002 and 2014, respectively, pointed to the need to search for unified approaches that allow the development of universal models. In the same vein, Muthukrishnan, Govindasamy and Mustapha [61], based on a review of 59 articles on predictive models of student performance, concluded that there is a huge shortage of portable predictive models, something confirmed by, among others, by Gasevic et al. and Gitinabard et al. [12,38]. Even analyzing different courses within the same institution, there are important differences between them, requiring predictive models tailored to each of them. ...
... Thus, the enormous difficulty, and probable impossibility, of developing universal predictive models seems to be confirmed, at least for using the results of previous evaluations as the only factors. The result coincides with that of previous research that, using data from learning platforms, concluded that the models have limited portability [38,39]. Therefore, the conclusion is that the limited portability of predictive models of academic performance is not due to the approach adopted. ...
Article
Full-text available
The portability of predictive models of academic performance has been widely studied in the field of learning platforms, but there are few studies in which the results of previous evaluations are used as factors. The aim of this work was to analyze portability precisely in this context, where preceding performance is used as a key predictor. Through a study designed to control the main confounding factors, the results of 170 students evaluated over two academic years were analyzed, developing various predictive models for a base group (BG) of 39 students. After the four best models were selected, they were validated using different statistical techniques. Finally, these models were applied to the remaining groups, controlling the number of different factors with respect to the BG. The results show that the models’ performance varies consistently with what was expected: as they move away from the BG (fewer common characteristics), the specificity of the four models tends to decrease.
... These diverse findings may be related to the large diversity of courses of different disciplines and learning designs. More importantly, they may be related to the speculation that platform-dependent interactions derived from log data are at best an indirect measurement of learning motivation and engagement, than a concrete measurement of pedagogical concepts matching the learning design (Conijn et al., 2017). ...
... Furthermore, Joksimovic, Gašević, Loughin, Kovanovic, and Hatala (2015) found that individual differences and course offerings can explain 18% and 22% of the final grade variance, respectively. It implies that the predictive models can be improved by considering individual differences in a single course (Conijn et al., 2017). In addition, earlier studies showed that about 30%-52% of the final grade variance can be explained by online interactions such as discussion forum posts (Morris et al., 2005;Zacharis, 2015). ...
... In addition, earlier studies showed that about 30%-52% of the final grade variance can be explained by online interactions such as discussion forum posts (Morris et al., 2005;Zacharis, 2015). However, the predictors and their effects on learning performance may vary by courses or institutions, implying that log data may not be a concrete measure of the pedagogical concepts used in course design (Conijn et al., 2017). ...
Article
Flipped classrooms supported by learning management systems (LMS) have been widely adopted by educational institutions. However, earlier studies have found problems with interpreting LMS log data to understand student approaches to learning within the context of a learning design. This study investigates whether it is possible to use LMS log data as a proxy to understand stu-dents' learning strategies over different periods of time in the flipped-classroom context. A total of 135 sophomores from two classes of a flipped programming course participated in this study. Exploratory factor analysis is first conducted on the log data to synthesize second-order predictors based on the total-effort model. Then, we investigate the extent to which these second-order predictors relate to students' learning outcomes over time. Four types of learning outcomes are considered, including a quiz, a midterm exam, a final exam and the final grade. For each type of learning outcome, multiple linear regression is used to construct a weekly prediction model from these predictors. Adjusted R-squared and RMSE (Root Mean Square Error) are the metrics used to compare the models. The results show that consistent second-order predictors can be derived from log data, implying that students' clicking events in LMS could manifest students' learning strategies understandable in the design context of a flipped classroom. Furthermore, compared with the first-order models, most of the models constructed using the second-order predictors have higher predictive performance, although with lower data fitness. In addition, the predictive performance of the models with MSLQ (Motivated Strategies for Learning Questionnaire) indicators and past assessment data are also examined. It is found that MSQL variables have a positive but short-termed effect on the models' predictive ability, while past assessment data greatly improve the models of all types of learning outcomes. Theoretical contributions and implications of the proposed approach for practice, research and future research are discussed.
... An LMS is a software application for the delivery, documentation, tracking, reporting and administration of educational courses and learning, training and development programs [2]. By using the Internet, LMSs represent a valuable tool to transform the traditional face-to-face courses into blended and online programs [3]. Moreover, LMSs are also used to support the majority of face-to-face courses, because of the functionalities they provide such as course content management, communication, assignment delivery and assessment, online questionnaires and quizzes, and student grading, among others [4]. ...
... LMSs generate log files that contain data about user interaction (e.g., course and resource view, assignment submission and evaluation, and quiz and forum interaction). Such information has been used to create predictive models for different purposes such as foreseeing student performance [3], detecting procrastination [5] and clustering students [6]. Data in log files describe patterns of how students interact with LMSs, and such patterns may involve some correlation with their performance. ...
... Conijn et al. analyze 17 blended courses with 4,989 students using Moodle LMS [3]. Their objective is to predict students' final grades from LMS predictor variables and from in-between assessment grades, using logistic (pass/fail) and standard regression (final grade) models. ...
Article
Full-text available
The early prediction of students’ performance is a valuable resource to improve their learning. If we are able to detect at-risk students in the initial stages of the course, we will have more time to improve their performance. Likewise, excellent students could be motivated with customized additional activities. This is why there are research works aimed to early detect students’ performance. Some of them try to achieve it with the analysis of LMS log files, which store information about student interaction with the LMS. Many works create predictive models with the log files generated for the whole course, but those models are not useful for early prediction because the actual log information used for predicting is different to the one used to train the models. Other works do create predictive models with the log information retrieved at the early stages of courses, but they are just focused on a particular type of course. In this work, we use machine learning to create models for the early prediction of students’ performance in solving LMS assignments, by just analyzing the LMS log files generated up to the moment of prediction. Moreover, our models are course agnostic, because the datasets are created with all the University of UniversityName¹ courses for one academic year. We predict students’ performance at 10%, 25%, 33% and 50% of the course length. Our objective is not to predict the exact student’s mark in LMS assignments, but to detect at-risk, fail and excellent students in the early stages of the course. That is why we create different classification models for each of those three student groups. Decision tree, nave Bayes, logistic regression, multilayer perceptron (MLP) neural network, and support vector machine models are created and evaluated. Accuracies of all the models grow as the moment of prediction increases. Although all the algorithms but nave Bayes show accuracy differences lower than 5%, MLP obtains the best performance: from 80.1% accuracy when 10% of the course has been delivered to 90.1% when half of it has taken place. We also discuss the LMS log entries that most influence the students’ performance. By using a clustering algorithm, we detect six different clusters of students regarding their interaction with the LMS. Analyzing the interaction patterns of each cluster, we find that those patterns are repeated in all the early stages of the course. Finally, we show how four out of those six student-LMS interaction patterns have a strong correlation with students’ performance.
... However, the ability to make claims about student learning or behaviour based solely on LMS data has strict limitations (Conijn et al., 2017;Reiman et al., 2014, p. 529;Gašević et al., 2015;Lukarov et al., 2015;Lodge & Lewis, 2012). For instance, interactions (Agudo-Peregrina et al., 2014) are the most basic unit of learning data in virtual learning environments, but "there is no consensus yet on which interactions are relevant for effective learning" (2014, p. 542). ...
... These results are important because there is a need to improve LMS data analysis claims about students LMS observed behaviours (Conijn et al., 2017;Gašević et al., 2015;Lukarov et al., 2015;Reiman et al., 2014;Lodge & Lewis, 2012) to better process, analyze and translate data into actionable knowledge for teachers and researchers (Howard et al., 2018); because teachers need to make sure that students are in condition to learn actively and independently (Patrick et al., 2012;Tanner, 2012;Weimer, 2013) and because there is a need for a parsimonious theory of performance, of student characteristics and learning environment conditions that elicit optimal performance in students (Pardo et al., 2015;Corno et al., 2002); and because there is a need for instructors who observe students' behaviours and continuously ask what motivates these behaviours to understand them (Prizant & Fields-Meyer, 2015;Clark, 2016), instructors who may be analytical about their own practice and make iterative instructional decisions based on evidence (Tanner, 2011), who are engaged in sustained and intentional processes of identifying and checking the accuracy and validity of their teaching assumptions (Brookfield, 2017). ...
Thesis
In Higher Education, instructors provide students with opportunities to develop essential knowledge, competencies and skills. To offer students the highest quality learning experiences, effective instructors analyze their practice, intentionally seek to identify and check their teaching assumptions, and make iterative instructional decisions based on evidence. However, teaching and learning situations are complex and ill-defined and there is a lack of a parsimonious theory of student characteristics and learning conditions that elicit optimal performance in students. Moreover, learning analytics support the processing, analysis and translation of data into actionable knowledge but there is no consensus yet on which interactions are relevant for effective learning. Thus, this study sought to gain a deeper understanding on how and why students thrived and were productively engaged with insights from psychometric information and course trace data. Findings of this study contribute to the literature that seek to 1) translate trace data into actionable knowledge, 2) understand those characteristics and conditions that elicit optimal student performance, or 3) demonstrate how to use academic achievement, trace data, and psychometric characteristics to analyze an instructor’s practice. This study reports on research into 4,150 unique student course interactions clustered within 46 undergraduate student trajectories during an elective blended-learning course. It sought to describe changes in students’ active and independent online interaction behaviours; explore differences in interaction trajectories between students; and examine the relationship between students’ interaction trajectories, psychometric characteristics and levels of achievement. Students’ course interaction trace data was captured by a Learning Management System (LMS). Student characteristics were collected through self-report psychometric instruments completed as supplemental, non-graded, in-class learning activities. Finally, student achievement through total course, summative exams and formative assignment grades. Restricted Maximum Likelihood (REML) linear regressions described interindividual differences in students’ growing proportion of course objects accessed across time (interaction trajectories). Maximum Likelihood (ML) multilevel longitudinal regression models, with changes in the proportion of course objects nested within individuals, significantly described students’ average and individual trajectories of interaction and differences between course assessment periods and conscientiousness levels. Pearson and Spearman correlations found significant relationships between interaction trajectories and personality traits, psychosocial maturity resolutions, self-efficacy, self-regulation, reasons for studying, and major life goals, and between interaction trajectories and student achievement (knowledge/exam grades). Significant negative relationships were found between academic achievement, psychosocial intimacy-isolation resolutions, and major life aspirations to have a family life, to make meaningful contributions, and to have fun. After reflecting on these results, this instructor concluded that the courses, although beneficial, could have better promoted students’ optimal performance by shifting to a more streamlined set of outcomes and a clearer learning path; and by realigning learning activities and intended learning outcomes to better match students’ long-range aspirations. Findings from this study suggest that students should be treated not only as cognitive systems but that students may be productively engaged as human beings continually seeking to realize their own possibilities. Although these propositions may not be statistically generalizable, they may be analytically generalized if replicated in more education contexts.
... Machine learning algorithms used in the review are divided either as classification (Asif et al., 2017;Burgos et al., 2018;Hlosta et al., 2017) or as regression methods (Ioanna Lykourentzou et al., 2009;Kotsiantis et al., 2004;Mayilvaganan & Kalpanadevi, 2015). Some of the research studies have used correlation analysis to measure the correlation of features with the final performance of students (Conijn et al., 2017;Macfadyen & Dawson, 2010). Few studies have used voting methods to combine decisions from different methods (e.g. ...
... Therefore, simply accuracy measures are considered biased in situations where data are imbalanced between the numbers of samples in different classes. Many studies (Conijn et al., 2017;Jarvela & Hakkinen, 2003;Lopez Guarin et al., 2015;Márquez-Vera et al., 2016) have used other metrics that address the classification error of both majority and minority class. Other metrics used for classification are determined using the confusion matrix that includes true positive (TP), true negatives (TN), false positives (FP) and false negatives (FN) (Figure 9). ...
Article
Predictive models on students’ academic performance can be built by using historical data for modelling students’ learning behaviour. Such models can be employed in educational settings to determine how new students will perform and in predicting whether these students should be classed as at-risk of failing a course. Stakeholders can use predictive models to detect learning difficulties faced by students and thereby plan effective interventions to support students. In this paper, we present a systematic literature review on how predictive analytics have been applied in the higher education domain. Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we conducted a literature search from 2008 to 2018 and explored current trends in building data-driven predictive models to gauge students’ performance. Machine learning techniques and strategies used to build predictive models in prior studies are discussed. Furthermore, limitations encountered in interpreting data are stated and future research directions proposed.
... Such measures are commonly viewed through the broader framework of self-regulated learning, such that the quality and extent of activity can provide a composite measure of a student's motivation, engagement, time management, learning strategies, study behavior, and more (Roll & Winne, 2015;Winne, 2017). In general, studies find that these objective measures of a student's LMS activity are positively associated with engagement and achievement (Cerezo et al., 2016;Conijn et al., 2017;Joksimović et al., 2015;You, 2016;Yu & Jo, 2014), even when controlling for the amount of work assigned within a course (Motz et al., 2019). ...
... Normally, one might expect that an increased number of learning activities and increased effort on these learning activities would correspond with improved academic outcomes, as is typically the case in learning analytics examinations (Cerezo et al., 2016;Conijn et al., 2017;Joksimović et al., 2015;Motz et al., 2019;You, 2016;Yu & Jo, 2014). However, we observed precisely the opposite. ...
Article
Full-text available
Under normal circumstances, when students invest more effort in their schoolwork, they generally show evidence of improved academic achievement. But when universities abruptly transitioned to remote instruction in Spring 2020, instructors assigned rapidly-prepared online learning activities, disrupting the normal relationship between effort and outcomes. In this study, we examine this relationship using data observed from a large-scale survey of undergraduate students, from logs of student activity in the online learning management system, and from students' estimated cumulative performance in their courses (n = 4,636). We find that there was a general increase in the number of assignments that students were expected to complete following the transition to remote instruction, and that students who spent more time and reported more effort carrying out this coursework generally had lower course performance and reported feeling less successful. We infer that instructors, under pressure to rapidly put their course materials online, modified their courses to include online busywork that did not constitute meaningful learning activities, which had a detrimental effect on student outcomes at scale. These findings are discussed in contrast with other situations when increased engagement does not necessarily lead to improved learning outcomes, and in comparison with the broader relationship between effort and academic achievement.
... A contrast between traditional classification and clustering algorithms implemented in Weka was performed, together with various approaches for instance, and attribute selection. Conijn et al. (2016) explored students' usage data in Moodle LMS as a forecaster for their grade of exam. 438 students from seven engineering courses were included in the study. ...
Article
Full-text available
For a productive life, education plays a critical role to fill individual life with value and excellence. Education is compulsory to provide things that individuals partake in to compete in the modern world. Predicting the academic performance of the student is the most successive research in this era. A different set of approaches and methods are incorporated to increase student performance. However, this is a challenging task due to the wrong course selection. In the proposed study, we have used the hybrid approach consisting of Cluster-based Linear Discriminant Analysis (CLDA) and Artificial Neural Network (ANN) to provide the prospective students with the motivational comments and the video recommendations by which students can choose the right subject and the comments will facilitate the students with the insight reasons of dropout opted by other students for this course. The outcomes of this study will help in the reduction of the number of dropouts. The students will be able to choose an appropriate course for performance enhancement and carrier excel.
... In the practice of hybrid teaching, various learning platforms have appeared in recent years, including Moodle [13], Blackboard [14], WeChat Public Platform [15], Mosoteach [16], and so on. From the perspective of students, however, the teaching platform is not the main factor affecting blended teaching. ...
... Predictive models for blended courses are still limited. Conijn et al. (2017) analyzed 17 blended courses and concluded that the portability of the prediction models across courses is low. ...
Conference Paper
The Texas construction market is the second-largest hub inside the U.S. Nearly 750 thousand people are working in different sectors of the Texas construction industry. Although the big picture indicates steady growth in Texas hiring size in the last 30 years, the Texas construction market's volatility has been an issue for construction companies and their hiring plans. Rather than seasonal patterns inherent to construction activities, factors such as economic recessions and crises, tropical hurricanes, and outbreaks of pandemics are potential reasons for fluctuations in construction companies' demand to hire. The impact of each factor on the cities varies due to geographical and demographical diversity inside Texas. This paper focuses on understanding workforce migration behaviors following local disasters because it relies heavily on the local workforce. To determine each factor's significance is to find if they created an anomaly in the dataset after they occurred. This research implemented an outlier detection analysis on Texas cities and compared the resulting outlier dates with the timeline of Texas's extreme events in the last 30 years. The results show that economic crises with national scales such as the dot-com bubble at the start of the century and the 2008 economic crisis mostly affected four major cities (Austin, Houston, Dallas/Fort Worth, and San Antonio) of Texas. Multi-state local disasters such as hurricane Harvey impacted both major cities and their satellite cities, suggesting the migration of the workforce to the disaster-areas. The research found that low population cities have been affected by local disasters.
... Interpretations: According to [27], the p-value is a good parameter for measuring the relevance of a predictor variable in a regression model. If its value is less than 0.05, we say that this variable is relevant, so it has an influence on the explained variable (the final grade of the students in our case). ...
Conference Paper
Nowadays, a new form of learning has emerged in higher education. This is e-Learning. Lessons are taught on a Learning Content Management Systems (LCMS). These platforms generate a large variety of data at very high speed. This massive data comes from the interactions between the system and the users and between the users themselves (Learners, Tutors, Teachers, administrative Agents). Since 2013, UVS (Virtual University of Senegal), a digital university that offers distance learning through Moodle and Blackboard Collaborate platforms, has emerged. In terms of statistics, it has 29340 students, more than 400 active Tutors and 1000 courses. As a result, a large volume of data is generated on its learning platforms. In this article, we have set up an architecture allowing us to execute all types of queries on all data from platforms (historical data and real-time data) in order to set up intelligent systems capable of improving learning in this university. We then set up a machine learning model as a use case which is based on multiple regression in order to predict the most influential learning objects on the learners' final mark according to his learning activities.
... They all show a high accuracy of the prediction of students' success early in the semester. [13] analyze the Moodle Learning Managment System (LMS), in which they predict students' performance. They show that for the purpose of early intervention or conditional on in-between assessment grades, LMS data are of little value. ...
Preprint
Full-text available
We use learning data of an e-assessment platform for an introductory mathematical statistics course to predict the probability of passing the final exam for each student. Based on these estimated probabilities we sent warning emails to students in the next cohort with a low predicted probability to pass. We analyze the effect of this treatment and propose statistical models to quantify the effect of the email notification. We detect a small but imprecisely estimated effect suggesting effectiveness of such interventions only when administered more intensively.
... One of the most important lessons that we have learned is that context matters: models obtained in one context are barely transferable to other contexts [2]. Researchers have failed to replicate the results of predictive models (e.g., for estimating student performance) across multiple learning settings due to the remarkable diversity in the data generated by students' learning activates, the obtained predictors, as well as the levels of statistical significance [3,4]. These inconsistencies have made the efforts towards offering adaptive learning or personalizing support an arduous endeavor. ...
Conference Paper
Full-text available
Recent findings in the field of learning analytics have brought to our attention that conclusions drawn from cross-sectional group-level data may not capture the dynamic processes that unfold within each individual learner. In this light, idiographic methods have started to gain grounds in many fields as a possible solution to examine students’ behavior at the individual level by using several data points from each learner to create person-specific insights. In this study, we introduce such novel methods to the learning analytics field by exploring the possible potentials that one can gain from zooming in on the fine-grained dynamics of a single student. Specifically, we make use of Gaussian Graphical Models —an emerging trend in network science— to analyze a single student's dispositions and devise insights specific to him/her. The results of our study revealed that the student under examination may be in need to learn better self-regulation techniques regarding reflection and planning.
... In this context, learning management systems (LMSs) are very useful to support the teaching-learning process of students (Piña, 2010) Moodle has been extensively studied from different points of view and disciplines (Carvalho et al., 2011;Conijn et al., 2017;De Medio et al., 2020;Lu & Law, 2012;Martín-Blas & Serrano-Fernández, 2009), its acceptance and use as a technological product by end users being one of the topics that was of particular interest to the most recent researchers from the opinions of teachers and students of higher education (Nistor et al., 2019). ...
Article
Full-text available
Moodle is one of the most widely used learning management systems currently and has traditionally been studied through the Technology Acceptance Model. Existing literature is scattered and does not allow us to clearly conclude what characteristics this technological acceptance has as well as its main progress. The study aimed to provide an overview of the scientific literature on the application of TAM in the study of the acceptance and technological use of Moodle through a Systematic Mapping Study (SMS). Results from 24 selected studies indicate that the topic is of increasing interest and that the literature is characterized by studies that 1) use extended versions of TAM; 2) are based on university students from different programs; 3) take place in Europe or Asia; 4) are published in journals with different impact factors; and 5) have focused on testing mainly the original TAM hypotheses. Although a lot of hypotheses have been studied (271), only 16 have been accepted more times than rejected by 2 or more studies, with Perceived Ease of Use being the most commonly present construct in the hypotheses. Results imply that although TAM remains as a robust model for studying Moodle there are still important gaps to be addressed.
... One of the most important lessons that we have learned is that context matters: models obtained in one context are barely transferable to other contexts (Gašević et al., 2016). Researchers have failed to replicate the results of predictive models (e.g., for estimating student performance) across multiple learning settings due to the remarkable diversity in the data generated by students' learning activates, the obtained predictors, as well as the levels of statistical significance (Conijn et al., 2017;Dawson et al., 2019). These inconsistencies have made the efforts towards offering adaptive learning or personalizing support an arduous endeavor. ...
Preprint
Full-text available
Recent findings in the field of learning analytics have brought to our attention that conclusions drawn from cross-sectional group-level data may not capture the dynamic processes that unfold within each individual learner. In this light, idiographic methods have started to gain grounds in many fields as a possible solution to examine students' behavior at the individual level by using several data points from each learner to create person-specific insights. In this study, we introduce such novel methods to the learning analytics field by exploring the possible potentials that one can gain from zooming in on the fine-grained dynamics of a single student. Specifically, we make use of Gaussian Graphical Models-an emerging trend in network science-to analyze a single student's dispositions and devise insights specific to him/her. The results of our study revealed that the student under examination may be in need to learn better self-regulation techniques regarding reflection and planning.
... Similar methods as in subsection 4.3 were used in this category. For example, Conijn, Snijders, Kleingeld, and Matzat (2017) computed correlation analysis and multi-level analyses with cross-random effects, multiple linear regressions techniques on datasets of students' online behaviour (from Moodle LMS). Key indicators: LMS data and assessment data (including in-between grades, final exam grades and overall course grade). ...
Conference Paper
Common factors, which are related to study success include students’ sociodemographic factors, cognitive capacity, or prior academic performance, and individual attributes as well as course related factors such as active learning and attention or environmental factors related to supportive academic and social embeddedness. The aim of this research is to gain a deeper understanding of not only if learning analytics can support study success, but which aspects of a learner’s learning journey can benefit from the utilisation of learning analytics. We, therefore, examined different learning analytics indicators to show which aspect of the learning journey they were successfully supporting. Key indicators may include GPA, learning history, and clickstream data. Depending on the type of higher education institution, and the mode of education (face-to-face and/or distance), the chosen indicators may be different due to them having different importance in predicting the learning out-comes and study success.
... The syllabi-based tabular method is popular for problem-based teaching [19] and problem-solving environments [20], which aim to collect students' collaborative study activities and physical interactions [21], and teaching resources, students' computational practice, and interaction results are collected in a table or list [22]. The learning management system (LMS) is a popular method used for learning resource organization, and it is broadly adopted to support teaching activities [23], to collect students' online behaviors [24], and to predict academic performance [25]. These software programs provide learners with a browser-based environment they can use to review course objectives, download course materials, and submit assignments through hyperlinks that are dynamically added to or removed from the webpage based on resource availability [26]. ...
Article
Full-text available
Online learning and teaching have become the primary forms of education during the global pandemic, and online learning systems, which can provide fair educational opportunities for everyone, are increasingly important for sustainability in education. The amount of time a student spends on online learning systems affects the fairness and persistence of sustainability in education. To support personalized learning opportunities, interactive learning, and easy-to-access resources, we propose a map organization and visualization method called MapOnLearn for online learning systems. First, we converted tree-like hierarchical course units (HCUs) and knowledge points (KPs) into a fundamental two-dimensional (2D) map of hierarchically divided polygons and used the map to form containers to manage all learning resources. Then, we used the zoom feature of the map to express the hierarchical structures of knowledge and formulated corresponding rules for displaying information at different levels. Path analysis was applied to express the learning process, and topological processing was adapted to represent the relationships among HCUs and KPs. We developed maps for a high-level math course, a course on data structures, and an English course at a university in China and investigated 264 students and 27 teachers for a semester by using the technology acceptance model (TAM). We found that the map visualization and organization method had a positive impact on the way teachers and students use online learning systems and improved the online learning experience. To attract more students to spend more time on online learning, we hope our method can promote the sustainable development of education.
...  Identificirati sesiju svakog korisnika [34]. Sesija počinje prvim klikom korisnika na konkretnom kursu (ne na nekom drugom dijelu sistema) do trenutka odjave (bilo sistemske ili da se korisnik sam odjavi sa sistema) ili najviše 30 minuta od posljednje akcije na kursu. ...
Conference Paper
Full-text available
Ovaj rad istražuje cjelokupni postupak korištenja rudarjenja podataka (K-means klasteriranjem) za ekstrahiranje i pronalaženje korisnog znanja i zanimljivih uvida iz velike količine podataka o studentima prikupljenih tijekom online učenja, zabilježenih u datotekama dnevnika događaja (log files) poslužitelja Moodle LMS-a koji se koristi u programima učenja na daljinu na Ekonomskom fakultetu. Kao nesupervizirana tehnika, algoritam grupiranja K-means algoritam nema "učitelja", moguće je da ti odvojivi klasteri zajedničke distribucije tih četiriju varijabli ne slijede jasno razdvajanje pri prvom pokušaju studentskog ispita. Stoga su rezultati algoritma grupiranja potvrđeni slijedećim kriterijem-uspješnost na prvom pokušaju studentskog ispita. Dakle, na kraju je provjereno da pronađeni klasteri studenata koji koriste četiri varijable također odgovaraju razlici uspjeha na prvom ispitnom pokušaju, tj. da li i ti klasteri "razdvajaju" studente u svojoj procjeni kao na prvom ispitu. Takvi uvidi predstavljaju korisne informacije nastavnicima i menadžmentu visokih učilišta pri donošenju informiranih odluka o tome kako unaprijediti nastavu ili podržati manje uspješne studente da ispune zahtjeve nastavnog plana i programa. Ključne riječi-tehnika klasteriranja; rudarenje obrazovnih podataka; edukacijska analitika; blended learning.
... 19,20 These predictions can be conducted toward the different types of learning environment, such as fully online or blended learning. 21 Previous studies have successfully predicted student first-year academic performance. 22 According to the literature, predicting student learning performance can use the following data: (i) student historical records and relevant information such as high school grades and parents' education 23 and (ii) students' log data from learning management systems and other learning environments. ...
Article
Over the past decade, the field of education has seen stark changes in the way that data are collected and leveraged to support high-stakes decision-making. Utilizing big data as a meaningful lens to inform teaching and learning can increase academic success. Data-driven research has been conducted to understand student learning performance, such as predicting at-risk students at an early stage and recommending tailored interventions to support services. However, few studies in veterinary education have adopted Learning Analytics. This article examines the adoption of Learning Analytics by using the retrospective data from the first-year professional Doctor of Veterinary Medicine program. The article gives detailed examples of predicting six courses from week 0 (i.e., before the classes started) to week 14 in the semester of Spring 2018. The weekly models for each course showed the change of prediction results as well as the comparison between the prediction results, and students' actual performance. From the prediction models, at-risk students were successfully identified at the early stage, which would help inform instructors to pay more attention to them at this point.
... The growing adoption of LMS, MOOC, and OLE has significantly increased the possibility to collect a wider amount of log data, and several authors have applied data mining algorithms to discover patterns in actions performed by students. They investigated features like resource usage, action frequency, average latency, login frequency, number of module accesses, login time, login regularity, total studying time, and regularity of learning interval Bernardini and Conati (2010), Conijn et al. (2017), Cooley et al. (1999), Darlington (2017), Long (2011), Sael et al. (2013), Bovo et al. (2013), Yu and Jo (2014), Fincham et al. (2019). The authors have developed specific analytical tools for their objectives or case studies, in general, related to predicting student academic success and detecting at-risk students to avoid academic failure or drop-outs. ...
Preprint
Full-text available
Research is constantly engaged in finding more productive and powerful ways to support quality learning and teaching. However, although researchers and data scientists try to analyse educational data most transparently and responsibly, the risk of training machine learning algorithms on biased datasets is always around the corner and may lead to misinterpretations of student behaviour. This may happen in case of partial understanding of how learning log data is generated. Moreover, the pursuit of an ever friendlier user experience moves more and more Learning Management Systems functionality from the server to the client, but it tends to reduce significant logs as a side effect. This paper tries to focus on these issues showing some examples of learning log data extracted from Moodle and some possible misinterpretations that they hide with the aim to open the debate on data understanding and data knowledge loss.
... One of the most important lessons that we have learned is that context matters: models obtained in one context are barely transferable to other contexts . Researchers have failed to replicate the results of predictive models (e.g., for estimating student performance) across multiple learning settings due to the remarkable diversity in the data generated by students' learning activates, the obtained predictors, as well as the levels of statistical significance (Conijn et al., 2017;. These inconsistencies have made the efforts towards offering adaptive learning or personalizing support an arduous endeavor. ...
Conference Paper
Full-text available
Network analysis simulations were used to guide decision-makers while configuring instructional spaces on our campus during COVID-19. Course enrollment data were utilized to estimate metrics of student-to-student contact under various instruction mode scenarios. Campus administrators developed recommendations based on these metrics; examples of learning analytics implementation are provided.
... In addition to these assessments, prior work suggests that non-academic characteristics should also be incorporated along with academic factors from different university databases (Lee et al. 2015;Xue 2018). Examples include (1) student financial aid data (Adekitan and Noma-Osaghae 2019), (2) demographic characteristics (Al-Shabandar et al. 2017), and (3) learning management system (LMS) variables (Conijn et al. 2016). LMS data, in conjunction with academic characteristics (e.g., grade point average) and personal data records have been shown to improve prediction success (Vovides et al. 2017;Zhai et al. 2020). ...
Article
Full-text available
High levels of attrition characterize undergraduate science courses in the USA. Predictive analytics research seeks to build models that identify at-risk students and suggest interventions that enhance student success. This study examines whether incorporating a novel assessment type (concept inventories [CI]) and using machine learning (ML) methods (1) improves prediction quality, (2) reduces the time point of successful prediction, and (3) suggests more actionable course-level interventions. A corpus of university and course-level assessment and non-assessment variables (53 variables in total) from 3225 students (over six semesters) was gathered. Five ML methods were employed (two individuals, three ensembles) at three time points (pre-course, week 3, week 6) to quantify predictive efficacy. Inclusion of course-specific CI data along with university-specific corpora significantly improved prediction performance. Ensemble ML methods, in particular the generalized linear model with elastic net (GLMNET), yielded significantly higher area under the curve (AUC) values compared with non-ensemble techniques. Logistic regression achieved the poorest prediction performance and consistently underperformed. Surprisingly, increasing corpus size (i.e., amount of historical data) did not meaningfully impact prediction success. We discuss the roles that novel assessment types and ML techniques may play in advancing predictive learning analytics and addressing attrition in undergraduate science education.
Article
Full-text available
Artificial Intelligence is the domain of computer science which includes the solving of problems in reasoning, knowledge representation, prediction, learning and perception areas. The large volume of data can be used for social media, e-learning, distance learning and e-commerce environment. Our research work includes the classification and prediction of students' performance in educational sciences. The analyzed results are forecasting the future plan in higher studies. In this work, we use TensorFlow Artificial Intelligence engine for classification. Deep learning is used for measuring academic performance in core courses such as mathematics, physics, chemistry, biology and computer Science. The performance can be measured in nonacademic activities also such as sports, yoga, art and social services. These papers gives prediction result using machine learning tools and give more comprehensive study of both academic and nonacademic activities. Here we take number of intermediate nodes from students' performance and number of deep learning objects from students' activities. The result is generated and compared using TensorFlow. The input of two thousand five hundred students' data is taken from Tamil Nadu Nagapattinam and Thirvarur Districts from education science department, 65% of data is trained data and 35% of data are test data. The accuracy factor is 75% to 85%. The prediction factor accuracy can be determined by using optimal configuration of TensorFlow engine. This result can be used for the benefit of the students to select their future studies and career development of students based on their higher secondary academic and nonacademic performance factors.
Article
This decade, e-learning systems provide more interactivity to instructors and students than traditional systems and make possible a completely online (CO) education. However, instructors could not warn if a CO student is engaged or not in the course, and they could not predict his or her academic performance in courses. This work provides a collection of models (exploratory factor analysis, multiple linear regressions, cluster analysis, and correlation) to early predict the academic performance of students. These models are constructed using Moodle interaction data, characteristics, and grades of 802 undergraduate students from a CO university. The models result indicated that the major contribution to the prediction of the academic student performance is made by four factors: Access, Questionnaire, Task, and Age. Access factor is composed by variables related to accesses of students in Moodle, including visits to forums and glossaries. Questionnaire factor summarizes variables related to visits and attempts in questionnaires. Task factor is composed of variables related to consulted and submitted tasks. The Age factor contains the student age. Also, it is remarkable that Age was identified as a negative predictor of the performance of students, indicating that the student performance is inversely proportional to age. In addition, cluster analysis found five groups and sustained that number of interactions with Moodle are closely related to performance of students.
Conference Paper
Full-text available
Cloud Computing has become an important element of computer science education. This notion is supported by the main cloud service providers that offer resources to facilitate cloud-based module instruction. They focus however on specific topics and do not yet cope with final year projects (or dissertations), a semester or year-long task with particularities: the student works individually and not as part of a class and dives deeper into multiple and diverse technologies. We present a modular methodology to fill in this gap and address the end-to-end delivery of such projects in a way that can be evaluated through a set of assessment criteria and is transferable to other academic institutions. This methodology consists of six phases: from preparing and attracting students to undertake a cloud-based project, through their on-boarding and initial training to monitoring project work and activities beyond the project completion. By addressing this issue, we simplify the upskilling of students (and supervisors) and ease their adaptation to cloud related career pathways.
Article
Full-text available
This paper contains the design and development of an Adaptive Virtual Learning Environment (AdaptiveVLE) framework to assist educators of all disciplines with creating adaptive VLEs tailored to their needs and to contribute towards the creation of a more generic framework for adaptive systems. Fully online education is a major trend in education technology of our times. However, it has been criticised for its lack of personalisation and therefore not adequately addressing individual students’ needs. Adaptivity and intelligence are elements that could substantially improve the student experience and enhance the learning taking place. There are several attempts in academia and in industry to provide adaptive VLEs and therefore personalise educational provision. All these attempts require a multiple-domain (multi-disciplinary) approach from education professionals, software developers, data scientists to cover all aspects of the system. An integrated environment that can be used by all the multiple-domain users mentioned above and will allow for quick experimentation of different approaches is currently missing. Specifically, a transparent approach that will enable the educator to configure the data collected and the way it is processed without any knowledge of software development and/or data science algorithms implementation details is required. In our proposed work, we developed a new language/framework using MPS JetBrains Domain-Specific Language (DSL) development environment to address this problem. Our work consists of the following stages: data collection configuration by the educator, implementation of the adaptive VLE, data processing, adaptation of the learning path. These stages correspond to the adaptivity stages of all adaptive systems such as monitoring, processing and adaptation. The extension of our framework to include other application areas such as business analytics, health analytics, etc. so that it becomes a generic framework for adaptive systems as well as more usability testing for all applications will be part of our future work.
Conference Paper
The province of Quebec in Canada has begun to implement an important plan to bring a digital shift to the educational system. One of the key aspects of this plan is to implement a global electronic student file system. These electronic files encompass a lot of information that can in turn be used to monitor the progress of the students. In this paper, our team was able to obtain a large dataset from this new technological platform and used it to predict the grade of students. We tested up to 328 features and produced different datasets for classification. Moreover, different features selection methods were used. Finally, we were able to predict the end of the year final grade with up to 75% accuracy.
Article
Full-text available
Learning Analytics (LA) approaches in Blended Learning (BL) research is becoming an established field. In the light of previous critiqued toward LA for not being grounded in theory, the General Data Protection and a renewed focus on individuals’ integrity, this review aims to explore the use of theories, the methodological and analytic approaches in educational settings, along with surveying ethical and legal considerations. The review also maps and explores the outcomes and discusses the pitfalls and potentials currently seen in the field. Journal articles and conference papers were identified through systematic search across relevant databases. 70 papers met the inclusion criteria: they applied LA within a BL setting, were peer-reviewed, full-papers, and if they were in English. The results reveal that the use of theoretical and methodological approaches was disperse, we identified approaches of BL not included in categories of BL in existing BL literature and suggest these may be referred to as hybrid blended learning, that ethical considerations and legal requirements have often been overlooked. We highlight critical issues that contribute to raise awareness and inform alignment for future research to ameliorate diffuse applications within the field of LA.
Article
Introduction This study examined student access to online resources of a faculty’s learning management system (LMS). Issues relating to current e‐learning resources usage were identified and formed the basis for recommendations to help assist stakeholders in teaching, learning and research. Methods Learning analytics from four cohorts of undergraduate dental students were extracted from the database of a LMS spanning between 2012‐2016. Individual datasets were combined into one master file, re‐categorized, filtered and analyzed based on cohort, year of study, course and nature of online resource. Results A total of 157,293 access events were documented. The proportion of administrative to learning data varied across cohorts, with oldest cohort having the highest ratio (82:18) in their final year and most recent cohort having a ratio of 33:67 in their 4th year demonstrating a higher proportion to learning. Seven Learning domains were identified in the access data: access to problem‐based learning resources was the highest and next was fixed prosthodontics videos. The prosthodontics discipline had the highest access across the curriculum while some others had very limited or even no learning access events. Conclusion A number of limitations have been identified with the analytics and learning resources in this LMS and engagement with learning resource provision. More detailed data capture of access use and unique identifiers to resources as well as keyword tagging of the resources are required to allow accurate mapping and support of students learning. Moreover, motivation or nudging of students behavior to more actively engage with learning content needs exploration.
Chapter
This paper investigates the problem of predicting student flight performance in a training simulation from multimodal features, including flight controls, visual attention, and knowledge acquisition tests. This is a key component of developing next generation training simulations that can optimize the usage of student training time. Two types of supervised machine learning classifiers (random forest and support vector machines) were trained to predict the performance of twenty-three students performing simple flight tasks in virtual reality. Our experiments reveal the following: 1) features derived from gaze tracking and knowledge acquisition tests can serve as an adequate substitute for flight control features; 2) data from the initial portion of the flight task is sufficient to predict the final outcome; 3) our classifiers perform slightly better at predicting student failure than success. These results indicate the feasibility of using machine learning for early prediction of student failures during flight training.
Chapter
Predicting student performance is a very important but yet challenging task in education. In this paper, we propose a Multi-View Network Embedding (MVNE) method for student performance prediction, which effectively fuses multiple data sources. We first construct three networks to model three different types of data sources correlated with student performance, ranging from class performance data, historical grades, to students’ campus social relationships. Then we use joint network embedding to learn the embedding representation of students and questions based on the proposed separated random walk sampling. Student performance is predicted based on both student and question similarities in the low-dimensional representation. Experimental results on the real-world datasets demonstrate the effectiveness of the proposed method.
Article
Full-text available
Coronavirus (Covid-19) pandemic has imposed a complete shut-down of face-to-face teaching to universities and schools, forcing a crash course for online learning plans and technology for students and faculty. In the midst of this unprecedented crisis, video conferencing platforms (e.g., Zoom, WebEx, MS Teams) and learning management systems (LMSs), like Moodle, Blackboard and Google Classroom, are being adopted and heavily used as online learning environments (OLEs). However, as such media solely provide the platform for e-interaction, effective methods that can be used to predict the learner’s behavior in the OLEs, which should be available as supportive tools to educators and metacognitive triggers to learners. Here we show, for the first time, that Deep Learning techniques can be used to handle LMS users’ interaction data and form a novel predictive model, namely DeepLMS, that can forecast the quality of interaction (QoI) with LMS. Using Long Short-Term Memory (LSTM) networks, DeepLMS results in average testing Root Mean Square Error (RMSE) $$<0.009$$ < 0.009 , and average correlation coefficient between ground truth and predicted QoI values $$r\ge 0.97$$ r ≥ 0.97 $$(p<0.05)$$ ( p < 0.05 ) , when tested on QoI data from one database pre- and two ones during-Covid-19 pandemic. DeepLMS personalized QoI forecasting scaffolds user’s online learning engagement and provides educators with an evaluation path, additionally to the content-related assessment, enriching the overall view on the learners’ motivation and participation in the learning process.
Article
This paper investigates the compatibility of e-learning platforms with a competency-based educative (CBE) approach. Using the European Tuning program model derived from the Bologna Process declaration, an integrative framework for creating a degree curriculum based on the CBE approach has been designed such as to include degree competencies, learning outcomes, courses and learning activities. This framework and its underlying principles were then experimented on Moodle in order to explore to what extent the platform offers features that are adapted to implement the CBE approach. For comparison purposes, the compatibility of three other learning management systems (LMSs), namely Blackboard , Canvas and Brightspace, with the CBE approach was also examined. The overall results showed that the explored e-learning platforms offer a relatively compatible environment to the elaborated framework for online courses, thereby adding a boost of broadness and validity to the suggested CBE model.
Article
Industrial electronics (IE) covers a wide range of technologies and applications, being a key enabling technology for numerous industrial, domestic, and biomedical uses, among others. In this context, IE education has become a relevant and challenging topic for society and industry. This article covers its evolution and state-of-the-art methodologies and provides an overall view of its status around the world. Finally, future trends and challenges in IE education are discussed.
Article
Effective candidates screening is critical for any IT firm as it impacts the future growth and productivity of that firm. Currently majority of these firms follow a manual approach of hiring employees which is more prone to errors and time consuming. Prime purpose of the research is to develop an intelligent predictive model to decide upon candidate’s suitability for an applied It based job. A sample size of 13,168 instances with 19 attributes of job seekers data are used for study. An ensemble predictive meta model approach is presented in the research where a stacked KNN (K-Nearest Neighbours) algorithm combined with hard voting approach is employed. The idea is to use a single unified predictive model in place of separate classification models by considering the predicted class with maximum votes against each class label. The proposed approach comprises two functional modules. ‘Stacked KNN Learner Module’ involves the use of five variants of KNN which include 1-NN, 3-NN, 5-NN, 7-NN and 9-NN. Individual prediction of the job seeker data from these variants of KNN algorithm are input to the ‘Hard Voting Ensemble Predictive unit’ which eventually aggregates majority of predicted class votes determined for each class label to generate the final output predicted class label on basis of maximum votes. The developed ensemble meta model is successfully implemented using python programming language and its performance evaluation is done using various metrics. The implemented model generated a 96.96% prediction accuracy rate. The specificity, sensitivity and f-score value recorded was 96.36%, 96.06% and 96.26% respectively. Mean Absolute Error (MAE) and Root mean Square Error (RMSE) value observed with our proposed meta model was 0.0048 and 0.0102 respectively. A comparative analysis of the proposed model was done by varying the data sample size and it gave a consistent performance throughout. It was observed that proposed ensemble model gave an impressive 96.96% mean accuracy rate and the overall performance of other variants was consistent. Performance of proposed meta model was also compared with some existing techniques like C.45, CART and K-Means. It outperformed other techniques in terms of accurately predicting the class label of candidates applying for job. This ensemble meta model can be of great help for growing IT firms and assist the organization unit in hiring right and deserving candidates in future. Proposed work can act as a decision making framework in multi sector units with enormous work force to streamline performance appraisal process thereby enabling in hiring right people for right job.
Chapter
This study aims to address the e-inclusion problem related to digital skills improvement and meaningful use. In professional work, more and more jobs require the use of digital skills. Combining e-learning and face-to-face training is a convenient and affordable way to learn new digital skills. However, the problem is the low number of e-learning graduates and the even lower number of those who use the newly acquired skills for professional or personal purposes. Machine learning approach is used to predict student achievement and other events. However, no comprehensive study has been conducted, analyzing how to improve digital skills training by ensuring that newly acquired skills are meaningfully used in professional life.
Chapter
Online learning has developed rapidly, but the participation of learners is very low. So it is of great significance to construct a prediction model of learning results, to identify students at risk in time and accurately. We select nine online learning behaviors from one course in Moodle, take one week as the basic unit and 5 weeks as the time node of learning behavior, and the aggregate data and sequence data of the first 5 weeks, the first 10 weeks, the first 15 weeks, the first 20 weeks, the first 25 weeks, the first 30 weeks, the first 35 weeks and the first 39 weeks are formed. Eight classic machine learning methods, i.e. Logistic Regression (LR), Naive Bayes (NB), Radom Forest (RF), K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Iterative Dichotomiser3 (ID3), Classification and Regression Trees (CART), and Neural Network (NN), are used to predict the learning results in different time nodes based on aggregate data and sequence data. The experimental results show that sequence data is more effective than aggregate data to predict learning results. The prediction AUC of RF model on sequence data is 0.77 at the lowest and 0.83 at the highest, the prediction AUC of CART model on sequence data is 0.70 at the lowest and 0.83 at the highest, which are the best models of the eight classic prediction models. Then Radom Forest (RF) model, Classification and Regression Trees (CART) model, recurrent neural network (RNN) model and long short term memory (LSTM) model are used to predict learning results on sequence data; the experimental results show that long short term memory (LSTM) is a model with the highest value of AUC and stable growth based on sequence data, and it is the best model of all models for predicting learning results.
Article
Full-text available
As of late, with the progression of AI and man-made brainpower, there has been a developing spotlight on versatile e-learning. As all ways to deal with e-learning lose their allure and the level of online courses builds, they move towards more customized versatile learning so as to collaborate with students and achieve better learning results. The schools focus on the examination, mindfulness, and arranging techniques that infuse innovation into the vision and educational program. E-learning issues are a standard examination issue for us all. The motivation behind this research analysis is to separate the potential outcomes of assessing e-learning models utilizing AI strategies such as Supervised, Semi Supervised, Reinforced Learning advances by investigating upsides and downsides of various methods organization. The literature review methodology is to review the cross sectional impacts of e-learning and Machine learning algorithms from existing literatures from the year 1993 to 2020 and to assess the essentialness of e-learning features to optimize the e-learning models with available Machine learning techniques from peer-inspected journals, capable destinations, and books. Second, it legitimizes the chances of e-learning structures introduction, and changes demonstrated through AI and Machine Learning algorithms. This examination assists in providing helpful new highlights to analysts, researchers and academicians. It gives an exhaustive structure of existing e-learning frameworks for the most recent innovations identified with learning framework capacities and learning tasks to envision ML research openings in appropriate spaces. The survey paper identifies and demonstrates the important role of different types of e-learning features such as Individual pertinent feature, Course pertinent feature, Context pertinent feature and Technology pertinent feature in framework performance tuning. The performance of Machine Learning algorithms to optimize the features of E-Learning models were reviewed in previous literatures and Support Vector Machine technique was found to be the one of the best to predict the input and output parameters of e-learning models and it is found that Fuzzy C Means, Deep Learning algorithms are producing better results for Big Data sets.
Article
Full-text available
Moodle is a widely deployed distance learning platform that provides numerous opportunities to enhance the learning process. Moodle’s importance in maintaining the continuity of education in states of emergency and other circumstances has been particularly demonstrated in the context of the COVID-19 virus’ rapid spread. However, there is a problem with personalizing the learning and monitoring of students’ work. There is room for upgrading the system by applying data mining and different machine-learning methods. The multi-agent Observer system proposed in our paper supports students engaged in learning by monitoring their work and making suggestions based on the prediction of their final course success, using indicators of engagement and machine-learning algorithms. A novelty is that Observer collects data independently of the Moodle database, autonomously creates a training set, and learns from gathered data. Since the data are anonymized, researchers and lecturers can freely use them for purposes broader than that specified for Observer. The paper shows how the methodology, technologies, and techniques used in Observer provide an autonomous system of personalized assistance for students within Moodle platforms.
Article
Prediction models that underlie “early warning systems” need improvement. Some predict outcomes using entrenched, unchangeable characteristics (e.g., socioeconomic status) and others rely on performance on early assignments to predict the final grades to which they contribute. Behavioral predictors of learning outcomes often accrue slowly, to the point that time needed to produce accurate predictions leaves little time for intervention. We aimed to improve on these methods by testing whether we could predict performance in a large lecture course using only students’ digital behaviors in weeks prior to the first exam. Early prediction based only on malleable behaviors provides time and opportunity to advise students on ways to alter study and improve performance. Thereafter, we took the not-yet-common step of applying the model and testing whether providing digital learning support to those predicted to perform poorly can improve their achievement. Using learning management system log data, we tested models composed of theory-aligned behaviors using multiple algorithms and obtained a model that accurately predicted poor grades. Our algorithm correctly identified 75% of students who failed to earn the grade of B or better needed to advance to the next course. We applied this model the next semester to predict achievement levels and provided a digital learning strategy intervention to students predicted to perform poorly. Those who accessed advice outperformed classmates on subsequent exams, and more students who accessed the advice achieved the B needed to move forward in their major than those who did not access advice.
Article
Full-text available
This study examined the extent to which instructional conditions influence the prediction of academic success in nine undergraduate courses offered in a blended learning model (n = 4134). The study illustrates the differences in predictive power and significant predictors between course-specific models and generalized predictive models. The results suggest that it is imperative for learning analytics research to account for the diverse ways technology is adopted and applied in course-specific contexts. The differences in technology use, especially those related to whether and how learners use the learning management system, require consideration before the log-data can be merged to create a generalized model for predicting academic success. A lack of attention to instructional conditions can lead to an over or under estimation of the effects of LMS features on students' academic success. These findings have broader implications for institutions seeking generalized and portable models for identifying students at risk of academic failure.
Article
Full-text available
Contemporary literature on online and distance education almost unequivocally argues for the importance of interactions in online learning settings. Nevertheless, the relationship between different types of interactions and learning outcomes is rather complex. Analyzing 204 offerings of 29 courses, over the period of six years, this study aimed at expanding the current understanding of the nature of this relationship. Specifically, with the use of trace data about interactions and utilizing the multilevel linear mixed modeling techniques, the study examined whether frequency and duration of student-student, student-instructor, student-system, and student-content interactions had an effect of learning outcomes, measured as final course grades. The findings show that the time spent on student-system interactions had a consistent and positive effect on the learning outcome, while the quantity of student-content interactions was negatively associated with the final course grades. The study also showed the importance of the educational level and the context of individual courses for the interaction types supported. Our findings further confirmed the potential of the use of trace data and learning analytics for studying learning and teaching in online settings. However, further research should account for various qualitative aspects of the interactions used while learning, different pedagogical/media features, as well as for the course design and delivery conditions in order to better explain the association between interaction types and the learning achievement. Finally, the results might imply the need for the development of the institutional and program-level strategies for learning and teaching that would promote effective pedagogical approaches to designing and guiding interactions in online and distance learning settings.
Conference Paper
Full-text available
Annie.Bryan@open.ac.uk ABSTRACT While substantial progress has been made in terms of predictive modeling in the Learning Analytics Knowledge (LAK) community, one element that is often ignored is the role of learning design. Learning design establishes the objectives and pedagogical plans which can be evaluated against the outcomes captured through learning analytics. However, no empirical study is available linking learning designs of a substantial number of courses with usage of Learning Management Systems (LMS) and learning performance. Using cluster-and correlation analyses, in this study we compared how 87 modules were designed, and how this impacted (static and dynamic) LMS behavior and learning performance. Our findings indicate that academics seem to design modules with an " invisible " blueprint in their mind. Our cluster analyses yielded four distinctive learning design patterns: constructivist, assessment-driven, balanced-variety and social constructivist modules. More importantly, learning design activities strongly influenced how students were engaging online. Finally, learning design activities seem to have an impact on learning performance, in particular when modules rely on assimilative activities. Our findings indicate that learning analytics researchers need to be aware of the impact of learning design on LMS data over time, and subsequent academic performance.
Conference Paper
Full-text available
All forms of learning take time. There is a large body of research suggesting that the amount of time spent on learning can improve the quality of learning, as represented by academic performance. The wide-spread adoption of learning technologies such as learning management systems (LMSs), has resulted in large amounts of data about student learning being readily accessible to educational researchers. One common use of this data is to measure time that students have spent on different learning tasks (i.e., time-on-task). Given that LMS systems typically only capture times when students executed various actions, time-on-task measures are estimated based on the recorded trace data. LMS trace data has been extensively used in many studies in the field of learning analytics, yet the problem of time-on-task estimation is rarely described in detail and the consequences that it entails are not fully examined. This paper presents the results of a study that examined the effects of different time-on-task estimation methods on the results of commonly adopted analytical models. The primary goal of this paper is to raise awareness of the issue of accuracy and appropriateness surrounding time-estimation within the broader learning analytics community, and to initiate a debate about the challenges of this process. Furthermore, the paper provides an overview of time-on-task estimation methods in educational and related research fields.
Article
Full-text available
Learning analytics seek to enhance the learning processes through systematic measurements of learning related data and to provide informative feedback to learners and educators. Track data from Learning Management Systems (LMS) constitute a main data source for learning analytics. This empirical contribution provides an application of Buckingham Shum and Deakin Crick's theoretical framework of dispositional learning analytics: an infrastructure that combines learning dispositions data with data extracted from computer-assisted, formative assessments and LMSs. In a large introductory quantitative methods module, 922 students were enrolled in a module based on principles of blended learning, combining face-to-face Problem-Based Learning sessions with e-tutorials. We investigated the predictive power of learning dispositions, outcomes of continuous formative assessments and other system generated data in modelling student performance and their potential to generate informative feedback. Using a dynamic, longitudinal perspective, computer-assisted formative assessments seem to be the best predictor for detecting underperforming students and academic performance, while basic LMS data did not substantially predict learning. If timely feedback is crucial, both use-intensity related track data from e-tutorial systems, and learning dispositions, are valuable sources for feedback generation.
Article
Full-text available
The Open Academic Analytics Initiative (OAAI) is a collaborative, multi‐year grant program aimed at researching issues related to the scaling up of learning analytics technologies and solutions across all of higher education. The paper describes the goals and objectives of the OAAI, depicts the process and challenges of collecting, organizing and mining student data to predict academic risk, and report results on the predictive performance of those models, their portability across pilot programs at partner institutions, and the results of interventions on at‐risk students.
Conference Paper
Full-text available
The aim of this study is to suggest more meaningful components for learning analytics in order to help learners improving their learning achievement continuously through an educational technology approach. Multiple linear regression analysis is conducted to determine which factors influence student's academic achievement. 84 undergraduate students in a women's university in South Korea participated in this study. The six-predictor model was able to account for 33.5% of the variance in final grade, F(6, 77) = 6.457, p < .001, R2 = .335. Total studying time in LMS, interaction with peers, regularity of learning interval in LMS, and number of downloads were determined to be significant factors for students' academic achievement in online learning environment. These four controllable variables not only predict learning outcomes significantly but also can be changed if learners put more effort to improve their academic performance. The results provide a rationale for the treatment for student time management effort.
Article
Full-text available
This article considers the developing field of learning analytics and argues that to move from small-scale practice to broad scale applicability, there is a need to establish a contextual framework that helps teachers interpret the information that analytics provides. The article presents learning design as a form of documentation of pedagogical intent that can provide the context for making sense of diverse sets of analytic data. We investigate one example of learning design to explore how broad categories of analytics—which we call checkpoint and process analytics—can inform the interpretation of outcomes from a learning design and facilitate pedagogical action.
Article
Full-text available
Growing interest in data and analytics in education, teaching, and learning raises the priority for increased, high-quality research into the models, methods, technologies, and impact of analytics. Two research communities -- Educational Data Mining (EDM) and Learning Analytics and Knowledge (LAK) have developed separately to address this need. This paper argues for increased and formal communication and collaboration between these communities in order to share research, methods, and tools for data mining and analysis in the service of developing both LAK and EDM fields.
Article
Full-text available
Theoretical and empirical evidence in the learning sciences substantiates the view that deep engagement in learning is a function of a complex combination of learners' identities, dispositions, values, attitudes and skills. When these are fragile, learners struggle to achieve their potential in conventional assessments, and critically, are not prepared for the novelty and complexity of the challenges they will meet in the workplace, and the many other spheres of life which require personal qualities such as resilience, critical thinking and collaboration skills. To date, the learning analytics research and development communities have not addressed how these complex concepts can be modelled and analysed, and how more traditional social science data analysis can support and be enhanced by learning analytics. We report progress in the design and implementation of learning analytics based on a research validated multidimensional construct termed "learning power". We describe, for the first time, a learning analytics infrastructure for gathering data at scale, managing stakeholder permissions, the range of analytics that it supports from real time summaries to exploratory research, and a particular visual analytic which has been shown to have demonstrable impact on learners. We conclude by summarising the ongoing research and development programme and identifying the challenges of integrating traditional social science research, with learning analytics and modelling.
Article
Full-text available
In this article we examine educational assessment in the 21st century. Digital learning environments emphasize learning in action. In such environments, assessments need to focus on performance in context rather than on tests of ed and isolated skills and knowledge. Digital learning environments also provide the potential to assess performance in context, because digital tools make it possible to record rich streams of data about learning in progress. But what assessment methods will use this data to measure mastery of complex problem solvingthe kind of thinking in action that takes place in digital learning environments? Here we argue that one way to address this challenge is through evidence-centered design1a framework for developing assessments by systematically linking models of understanding, observable actions, and evaluation rubrics to provide evidence of learning. We examine how evidence-centered design can address the challenge of assessment in new media learning environments by presenting one specific theory-based approach to digital learning, known as epistemic games (http://epistemicgames.org/eg/), and describing a method, epistemic network analysis (ENA), to assess learner performance based on this theory. We use the theory and its related assessment method to illustrate the concept of a digital learning systema system composed of a theory of learning and its accompanying method of assessment, linked into an evidence-based, digital intervention. We argue that whatever tools of learning and assessment digital environments use, they need to be integrated into a coherent digital learning system linking learning and assessment through evidence-centered design.
Article
Full-text available
Since the beginning of the century, feedback interventions (FIs) produced negative--but largely ignored--effects on performance. A meta-analysis (607 effect sizes; 23,663 observations) suggests that FIs improved performance on average ( d  = .41) but that over one-third of the FIs decreased performance. This finding cannot be explained by sampling error, feedback sign, or existing theories. The authors proposed a preliminary FI theory (FIT) and tested it with moderator analyses. The central assumption of FIT is that FIs change the locus of attention among 3 general and hierarchically organized levels of control: task learning, task motivation, and meta-tasks (including self-related) processes. The results suggest that FI effectiveness decreases as attention moves up the hierarchy closer to the self and away from the task. These findings are further moderated by task characteristics that are still poorly understood. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
Networked learning is much more ambitious than previous approaches of using technology in education. It is, therefore, more difficult to evaluate the effectiveness and efficiency of the networked learning activities. Evaluation of learners' interactions in networked learning environments is a resource and expertise demanding task. Educators participating in networked learning communities, have very little support by integrated tools to evaluate students' learning activities flow and identify learners' online browsing behaviour and interactions. As a consequence, educators are in need for non-intrusive and automatic ways to become informed about learners' progress in order to better follow their learning process and appraise the online course effectiveness. They also need specialized tools for gathering and analysing data for evaluating the learning effectiveness of networked learning instructional models. The aim of this paper is to present a conceptual framework and an innovative tool, called AnalyticsTool, which is based on this framework, that allow teacher and evaluator to easily track the learners' online behaviour, make judgements about learners' activity flow and have a better insight about the knowledge constructed and skills acquired in a networked learning environment. The innovation of the proposed tool is that interoperates with the Moodle learning management system and that it guides the educator perform the interaction analysis of collaborative learning scenarios that have been designed following specific learning strategies such as TPS, Jigsaw, Pyramid, etc.
Article
Full-text available
Enterprise wide learning management systems are integral to university learning and teaching environments. Griffith University and the University of Western Sydney (UWS) are predominantly face-to-face, multi-campus teaching institutions with similar size student bodies and academic communities. Both Griffith and UWS utilise a single enterprise wide e-learning system, although the systems are different. This paper describes a benchmarking activity between the two universities to determine the level and quality of the uptake of the e-learning system. A framework was developed as a product of the partnership and applied to a representative sample of e-learning sites. The results of the benchmarking exercise showed that there are parallel trends between the two institutions in how the LMS is being used, however with distinct differences in specific areas.
Article
Full-text available
We propose that the design and implementation of effective Social Learning Analytics presents significant challenges and opportunities for both research and enterprise, in three important respects. The first is the challenge of implementing analytics that have pedagogical and ethical integrity, in a context where power and control over data is now of primary importance. The second challenge is that the educational landscape is extraordinarily turbulent at present, in no small part due to technological drivers. Online social learning is emerging as a significant phenomenon for a variety of reasons, which we review, in order to motivate the concept of social learning, and ways of conceiving social learning environments as distinct from other social platforms. This sets the context for the third challenge, namely, to understand different types of Social Learning Analytic, each of which has specific technical and pedagogical challenges. We propose an initial taxonomy of five types. We conclude by considering potential futures for Social Learning Analytics, if the drivers and trends reviewed continue, and the prospect of solutions to some of the concerns that institution-centric learning analytics may provoke.
Article
Full-text available
This is a study of an online, Web based learning environment developed for an introductory business information systems course. The development of the environment was guided by a design principle that emphasizes choices regarding hypertextuality, centralization in client-server, interactivity, multimedia, and synchronicity. The environment included a several hundred page textbook and individual and group online learning aids and implements. Following the development period and a pilot class, the system was used by students in three large classes of 50 -100 students each. We examined student grades, attitudes, and usage logs and their intercorrelations. Our findings indicate that online, Web-based learning environments are not just feasible. Employing such a system to complement lectures yields measurable enhancements. We propose a focus on logged, machine-collected usage statistics. Such statistics allow a measurement of actual reading behavior and linearity in the learning process. Reading amount is highly correlated with student achievement. A-synchronous conferencing tools enhance instructor-student and student-to-student interaction. This interaction is, itself, a correlate of success in the course. Furthermore, more mature student groups seem to make better use of the online environment, and use it less linearly.
Conference Paper
Full-text available
The trend to greater adoption of online learning in higher education institutions means an increased opportunity for instructors and administrators to monitor student activity and interaction with the course content and peers. This paper demonstrates how the analysis of data captured from various IT systems could be used to inform decision making process for university management and administration. It does so by providing details of a large research project designed to identify the range of applications for LMS derived data for informing strategic decision makers and teaching staff. The visualisation of online student engagement/effort is shown to afford instructors with early opportunities for providing additional student learning assistance and intervention – when and where it is required. The capacity to establish early indicators of 'at-risk' students provides timely opportunities for instructors to re-direct or add resources to facilitate progression towards optimal patterns of learning behaviour. The project findings provide new insights into student learning that complement the existing array of evaluative methodologies, including formal evaluations of teaching. Thus the project provides a platform for further investigation into new suites of diagnostic tools that can, in turn, provide new opportunities to inform continuous, sustained improvement of pedagogical practice.
Article
Full-text available
Academic failure among first-year university students has long fuelled a large number of debates. Many educational psychologists have tried to understand and then explain it. Many statisticians have tried to foresee it. Our research aims to classify, as early in the academic year as possible, students into three groups: the 'low-risk' students, who have a high probability of succeeding, the 'medium-risk' students, who may succeed thanks to the measures taken by the university, and the 'high-risk' students, who have a high probability of failing (or dropping out). This article describes our methodology and provides the most significant variables correlated to academic success among all the questions asked to 533 first-year university students during the month of November of academic year 2003-04. Finally, it presents the results of the application of discriminant analysis, neural networks, random forests and decision trees aimed at predicting those students' academic success.
Article
Full-text available
The beneficial effects of learners interacting in online programmes have been widely reported. Indeed, online discussion is argued to promote student-centred learning. It is therefore reasonable to suggest that the benefits of online discussion should translate into improved student performance. The current study examined the frequency of online interaction of 122 undergraduates and compared this with their grades at the end of the year. The findings revealed that greater online interaction did not lead to significantly higher performance for students achieving passing grades; however, students who failed in their courses tended to interact less frequently. Other factors that may be salient in online interactions are discussed.
Article
The rapid uptake of campus-wide Learning Management Systems (LMS) is changing the character of the on-campus learning experience. The trend towards LMS as an adjunct to traditional learning modes has been the subject of little research beyond technical analyses of alternative software systems. Drawing on Australian experience, this paper presents a broad, critical examination of the potential impact of these online systems on teaching and learning in universities. It discusses in particular the possible effects of LMS on teaching practices, on student engagement, on the nature of academic work and on the control over academic knowledge.
Article
In this chapter, the reader is taken through a macro level view of learning management systems, with a particular emphasis on systems offered by commercial vendors. Included is a consideration of the growth of learning management systems during the past decade, the common features and tools contained within these systems, and a look at the advantages and disadvantages that learning management systems provide to institutions. In addition, the reader is presented with specific resources and options for evaluating, selecting and deploying learning management systems. A section highlighting the possible advantages and disadvantages of selecting a commercial versus an open source system is followed by a series of brief profiles of the leading vendors of commercial and open source learning management systems.
Article
Student engagement has become synonymous with the measurement of teaching and learning quality at universities. The almost global adoption of learning management systems as a technical solution to e-learning within universities and their ability to record and track user behaviour provides the academy with an unprecedented opportunity to harness captured data relating to student engagement. This is an exploratory study that aims to show how data from learning management systems can be used as an indicator of student engagement and how patterns in the data have changed with CQUniversity's recent adoption of Moodle as its single learning management system.
Article
This study sought to identify significant behavioral indicators of learning using learning management system (LMS) data regarding online course achievement. Because self-regulated learning is critical to success in online learning, measures reflecting self-regulated learning were included to examine the relationship between LMS data measures and course achievement. Data were collected from 530 college students who took an online course. The results demonstrated that students' regular study, late submissions of assignments, number of sessions (the frequency of course logins), and proof of reading the course information packets significantly predicted their course achievement. These findings verify the importance of self-regulated learning and reveal the advantages of using measures related to meaningful learning behaviors rather than simple frequency measures. Furthermore, the measures collected in the middle of the course significantly predicted course achievement, and the findings support the potential for early prediction using learning performance data. Several implications of these findings are discussed.
Article
This paper outlines how teachers can use the learning management system (LMS) to identify at risk students in the first week of a course. Data is from nine second year campus based business courses that use a blend of face-to-face and online learning strategies. Students that used the LMS in the first week of the course were more likely to pass. For the rest of the course the pattern of usage is then largely similar for students who pass and those that do not pass. This paper identifies how a LMS can identify at risk students in the first week of the course and provides some strategies to motivate these students. © 2012 John Milne, Lynn M Jeffrey, Gordon Suddaby & Andrew Higgins.
Article
The role of discussion forums is an essential part of online courses in tertiary education. Activities in discussion forums help learners to share and gain knowledge from each other.. In fully online courses, discussion forums are often the only medium of interaction. However, merely setting up discussion forums does not ensure that learners interact with each other actively and investigation into the type of participation is required to ensure quality participation. This paper provides a general overview of how fully online students participate in discussion forums and the correlation between their activity online and achievement in terms of grades. The main benefit of this research is that it provides a benchmark for the trend of participation expected of the fully online introductory information technology and programming students. Investigating the participation and the factors behind online behaviour can provide guidelines for continual development of online learning systems.. The results of the data analysis reveal that a high number of students are not accessing or posting in the discussion board. Results also show that there is a correlation between activity of students' in online forums and the grades they achieve. Discussion about the findings of data analysis and the lessons learned from this research are presented in this paper.
Article
This issue is a call for researchers and practitioners to reflect on progress to date and understand the criticality of theory – how it facilitates interpretation of findings but also how it can also restrict and confine our thinking through the assumptions many theoretical models bring. As education paradigms further shift and juxtapose informal and formal learning settings there is a need to re-visit any underlying theoretical assumptions.
Article
Blended learning (BL) is recognized as one of the major trends in higher education today. To identify how BL has been actually adopted, this study employed a data-driven approach instead of model-driven methods. Latent Class Analysis method as a clustering approach of educational data model-driven methods. Latent Class Analysis method as a clustering approach of educational data mining was employed to extract common activity features of 612 courses in a large private university located in South Korea by using online behavior data tracked from Learning Management System and institution's course database. Four unique subtypes were identified. Approximately 50% of the courses manifested inactive utilization of LMS or immature stage of blended learning implementation, which is labeled as Type I. Other subtypes included Type C - Communication or Collaboration (24.3%), Type D - Delivery or Discussion (18.0%), and Type S - Sharing or Submission (7.2%). We discussed the implications of BL based on data-driven decisions to provide strategic institutional initiatives.
Article
We propose that the design and implementation of effective Social Learning Analytics (SLA) present significant challenges and opportunities for both research and enterprise, in three important respects. The first is that the learning landscape is extraordinarily turbulent at present, in no small part due to technological drivers. Online social learning is emerging as a significant phenomenon for a variety of reasons, which we review, in order to motivate the concept of social learning. The second challenge is to identify different types of SLA and their associated technologies and uses. We discuss five categories of analytic in relation to online social learning; these analytics are either inherently social or can be socialised. This sets the scene for a third challenge, that of implementing analytics that have pedagogical and ethical integrity in a context where power and control over data are now of primary importance. We consider some of the concerns that learning analytics provoke, and suggest that Social Learning Analytics may provide ways forward. We conclude by revisiting the drivers andtrends, and consider future scenarios that we may see unfold as SLA tools and services mature. © International Forum of Educational Technology & Society (IFETS).
Article
This study aimed to develop a practical model for predicting students at risk of performing poorly in blended learning courses. Previous research suggests that analyzing usage data stored in the log files of modern Learning Management Systems (LMSs) would allow teachers to develop timely, evidence-based interventions to support at risk or struggling students. The analysis of students' tracking data from a Moodle LMS-supported blended learning course was the focus of this research in an effort to identify significant correlations between different online activities and course grade. Out of 29 LMS usage variables, 14 were found to be significant and were input in a stepwise multivariate regression which revealed that only four variables – Reading and posting messages, Content creation contribution, Quiz efforts and Number of files viewed – predicted 52% of the variance in the final student grade.
Article
In the past decades, online learning has transformed the educational landscape with the emergence of new ways to learn. This fact, together with recent changes in educational policy in Europe aiming to facilitate the incorporation of graduate students to the labor market, has provoked a shift on the delivery of instruction and on the role played by teachers and students, stressing the need for development of both basic and cross-curricular competencies. In parallel, the last years have witnessed the emergence of new educational disciplines that can take advantage of the information retrieved by technology-based online education in order to improve instruction, such as learning analytics.
Book
This unique and ground-breaking book is the result of 15 years research and synthesises over 800 meta-analyses on the influences on achievement in school-aged students. It builds a story about the power of teachers, feedback, and a model of learning and understanding. The research involves many millions of students and represents the largest ever evidence based research into what actually works in schools to improve learning. Areas covered include the influence of the student, home, school, curricula, teacher, and teaching strategies. A model of teaching and learning is developed based on the notion of visible teaching and visible learning. A major message is that what works best for students is similar to what works best for teachers - an attention to setting challenging learning intentions, being clear about what success means, and an attention to learning strategies for developing conceptual understanding about what teachers and students know and understand. Although the current evidence based fad has turned into a debate about test scores, this book is about using evidence to build and defend a model of teaching and learning. A major contribution is a fascinating benchmark/dashboard for comparing many innovations in teaching and schools.
Article
Learning analytics, the analysis and representation of data about learners in order to improve learning, is a new lens through which teachers can understand education. It is rooted in the dramatic increase in the quantity of data about learners and linked to management approaches that focus on quantitative metrics, which are sometimes antithetical to an educational sense of teaching. However, learning analytics offers new routes for teachers to understand their students and, hence, to make effective use of their limited resources. This paper explores these issues and describes a series of examples of learning analytics to illustrate the potential. It argues that teachers can and should engage with learning analytics as a way of influencing the metrics agenda towards richer conceptions of learning and to improve their teaching.
Article
An early warning system can help to identify at-risk students, or predict student learning performance by analyzing learning portfolios recorded in a learning management system (LMS). Although previous studies have shown the applicability of determining learner behaviors from an LMS, most investigated datasets are not assembled from online learning courses or from whole learning activities undertaken on courses that can be analyzed to evaluate students' academic achievement. Previous studies generally focus on the construction of predictors for learner performance evaluation after a course has ended, and neglect the practical value of an "early warning" system to predict at-risk students while a course is in progress. We collected the complete learning activities of an online undergraduate course and applied data-mining techniques to develop an early warning system. Our results showed that, time-dependent variables extracted from LMS are critical factors for online learning. After students have used an LMS for a period of time, our early warning system effectively characterizes their current learning performance. Data-mining techniques are useful in the construction of early warning systems; based on our experimental results, classification and regression tree (CART), supplemented by AdaBoost is the best classifier for the evaluation of learning performance investigated by this study.
Article
Considering the increasingly challenging resource environments in many business schools, this study examined whether course technologies, learner behaviors or instructor behaviors best predict online course outcomes so that administrators and support personnel can prioritize their efforts and investments. Based on reviewing prior online and blended management education literature, we hypothesized that instructor behaviors would be most predictive of online course outcomes. However, our study of 48 online Master of Business Administration courses found that although instructor behaviors (operationalized as teaching presence) was the strongest predictor of any of our three outcome variables (perceived learning), only student behaviors (operationalized as social presence) significantly predicted all three (course grades, perceived learning and delivery medium satisfaction). Technological characteristics, operationalized using variables from the technology acceptance model and media variety, predicted perceived learning (perceived ease of use) and delivery medium satisfaction (perceived usefulness). The paper provides recommendations for instructors and administrators based on the findings, specifically advocating for a balance between instructor and administrator involvement in course design, presentation and conduct. The paper concludes by describing research opportunities regarding the roles of social presence in and collaborative approaches to online learning in business schools.
Article
Feedback is one of the most powerful influences on learning and achievement, but this impact can be either positive or negative. Its power is frequently mentioned in articles about learning and teaching, but surprisingly few recent studies have systematically investigated its meaning. This article provides a conceptual analysis of feedback and reviews the evidence related to its impact on learning and achievement. This evidence shows that although feedback is among the major influences, the type of feedback and the way it is given can be differentially effective. A model of feedback is then proposed that identifies the particular properties and circumstances that make it effective, and some typically thorny issues are discussed, including the timing of feedback and the effects of positive and negative feedback. Finally, this analysis is used to suggest ways in which feedback can be used to enhance its effectiveness in classrooms.
Article
Learning analytics is the analysis of electronic learning data which allows teachers, course designers and administrators of virtual learning environments to search for unobserved patterns and underlying information in learning processes. The main aim of learning analytics is to improve learning outcomes and the overall learning process in electronic learning virtual classrooms and computer-supported education. The most basic unit of learning data in virtual learning environments for learning analytics is the interaction, but there is no consensus yet on which interactions are relevant for effective learning. Drawing upon extant literature, this research defines three system-independent classifications of interactions and evaluates the relation of their components with academic performance across two different learning modalities: virtual learning environment (VLE) supported face-to-face (F2F) and online learning. In order to do so, we performed an empirical study with data from six online and two VLE-supported F2F courses. Data extraction and analysis required the development of an ad hoc tool based on the proposed interaction classification. The main finding from this research is that, for each classification, there is a relation between some type of interactions and academic performance in online courses, whereas this relation is non-significant in the case of VLE-supported F2F courses. Implications for theory and practice are discussed next.
Article
crossfold performs k-fold cross-validation on a specified model in order to evaluate a model's ability to fit out-of-sample data. This procedure splits the data randomly into k partitions, then for each partition it fits the specified model using the other k-1 groups and uses the resulting parameters to predict the dependent variable in the unused group.
Article
This article reports a study which sought to evaluate the different learning strategies adopted by students when accessing virtual learning environment (VLE)-hosted resources because, if student achievement corresponds to the learning strategy that is adopted whilst accessing VLE resources, directed tasks can be put in place that will encourage students to adopt the learning strategy that is most closely associated with this higher achievement. The different approaches adopted by undergraduates (n = 46) in their use of online learning resources for a single second-year module are evaluated by analysing their record of accessing these resources at three time-points during the module, and their performance in coursework and exams. From these longitudinal data, three learning approaches are identified (two surface and one deep). It is found that students who adopted a deep learning approach, in which online resources were accessed consistently throughout the module, performed markedly higher than surface learners who focused their online activity at the beginning or end of the module’s duration. Implications for teaching strategies are discussed.
Article
In this paper we report ongoing research on the Open Academic Analytics Initiative (OAAI), a project aimed at increasing college student retention by performing early detection of academic risk using data mining methods. The paper describes the goals and objectives of the OAAI, and lays out a methodological framework to develop models that can be used to perform inferential queries on student performance using open source course management system data and student academic records. Preliminary results on initial model development using several data mining algorithms for classification are presented.
Article
Achievement behavior is defined as behavior directed at developing or demonstrating high rather than low ability. Ability can be conceived either with reference to the individual's own past performance or knowledge, a context in which gains in mastery indicate competence, or as capacity relative to that of others, a context in which a gain in mastery alone does not indicate high ability. To demonstrate high capacity, one must achieve more with equal effort or use less effort than do others for an equal performance. The conditions under which these different conceptions of ability function as individuals' goals and the nature of subjective experience in each case are specified. Different predictions of task choice and performance are derived and tested for each case using data from previously published studies. The effects of task and ego involvement, task choice, and self-perceptions are discussed. (125 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Student motivation is an important factor for the successful completion of an e-learning course. Detecting motivational problems for particular students at an early stage of a course opens the door for instructors to be able to provide additional motivating activities for these students. This paper analyzes how the behavior patterns in the interaction of each particular student with the contents and services in a learning management system (LMS) can be used to predict student motivation and if this student motivation can be used to predict the successful completion of an e-learning course. The interactions of 180 students of six different universities taking a course in three consecutive years are analyzed.
Article
Blended learning has become an increasingly popular form of e-learning, and is particularly suitable to the process of transitioning towards e-learning from traditional forms of learning and teaching. This paper describes the use of the blended e-learning model, which is based on a mixture of collaborative learning, problem-based learning (PBL) and independent learning, in a course ldquoTeaching Methods in Information Science,rdquo given at the University of Rijeka, Rijeka, Croatia. This model is realized as a combination of a face-to-face environment and online learning, using a proprietary learning management system (LMS) named adaptive hypermedia courseware (AHyCo). AHyCo is based on adaptive hypermedia and in addition to supporting learning and testing, introduces completely new constructivist and cognitivist elements to education. By supporting collaborative and project-oriented activities AHyCo promotes students' motivation for learning and establishes learning as an active and interactive process. This paper describes both the technology for, and the methodological approach to, course design and development which is aimed at supporting the evolution from traditional teaching to active learning, and raising interest in the topics of e-learning and Web courseware development among IT students. A survey conducted in the end of the course showed that students were satisfied with the pedagogical approach, and their academic achievements were also better than expected. Particularly important is that the dropout rate was greatly diminished, which could be related to students' satisfaction with the support they received from the instructor and the system.