Content uploaded by Martin Ebner
Author content
All content in this area was uploaded by Martin Ebner on Jun 11, 2016
Content may be subject to copyright.
ISSN 2414-6099
294
What is Learning Analytics about? A Survey of Different Methods
Used in 2013-2015
Mohammad Khalil
Graz University of Technology,Graz, Austria
Martin Ebner
Graz University of Technology,Graz, Austria
Abstract
The area of Learning Analytics has developed enormously since the first International Conference
on Learning Analytics and Knowledge (LAK) in 2011. It is a field that combines different
disciplines such as computer science, statistics, psychology and pedagogy to achieve its intended
objectives. The main goals illustrate in creating convenient interventions on learning as well as its
environment and the final optimization about learning domain’s stakeholders (Khalil & Ebner,
2015b). Because the field matures and is now adapted in diverse educational settings, we believe
there is a pressing need to list its own research methods and specify its objectives and dilemmas.
This paper surveys publications from Learning Analytics and Knowledge conference from 2013 to
2015 and lists the significant research areas in this sphere. We consider the method profile and
classify them into seven different categories with a brief description on each. Furthermore, we show
the most cited method categories using Google scholar. Finally, the authors raise the challenges
and constraints that affect its ethical approach through the meta-analysis study.
It is believed that this paper will help researchers to identify the common methods used in Learning
Analytics, and it will assist by establishing a future forecast towards new research work taking into
account the privacy and ethical issues of this strongly emerged field.
Keywords: Learning Analytics, survey, publications, literacy, techniques
Introduction
Since the first Learning Analytics and Knowledge (LAK) conference in 2011 as well as the Horizon
Report in 2013 (Johnson et al., 2013), learning analytics is considered to be an emerging field that
would be applied in the different educational settings. This field provides tools and technologies
that offer the potentials to do proper interventions and improve education in general. The Society
for Learning Analytics and Research (SoLAR) defined it as “the measurement, collection, analysis
and reporting of data about learners and their contexts, for purposes of understanding and
optimizing learning and the environments in which it occurs”. Several studies exchanged views
about learning analytics goals. For instance, Khalil and Ebner (2015) introduced learning analytics
lifecycle and listed the main surveyed objectives of the past four years of the LAK conferences.
ISSN 2414-6099
295
They listed interventions, predictions, reflection, awareness, personalization, recommendation and
benchmarking as the main goals. These goals conformed to Siemen’s defined techniques back in
2012 (Siemens, 2012). In addition to that, different frameworks have been introduced to define both
key objectives and dilemmas of this field. In his paper “The Learning Analytics Cycle: Closing the
loop effectively”, Clow argued that a successful learning analytics should be winding up into
feeding back the product to learners in order to make effective intervention(s) (Clow, 2012). Whiles
Ferguson indexed remarkable challenges of ethics, distinct perspectives from stakeholders’ field of
vision and the methods to use in order to make these goals achievable (Ferguson, 2012).
For the time being, there is a large variety of educational environments such as MOOC-
platforms, LMS, virtual environments, etc. These educational information systems hold “Big Data”
of learners that create huge data repositories. According to learning analytics definition, the data
need to be analysed by typical methodologies in order to reflect benefits on learning and teaching.
The beginning of lively discussions on the differences between learning analytics and educational
data mining were mainly residing to the opposing opinions of using tools and methodologies in both
fields (Baker et al., 2012). Nevertheless, educational data mining and learning analytics are
enriched by the methods of data mining and analytics in general (Baker & Siemens, 2013). In its
first stages, researchers of learning analytics frameworks and structure were discussing methods
such as visualizations, data mining techniques (Elias, 2011), social network analysis (Ferguson,
2012), and sentiment analysis (Siemens, 2012), in addition to statistics which was also mentioned as
a required tool to build learning prediction models (Campbell, DeBlois & Oblinger, 2007).
SoLAR brought to success the annual organization of LAK conferences since 2011.
Accordingly, several categories of methods to analyse educational datasets were used. Most of these
methods tend to process data quantitatively and qualitatively to discover interesting hidden patterns.
Baker and Siemens (2013) mentioned that educational data is what drives new methods to be used
in learning analytics. They said: “The specific characteristics of educational data have resulted in
different methods playing a prominent role in EDM/LA than in data mining in general, or have
resulted in adaptations to existing psychometrics methods”. In this paper, we survey publications
from LAK conference from 2013 to 2015. The purpose is to list the most common methods used in
the field of learning analytics in the last three years. We believe this paper provides different
benefits because learning analytics becomes an important field by itself and is now completely
matured into being adapted in different educational institutions and applications. The main
advantages are:
1. It helps learning analytics researchers to identify common methods in use in order to reach
intended goals.
2. It determines methods that are highly cited, e.g. by Google scholar (http://scholar.google.com),
and establish a future forecast towards new research work.
3. Finally, it assists to compare the beginning view about learning analytics methods and the on-
going current version.
In addition, the paper aims to guide future researchers into further advances in this field and
meets the “Smart Learning Excellence” theme of Innovation Arabia 9 conference which is “The
next wave of innovations in Smart-Learning” that mainly considers Big Data and Learning
Analytics as a new wave in educational technology.
ISSN 2414-6099
296
We have organized this paper into the following sections. First, we list the methodology
employed to extract methods used in learning analytics publications. We then show statistical data
and describe methods in detail with remarks about their types. Finally, we discuss and summarize
the main conclusions and list the constraints and the ethical issues of learning analytics.
Research Design
As mentioned before, the conference of learning analytics and knowledge is considered to be the
first and the largest repository of learning analytics publications. We mainly focused on it, and
surveyed 91 papers from LAK 2013 (Suthers et al., 2013), LAK 2014 (Pistilli, Willis & Koch,
2014), and LAK 2015 (Baron, Lynch & Maziarz, 2015), with an additional supplementary literature
from other sources. We excluded papers with topics philosophy, frameworks and conceptual studies
of learning analytics for the reason that they address structures and do not accommodate a
mechanism for revealing patterns. We also faced papers with unclear methods, and these were
excluded too. At the end, 78 publications were set for examination. This study was influenced by
the work of Romero and Ventura (Romero & Ventura, 2010), and Dawson et al. (Dawson et al.,
2014). The classification of methods was based on reading the abstract, keywords, general terms,
methodology section and the conclusion of each paper. In some publications, we paid more analysis
into examining literature and the reference list. Furthermore, we collected the total number of
citations for each analysed paper from Google scholar and observed the trending topics.
Learning Analytics Methods
Learning analytics is a combination of different disciplines like computer science, statistics,
psychology, and education. As a result, we realized different analysis methods that do not only tend
to be too technical but rather pedagogical. Before classifying the analysis methods, we have been
gravitated towards the beginning topics of the emergence of learning analytics, which briefly
described methods and tools for collecting data and analyzing them (Ferguson, 2012; Siemens,
2012). However, the survey reveals more methods being used to examine learners’ data. Our main
methods categories, which will be explained in detail in section 3.1, are: (a) data mining techniques;
(b) statistics and mathematics; (c) text mining, semantics and linguistic analysis; (d) visualization;
(e) social network analysis; (f) qualitative analysis; and (g) gamification. Figure 1 shows grouping
of the methods used in learning analytics for LAK publications with the number of papers in each
category. It should be noted that some publications might be referenced in a different category.
Moreover, a paper could be referenced in multiple methods category.
The bar plot in the figure shows that researchers of 31 publications used data mining techniques
and 26 research studies used statistics and mathematics to analyse their data. This makes both of
these two methods as the top most employed techniques of analysis. We also see that “Text Mining,
Semantics and Linguistic” analyses as well as visualizations are being used in 13 LAK publications
equally. However, social network analysis and qualitative analysis as well as gamification were the
least used techniques.
ISSN 2414-6099
297
Categories
This section lists the methods categories in more detail and states relevant publications under
each method.
Data Mining Techniques: Data mining tends to make sense out of data. The definition of learning
analytics cited the similar idea, namely understanding the data, but in this case, about learners. The
survey shows that data mining techniques are the most used method for analyzing and interpreting
the learners’ log data. The decision tree algorithm was used to predict the performance drop and the
final outcome of students in a Virtual Learning Environment (VLE) (Wolff et al., 2013). Other
researchers used several classification techniques such as step regression, Naive Bayes and REP-
Trees to study students’ behavior and detect learners who game the system (Pardos et al., 2013).
While clustering was used to propose an approach for the purposes of enhancing educational
process mining based on the collected data from logs and detecting students at risk (Bogarín et al.,
2014). Discovering relations between two factors was observed by using multiple linear regression
analysis to forecast the relation between studying time and learning performance (Jo, Kim & Yoon,
2014). Moreover, data mining is used for assessment such as the work at the University of
Missouri-Columbia, which proposed an automated tool to enable teachers assess students in online
environments (Xing, Wadholm & Goggins, 2014). It was remarked that regression analysis was the
common mechanism among data mining techniques.
Statistics and Mathematics: Statistics is the science of measuring, controlling, communicating and
understanding the data (Davidian & Louis, 2012). Publications show that researchers have been
using descriptive statistics and mathematics, such as the mean, median and standard deviation to
signify their results. In addition, inferential statistics was used side by side with data mining in
Figure 1. Number of the examined LAK papers grouped by methods. Some papers share
more than single category
ISSN 2414-6099
298
some of the publications. Markov chain was used to study school students behavior in solving
multiplication (Taraghi et al., 2014). Different statistical techniques were operated to build a
grading system (Vogelsang & Ruppertz, 2015). Additionally, statistical discourse analysis with
Markov chain was employed to study online discussions and summarize demographics (Chiu &
Fujita, 2014), as well as examining student problem solving behavior and adapting it into tutoring
systems (Eagle et al., 2015).
Text mining, Semantics & Linguistic Analysis: Publications which refer to ontologies, mining
texts, discourse analysis, Natural Language Processing, or study of languages are set to be in this
category. Some studies refer to text analysis for assessment purposes of short answer questions
(Leeman-Munk, Wiebe & Lester, 2014), or to enhance collaborative writing between students
(Southavilay et al., 2013). Contextualizing user interactions based on ontologies to illustrate a
learning analytics approach (Renzel & Klamma, 2013). Linguistic analysis was clearly used in
parsing posts from students for prediction purposes (Joksimović et al., 2015). Finally, online
discussion forums were analysed to pioneer an automatic dialogue detection system in order to
develop a self-training approach (Ferguson et al., 2013).
Visualization: When the information is visually presented to the field experts, efficient human
capabilities rise to perceive and process the data (Kapler & Wright, 2015). Visual representations
take the advantages into expanding human decisions within a large amount of information at once
(Romero & Ventura, 2010). There are several studies that cited visualization as a method to analyse
the data and deliver information to end users, such as: building a student explorer screen to prepare
meetings and identifying at-risk students by the teachers (Aguilar, Lonn & Teasley, 2014). Studying
MOOC’s attrition rate and learners’ activities (Santos et al., 2014); Building an awareness tool for
teachers and learners (Martinez-Maldonado et al., 2015), and a dashboard for self-reflection goals
(Santos et al., 2013). Information can be interpreted into heat maps, scatterplots, diagrams, and
flowcharts which were observed in most of the statistical, mathematical and data mining based
publications.
Social Network Analysis: Abbreviated as SNA. It focuses on relationships between entities. In
learning analytics, SNA can be used to promote collaborative learning and investigate connections
between learners, teachers and resources (Ferguson, 2012). Moreover, it can be employed in
learning environments to examine relationships of strong or weak ties (Khalil & Ebner, 2015). This
category includes network analysis in general and Social Learning Analytics (SLA). The survey
observed researchers who: built a collaborative learning environment by visualizing relationships
between students about the same topic (Schreurs et al., 2013). A two-mode network was used to
study students’ patterns and to classify them into particular groups (Hecking, Ziebarth & Hoppe,
2014). It was also used with a grading system in a PLE to examine the centrality of students and
grades (Koulocheri & Xenos, 2013). Again, not so far from this survey study, a network analysis
was done to analyse citations of LAK conference papers (Dawson et al., 2014). The authors studied
the degree centrality and pointed out the emergence and isolated disciplines in learning analytics.
SNA was used to analyse data of connectivist MOOCs by examining interactions of learners from
social media websites (Joksimović et al., 2015).
ISSN 2414-6099
299
Qualitative Analysis: This category is related to the decisions based on explained descriptions of
the analysts. For instance: i) a qualitative evaluation with data mining techniques was made to
understand the nature of discussion forums of MOOCs (Ezen-Can et al., 2015); ii) Usage of
qualitative interviews, which are answered by words to build a learning analytics module of
understanding fractions for school children (Mendiburo, Sulcer & Hasselbring, 2014); iii)
Qualitative meta-analysis to investigate teachers’ needs in technology enhanced learning covered by
the umbrella of learning analytics (Dyckhoff et al., 2013).
Gamification: It is the use of game mechanics and tools to make learning and instruction attractive
and fun (Kapp, 2012). This method is considered as a technique on its own because of its relevant
appearance in educational workshops and the requests to make learning entertaining. Some
examples are using rewards points and progress bar to enhance the retention rate and building a
gamified grading system (Holman, Aguilar & Fishman, 2013), or presenting a competency map
with progress bars, pie charts, labels and hints to improve students’ performance (Grann &
Bushway, 2014). A significant study on monitoring students in a 3D immersive environment was
also advised as another type of gamification techniques (Camilleri et al., 2013).
Prominent Methods and Discussion
In this section, we consider learning analytics methods that have been frequently cited. We used the
Google scholar as a foundation to check methods’ popularity. All the data were collected recently
and retrieved before the submission date. Figure 2 shows Google scholar citations for the analysed
LAK conference papers based on the methods category. The publications with the method type
Data Mining and Techniques, were the most cited articles (452 citations). The ultimate number of
citations in this survey belongs to the paper of (Kizilcec, Piech & Schneider, 2013) with 236
citations. Although we took into consideration the time span of publications, we see that articles
that belong to MOOCs are the most cited papers. Statistics and Mathematics publications were cited
363 times. Qualitative analysis and gamification publications were the least cited articles. In figure
3, we show a density plot of publications’ citations grouped by year. The x-axis records number of
citations converted into logarithmic scale to ease the reading. The y-axis records the density of
publications per year. Since we did not survey a fair number of publications per year, we intended
to use this plot instead of histogram plot, which is highly sensitive to bin size.
Figure 2. Number of Google scholar citations of the examined LAK papers based on
methods category. Retrieved on 26
th
October, 2015.
ISSN 2414-6099
300
Figure 3. Density plot of citations grouped by year
ISSN 2414-6099
301
Some of 2013 publications attracted numerous citations which exceed the expectations such as
(Kizilcec, Piech & Schneider, 2013; Pardos et al., 2013). A descriptive analysis of the articles in
2013 leads to: median=8, mean=24.37, max=236; articles in 2014: median=5, mean=5.56, max=17;
and articles of 2015: median =1, mean=1.42, max=4. The low number of citations for 2015 is
reasonable as the time span between this survey and 2015 LAK publication is around six months.
Challenges & Constraints
The data collection and analysis through learning analytics methods lead to questions related to
ownership, privacy and ethical issues (Khalil & Ebner 2015; Khalil & Ebner, 2016). We summarize
our experience and previous research studies as the following: A) Privacy, in which the learning
analytics specialists need to carefully deliberate the potential privacy issues while collecting,
analyzing, and intervene of the students’ data. B) Transparency of disclosing information of
learners and the needs to proclaim consent by the students. C) Assuring security and achieving the
CIA which is an acronym that refers to Confidentiality, Integrity and Accessibility such as storing
of learners records. D) The ownership of the collected and analysed data.
Conclusion
Learning Analytics is a promising area which provides the adequate tools and methods to optimize
the learning mechanism in the different environments of educational technology platforms. In this
paper, we did a meta-analysis study on publications in the last two years and classified seven
different categories of techniques that have been used in that period. We noticed that learning
analytics researchers adopt data mining and statistics more often than other techniques.
Additionally, 2013 was a stimulating year in showing MOOCs as a desirable article by a distinct
number of citations. Moreover, we also see that some publications have had a high impact on
education with their peak Google scholar score. In fact, the upcoming learning analytics events
might show extinction of methods and an uprising appearance or emergence of new techniques,
which can be allocated in our defined categories. Finally, we summarized our experience in this
field and listed some of the constraints and dilemmas that negatively affect learning analytics
approaches.
References
Aguilar, S., Lonn, S., & Teasley, S. D., 2014. Perceptions and use of an early warning system during a
higher education transition program. In Proceedings of the fourth international conference on learning
analytics and knowledge (pp. 113-117). ACM.
Baker, R. S. J. D., Duval, E., Stamper, J., Wiley, D., & Buckingham Shum, S., 2012. Panel: educational data
mining meets learning analytics. In Proceedings of 2nd International Conference on Learning Analytics &
Knowledge (LAK'12), New York, USA (p. 20).
Baker, R. S., & Siemens, G., 2013. Educational Data Mining and Learning Analytics.
Baron, J., Lynch, G., Maziarz, N., 2015. Proceedings of the Fifth International Conference on Learning
Analytics and Knowledge. ACM, New York, USA.
Bogarín, A., Romero, C., Cerezo, R., & Sánchez-Santillán, M., 2014. Clustering for improving educational
process mining. In Proceedings of the Fourth International Conference on Learning Analytics and
Knowledge (pp. 11-15). ACM.
ISSN 2414-6099
302
Camilleri, V., de Freitas, S., Montebello, M., & McDonagh-Smith, P., 2013. A case study inside virtual
worlds: use of analytics for immersive spaces. In Proceedings of the Third International Conference on
Learning Analytics and Knowledge (pp. 230-234). ACM.
Campbell, J. P., DeBlois, P. B., & Oblinger, D. G., 2007. Academic analytics: A new tool for a new
era. EDUCAUSE review, 42(4), 40.
Chiu, M. M., & Fujita, N., 2014. Statistical discourse analysis of online discussions: informal cognition,
social metacognition and knowledge creation. In Proceedings of the Fourth International Conference on
Learning Analytics And Knowledge (pp. 217-225). ACM.
Clow, D., 2012. The learning analytics cycle: closing the loop effectively. In Proceedings of the 2nd
international conference on learning analytics and knowledge (pp. 134-138). ACM.
Davidian, M., & Louis, T. A., 2012. Why statistics?. Science, 336(6077), 12-12.
Dawson, S., Gašević, D., Siemens, G., & Joksimovic, S., 2014. Current state and future trends: A citation
network analysis of the learning analytics field. In Proceedings of the Fourth International Conference on
Learning Analytics and Knowledge (pp. 231-240). ACM.
Dyckhoff, A. L., Lukarov, V., Muslim, A., Chatti, M. A., & Schroeder, U., 2013. Supporting action research
with learning analytics. In Proceedings of the Third International Conference on Learning Analytics and
Knowledge (pp. 220-229). ACM.
Eagle, M., Hicks, D., Peddycord III, B., & Barnes, T., 2015. Exploring networks of problem-solving
interactions. In Proceedings of the Fifth International Conference on Learning Analytics And
Knowledge (pp. 21-30). ACM.
Elias, T., 2011. Learning Analytics: Definitions, Processes and Potential. Available from:
http://learninganalytics.net/LearningAnalyticsDefinitionsProcessesPotential.pdf. [25 October 2015].
Ezen-Can, A., Boyer, K. E., Kellogg, S., & Booth, S., 2015. Unsupervised modeling for understanding
MOOC discussion forums: a learning analytics approach. In Proceedings of the Fifth International
Conference on Learning Analytics And Knowledge (pp. 146-150). ACM.
Ferguson, R., 2012. Learning analytics: drivers, developments and challenges. In International Journal of
Technology Enhanced Learning, 4(5-6), 304-317.
Ferguson, R., Wei, Z., He, Y., & Buckingham Shum, S., 2013. An evaluation of learning analytics to identify
exploratory dialogue in online discussions. In Proceedings of the Third International Conference on
Learning Analytics and Knowledge (pp. 85-93). ACM.
Grann, J., & Bushway, D., 2014. Competency map: visualizing student learning to promote student success.
In Proceedings of the fourth international conference on learning analytics and knowledge (pp. 168-172).
ACM.
Hecking, T., Ziebarth, S., & Hoppe, H. U., 2014. Analysis of dynamic resource access patterns in a blended
learning course. In Proceedings of the Fourth International Conference on Learning Analytics and
Knowledge. ACM.
ISSN 2414-6099
303
Holman, C., Aguilar, S., & Fishman, B., 2013. GradeCraft: what can we learn from a game-inspired learning
management system?. In Proceedings of the Third International Conference on Learning Analytics and
Knowledge (pp. 260-264). ACM.
Jo, I. H., Kim, D., & Yoon, M., 2014. Analyzing the log patterns of adult learners in LMS using learning
analytics. In Proceedings of the Fourth International Conference on Learning Analytics And Knowledge (pp.
183-187). ACM.
Johnson, L., Adams Becker, S., Cummins, M., Estrada, V., Freeman, A., & Ludgate, H., 2013. NMC
Horizon Report: 2013 Higher Education Edition. Texas: The New Media.
Joksimović, S., Dowell, N., Skrypnyk, O., Kovanović, V., Gašević, D., Dawson, S., & Graesser, A. C., 2015.
How do you connect?: analysis of social capital accumulation in connectivist MOOCs. In Proceedings of the
Fifth International Conference on Learning Analytics And Knowledge (pp. 64-68). ACM.
Kapler, T., & Wright, W., 2005. GeoTime information visualization. Information Visualization, 4(2), 136-
146.
Kapp, K. M., 2012. The gamification of learning and instruction: game-based methods and strategies for
training and education. John Wiley & Sons.
Khalil, M., & Ebner, M., 2015. Learning Analytics: Principles and Constraints. In Proceedings of World
Conference on Educational Multimedia, Hypermedia and Telecommunications (pp. 1326-1336).
Khalil, M., & Ebner, M., 2016. De-Identification in Learning Analytics. Journal of Learning Analytics.
Accepted, in Print.
Kizilcec, R. F., Piech, C., & Schneider, E., 2013. Deconstructing disengagement: analyzing learner
subpopulations in massive open online courses. In Proceedings of the third international conference on
learning analytics and knowledge (pp. 170-179). ACM.
Koulocheri, E., & Xenos, M., 2013. Considering formal assessment in learning analytics within a PLE: the
HOU2LEARN case. In Proceedings of the Third International Conference on Learning Analytics and
Knowledge (pp. 28-32). ACM.
Leeman-Munk, S. P., Wiebe, E. N., & Lester, J. C., 2014. Assessing elementary students' science
competency with text analytics. In Proceedings of the Fourth International Conference on Learning
Analytics And Knowledge(pp. 143-147). ACM.
Martinez-Maldonado, R., Pardo, A., Mirriahi, N., Yacef, K., Kay, J., & Clayphan, A., 2015. The LATUX
workflow: designing and deploying awareness tools in technology-enabled learning settings. In Proceedings
of the Fifth International Conference on Learning Analytics And Knowledge (pp. 1-10). ACM.
Mendiburo, M., Sulcer, B., & Hasselbring, T., 2014. Interaction design for improved analytics.
In Proceedings of the Fourth International Conference on Learning Analytics And Knowledge (pp. 78-82).
ACM.
Pardos, Z. A., Baker, R. S., San Pedro, M. O., Gowda, S. M., & Gowda, S. M., 2013. Affective states and
state tests: Investigating how affect throughout the school year predicts end of year learning outcomes. In
Proceedings of the Third International Conference on Learning Analytics and Knowledge (pp. 117-124).
ACM.
ISSN 2414-6099
304
Pistilli, M., Willis, J., Koch, D., 2014. Proceedings of the Fourth International Conference on Learning
Analytics And Knowledge. ACM, New York, USA.
Renzel, D., & Klamma, R., 2013. From micro to macro: Analyzing activity in the ROLE sandbox.
In Proceedings of the Third International Conference on Learning Analytics and Knowledge (pp. 250-254).
ACM.
Romero, C., & Ventura, S., 2010. Educational data mining: a review of the state of the art. Systems, Man,
and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on, 40(6), 601-618.
Santos, J. L., Klerkx, J., Duval, E., Gago, D., & Rodríguez, L., 2014. Success, activity and drop-outs in
MOOCs an exploratory study on the UNED COMA courses. In Proceedings of the Fourth International
Conference on Learning Analytics and Knowledge (pp. 98-102). ACM.
Santos, J. L., Verbert, K., Govaerts, S., & Duval, E., 2013. Addressing learner issues with StepUp!: an
Evaluation. In Proceedings of the Third International Conference on Learning Analytics and Knowledge (pp.
14-22). ACM.
Schreurs, B., Teplovs, C., Ferguson, R., De Laat, M., & Buckingham Shum, S., 2013. Visualizing social
learning ties by type and topic: Rationale and concept demonstrator. In Proceedings of the Third
International Conference on Learning Analytics and Knowledge (pp. 33-37). ACM.
Siemens, G., 2012. Learning analytics: envisioning a research discipline and a domain of practice.
In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (pp. 4-8). ACM.
Southavilay, V., Yacef, K., Reimann, P., & Calvo, R. A., 2013. Analysis of collaborative writing processes
using revision maps and probabilistic topic models. In Proceedings of the Third International Conference on
Learning Analytics and Knowledge (pp. 38-47). ACM.
Suthers, D., Verbert, K., Duval, E., & Ochoa, X., 2013. Proceedings of the Third International Conference
on Learning Analytics and Knowledge. ACM, New York, USA.
Taraghi, B., Ebner, M., Saranti, A., & Schön, M., 2014, March. On using markov chain to evidence the
learning structures and difficulty levels of one digit multiplication. In Proceedings of the Fourth
International Conference on Learning Analytics And Knowledge (pp. 68-72). ACM.
Vogelsang, T., & Ruppertz, L., 2015. On the validity of peer grading and a cloud teaching assistant system.
In Proceedings of the Fifth International Conference on Learning Analytics And Knowledge (pp. 41-50).
Wolff, A., Zdrahal, Z., Nikolov, A., & Pantucek, M., 2013. Improving retention: predicting at-risk students
by analysing clicking behaviour in a virtual learning environment. In proceedings of the third international
conference on learning analytics and knowledge (pp. 145-149). ACM.
Xing, W., Wadholm, B., & Goggins, S., 2014. Learning analytics in CSCL with a focus on assessment: an
exploratory study of activity theory-informed cluster analysis. In Proceeding of LAK’14. ACM.
