Figure 1 - uploaded by Garron Hillaire
Content may be subject to copyright.
Source publication
Prototype work can support the creation of data visualizations throughout the
research and development process through paper prototypes with sketching, designed
prototypes with graphic design tools, and functional prototypes to explore how the
implementation will work. One challenging aspect of data visualization work is coordinating the
expertise...
Similar publications
The increasing progress in ubiquitous technology makes it easier and cheaper to track students' physical actions unobtrusively, making it possible to consider such data for supporting research, educator interventions, and provision of feedback to students. In this paper, we reflect on the underexplored, yet important area of learning analytics appl...
Citations
... Our aim here is to shed light on how we went about turning a concept designed from and for manual analysis into automated coding, including the difficulties and achievements met along the way. Third, we discuss a prototype visualization of results offered by the resulting automated analysis (see Hillaire et al., 2016). ...
Education is offering evermore possibilities for generating learning analytics. However, to ensure that patterns revealed by analysing large-scale data are meaningful for education and to determine their educational implications requires educational theories. This chapter draws on a framework already having significant impact in education: Legitimation Code Theory (LCT). Like other educational theories, LCT is limited by its reliance on time-intensive manual analysis. Unlike many other theories, LCT concepts have clear empirical referents, lending themselves to automation. In this chapter we describe a pilot study to automate theoretical coding, using a concept from LCT that explores changes in the complexity of knowledge being expressed in, for example, classroom discourse and student assessments. First, we introduce LCT and why its concepts lend themselves to automation. Second, we outline how, through manual analyses and machine learning, we iteratively trained and tested an algorithm to support research using a specific concept. Third, we discuss a prototype visualization of results offered by the resulting automated analysis. Our aim is to show educational scholars how automated support for their analyses is within reach and to illustrate to learning analytics scholars how proven educational theories may offer a powerful resource to create meaningful and actionable insights.
... Although crucial to consider explicitly issues of inclusion or exclusion, we agree also with Hillaire et al., (2016) that implementation of learning analytics applications for disadvantaged students should be done in an inclusive manner to "challenge, motivate, support, and educate not only students with learning disabilities, but their peers (and teachers) too" (Hillaire et al., 2016, p.119). Our systematic review on what is known of learning analytics with regard to broader aspects of inclusiveness and disability highlights that much remains to be done. ...
This article maps considerations of inclusiveness and support for students with disabilities by reviewing articles within the field of learning analytics. The study involved a PRISMA-informed systematic review of two popular digital libraries, namely Clarivate’s Web of Science, and Elsevier’s Scopus for peer-reviewed journal articles and conference proceedings. A final corpus of 26 articles was analysed. Findings show that although the field of learning analytics emerged in 2011, none of the studies identified here covered topics of inclusiveness in education before the year of 2016. Screening also shows that learning analytics provides great potential to promote inclusiveness in terms of reducing discrimination, increasing retention among disadvantaged students, and validating particular learning designs for marginalised groups. Gaps in this potential are also identified. The article aims to provide valuable insight into what is known about learning analytics and inclusiveness and contribute knowledge to this particular nascent area for researchers and institutional stakeholders.
... In recent years, visual learning analytics has become an important technical field to support the online learning process [8]. Visual learning analytics emphasizes the guiding and regulating role of learning theory in data exploration, and analysis rather than guiding and learning based on simple personal experience [9]. Visual learning analytics can gain insight into students' learning process, help students understand learning progress, and promote self-reflection [10,11]. ...
Online Judge (OJ) is an important aid for programming learning that can help students evaluate learning effects in real-time, while teachers can adjust practice tasks in time according to the records of the tool. With these advantages, OJ shows great value for promoting teaching and learning in programming. The existing OJ system usually only provides information such as problem status list and recent rank list. However, it is unable to provide teachers with more fine-grained analysis information, such as the distribution of students’ incorrect responses and level of knowledge mastery. And it also cannot provide students with effective comparative information on their learning status. This research developed a visual learning analytics dashboard named VisOJ for the OJ system, which includes two types of user interfaces: teacher and student. The teacher interface presents students' learning status and ranking trends, which help teachers monitor and give feedback on their learning activities. The student interface provides views such as error type analysis and evaluation, which promote students' self-reflection and self-regulation. Preliminary case studies and expert interviews prove the usability of the dashboard. In the end, we summarize our main work and suggest future research directions.
... Consequently, these four elements form the basic structure of our review. We also made the adaptation of changing knowledge to goal to better incorporate the goal-relevant considerations of visual analysis approaches in a learning context, such as target users, target problems, and theoretical background (e.g., Hillaire et al., 2016;Vieira et al., 2018). Theoretical considerations largely inform and justify the knowledge that users desire to gain from visual analytics approaches. ...
... The development of learning tools must be guided by specific learning theories (Hillaire et al., 2016;Shaffer & Ruis, 2017;Wise & Schaffer, 2015). Therefore, we further examined whether the goal of VRCD involved any theoretical considerations. ...
... Innovative approaches have started to achieve this goal. For example, Hillaire et al. (2016) proposed a six-step model to guide the development of visual learning analytics tools. In this model, the first step is to define an educational goal informed by educational theories, and subsequent steps involve the definition of the target users, an interdisciplinary paper prototyping process, a formative evaluation, mock data, and implementation. ...
Visual analytics combines automated data analysis and human intelligence through visualisation techniques to address the complexity of current real-world problems. This review uses the lens of visual analytics to examine four dimensions of visual representations of collaborative discourse: goals, data sources, visualisation designs, and analytical techniques. We found visual analysis approaches to be suitable and advantageous for decomposing the temporality of collaborative discourse. However, it has been challenging for current research to simultaneously consider learning theories and follow visualisation design principles when adopting visualisations to analyse collaborative discourse. At the same time, existing visual analysis approaches have mainly targeted learners or researchers and mainly focused on mirroring collaborative discourse rather than providing advanced affordances such as alerting or advising. Informed by these findings, we propose a possible future research agenda and offer suggestions for the features of successful collaboration to guide the design of advanced affordances.
... The digital literacy platform used in the current study was built on the principles of UDL, with a particular focus on providing multiple ways for students to engage with text. Called Udio, the intervention was developed through a process of design-based research and targeted to middleschool students identified by their schools as in need of additional support in reading comprehension (see Hillaire, Rappolt-Schlichtmann, & Ducharme, 2016). The resulting online environment builds on prior approaches to improving reading comprehension in this population by emphasizing, as the Edmonds et al. (2009) meta-analysis describes, a focus on engaging students, "in thinking about text, learning from text, and discussing what they know" (p. ...
We use the affordances of a supplemental digital literacy platform to study the dynamics of behavioral engagement and reading comprehension among middle-school students in remedial reading classes. All participating students (n = 315) were identified by their schools as needing additional reading support; 56% received special education services. They used the digital literacy platform designed using the Universal Design for Learning (UDL) framework for approximately 1 hr per week for an academic year. Embedded indicators of activity in the digital platform captured whether and how students engaged with available text-related activities. Higher levels of behavioral engagement were not associated with improved reading comprehension in the sample overall. An interaction effect indicated that students who started the year with lower reading comprehension skills were likely to benefit more from higher reading-related behaviors and more use of text-to-speech than those starting with relatively higher comprehension skills.
... With the trend of shifting beneficiary, more types of objectives will be produced because different stakeholders have different expectation on what LA can bring to them (Kwok, 2015). For example, teachers may want to improve the learning design (Hillaire et al., 2016), system designers may want to improve the learning platform (Dowell et al., 2016), parents may want to monitor their children's learning status and senior managements of institutions may want to understand the way to reduce turnover rate and drop-out rate and (Kwok, 2015). ...
Learning analytics is a new research stream developed from other
previous related streams such as academic analytics and educational data mining. We conducted a literature review on recent publications related to data analytics in education in various journals and conferences. With the reference model proposed by Chatti et al (2012), we proposed a refined model by introducing the second layer elements to two out of the four dimensions and propose changes to the list of options of some dimensions. We then analysed publications of the initial three years (2014–2016) in the Journal of Learning Analytics based on the refined model. We discovered five trends in learning analytics applications including extending the data landscape to non-academic data and shifting of objectives and analysis methods. Finally, we proposed four
future research directions in this area and rationalised the relationship between dimensions in the model.
... When designing data visualizations for learning analytics research or practice, other practical guidelines in the literature would be worth being synthesized with the present paper in guiding practice. See Klerx, Verbet, and Duval (2017) for practical guidelines on how to get started on developing data visualizations for this purpose ;and Hillaire, Rappolt-Schlichtmann, and Ducharme (2016) for prototyping guidelines. ...
Understanding human judgement and decision making during visual inspection of data is of both practical and theoretical interest. While visualizing data is a commonly employed mechanism to support complex cognitive processes such as inference, judgement, and decision making, the process of supporting and scaffolding cognition through effective design is less well understood. Applying insights from cognitive psychology and visualization science, this paper critically discusses the role of human factors — visual attention, perception, judgement, and decision making — toward informing methodological choices when visualizing data. The value of visualizing data is discussed in two key domains: 1) visualizing data as a means of communication; and 2) visualizing data as research methodology. The first applies cognitive science principles and research evidence to inform data visualization design for communication. The second applies data- and cognitive-science to deepen our understanding of data, of its uncertainty, and of analysis when making inferences. The evidence for human capacity limitations — attention and cognition — are discussed in the context of data visualizations to support inference-making in both domains, and are followed by recommendations. Finally, how learning analytics can further research on understanding the role data visualizations can play in supporting complex cognition is proposed.
... For this paper, we were interested in identifying how visualization tools presented in the literature informed their designs using educational theories-e.g., theories regarding how people learn or which pedagogical practices are most effective. Effective visual learning analytics systems need to be informed by pedagogical practices and educational theories so that instructors and students can use them to enrich the learning process (Dawson, 2010;Hillaire, Rappolt-Schlichtmann, & Ducharme, 2016;Lockyer, Heathcote, & Dawson, 2013;Wang & Jacobson, 2011). Hence, the research team designed an assessment rubric aimed at characterizing the reviewed literature based on three dimensions (see Table 1): (1) Connection with Visualization Background (CVB); (2) Connection with Educational Theory (CET); and (3) Sophistication of Visualization (SoV). ...
... Promote Reflection: When the visualizations were designed for students, these were often intended to promote student reflection. Seven studies (Beheshitha, Hatala, Gašević, & Joksimović, 2016;Bull et al., 2016;Ruiz et al., 2016;Hillaire et al., 2016;Hsiao et al., 2016;Nagy, 2016) focused on using the visual learning analytics tool for this purpose. For instance, Nagy (2016) showed students a visualization of their effort in several dimensions, and asked them to reflect on the amount of effort they had exerted during a period of time. ...
We present a systematic literature review of the emerging field of visual learning analytics. We review existing work in this field from two perspectives: First, we analyze existing approaches, audiences, purposes, contexts, and data sources—both individually and in relation to one another—that designers and researchers have used to visualize educational data. Second, we examine how established literature in the fields of information visualization and education has been used to inform the design of visual learning analytics tools and to discuss research findings. We characterize the reviewed literature based on three dimensions: (a) connection with visualization background; (b) connection with educational theory; and (c) sophistication of visualization(s). The results from this systematic review suggest that: (1) little work has been done to bring visual learning analytics tools into classroom settings; (2) few studies consider background information from the students, such as demographics or prior performance; (3) traditional statistical visualization techniques, such as bar plots and scatter plots, are still the most commonly used in learning analytics contexts, while more advanced or novel techniques are rarely used; (4) while some studies employ sophisticated visualizations, and some engage deeply with educational theories, there is a lack of studies that both employ sophisticated visualizations and engage deeply with educational theories. Finally, we present a brief research agenda for the field of visual learning analytics based on the findings of our literature review.
... Visual LA (Hillaire, Rappolt-Schlichtmann, & Ducharme, 2016) Supports pedagogical decisions by interactive visualizations that claim information design to acquire, parse, filter, mine, depict, and interact with a data collection. ...
Before the emergence of computer‐based educational systems (CBES) whose aims of providing teaching and learning experiences to hundreds even thousands of users, an explosion of information (e.g., students' log data) demands sophisticated methods to gather, analyze, and interpret learners' traces to regulate and enhance education. Thus, learning analytics (LA) arises as a knowledge discovery paradigm that provides valuable findings and facilitates stakeholders to understand the learning process and its implications. Therefore, a landscape of the LA nature, its underlying factors, and applications achieved is outlined in this paper according to a suggested LA Taxonomy that classifies the LA duty from a functional perspective. The aim is to provide an idea of the LA toil, its research lines, and trends to inspire the development of novel approaches for improving teaching and learning practices. Furthermore, the scope of this review covers recently published papers in prestigious journals and conferences, where the works dated from 2016 are summarized and those corresponding to 2014–2015 are cited according to the proposed LA taxonomy. A glimpse is sketched of LA, where underlying elements frame the field foundations to ground the approaches. Moreover, LA strengths, weaknesses, challenges, and risks are highlighted to advice how the LA arena could be enhanced and empowered. In addition, this review offers an insight of the recent LA labor, as well as motivates readers to enrich the LA achievements. This work promotes the LA practice giving an account of the job being achieved and reported in literature, as well as a reflection of the state‐of‐the‐art and an acumens vision to inspire future labor.
This article is categorized under: Application Areas > Education and Learning
Application Areas > Science and Technology
Fundamental Concepts of Data and Knowledge > Human Centricity and User Interaction
... An obstacle to adoption of our method by a broader audience of instructors, without researcher support, is the rather careful process by which the parameters of the visualization were initially tuned. Since this was a collaborative research endeavour, guided by the educational theory informed goals of the instructor (Hillaire, Rappolt-Schlichtmann, & Ducharme, 2016), it was justified for this laborious effort to be undertaken by a member of the instructional staff; however, this should not be a pre-requisite, in practice, for a typical instructor to take on before interfacing with these analytics. The technique needs to be applied to more courses and more settings that vary by the type of students, course material, and platform to ascertain if common useful settings emerge in general contexts which would reduce or eliminate the upfront manual tuning effort. ...
We introduce a novel approach to visualizing temporal clickstream behaviour in the context of a degree-satisfying online course, Habitable Worlds, offered through Arizona State University. The current practice for visualizing behaviour within a digital learning environment has been to generate plots based on hand engineered or coded features using domain knowledge. While this approach has been effective in relating behaviour to known phenomena, features crafted from domain knowledge are not likely well suited to make unfamiliar phenomena salient and thus can preclude discovery. We introduce a methodology for organically surfacing behavioural regularities from clickstream data, conducting an expert in-the-loop hyperparameter search, and identifying anticipated as well as newly discovered patterns of behaviour. While these visualization techniques have been used before in the broader machine learning community to better understand neural networks and relationships between word vectors, we apply them to online behavioural learner data and go a step further; exploring the impact of the parameters of the model on producing tangible, non-trivial observations of behaviour that are suggestive of pedagogical improvement to the course designers and instructors. The methodology introduced in this paper led to an improved understanding of passing and non-passing student behaviour in the course and is widely applicable to other datasets of clickstream activity where investigators and stakeholders wish to organically surface principal patterns of behaviour.
