Conference PaperPDF Available

Development of a Learning Dashboard Prototype Supporting Meta-cognition for Students

  • March 2020
  • Conference: Companion Proceedings 10th International Conference on Learning Analytics & Knowledge (LAK20), pp.104-106
  • At: Frankfurt (online), Germany
Authors:
Min Lu at Akita University
Min Lu
Li Chen at Kyushu University
Li Chen
Yoshiko Goda at Kumamoto University
Yoshiko Goda
Atsushi Shimada
Atsushi Shimada
Atsushi Shimada
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Show all 5 authorsHide
Download full-text PDF
Read full-text
Download citation

Abstract and Figures

This poster presents an initial prototype design and development of a learning analytics dashboard supporting self-regulated learning, from which the students can benefit from LA more directly. The current stage of the development focuses on providing visualizations of learning processes and behaviors extracted from operation log data of an e-book system for self-monitoring. An overview of the reading paths and time of the slide pages in a class and a detailed view of the activities and learner-created content on the selected page are provided with a comparison of the class overall states and those of the learner. This work is expected to invoke the future developments and practical experiments of an LA dashboard supporting different phases of self-regulated learning.
Figures - uploaded by Masanori Yamada
Author content
All figure content in this area was uploaded by Masanori Yamada
Content may be subject to copyright.
Content uploaded by Masanori Yamada
Author content
All content in this area was uploaded by Masanori Yamada on Mar 21, 2020
Content may be subject to copyright.
Companion Proceedings 10th International Conference on Learning Analytics & Knowledge (LAK20)
Creative Commons License, Attribution - NonC ommercial-NoDer ivs 3.0 Unported (CC BY-NC-ND 3.0)
Development of a Learning Dashboard Prototype Supporting
Meta-cognition for Students
Min Lu1*, Li Chen1, Yoshiko Goda2, Atsushi Shimada1, Masanori Yamada1**
1 Kyushu University, Japan; 2 Kumamoto University, Japan
* lu@artsci.kyushu-u.ac.jp, ** mark@mark-lab.net
ABSTRACT: This poster presents an initial prototype design and development of a learning
analytics dashboard supporting self-regulated learning, from which the students can benefit
from LA more directly. The current stage of the development focuses on providing
visualizations of learning processes and behaviors extracted from operation log data of an e-
book system for self-monitoring. An overview of the reading paths and time of the slide
pages in a class and a detailed view of the activities and learner-created content on the
selected page are provided with a comparison of the class overall states and those of the
learner. This work is expected to invoke the future developments and practical experiments
of an LA dashboard supporting different phases of self-regulated learning.
Keywords: self-monitoring, dashboard, learning analytics, visualization
1 INTRODUCTION
Learning analytics (LA) with the large-scale educational log data obtained from e-learning
environments can benefit both the instructors and learners with different kinds of feedback.
Although researches of LA dashboards have become popular in recent years; however, as the most
visible results are designed for the instructors, the learners cannot benefit from LA in a direct
manner. On the other hand, monitoring the learner's own learning behaviors and processes is an
important aspect of self-regulated learning because it helps learners to be aware of their
weaknesses or deficiencies in their learning processes and regulates their learning strategies (Hofer
et al., 1998). Visualization is useful for learners to be aware of what they have been doing and what
they should do by making such information salient for them (Yen et al., 2018). Our prior research has
designed a learning analytics dashboard supporting metacognition to improve self-regulated
learning in online environments through the collection, analysis, and visualization of learning log
data (Chen et al., 2019). This poster presents an initial prototype of the dashboard, focusing on
supporting self-monitoring, which invokes future experiments, evaluations, and developments.
2 UI DESIGN OF THE LA DASHBOARD FOR STUDENTS
For self-monitoring, learners are expected to focus on the learning processes rather than the
outcome only (Zimmerman, 1998). Thus, the dashboard intends to provide the students with the
processes and behaviors visualized from learning log data in two types of views (Chen et al., 2019).
The first one is an overview of the learning activities on all the slide pages in a class. The second one
is the learning behaviors on a single page. As the point of self-evaluation is a comparison of one’s
own performance or behaviors based on certain criteria or standards (Belfiore & Hornyak, 1998),
both views provide comparisons between the class's overall situations and the user’s behaviors.
104
Companion Proceedings 10th International Conference on Learning Analytics & Knowledge (LAK20)
Creative Commons License, Attribution - NonC ommercial-NoDer ivs 3.0 Unported (CC BY-NC-ND 3.0)
Figure 1: Overview of reading path and time on all the slide pages in a class
Figure 2: Detailed view of the activities and learner-created content on the selected page
2.1 Overview of All the Slide Pages in a Class
A graph to visualize the slide reading path and time with the nodes on a circle stand for pages and
the links between the nodes stand for the reading path (Figure 1). The intensity of a node’s color
indicates the reading time spent on the page, and the thickness of a link shows the number of the
page transit. The accessories, which are smaller circles attached to a page node, present the
recorded learning behaviors, including highlight markers, memo annotations, on the page. The
intensity of an accessory’s color indicates the number of corresponded behaviors.
2.2 Detailed View of Each Slide Page
When the learner clicks a page node in the overview, the details of the reading time and learning
behaviors will be displayed with the learner-created content overlapped on the slide page (Figure 2).
The left view shows the average reading time of the class and all the highlight markers and memo
annotations created on the page, while the right view shows those of the learner.
3 PROTOTYPE DEVELOPMENT
We mainly developed a data processing module and a web-based visualization module to realize the
above design, mainly with the operation event logs from the e-book system of our university.
105
Companion Proceedings 10th International Conference on Learning Analytics & Knowledge (LAK20)
Creative Commons License, Attribution - NonC ommercial-NoDer ivs 3.0 Unported (CC BY-NC-ND 3.0)
3.1 Data Processing Module
As the accumulated operation event logs are from different courses and students, this module at
first filter the records according to slides and students’ IDs, and then extract the sequences of page-
transit related events by time after a data cleansing of events with too short intervals (<0.5s). From
such sequences, the time spent on each page and the numbers of the “from-to” links between each
pair of pages can be calculated. The overall states of the class can be obtained by summarizing the
result from all the students. The results are stored in JSON files for the visualization module.
3.2 Web-based Visualization Module
This module is developed based on D3.js, and the visual elements (e.g., the nodes and links) are
implemented as Scalable Vector Graphics (SVG), thus can be interactive to the user’s mouse
operation. This module also provides programming interfaces for setting up parameters of the
visualization, such as the size of the visual elements, colors of the nodes, accessories, links, and
intensity scales, and so on, for future developments.
4 PRELIMINARY EVALUATION AND FUTURE WORK
In the interviews with graduate students and teachers in our university, we obtained useful
comments and suggestions. For example, the links should present the direction of page transits
more clearly; the number displayed should be more meaningful to students; hyperlinks from the
graph to other useful plugins of the LMS and e-book can be helpful; and so on. With the refined
prototype, we plan to conduct formative experiments in the next step to clarify its effectiveness. In
the future, the functions of the LA dashboard to support other phases of self-regulated learning,
including knowledge monitoring, planning, and regulation, will be studied and developed.
ACKNOWLEDGMENT
This research is supported by a JST AIP Grant No. JPMJCR19U1, Japan.
REFERENCES
Chen, L., Lu, M., Goda, Y., & Yamada, M. (2019). Design of learning analytics dashboard supporting
metacognition. Proceedings of 16th International Conference Cognition and Exploratory
Learning in Digital Age (CELDA 2019), 175-182.
Belfiore, P. J., & Hornyak, R. S. (1998). Operant theory and application to self-monitoring in
adolescents. In: D. H. Schunk & B. J. Zimmerman (Eds.), Self-regulated learning: From
teaching to self-reflective practice (pp. 184-202). New York: Guilford.
Hofer, B. K., Yu, S. L., & Pintrich, P. R. (1998). Teaching college students to be self-regulated learners.
In: D. H. Schunk & B. J. Zimmerman (Eds.), Self-regulated learning: From teaching to self-
reflective practice (pp. 57-85). New York: Guilford.
Zimmerman, B. J. (1998). Developing self-fulfilling cycles of academic regulation: An analysis of
exemplary instructional models. In: D. H. Schunk & B. J. Zimmerman (Eds.), Self-regulated
learning: From teaching to self-reflective practice (pp. 1-19). New York: Guilford.
Yen, M. H., Chen, S., Wang, C. Y., Chen, H. L., Hsu, Y. S., & Liu, T. C. (2018). A Framework for Self-
Regulated Digital Learning (SRDL). Journal of Computer Assisted Learning, 34(5), 580-589.
106
... The systems used here include e-book viewers (Teasley 2017) and learning management systems (LMS). Moreover, a personalized learning analytics dashboard (LAD) for students was developed by integrating data from a student information system into a basic algorithm , Chen et al. 2019, Lu et al. 2020. The LAD can visualize information about the acquisition and use of learning skills. ...
ESTIMATING LEARNING ASSISTANCE SKILLS USING LEARNING ANALYTICS
Conference Paper
Full-text available
  • Oct 2021
Learning assistance is an essential part of higher education. Tutors, the core of the assistance staff, need to have learning assistance skills. If the potential for these skills can be identified when selecting tutors, the training method will be more efficient. Also, learning assistance skills are thought to be related to learning skills. Therefore, in this paper, we used previous research to identify the relationship between learning skills and learning assistance skills to clarify this. Then, using learning analytics, we explored whether learning support skills can be determined from learning skills.
... BR refers to electronic textbooks, which are allowed to be used to record the use of digital lecture materials such as slides and notes (Ogata et al., , 2017a(Ogata et al., , 2017b. MB is a learning analytics dashboard that aims to provide the visualization of processes and behaviors based on the learning log data of the BR system (Chen et al., 2019bLu et al., 2020). Data from the BR and MB systems can be obtained and visualized on the MAI Helper and is displayed in Figure 1. ...
Learning Analytics in the Context of COVID-19: A Case Study of Using Network Analysis to Guide Campus Course Offering Plans.
Conference Paper
Full-text available
  • Apr 2021
Network analysis simulations were used to guide decision-makers while configuring instructional spaces on our campus during COVID-19. Course enrollment data were utilized to estimate metrics of student-to-student contact under various instruction mode scenarios. Campus administrators developed recommendations based on these metrics; examples of learning analytics implementation are provided.
... BR refers to electronic textbooks, which are allowed to be used to record the use of digital lecture materials such as slides and notes (Ogata et al., , 2017a. MB is a learning analytics dashboard that aims to provide the visualization of processes and behaviors based on the learning log data of the BR system (Chen et al., 2019bLu et al., 2020). Data from the BR and MB systems can be obtained and visualized on the MAI Helper and is displayed in Figure 1. ...
... The systems used include an eBook viewer [23] and a learning management system (LMS). In addition, personalized student-facing LADs were developed by integrating data from the student information system into an underlying algorithm [24], [25], [26]. Thus, the utilization of the LAD provides the visualization of information regarding the acquisition and use of learning skills. ...
Toward Identification of Learning Skills from Learning Analytics Approach: A Research Review
Conference Paper
Full-text available
  • Dec 2020
This study aims to identify the types of learning logs that can measure learning skills. By using the learning analytics approach, the learning process can be analyzed. Identifying learning skills from the behavioral level can be linked to students' learning behaviors and how they support their own learning. However, the skills identified from those logs were not particularly clear. Therefore, we reviewed previous studies from the databases of educational papers "ERIC" and "Google Scholar", linked the behaviors with the skills and identified the logs that can objectively reflect the behaviors. In this study, the learning skills that could be analyzed through learning logs were presented.
MAI Helper: Learning Support System for Time Management Skill Acquisition Using Learning Analytics
Chapter
  • Jan 2022
Learning skills include abilities, habits, understanding, and attitudes that are utilized to achieve learning. Students will not achieve good grades unless they manage their limited study time properly. However, it is not easy for them to organize their own study time and use it efficiently. Previous research on time management has not examined the effect of learning analytics. Learning analytics provides students with accurate data that they can use to manage their study time. Therefore, in this study, we designed and developed a system based on a learning analytics approach with functions that allow students to make their learning schedules and reflect on their learning behaviors. The system has visualization, calendars, and reflection functions, which can be used by both students and their instructors. After developing these systems, we conducted a formative evaluation of the students. For the students, we imposed tasks and applied the system usability scale (SUS), one of the measures used universally. As a result, a certain effect is observed.KeywordsLearning analyticsLearning skillsLearning analytics dashboardsTime management skillsFormative evaluationUsability scale
Exploring jump back behavior patterns and reasons in e-book system
Article
Full-text available
  • Jan 2022
With the increasing use of digital learning materials in higher education, the accumulated operational log data provide a unique opportunity to analyzing student learning behaviors and their effects on student learning performance to understand how students learn with e-books. Among the students’ reading behaviors interacting with e-book systems, we find that jump-back is a frequent and informative behavior type. In this paper, we aim to understand the student’s intention for a jump-back using user learning log data on the e-book materials of a course in our university. We at first formally define the “jump-back” behaviors that can be detected from the click event stream of slide reading and then systematically study the behaviors from different perspectives on the e-book event stream data. Finally, by sampling 22 learning materials, we identify six reading activity patterns that can explain jump backs. Our analysis provides an approach to enriching the understanding of e-book learning behaviors and informs design implications for e-book systems.
Learning Analytics Dashboard Supporting Metacognition
Chapter
  • Feb 2021
Mastering self-regulated learning (SRL) can help learners achieve their academic goals. In this paper, we discuss the designs and development of a learning analytics dashboard (LAD) we created to support learners’ metacognition, the Metaboard. We tested the Metaboard with four student participants, then surveyed their perceptions of the LAD and its usefulness as a formative evaluation in the later design cycle. They reported that the Metaboard was a useful tool for learning digital learning materials, proposed additions and revisions, and identified the four most important elements of the LAD: (1) appropriate graphic representations; (2) comparison functions; (3) monitoring functions that relate to goal attainment; and (4) consistent feedback.
DESIGN OF THE TIME MANAGEMENT SKILLS ACQUISITION SYSTEM USING LEARNING ANALYTICS
Conference Paper
Full-text available
  • Nov 2020
Learning skills include abilities, habits, understanding, and attitudes which are utilized to achieve learning. Students will not achieve good grades unless they properly manage their limited study time. However, it is not easy for them to organize their own study time and learn how to use it efficiently. Notably, learning analytics has not been used in previous research for time management. Learning analytics provides students with accurate data that they can use to manage their study time. Therefore, in this study, we designed a system based on a learning analytics approach with functions that allow students to make their own learning schedules and reflect on their learning behaviors. The system has visualization, calendars, and reflection functions, which are used by both students and their instructors.
FACTORS OF THE USE OF LEARNING ANALYTICS DASHBOARD THAT AFFECT METACOGNITION
Conference Paper
Full-text available
  • Nov 2020
In this study, we used a learning analytics dashboard (LAD) in a higher education course to support students' metacognition and evaluated the effects of its use. The LAD displays students' reading path and specific behaviors when viewing digital learning materials. The study was conducted on 53 university students to identify the factors that affected metacognition changes in terms of their awareness and behavior dimensions when using the LAD. In terms of results, first, the students' perception of visual attraction for the LAD, and behaviors related to reflection such as deleting annotations they had previously added, positively affected the changes in the knowledge of cognition dimension of metacognition. Second, students' perception of behavioral changes by using the LAD had positive effects on the regulation of cognition dimension of metacognition. However, the behaviors of using some cognitive tools, negatively affected knowledge of cognition, which indicated the necessity to provide more guidance or feedback to students.
Visualizing Studying Activities for a Learning Dashboard Supporting Meta-cognition for Students
Chapter
  • Jul 2020
The existing researches and developments of dashboard visualizing results from learning analytics mainly serve the instructors instead of learners in a direct manner. Effective visualizations extracted from learning log data can help the students to reflect and compare studying activities and access their metacognition to improve their self-regulated learning. For such purposes, we designed a reading path graph for visualizing the studying activities on slide pages used as teaching materials in classes intuitively, as one of the key functions of the learning dashboard. By providing the comparisons between the user’s own situation and the class overview, the visualization is expected to motivate the further actions of using other tools of the learning dashboard and reflecting studies. This paper introduces our exploration of the data process flows of extracting necessary data from a large number of operational logs for the visualization, and the techniques and strategies applied for rendering the graphics effectively. We implemented the data processing module with Python3 and the web-based visualization module of the reading path graph with JavaScript based on D3.js considering the extensibilities. The issues engaged in the development of prototypes are discussed, which will lead to the improvement of future prototypes and better designs of user experiments for formative evaluations as the next step of this research.
DESIGN OF LEARNING ANALYTICS DASHBOARD SUPPORTING METACOGNITION
Conference Paper
Full-text available
  • Nov 2019
Metacognition is an aspect in self-regulated learning and is necessary to achieve such learning in an effective and efficient manner. However, it is not always easy and accurate for learners to monitor or assess their own metacognition. In this study, we designed a learning analytics dashboard to improve self-regulated learning in online environments through the collection and analysis of learning log data. There are two separate dashboards used in our system: a knowledge monitoring dashboard and a strategy use dashboard. The knowledge monitoring dashboard is designed to support the knowledge monitoring skills of learners, allowing them to monitor their prior knowledge, whereas the strategy use dashboard is designed to help learners develop metacognitive skills of planning, monitoring, and regulation.
Developing self-fulfilling cycles of academic regulation: An analysis of exemplary instructional models.
Article
Full-text available
Academic self-regulation is not a mental ability, such as intelligence, or an academic skill, such as reading proficiency; rather, it is the self-directive process through which learners transform their mental abilities into academic skills. What is this process, and how can students become more self-regulatory? In this chapter, the author presents a cyclical phase analysis of self-regulation, identifies key self-regulatory processes used during each phase, and compares how skillful self-regulators differ from their classmates in the use of these processes. He also describes the social, environmental, and personal conditions from which self-regulation emerges and analyzes exemplary instructional models that have been used to develop students' self-regulatory skill. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
A framework for self-regulated digital learning (SRDL)
Article
  • May 2018
This article develops a framework for self‐regulated digital learning, which supports for self‐regulated learning (SRL) in e‐learning systems. The framework emphasizes 8 features: learning plan, records/e‐portfolio and sharing, evaluation, human feedback, machine feedback, visualization of goals/procedures/concepts, scaffolding, and agents. Each feature facilitates or supports one or more SRL skills, including planning, monitoring and evaluating learning, applying appropriate cognitive strategies, and setting standards of products or performance. The implementation in domain‐general and ‐specific systems as illustrated by web‐based inquiry and problem‐solving are discussed. Examples and learning effects are elicited from the literature to demonstrate various designs. Approaches for designing SRL systems, educational implications, and new directions for future research incorporating SRL into digital learning are presented. Lay Description What is already known about this topic: • An increasing amount of information and learning materials is delivered in a digital format, which requires self‐regulation for effective learning. • Individual efforts have been made to facilitate self‐regulated learning in domain‐general/specific learning systems. What this paper adds: • A synthesized framework is developed to support self‐regulation in e‐learning. Implications for practice and/or policy: • Online learning via MOOCS or open courses would be enhanced by integrating the SRDL framework into the systems.
Self-regulated learning: From teaching to self-reflective practice
  • Jan 1998
  • 184-202
  • P J Belfiore
  • R S Hornyak
Belfiore, P. J., & Hornyak, R. S. (1998). Operant theory and application to self-monitoring in adolescents. In: D. H. Schunk & B. J. Zimmerman (Eds.), Self-regulated learning: From teaching to self-reflective practice (pp. 184-202). New York: Guilford.