Content uploaded by Masanori Yamada
Author content
All content in this area was uploaded by Masanori Yamada on Mar 21, 2020
Content may be subject to copyright.
Companion Proceedings 10th International Conference on Learning Analytics & Knowledge (LAK20)
Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0)
For Evidence-Based Class Design with Learning Analytics:
A Proposal of Preliminary Practice Flow Model in High School
Satomi Hamada1, Yufan Xu1, Xuewang Geng1, Li Chen1,
Hiroaki Ogata2, Atsushi Shimada3, Masanori Yamada1,4
Graduate School of Human-Environment Studies, Kyushu University1
Academic Center for Computing and Media Studies, Kyoto University2
Faculty of Information Science and Electrical Engineering, Kyushu University3
Faculty of Arts and Science, Kyushu University4
satomi.hamada@mark-lab.net, xuyufan@mark-lab.net, geng@mark-lab.net, chenli@mark-
lab.net, hiroaki.ogata@gmail.com, atsushi@ait.kyushu-u.ac.jp, mark@mark-lab.net
ABSTRACT: This paper introduces a practice to incorporate a learning analytics dashboard
that analyzes and visualizes learning logs using digital textbooks for high school students.
Based on the knowledge gained through the practice over the past six months, an important
model for incorporating learning analytics in high schools is proposed. In the future, we plan
to examine the model, including how learning logs can be used for each teacher's class.
Keywords: Learning Analytics, Study Logs, Digital Textbooks, Proposal Models, High School
1 INTRODUCTION
As a promising field, learning analytics (LA) is widely applied to understand and improve the learning
environment, which involves educational data collection and analysis. With the support of
technology, it enables the collection of educational data from various resources, for example,
learners’ profiles (such as their education level and motivation), curriculum profiles (such as
information of curriculum and learning materials), and learning logs collected from online systems or
learning material reader system that identifies online learning behaviors, such as time students
spent on each activity or the operations on reading learning materials (Ifenthaler and
Widanapathirana 2014; Yamada, 2017). To utilize LA approach to improve the learning environment,
Dunbar et al. (2014) indicated that curriculum designers play an important role in bridging data
collection and analysis with instructional design considering students’ learning conditions. The
primary/secondary education involves various stakeholders in addition to students and teachers,
and their decision-making needs to be included for effective and efficient LA-based practice. This
paper proposes a preliminary work flow model of the practical development of LA for decision-
making.
2 CASE STUDIES
This practical research project is a collaborative project that involves three university laboratories
(one lab: educational technology, other labs: information science), a public high school, and local
government. This practice was conducted on eighty 10th grade students. Each student is provided
with a tablet device, and mathematics (one class/day) and English (two classes/week) classes are
conducted. In mathematics, students are divided into three classes by proficiency level, and in
13
Companion Proceedings 10th International Conference on Learning Analytics & Knowledge (LAK20)
Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0)
Table 1: The characteristics of the four teachers
English classes, students are taught together regardless of proficiency.
2.1 The system used in this practice
There are three kinds of systems used. First, a learning management system, “Moodle”, is used. In
this practice, in addition to its role as a portal for the learning system, it is also used as a test plug-in
and an assignment submission plug-in. The second is the digital teaching material viewer “BookRoll”
(Ogata et al., 2017). “BookRoll” has the ability to mark the displayed digital teaching materials, make
notes (typing and handwriting formats), as well as a bookmark function, a search function, and a
recommendation information presentation function. The third is the “Dashboard system” (Flanagan
and Ogata, 2018). The dashboard displays a graph of the learning logs accumulated by textbooks and
uploaded to the “BookRoll”. Specific learning logs include the actions taken by the students on each
page (marker count, marker point, number of notes, and note content) and the time spent by them
when they were looking at a page. These systems were introduced at the start of this practice.
2.2 The teachers who cooperated in practice
The characteristics of the four teachers who cooperated in the practice in this study are given in
Table 1. All students read textbooks using “BookRoll” to write assignments outside the class and
clarify their understandings. All teachers instructed them to submit assignments on Moodle. Before
a class starts, each teacher views the dashboard, decides the class plan, and then conducts the class.
2.3 The educational design assistant
In this practice, educational design assistants are provided by the university to introduce three
systems and other assistance in the high school every weekday. There are five roles of the
educational design assistant: 1. To provide information about the three systems to teachers, 2. To
create system manuals, 3. To answer any question about system usage for teachers and students, 4.
To handle problems during classes when the systems are used; if any problem occurs, then they
need to take it back to the university and get in touch with the system developer, and 5. To provide
a suggestion regarding instructional design based on the dashboard and learning analytics data,
such as the data shown in Table 2. Referring to Table 1 and Table 2, it is suggested that the usage
Teacher A
Teacher B
Teacher C
Teacher D
Subject
Mathematics
Mathematics
Mathematics
English
Teaching
Experience
20 years
30 years
8 years
4 years
Teaching style
Mainly using textbooks,
Solving questions during
preparation and explaining them in
class,
There are few questions from
students,
The system is not used much
Mainly using textbooks,
An environment where students
can easily ask questions,
Teacher frequently comments on
system usage evaluation Mainly
Using my own prints,
An environment where it is
difficult for students to ask
questions,
The system is not used much
Mainly using textbooks,
A small test of English words,
Student learn in pairs,
Speaking in class is not permitted,
Regular note check.
System usage
Submit assignments using Moodle,
Review based on student markers
Use BookRoll in preparation,
Submit assignments using Moodle,
Use BookRoll for assignments,
Review based on student markers
Submit assignments using Moodle,
Review based on student markers,
Distribute question paper with
iPad’s airdrop
Use BookRoll in preparation,
Submit assignments using Moodle,
Use BookRoll for assignments,
Review based on student markers,
Conduct a small test with Moodle
14
Companion Proceedings 10th International Conference on Learning Analytics & Knowledge (LAK20)
Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0)
frequency and usage method of the system varies greatly depending on the teacher, and the view of
education also differs. After all classes are complete, educational assistants discuss the problems of
class with teachers and sometimes the head of the school, in order to improve class design and
intervention during class. In the future, educational assistants will mainly implement interventions
to develop teachers' ability based on analysis tools, such as “Dashboard”, rather than providing
support during class.
2.4 The workshop
In this practice, learning analytics workshops were held. Teachers, educational design assistants,
other stakeholders, such as the head of the school, and the municipal board of education attended
these workshops. In these workshops, issues and problems arising from the practice will be
considered and discussed together with the head of the high school and the municipal board of
education to design an effective curriculum and methods for instruction and how this can be used in
other schools. For example, participants, based on the data as displayed on the dashboard, spoke
about the educational practices that uses learning analytics and discussed effective seamless
learning design that bridges in- and out-of-class practices using the three systems. To make the
practice with learning analytics more effective, teachers suggested the educational technology and
information technology researchers to add new functions to these three systems and the municipal
board of education to improve the viewer and dashboard. The head of the school and municipal
board members also asked other stakeholders to proceed with the evidence-based instruction and
analyze the effects of these instruction using learning analytics. These workshops were conducted
once a month to implement an effective organizational learning analytics. This workshop seems to
contribute to enhancement of teachers’ motivation for the practice, and new instructional design
with LA platform were proposed by sharing the concerns of each school. This Workshop activities
beyond their region are encouraged.
3 PRELIMINARY FLOW MODEL FOR EVIDENCE-BASED CLASS DESIGN
Based on the practice so far, it will introduce an ideal model for implementing class designs using
learning analytics by introducing the system in a school (Figure 1). There are two important points in
this model. First, activities are performed at different granularities, and the research team including
educational design assistants manages activities at each granularity. Second, there is interaction
among teachers who are using the system. The practice is promoted by sharing ways to incorporate
Week 1
FLIP NEXT
FLIP PREV
ADD MK
ADD MEMO
ADD BM
DELETE MK
DELETE MEMO
DELETE BM
In class
M Class A
546
185
468
21
12
353
8
10
M Class B
975
354
244
19
9
123
2
8
M Class C
172
15
20
45
0
1
2
0
E Class 1
300
102
283
11
2
288
11
2
E Class 2
94
35
275
8
0
213
0
0
Out-of-
M Class A
575
96
480
8
1
137
2
1
class
M Class B
387
80
128
17
2
39
0
2
M Class C
749
143
276
0
1
93
0
1
E Class 1
652
189
644
18
3
371
12
3
E Class 2
397
80
509
5
2
300
0
1
Note: M Class: Mathematics Class, E Class: English Class, MK: MARKER, BM: Bookmark
Table 2: Sample learning logs for educational design
15
Companion Proceedings 10th International Conference on Learning Analytics & Knowledge (LAK20)
Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0)
the system into lessons and raising concerns regarding the system. This practice provides the
opportunity to share information with practice schools in other prefectures. Using these efforts, it
could be possible that each organization can cooperate and practice smoothly.
4 CONCLUSION AND FUTURE WORK
In this paper, a flow model for evidence-based class design in high schools was considered based on
actual practical experience. As future works, the project members are required to discuss the
important elements, data and events through workshop, discussion, observation, intervention, and
reflection with data through formative evaluation, in order to modify the proposed preliminary flow
model. The negative and positive impact and effects of the feedback on stakeholders’ awareness in
terms of education, project team, and policy should be investigated.
ACKNOWLEDGMENT
We would like to thank everyone at F high school for their cooperation in this research. This research
is supported by JST AIP Grant No. JPMJCR19U1, Japan.
REFERENCES
Flanagan, B. & Ogata, H. (2018). Learning analytics platform in higher education in Japan. Knowledge
Management & ELearning, 10(4), 469-484
Ifenthaler, D., & Widanapathirana, C. (2014). Development and validation of a Learning Analytics
framework: Two case studies using support vector machines. Technology, Knowledge and
Learning, 19(1–2), 221–240.
Ogata, H., Taniguchi, Y., Suehiro, D., Shimada, A., Oi, M., Okubo, F., Yamada, M., & Kojima, K. (2017).
M2B System: A Digital Learning Platform for Traditional Classrooms in University,
Proceedings of LAK2017. 155–162
Yamada, M., Shimada, A., Okubo, F., Oi, M., Kojima, K. & Ogata, H. (2017). Learning analytics of the
relationships among self-regulated learning, learning behaviors, and learning performance.
Research and Practice in Technology Enhanced Learning, 12, 13.
Figure 1: Preliminary implementation flow model for evidence-based class design in high school
16
