Learning Analytics of the Relationships among Knowledge Constructions, Self-regulated Learning, and Learning Performance

Abstract

The concept map has a positive effect on the enhancement of self-regulated learning (SRL) and learning performance in terms of cognitive learning tools, according to previous research. However, the relationships between knowledge construction state, learning behaviors, psychological state, and learning performance have not been clearly investigated. Learning analytics (LA) can play an important role in addressing the issue of collecting learning behaviors. This study aims to investigate the relationships between them, using the LA approach. The results indicated that seven knowledge construction types were detected, and knowledge construction type had significant differences in performance, albeit no significant differences in the Tukey post-hoc analyses. Moreover, there is a significant correlation between knowledge map cluster and discussion, some of the factors of SRL (e.g., declarative knowledge, monitoring), and some learning behaviors, such as adding marker, memo, and red marker.

Full text

978-1-6654-3687-8/21/$31.00 ©2021 IEEE
Learning Analytics of the Relationships among
Knowledge Constructions, Self-regulated
Learning, and Learning Performance
Hao Hao Xuewang Geng Li Chen
Graduate School of Graduate School of Graduate School of
Human-Environment Studies Human-Environment Studies Human-Environment Studies
Kyushu University Kyushu University Kyushu University
Fukuoka, Japan Fukuoka, Japan Fukuoka, Japan
kakukou@mark-lab.net geng@mark-lab.net chenli@mark-lab.net
Atsushi Shimada Masanori Yamada
Faculty of Information Science and Faculty of Arts and Science
Electrical Engineering Kyushu University
Kyushu University Fukuoka, Japan
Fukuoka, Japan mark@mark-lab.net
atsushi@limu.ait.kyushu-u.ac.jp
Abstract— The concept map has a positive effect on the
enhancement of self-regulated learning (SRL) and learning
performance in terms of cognitive learning tools, according to
previous research. However, the relationships between
knowledge construction state, learning behaviors, psychological
state, and learning performance have not been clearly
investigated. Learning analytics (LA) can play an important role
in addressing the issue of collecting learning behaviors. This
study aims to investigate the relationships between them, using
the LA approach. The results indicated that seven knowledge
construction types were detected, and knowledge construction
type had significant differences in performance, albeit no
significant differences in the Tukey post-hoc analyses. Moreover,
there is a significant correlation between knowledge map cluster
and discussion, some of the factors of SRL (e.g., declarative
knowledge, monitoring), and some learning behaviors, such as
adding marker, memo, and red marker.
Keywords— concept map; self-regulated learning; learning
analytics
I. INTRODUCTION
A concept map is a tool that uses lines to connect the
learned knowledge and visualize the knowledge structure, and
it is widely used in classrooms [1]. As a cognitive learning tool,
the concept map is considered effective in many aspects.
Reference [2] has stated that when students use concept maps
to summarize knowledge, their creative and learning ability
can be improved. Using concept maps to summarize
knowledge is more helpful for students to recall central ideas
and details than using text [3]. Some studies indicate that
students can improve on their self-regulated learning (SRL)
and metacognition by making concept maps [4][5]. Learners
with high SRL can be aware of their weaknesses and actively
adjust their learning strategies. However, students are not
innate self-regulated learners, and they need teachers' support
to exercise this ability. For example, in the classroom, teachers
need to ask challenging questions and provide timely support
and feedback to help students improve their SRL skills [6].
Teachers need to realize students' learning behaviors and
habits before they can better support students to improve their
SRL. Although research on the concept map and SRL is
ongoing, there have not been sufficient studies on what type
of concept map and learning behaviors can enhance SRL
awareness and improve learning outcomes.
According [2]-[5], students can use the digital concept
map to sort out the knowledge they have learned. In this
process, students can review the knowledge they have learned
and then adjust their learning strategies. Therefore, not only
students' understanding of knowledge but also their SRL and
metacognition can be improved through this process. The
digital concept map is a useful tool for elucidating the
relationships between knowledge constructions, SRL, and
learning performance. Digital concept maps, which can be
used as a cognitive SRL tool, can present thematic, relational,
and informational relationships [7]. As digital concept maps
can store user logs, it can be used for learning analytics (LA)
to analyze dates. It is also possible to objectively grasp the
relationship between SRL and learning behaviors, as well as
knowledge constructions.
As we mentioned above, concept maps have been widely
used in classrooms. Students can summarize the knowledge
they have learned by making concept maps to improve their
self-regulation ability. Then enhancing students’ SRL
awareness can improve learning outcomes [8][9]. As
mentioned in [4] and [5], creating concept maps can help
improve students' SRL. Therefore, teachers can integrate the
concept map into their instructional design to improve
students’ SRL. However, teachers also need to provide
support for the learning process of creating a concept map [6].
In order to allow teachers to better support students’ SRL, this
study collected and analyzed the 7 weeks learning log in
BookRoll-Map (BR-Map), which is a concept map system
based on logs of the e-Book viewer BookRoll [4], to
investigate the relationship between different operations,
metacognition, and SRL to present suggestions for promoting
students' SRL in the future.

II. THEORETICAL BACKGROUND
A. Self-regulated Learning (SRL)
Most of the research on SRL have been based on the
theory of self-regulation proposed by Bandura in the 1980s
[10]. Reference [10] pointed out that self-regulation is a
process of influencing the external environment through
people's emotions and behaviors. Reference [11] concluded
that self-regulation can be divided into two parts: monitor and
control. Self-regulation can be described as the monitoring
and adjustment of one's emotions and behaviors.
According to researchers, the SRL concept is defined as
“the process whereby students activate and sustain cognitions
and behaviors systematically oriented toward the attainment
of their learning goals” (p. 465) [12]. Supporting the
development of students' metacognition and SRL can promote
independent learning [13]. Students who have high SRL levels
know their strengths and weaknesses and are able to motivate
themselves to participate and improve their learning.
Moreover, some studies have pointed out that there is a
positive relationship between supporting students' SRL and
students' performance [14][15].
Information and communications technology (ICT) is a
useful tool for teachers to support students to improve SRL.
The use of ICT can facilitate students' decision-making vis-à-
vis where and what time they learn [9]. Additionally, some
recent studies have begun to focus on analyzing the logs of
students' use of ICT to explore the relationship between SRL,
learning behavior, and learning performance. Reference [16]
developed the "gStudy" system with LA, which laid an early
foundation for the study of LA in the field of SRL. Reference
[17] proposed 13 SRL elements in computer-based learning,
including help-seeking, time and effort spent in planning, and
so on. Reference [18] proved that the SRL elements in the ICT
learning environment are helpful in predicting students'
performance.
Reference [19] proposed that the use of cognitive learning
strategies in the ICT environment, such as marking in learning
materials or controlling reading time, plays an important role
in improving SRL awareness. Notably, ICT can be used to
analyze and study learning behaviors that contribute to the
improvement of SRL from an LA perspective.
The purpose of this paper is to analyze the use of digital
content maps "BR-Map" and undertake a preliminary
investigation on the relationship between knowledge
construction state, learning behaviors, psychological state,
and learning performance.
B. Concept Map System
A concept map is an effective cognitive learning tool. It
uses text and graphic elements, such as lines and arrows, to
construct the relationship among the learned knowledges [1].
A concept map can help students to organize, evaluate,
communicate, and reconstruct knowledge [20]. With the
development of technology, an increasing number of
classrooms are adopting the digital concept map system.
The visualized digital concept map is considered as a tool
for in-depth understanding of students' metacognitive and
SRL levels [7]. Reference [4] reported a visualized concept
map system “BR-Map”, which is based on logs on the e-Book
viewer BookRoll. The formative evaluation of the BR-Map is
also conducted from an SRL perspective. The results showed
that the BR-Map can facilitate students’ understanding of the
learning content. Reference [21] developed and evaluated the
knowledge construction clustering tool “social knowledge
map” using the logs stored on the BR-Map. The results show
that visualization of types of knowledge construction has a
relatively complete function of making concept map; however,
it is difficult to observe the relationship between the use of the
concept map system and students' understanding of
knowledge.
Reference [23] proposed the use of LA to analyze the logs
of learning systems. It is possible to have a better
understanding of students' learning conditions so that teachers
can provide better support to students. Therefore, the trace of
students' use of the BR-Map system can be analyzed to deeply
understand the learning status of students to enable teachers to
improve the support to students.
This study aims to analyze and verify the relationship
between the differences in knowledge construction type and
learning behavior, metacognition, and SRL. It is envisaged
that we can have a deeper understanding of students' learning
habits through the results of the analysis to propound
suggestions for promoting students' SRL teaching in the future.
III. METHODS
A. Subjects and Course
Eighty-two university students took the class for this study,
and 46 among them made concept maps in this class. The topic
of this course is the Introductory Course of Pedagogy for the
2020 spring. The course consisted of seven classes; 1.
Education history over the world, 2. Educational
informalization in changing educational viewpoints, 3. What
is learning performance? Its definition and concept with ICT,
4. Learning theories and environment design, 5. Formal
education with ICT, 6. Informal learning with ICT, 7.
Educational technology research. All the classes were
provided using the online conference tool “Microsoft Teams”
because of the coronavirus disease pandemic. The main
learning objective was to understand the foundations of
education. There were three criteria for the grade: submitting
a one-minute paper after every class, grading of discussion
using text chat, and a report. Students had to submit the one-
minute paper within a day; late submission entailed that the
score would be reduced by half. Teachers and teaching
assistants also graded students' discussions. In the last class,
the students were required to summarize all the acquired
knowledge and their idea using the BR-Map.
B. BR-Map
In our study, students used the BR-Map to develop concept
maps. As [23] reports, the BR-map is a concept map system
using logs on the e-Book viewer “BookRoll”. There are three
steps to use the BR-Map in the course. Step 1: Students read,
highlight, and note on the teaching material with BookRoll,
before, during, and after each class; Step 2: Students construct
to sort out the knowledge that they have learned. The interface
of the BR-Map is shown in Fig. 1. As [4] introduced, the BR-
Map can read, and list all logs of highlights and memos from
the BookRoll database (the operations applied when learners
read learning materials) and display them as objects on the
left-side area. Students click an object on the left-side area and
drag-and-drop it into the right-side area, namely, the “concept
map area,” and create concept map(s) by connecting the
objects using “arrow(s).” Learners can add new concept maps

by clicking “new tab,” depending on the students’ desire. The
role of arrows is to link conceptual nodes and reflect the
connections between nodes. For example, if the arrows are
used to link between node 1 and node 2, it proves that the
learner believes that there is a connection between the two
nodes. This connection includes the following situations, 1.
Node 1 to node 2, which means that node 2 is learned after
node 1; 2. There is a double arrow between node 1 and node
2, which means that node 1 and node 2 promote each other; 3.
Multi-node to single node, which means that one node is
summed up from multiple nodes; 4. From single node to
multiple nodes, it means that a node contains multiple
subsidiary content; 5. Add note on the arrows to illustrate the
relationship between nodes.
Fig. 1. Interface of the BR-Map.
C. Clustering Concept Map
The traces of learners using the system and making maps
will be analyzed by the system and then classified. The system
used for classification is an analysis tool designed by [21][22].
The analysis process is as follows: 1. Morphological analysis
on the areas highlighted by the students and then integration
of nodes with similar words; 2. Calculate the weight of the
links between the keywords after the analysis; 3. Rebuild the
concept map of all students, focusing on the importance of
computing nodes (keywords) and the structure of the
knowledge graph; 4. Categorize the calculation results to get
different clusters of the concept map.
D. Data Collection
Three methods were used to collect data: a questionnaire,
grade, and learning logs. Students were asked to answer the
Metacognitive Awareness Inventory questionnaire [18] in the
first and last classes. The questionnaire consists of 19 items
divided into 10 factors. The 10 factors are: Declarative
knowledge (DK), Procedural knowledge (PK), Conditional
knowledge (CK) Knowledge of cognition (KC), Planning (P),
Information management strategies (IM), Monitoring (M),
Evaluation (E), Debugging strategies (DS), and Regulation of
cognition (RC). The total score of the questionnaire, Total
Metacognition (TM), was also calculated. (See appendix)
The student's grade is made up of three parts: minute-paper,
discussion, and report scores. Thereafter, the three parts are
added together to get the total performance score. A teacher
evaluated the minute-paper as a score in the range of 0 to 50,
the discussion in the range of 0 to 20, and the report in the
range of 0 to 30.
The learning log recorded behaviors of students while
reading learning materials on the BookRoll system and
making concept map: Node; link; memo; turning to the next
page (Next); turning to the previous page (Prev); adding a
marker (A_Ma); deleting a marker (D_Ma); adding a memo
(A_Me); deleting a memo (D_Me); adding a bookmark
(A_BM); deleting a bookmark (D_BM); adding a red marker
(A_MR) and yellow marker (A_MY); deleting a red marker
(D_MR) and yellow marker (D_MY).
IV. RESULTS
A. Result of Cluster
To understand the differences of the various learning
behaviors, we divided the students into seven groups named
G0–G6 based on the behaviors of the students using the BR-
Map. The system was grouped according to students' use of
the BR-Map, such as node, link, and memo. Students in Group
0 had the fewest use of traces, while those in Group 5 had the
most. Although in G6 group, there was no link trace in the BR-
Map, and the KJ method [24] was used to organize the
knowledge. The KJ method is devised to compile data, write
the data on a card, organize the cards into groups, illustrate
them, and summarize them in a treatise.
Students will draw the key points and notes in each class
into their concept map. The system will classify the students
according to the concept map made by the students.
The explanation of each cluster and the example is
presented below.
G0: Summarizing the topic of the lesson, but there is no
correlation between the items, such as arrows.
Fig. 2. Example of G0.
Memo learning
Knowledge points about
education and learning.
Effective use of ICT in education
Knowledge points
about educational
technology, psychological
theory and learning
outcomes.
Knowledge points about
educational technology and
utilizing ICT in the future.

G1: Linked the topic of each lesson to make a concept map.
Fig. 3. Example of G1.
G2: Using the summary of each lesson to make the concept
map. Have a decentralized knowledge construct.
Fig. 4. Example of G2.
G3: All courses are classified into two to three categories,
and concept map is made for each category. All students have
small pieces of knowledge constructs.
Fig. 5. Example of G3.
G4: The contents of the seven lessons were summarized
horizontally, and the content map was created. The students
have one large knowledge construct and three or more small
knowledge constructs.
Fig. 6. Example of G4.
Digital textbook
The point to put
digital and the point
to put analog
It does not mean that
we should proceed
with computerization
of everything.
ARCS model
Effective use of ICT in education
People are learning beings
·Instructional design
Some specific
advantages of ICT
Digital
textbook
Specific
content
related to
digital books
Knowledge points
about learning ability
and visualization
Knowledge points about
Activism, Cognitivism,
Constructivism and
Constructionism.
The merit of ICT is
that learning behavior
history remains
Effective use of ICT in education
The merit is that the
behavior history of
learning remains
Paraphrase
The strength of what can be
done with digital data
What was captured from
the psychological aspect
Effective use of ICT in education
Should be practical
Proper use
of analog
and digital
Evaluation method
Knowledge points
about personal studies
knowledge points
The relationship
between people,
tools and learning
Knowledge points about
career learning.
Effective use of ICT in education
Knowledge points
about education
Effective use of ICT in education

G5: The last half of the course materials is used to develop
the concept map. Students have less than two knowledge
constructs.
Fig. 7. Example of G5.
G6: The summary of every two to three lessons is made
into a concept map. Students have a single knowledge
construct.
Fig. 8. Example of G6.
B. Differences in Performance
，
Learning Behaviors, and
SRL among the Seven Clusters
Regarding the SRL factors, logs, and students' grade
among the seven groups, we conducted an analysis of variance
(ANOVA) and Tukey post-hoc analyses of learning behaviors.
Regarding the learning outcomes, the ANOVA revealed
that there was a significant difference in the total score among
the seven groups at the p=0.10 level (F=2.22, p<0.10).
However, no significant differences was found in the minute-
paper, discussion, and report among these groups (See Table
1). Surprisingly, ANOVA found no significant differences in
the factors of the SRL questionnaire among the seven different
BR-Map behavior groups (See Table 2).
TABLE I. ANOVA AMONG THE PERFORMANCE OF THE SEVEN
GROUPS.
Mean (SD)
F
G0
N=14
G1
N=7
G2
N=4
G3
N=4
G4
N=7
G5
N=3
G6
N=7
Minute
-paper
37.93
(4.46) 37.00
(4.16)
35.75
(6.08)
41.75
(0.50)
38.29
(3.59)
35.33
(3.51) 39.4
3
(4.2
9)
1.1
0
Discus
sion
18.09
(2.29) 18.53
(0.87)
18.35
(1.63)
19.84
(1.17)
19.14
(1.08)
17.97
(2.98)
19.5
9
(1.0
9)
1.0
9
Report
34.36
(3.77) 30.00
(10.0
7)
31.00
(3.56)
37.75
(3.86)
33.00
(2.45)
29.33
(2.31)
34.4
3
(2.9
4)
1.6
6
Total
90.21
(7.57) 85.53
(11.7
2)
85.00
(8.06)
99.35
(4.71)
90.37
(5.86)
82.33
(4.27)
93.2
4
(8.0
0)
2.2
2†
†p<0.1, * p<0.05, ** p <0.01
TABLE II. ANOVA AMONG THE SRL OF THE SEVEN GROUPS.
Mean (SD)
F
G0
N=14
G1
N=7
G2
N=4
G3
N=4
G4
N=7
G5
N=3
G6
N=7
DK -0.50
(2.56)
0.00
(2.08)
-0.25
(0.96)
1.25
(1.71)
0.71
(2.21)
-0.67
(1.53)
1.00
(1.53)
0.79
PK -0.36
(1.60)
-0.86
(1.57)
0.50
(0.58)
-1.25
(1.26)
-0.29
(1.80)
1.00
(2.00)
-1.14
(1.95)
1.08
CK -0.70
(1.14)
-1.29
(1.25)
0.75
(1.50)
-0.75
(0.50)
-0.86
(1.57)
0.00
(2.65)
-0.57
(1.51)
1.26
KC -0.93
(3.71)
-2.14
(3.02)
1.00
(0.82)
-0.75
(0.96)
-0.43
(4.31)
0.33
(3.21)
-0.71
(2.56)
0.48
P -0.21
(1.97)
-0.57
(2.23)
0.25
(2.06)
0.75
(0.50)
-0.14
(1.57)
1.33
(2.08)
0.43
(1.72)
0.58
IM -0.71
(3.12)
1.00
(2.38)
0.25
(1.26)
-0.25
(2.36)
-1.14
(1.68)
0.33
(1.53)
-0.86
(1.77)
0.69
M -0.43
(1.16)
-0.14
(1.57)
0.75
(0.50)
0.25
(1.26)
0.43
(0.53)
1.00
(1.00)
0.14
(1.21)
1.20
E 0.43
(1.65)
0.43
(1.72)
1.50
(1.00)
-0.75
(0.96)
1.29
(1.50)
0.33
(0.58)
-0.14
(1.86)
1.22
DS -0.36
(1.50)
-0.86
(1.57)
1.25
(1.26)
-1.25
(1.26)
0.00
(1.53)
-1.00
(1.73)
-0.14
(1.07)
1.43
RC -1.29
(6.04)
-0.14
(6.62)
4.00
(4.55)
-1.25
(2.87)
0.43
(4.20)
2.00
(6.00)
-0.57
(3.64)
0.65
T
M
-2.21
(7.38)
-2.29
(8.46)
5.00
(4.69)
-2.00
(3.46)
0.00
(7.92)
2.33
(8.74)
-1.29
(4.96)
0.74
a. †p<0.1, * p<0.05, ** p <0.01
How to use
Knowledge points about
evaluation
and evaluation
method by ICT
Advantages of ICT
Purpose of
utilization
Effective use of ICT in education
Effective use of ICT in education
Public
education
Private
education
Knowledge
points
about Class
design and
Learning
environment
Various
learning
activities
How do evaluate

Regarding the Learning Behaviors factors, there were
statistically significant differences in the node (F=2.70,
p<0.05) and link (F=3.52, p<0.05) among the seven groups
(See Table 3). We also performed Tukey post-hoc analyses
among the seven groups, but there was no statistically
significant difference among them.
TABLE III. ANOVA AMONG THE LEARNING BEHAVIORS OF THE SEVEN
GROUPS.
Mean (SD)
F
G0
N=14
G1
N=7
G2
N=4
G3
N=4
G4
N=7
G5
N=3
G6
N=7
Node 2.79
(1.89)
4.57
(2.07)
9.25
(7.59)
9.75
(6.70)
9.71
(8.56)
11.00
(7.00)
6.57
(4.20)
2.70*
Link 0.00
(0.00)
2.43
(1.90)
6.25
(5.68)
2.50
(1.00)
8.43
(9.47)
6.00
(4.00)
1.86
(3.48)
3.52*
Mem
o 2.07
(2.13)
2.00
(2.16)
2.50
(2.08)
2.00
(1.41)
5.14
(6.91)
0.00
(0.00)
3.57
(2.64)
1.20
Next
1409.
64
(846.
57)
1397.
14
(842.
99)
1078.
75
(699.
90)
1419.
25(11
11.40
)
1631.
29
(1031
.84)
1049.
00
(272.
45)
1287.
29
(665.
58)
0.28
Pre
548.7
1
(330.
53)
575.2
9
(435.
64)
365.7
5
(305.
67)
388.5
0
(472.
32)
673.4
3
(475.
80)
313.6
7
(207.
94)
442.8
6
(383.
78)
0.59
A_
Ma
90.64
(73.3
3)
60.86
(41.4
4)
178.7
5
(130.
89)
97.50
(38.5
1)
179.8
6
(133.
37)
108.6
7
(50.2
9)
145.7
1
(95.5
3)
1.77
D_
Ma
8.07
(12.8
0)
4.29
(7.52)
5.00
(8.72)
20.00
(31.0
2)
22.29
(18.6
3)
5.00
(5.00)
12.00
(12.6
6)
1.43
A_
Me
2.29
(2.95)
8.86
(17.2
9)
25.50
(29.4
9)
11.50
(17.7
7)
11.71
(12.4
2)
9.00
(12.2
9)
20.14
(34.9
0)
1.18
D_
Me
0.29
(0.61)
0.29
(0.76)
1.50
(2.38)
0.25
(0.50)
0.43
(0.53)
0.33
(0.58)
0.43
(0.53)
1.12
A_
BM
0.36
(1.08)
1.43
(2.51)
0.00
(0.00)
0.00
(0.00)
0.43
(0.79)
0.67
(1.15)
0.71
(0.76)
0.89
D_
BM
0.36
(1.08) 0.71
(1.50)
0.00
(0.00)
0.00
(0.00)
0.14
(0.38)
0.67
(1.15)
0.7
1(0.7
6)
0.61
A_
MR
54.07
(58.2
7)
31.14
(18.5
4)
148.0
0
(129.
99)
77.50
(37.5
0)
138.5
7
(141.
81)
78.33
(54.5
2)
105.2
9
(106.
21)
1.60
A_
MY
26.57
(46.6
4)
29.71
(31.0
5)
30.75
(41.2
9)
20.00
(10.3
0)
41.29
(54.1
7)
30.33
(15.0
4)
40.43
(27.1
9)
0.18
D_
MR
4.43
(11.1
2)
2.43
(4.16)
3.00
(4.76)
17.00
(28.1
1)
16.57
(19.1
2)
2.67
(3.79)
7.86
(9.96)
1.31
D_
MY
3.64
(7.74) 1.86
(3.76)
2.00
(4.00)
3.00
(3.56)
5.71
(10.1
8)
2.33
(2.08)
4.14
(3.72)
0.28
†p<0.1, * p<0.05, ** p <0.01
C. Relationships among Performance
，
Learning
Behaviors
，
SRL, and Clusters
Regarding the relationship between performance, SRL,
learning behaviors, and concept map cluster, we conducted a
correlation analysis using Polyserial correlation coefficients.
Table 2 presents the results of the correlation coefficients
which shows only statistically significant results less than the
significance value of 0.10.
In the results, cluster had a positive correlation with
performance factor discussion (0.315, p<0.05); SRL’s factor,
declarative knowledge (DK) (0.264, p<0.10), monitoring (M)
(0.296, p<0.05); and learning behaviors, adding a marker
(A_Ma) (0.280, p<0.05), adding a memo (A_Me) (0.325,
p<0.05), adding a red marker (A_MR) (0.273, p<0.05).
TABLE IV. POLYSERIAL CORRELATION COEFFICIENT AMONG
PERFORMANCE, SRL, LEARNING BEHAVIORS, AND CLUSTER
Discuss
ion
DK
M
A_Ma
A_Me
A_M
R
Cluster 0.315*
0.264†
0.296*
0.280*
0.325* 0.273*
†p<0.1, * p<0.05, ** p <0.01
V. CONCLUSION AND FUTURE RESEARCH
There are significant differences in performance between
different clusters. Among them, G3 has the highest
performance score. With the exception of G3, there are three
groups with performance scores of over 90. They are G6, G4,
and G0, while the other clusters which have performance
scores below 90 are G1, G2, and G5.
As introduced in the cluster results, the characteristic of
the concept maps of G3 is that students divide all courses into
several types and develop concept maps separately. Notably,
G6 also made a concept map by classifying all the courses.
Although the students in G4 did not make multiple concept
maps separately, they divided the courses into several small
parts in a large concept map. Students in G0 classified the
courses but did not indicate the relevance. Therefore, we can
see that a feature of the concept map with a score higher than
90 is that students classify all the knowledge they have learned.
This shows that after studying the content of the course, this
part of the students can organize the contents by making their
concept maps. Reference [25] reported that making a concept
map can help students recall the knowledge they have learned
and effectively improve the storage of knowledge. Students in
those clusters have a process of summarizing what they have
learned and self-adjustment when making concept maps. This
is consistent with the results of the correlation analysis. There
is a correlation between cluster and SRL's declarative
knowledge and monitoring. This is consistent with the
research results [4] that the BR-Map can support the
development of SRL skills, especially the use of cognitive
learning strategies. Many research results support that SRL
can improve the learning effect [19] [26] [27].
The feature of G1 is to link the topics of each lesson with
arrows. The feature of the G2 is that they summarize the
content of each lesson and link them with arrows. The feature
of G5 is that the students only use the learning materials in the
last half of the course to make a BR-Map. The common
feature of BR-Maps with a score of less than 90 is that students
directly use the title or course content to make the BR-Maps,
and they do not have their sorting and understanding of the

content. This also shows that these students did not use the
SRL skills when making the BR-Map, but simply listed the
content of the textbook on the BR-Map. As pointed out by [21],
utilizing the functions of the digital concept map more
efficiently and understanding how teachers can guide students
to exercise the ability of SRL by making BR-Maps still need
more research.
It is also evident from the results that there is a significant
difference between the link and node among clusters, but there
is no correlation between the link and performance, as well as
node and performance. Moreover, for G5 with the lowest
scores, the number of nodes ranked top in each group, and the
number of links ranked third. This is consistent with the result
of [21] that the use of BR-Maps seems to be influenced by the
habit of reading learning materials. However, the quality of
the knowledge composition is not determined by how many
nodes and links are used.
The topic of this course was the Introductory of Basic
Education for the 2020 spring semester. The course had
specific and correct definitions of various education concepts.
Nevertheless, the causality between concepts was weak.
Therefore, this makes it possible for the student's knowledge
map resulted in a situation like Fig. 2 of G0, which represents
the listing of concepts instead of using arrows to connect
knowledge. Future research is required to perform in more
courses and explore the difference between the concept maps
made by students in different disciplines.
Since there was no pre-questionnaire survey on students’
familiarity with concept map, it was difficult to distinguish
whether the experimental results are caused by SRL or
concept mapping. Alternatively, both SRL and concept
mapping might have an impact on the results. In future
research, the curriculum should be designed more carefully,
and the influence of SRL and concept map production on the
learning results should be distinguished.
In the present study, we observed and summarized
relationships among knowledge constructions, SRL, and
learning performance using LA to analyze BR-Map logs. The
following conclusions can be drawn from the present study:
students who categorize what they have learned when making
BR-Maps can better mobilize their SRL capabilities, such as
declarative knowledge and monitoring, thereby promoting the
improvement of academic performance. However, in the
present study, clarifying what type of learning behaviors can
better help mobilize SRL skills when making BR-Maps
remains elusive. Therefore, further work is required to explore
how the difference in BR-Map making behaviors influences
SRL.
ACKNOWLEDGEMENT
This research is supported by JSPS KAKENHI JP19H01716, JP 21K18134,
JP21KK0184 and JST AIP Acceleration Research JPMJCR19U1.
REFERENCES
[1] D. H. Jonassen, K. Beissner, and M. Yacci, Structural Knowledge:
Techniques for Representing, Conveying, and Acquiring Structural
Knowledge. Routledge, 2013.
[2] P. W. Foos, "The effect of variations in text summarization
opportunities on test performance," J. Exp. Educ., vol. 63, no. 2, pp.
89–95, 1995.
[3] J. C. Nesbit and O. O. Adesope, "Learning with concept and knowledge
maps: A meta-analysis," Rev. Educ. Res., vol. 76, no. 3, pp. 413–448,
2006.
[4] M. Yamada, A. Shimada, M. Oi, Y. Taniguchi, and S. Konomi, "BR-
Map: Concept map system using e-Book logs," Proceedings of CELDA
2018, pp. 248–254, 2018.
[5] R. Azevedo, M. Taub, N. V. Mudrick, G. C. Millar, A. E. Bradbury,
and M. J. Price, "Using data visualizations to foster emotion regulation
during self-regulated learning with advanced learning technologies," in
Informational Environments: Springer, 2017, pp. 225–247.
[6] A. Quigley, D. Muijs, and E. Stringer, Metacognition and self-
regulated learning. Guidance report. London, Education Endowment
Foundation, UK, 2018.
[7] G. C. Rakes, "Using the Internet as a tool in a resource-based learning
environment," Educ. Technol., vol. 36, no. 5, pp. 52–56, 1996.
[8] B. J. Zimmerman, (1986). “Becoming a self-regulated learner: Which
are the key subprocesses?” Contemporary Educational Psychology,
vol.11, no.4, pp. 307–313, 1986.
[9] D. H. Schunk and B. J. Zimmerman, Self-regulated Learning: From
Teaching to Self-reflective Practice. Guilford Press, 1998.
[10] New York, NY: The Guilford Press.A. Bandura, Social Foundations of
Thought and Action: A Social Cognitive Theory, Englewood Cliffs,
NJ: Prentice-Hall, 1986.
[11] D. L. Dinsmore, P. A. Alexander, and S. M. Loughlin, "Focusing the
conceptual lens on metacognition, self-regulation, and self-regulated
learning," Educ. Psychol. Rev., vol. 20, no. 4, pp. 391–409, 2008.
[12] D. H. Schunk, "Metacognition, self-regulation, and self-regulated
learning: Research recommendations," Educ. Psychol. Rev., vol. 20, no.
4, pp. 463–467, 2008.
[13] Education Endowment Foundation, Metacognition and Self-Regulated
Learning: Guidance Report. London: EEF. Available at:
educationendowmentfoundation.org.uk/tools/guidance-
reports/metacognition (Published 27 April, 2018).
[14] Y. Goda, M. Yamada, H. Kato, T. Matsuda, Y. Saito, and H. Miyagawa,
"Procrastination and other learning behavioral types in e-learning and
their relationship with learning outcomes," Learn. Individ. Differ., vol.
37, pp. 72–80, 2015.
[15] C. A. Cunningham and M. Billingsley, Curriculum Webs: A Practical
Guide to Weaving the Web into Teaching and Learning. Allyn &
Bacon, Inc., 2002.
[16] P. H. Winne, J. C. Nesbit, V. Kumar, A. F. Hadwin, S. P. Lajoie, R.
Azevedo, and N. E. Perry, “Supporting self-regulated learning with
gStudy software: The learning kit project,” Technology, Instruction,
Cognition and Learning, vol. 3, no. 1, pp. 105–113, 2006
[17] J. A. Greene and R. Azevedo, "A macro-level analysis of SRL
processes and their relations to the acquisition of a sophisticated mental
model of a complex system," Contemp. Educ. Psychol., vol. 34, no. 1,
pp. 18–29, 2009.
[18] Y. Goda, M. Yamada, T. Matsuda, H. Kato, Y. Saito, and H. Miyagawa,
"Effects of help seeking target types on completion rate and satisfaction
in e-learning," Proceedings of INTED, vol. 2013, pp. 1399–1403, 2013.
[19] M. Yamada, A. Shimada, F. Okubo, M. Oi, K. Kojima, and H. Ogata,
"Learning analytics of the relationships among self-regulated learning,
learning behaviors, and learning performance," Res. Practice in
Technol. Enhanced Learning, vol. 12, no. 1, pp. 1–17, 2017.
[20] S.-O. Tergan and T. Keller, Knowledge and Information Visualization:
Searching for Synergies. Springer, 2005.
[21] A. Onoue, M. Yamada, A. Shimada, and R.-i. Taniguchi, "The
integrated knowledge map for surveying students’ learning," in Society
for Information Technology & Teacher Education International
Conference, 2019: Association for the Advancement of Computing in
Education (AACE), pp. 1092–1100.
[22] A. Onoue, A. Shimada, T. Minematsu, R. Taniguchi, “Clustering of
learners based on knowledge maps”, Proceedings of CELDA 2019, pp.
363–370, 2019.
[23] H. Ogata, M. Oi, K. Mohri, F. Okubo, A. Shimada, M. Yamada, & S.
Hirokawa, "Learning analytics for e-book-based educational big data
in higher education," in Smart Sensors at the IoT Frontier: Springer,
2017, pp. 327–350.
[24] J, Kawakita, "Continued idea method-development and application of
KJ method," Chuko Shinsho, 1970.
[25] N. F. Huang, C. A. Lee, Y. W. Huang, P. W. Ou, H. H. Hsu, S. C. Chen,
& J. W. Tzenger, "On the automatic construction of knowledge-map
from handouts for MOOC courses," in International Conference on
Intelligent Information Hiding and Multimedia Signal Processing,
2017: Springer, pp. 107–114.

[26] M. Yamada, Y. Goda, T. Matsuda, Y. Saito, H. Kato, and H. Miyagawa,
"How does self-regulated learning relate to active procrastination and
other learning behaviors?," J. Comput. High. Educ., vol. 28, no. 3, pp.
326–343, 2016.
[27] C. A. Wolters, P. R. Pintrich, and S. A. Karabenick, "Assessing
academic self-regulated learning," in What Do Children Need to
Flourish? Springer, 2005, pp. 251–270.
APPENDIX : METACOGNITIVE
AWARENESS INVENTORY
 Knowledge of cognition dimension
• Declarative knowledge
1. I know what kind of information is most important to
learn.
2. I know what the teacher expects me to learn.
3. I have control over how well I learn.
4. I am a good judge of how well I understand something.
• Procedural knowledge
5. I am aware of what strategies I use when I study.
6. I find myself using helpful learning strategies
automatically.
• Conditional knowledge
7. I can motivate myself to learn when I need to.
8. I know when each strategy I use will be most effective.
 Regulation of cognition dimension
• Planning
9. I think about what I really need to learn before I begin a
task.
10. I set specific goals before I begin a task.
• Information management strategies
11. I try to translate new information into my own words.
12. I use the organizational structure of the text to help me
learn.
13. I ask myself if what I’m reading is related to what I
already know.
• Monitoring
14. I periodically review to help me understand important
relationships.
• Evaluation
15. I summarize what I’ve learned after I finish.
16. I ask myself if I learned as much as I could have once
I finish a task.
• Debugging strategies
17. I change strategies when I fail to understand.
18. I re-evaluate my assumptions when I get confused.
19. I stop and go back over new information that is not
clear.