Content uploaded by Wilk Oliveira
All content in this area was uploaded by Wilk Oliveira on Jul 15, 2021
Content may be subject to copyright.
Modeling students’ ﬂow experience through data
logs in gamiﬁed educational systems
Wilk Oliveira, Seiji Isotani
University of S˜
ao Carlos, Brazil
Olena Pastushenko, Tom´
Brno University of Technology
Brno, Czech Republic
Abstract—User modeling in gamiﬁed educational systems is a
contemporary challenge. In particular, modeling the students’
ﬂow experience (i.e., challenge-skill balance, action-awareness
merging, clear goals, unambiguous feedback, total concentration
on the task at hand, sense of control, loss of self-consciousness,
transformation of time, and autotelic experience) during a gam-
iﬁed system usage is highly challenging. It is because mea-
surement’ instruments usually are invasive, removing the users
from the ﬂow experience and/or cannot be applied massively
(e.g., participant observation, questionnaires or electroencephalo-
gram). We faced this challenge by conducting a data-driven
study (N = 23), where we used a robust statistical method
(i.e., partial least squares path modeling) to model the students’
ﬂow experience, based on their interaction data (e.g., number
of mouse clicks) in a gamiﬁed educational system. The main
results indicate a relationship between the interaction logs and
four ﬂow experience dimensions. Our ﬁnds contribute to the
area of gamiﬁed educational systems, through the students’
ﬂow experience modeling. Finally, based on our results, we also
provided a series of recommendations for future studies.
Index Terms—Gamiﬁed educational systems, Flow Theory,
Flow experience, Students’ experience, User modeling
In recent years, the number of different types of educational
systems (e.g., Massive Open Online Courses (MOOCs) ,
intelligent tutoring systems (ITS) , gamiﬁed educational
systems , and others) has grown, attracting the attention
of teachers/instructors and students around the world. That
is causing the increase of the number of students using this
type of systems . Therefore it is necessary to invest in
new methods to improve the student experience . One
of the commonly used methods to improve the quality of
such educational systems is gamiﬁcation (“The use of game
elements in non-game contexts” ) .
On the one hand, the design of online gamiﬁed educational
systems can help improve the teaching and learning process
. On the other hand, the large number of students using
the same system at the same time tends to hinder some tasks,
such as understanding students’ behavior and assessing their
experience (e.g., engagement, motivation, and ﬂow) during
the system usage . This is highlighting the importance of
The authors would like to thank the grant provided by S˜
ao Paulo Research
Foundation (FAPESP), Project: 2018/07688-1.
modeling students’ experience in gamiﬁed educational systems
through their data logs .
Faced with this challenge, recent research has attempted
to take advantage of the big amounts of data logs generated
during users interaction with educational systems to model
students’ experience , . One of the most complex pa-
rameters to be analyzed in such data sets is the ﬂow experience
. Flow is a deep engagement experience, composed of
nine associated dimensions : i) challenge-skill balance, ii)
action-awareness merging, iii) clear goals, iv) unambiguous
feedback, v) total concentration on the task, vi) sense of
control, vii) loss of self-consciousness, viii) transformation of
time and ix) autotelic experience. When a student is in a state
of ﬂow, they also tend to have a high learning experience (i.e.,
ﬂow state positively affects the learning process) .
This paper addresses the described challenge by presenting
the results of a data-driven study with a sample composed of
23 university students. For this research we used structural
equation modeling (i.e., partial least squares path modeling
) to associate the students’ ﬂow experience in a gamiﬁed
system with their interaction data logs. Afterwards, we answer
the following research question: Does possible the students’
ﬂow experience in a gamiﬁed educational system be mod-
eled based on their interaction data logs?
The study’s main results indicate a correlational model
between the group of data logs and four different ﬂow ex-
perience dimensions (i.e., unambiguous feedback, clear goals,
loss of self-consciousness, and action-awareness merging).
Thus, our study is a step towards modeling the students’ ﬂow
experience in gamiﬁed educational systems based on the data
logs. It also contributes to the development of computational
approaches for providing automatic students’ ﬂow experience
identiﬁcation in this kind of system. Based on the achieved
results, we proposed a series of recommendations for new
studies in the ﬁeld.
II. RELATED WORKS
To identify the main related works, we analyzed the sys-
tematic literature review about Flow Theory and educational
technologies conducted by Oliveira et al. , the literature
review about Flow Theory and game based systems conducted
by Perttula et al. , and the literature review about Flow
2021 International Conference on Advanced Learning Technologies (ICALT)
2161-377X/21/$31.00 ©2021 IEEE
Theory and gamiﬁcation conducted by Oliveira et al. . Ini-
tially, studies that aim to model the students’ ﬂow experience
in educational systems using data logs are relatively recent
–. The ﬁrst studies were published in 2014 and used
statistical analysis . After that, new studies used ontology
 and electroencephalogram  aiming to relate students’
ﬂow experience and their data logs. However, in these studies
the ﬂow experience was implemented only as challenge-skill
Most recent papers are proposing and evaluating theoretical
models that relate the students’ ﬂow experience in educational
systems with different data logs , as well as conducting
qualitative studies with the same objective . Finally, there
are studies which conduct similar experiments using machine
learning methods to model the students’ ﬂow experience .
Analysis of the existing research indicates that one of
the remaining challenges is the need to conduct different
experiments in various contexts (e.g., gamiﬁed educational
systems), as well as data analysis using different techniques.
As far as we know, our study is the ﬁrst to model the
students’ ﬂow experience in gamiﬁed educational systems
through structural equation modeling considering the nine
original ﬂow experience dimensions.
III. STUDY DESIGN
The main goal of this study is to model students’ ﬂow
experience in a gamiﬁed educational system through data logs
(i.e., the student interaction data logs during the system usage).
To achieve this goal, we choose a data-drive method (i.e.,a
research based on users’ data analysis) .
A. Research question and hypothesis
Based on the study goal, the following research question
(RQ) was deﬁned:
•RQ: Does possible the students’ ﬂow experience in a
gamiﬁed educational system be modeled based on their
interaction data logs?
Since a large number of students started to use online
educational platforms in recent years, as well as due to
the large amount of data generated by these interactions,
numerous studies have sought to use this data to analyze
students’ experience . In particular, concerning the ﬂow,
some recent studies have shown that there may be a direct
relationship between different dimensions of the students’ ﬂow
experience and the data logs , , . Therefore, in this
study, we hypothesized that is possible to model students’
ﬂow experience through data logs in gamiﬁed educational
B. Materials and procedure
To conduct this study, we used the gamiﬁed educational
system “bombsQuery”1, which is a tool for teaching the
different missions, each one devoted to a different topic. Each
mission has some theory and examples and a free text area
where students need to insert their proposed solutions. The
playful goal of the missions is to clear the mineﬁeld from
all bombs. If the student’s solution was wrong, they have an
unlimited amount of attempts to correct it. However, if their
answer was correct, the next level starts. The students can
always come back to any of the already solved levels. This
might be useful if they want to check the accepted answer
for inspiration or go through the theory once again . The
tool was chosen because it allowed the implementation of a
module to collect the students’ data logs. Moreover, it has
already been validated and used by other researchers .
Figure 1 illustrates an example for the mission.
Fig. 1. An example of the mission in the gamiﬁed educational system
To collect the students’ data logs, we implemented a new
module in the tool (described below). Data logs were collected
based on the theoretical model proposed by Oliveira et al. .
The theoretical model proposed by Oliveira et al.  presents
a series of data logs theoretically associated with the nine ﬂow
experience dimensions. The module proposes nine different
data logs that can be related to the nine ﬂow experience
dimensions. The collected data logs are: ArtAF: average
students’ response time after a feedback; NumCOB: number
of mouse clicks; ProWS: proportion of wrong steps/responses;
RF: received feedback; TotUSV: total unique session views;
UsdTFS: used time to ﬁnish a step/mission; and ActTS: active
time in the system.
To analyze the students’ ﬂow experience during working on
the assignment, we used the short ﬂow state scale (short FSS)
proposed by Jackson and Mars . This scale was chosen
because it was previously validated by Hamari and Koivisto
 to be used gamiﬁed settings, as well as being the most
popular scale in studies in the area of educational technologies
. As the data collection was done after performing a quick
activity (with less than an hour), following the recommenda-
tion of the “The Manual for the Flow Scales” , we chose to
use the short scale composed of nine questions (one for each
dimension of the ﬂow experience) presented in a ﬁve-points
Likert scale. To ensure the quality of the responses, inspired
by recent studies , we have included an “attention-check”
question (i.e., if you are ﬁlling out the form carefully, answer
3*) to eliminate responses from students who were not paying
due attention when answering the questions. This study was
organized in four different general steps: i) selection of the
gamiﬁed system, ii) students’ invitation, iii) data collection,
and iv) data analysis.
Our participants were 31 bachelor students of Brno Uni-
versity of Technology (Czech Republic), who volunteered to
participate in the experiment. Five responses were excluded
because students spent less than 5 minutes on the assignment,
which is an indication that they haven’t really used the system.
Three responses were excluded because the students answered
incorrectly the attention-check question. We, therefore, in-
cluded 23 participants (mean age = 21.54 years old, SD = 1.33;
6 women, 13 men, 0 non-binary, 4 preferred not to disclose
gender). To participate in the study, students received a link
to the questionnaires and the assignment, and they could work
on it online, at their pace and preferred time.
In order to deﬁne the best strategy to analyse our data,
we analysed the data normality. As recommended by Wohlin
, we used the Shapiro-Wilk test to check it. The tests
showed that data is within a non-normal distribution. Thus, we
measured the internal reliability for the scale (for the overall
ﬂow experience), using Cronbach’s alpha, thus obtain α=
0.621 (despite the low alpha value, this can occur because
the measured experiences on the scale behave independently).
Next, we measured the discriminant validity for the analyses.
The results are shown in Table I.
To answer the RQ, we modeled the students’ interaction
logs (as latent variables) and their ﬂow experience in the
gamiﬁed educational system. Partial Least Squares Path Mod-
eling (PLS-PM) analysis  was used to observe the relation
between the students’ data logs and their ﬂow experience.
PLS-PM was used because it is a reliable method for estimate
cause-effect relationship models with latent variable . At
the same time, this method can be selected to the exploratory
research and to studies with small sample sizes (as such
our case) . We used the software SmartPLS2to run the
analyses. Table II present the PLS-PM matrix and Figure 2
present the research path model with the results.
In this study, we used PLS-PM to model the user ﬂow
experience in the gamiﬁed educational system using gathered
data logs. The created model presented a signiﬁcant relation-
ship between the data logs and four different ﬂow experience
dimensions (see Table II): unambiguous feedback (β-0.435),
clear goals (β-0.530), loss of self-consciousness (β-0.707),
and action-awareness merging (β-0.283).
Initially, the data logs have a negative relationship (β-0.435)
with the unambiguous feedback. In this dimension, according
to Oliveira et al. , when a student takes a long time to
complete an activity after receiving feedback, they possibly
R2 = 0.121
R2 = 0.080
R2 = 0.281
R2 = 0.189
R2 = 0.005
R2 = 0.001
R2 = 0.500
R2 = 0.000
R2 = 0.017
Fig. 2. Research path model
received ambiguous feedback. Thus, the less time the student
spends in the system, or the lesser the number of received
feedback, the unambiguous feedback experience tends to be
less. On the other hand, this result has not yet been identiﬁed
in previous studies and needs to be conﬁrmed , , .
We also identiﬁed a negative relationship between the data
logs model and the the clear goals dimension (β-0.530).
This relationship can be explained based on the previous
relationship, because, once a student has not received unam-
biguous feedback, they will also possibly not be able to clearly
understand the objectives of the activity. This result can also
help to explain the results identiﬁed in the qualitative study
conducted by Oliveira et al. . Their study identiﬁed that
the average of correct steps affected the sense of “clear goals”.
The highest relationship (also negative) occurred between
the data logs and loss of self-consciousness dimension (β-
0.707). According to Jackson et al. , when an individual
is no longer concerned with what others think of them, self-
consciousness has been lost. This statement may be key to
interpreting the results as if there is a trend in data logs (e.g.,
the proportion of wrong steps/responses, average students’
response time after feedback, and active time in the system),
is low, students can be concerned with what others think of
them, and then, to have a low loss of self-consciousness.
The last relationship in the model in represented by the
correlation between data logs and action-awareness merging
dimension (β-0.283), an experience comes about through a
total absorption in what one is doing . This relationship
can also be explained according to the same explanation as
the previous experience. Our results demonstrate a tendency
regarding a relationship/pattern between students’ ﬂow expe-
rience and their data logs in a gamiﬁed educational system.
Therefore, following the trend of previous studies. However,
the relationships identiﬁed in our study are different from the
DISCRIMINANT VALIDITY (COMPLETE BOOTSTRAPPING,SAMPLE=5000)
A C CSB CTRL Datalogs F G LSC MMA T
CSB -0.135 0.030 1.000
CTRL 0.399 0.171 0.051 1.000
Datalogs 0.128 0.068 -0.348 0.035 1.000
F0.017 0.369 0.361 0.228 -0.435 1.000
G0.157 0.145 0.239 0.090 -0.530 0.536 1.000
LSC -0.014 0.064 0.181 0.072 -0.707 0.311 0.303 1.000
MMA 0.193 -0.130 0.586 0.294 -0.283 0.150 0.182 0.039 1.000
T0.644 0.209 0.002 0.138 -0.015 -0.043 0.113 0.246 -0.027 1.000
Key: CSB: challenge-skill balance, MMA: action-awareness merging, G: clear goals, F: unambiguous feedback, C:
total concentration on the task at hand, CTRL: sense of control, LSC: loss of self-consciousness, T: transformation
of time, and A: autotelic experience
PLS-PM MATRIX FOR DATA LOGS AND FLOW EXPERIENCE DIMENSIONS
βM SD P 2.5% 97.5%
DL →A0.128 0.058 0.248 0.605 -0.604 0.397
DL →C0.068 0.141 0.220 0.756 -0.233 0.534
DL →CSB -0.348 -0.384 0.190 0.067 -0.755 -0.034
DL →CTRL 0.035 0.010 0.201 0.862 -0.496 0.372
DL →F-0.435** -0.436 0.153 0.005 -0.707 -0.097
DL →G-0.530** -0.526 0.145 0.000 -0.756 -0.227
DL →LSC -0.707* -0.584 0.279 0.011 -0.908 0.034
DL →MMA -0.283* -0.341 0.134 0.036 -0.602 -0.101
DL →T-0.015 -0.026 0.135 0.913 -0.284 0.232
Key: DL: data logs, CSB: challenge-skill balance, MMA: action-awareness
merging, G: clear goals, F: unambiguous feedback, C: total concentra-
tion on the task at hand, CTRL: sense of control, LSC: loss of self-
consciousness, T: transformation of time, and A: autotelic experience, β:
regression coefﬁcient, M: meam, SD: standard deviation, P: p-value,CI:
conﬁdence interval, * p<0.5, ** p<0.005
relationships observed in some other studies.
This result may indicate that the relationship between data
logs and ﬂow experience can be different according to the type
of system, participants’ age, and other factors (since each pre-
vious study was conducted in different settings). Another way
of thinking is related to the number of participants (especially
in our study). The sample size needs to be increased to also
increase the conﬁdence in the results. Regardless, our results
are promising for modeling the students’ ﬂow experience
in gamiﬁed educational systems based on data logs, while
highlighting the need for further research.
The study presented in this paper has some limitations,
which we seek to mitigate. The experience measured in the
study (i.e., ﬂow experience) is a complex parameter to be
measured. To mitigate this limitation, we use only previously
validated methods (i.e., the short FSS validated by Hamari
and Koivisto  for the gamiﬁcation domain and theoret-
ical model proposed by Oliveira  to collect data logs
in educational systems). At the same time, to ensure the
quality of responses and to avoid external threats (e.g., lack
of attention from students), we insert an “attention checking
question” within the scale and used other methods (e.g.,
remove responses from students who used the system for less
than ﬁve minutes) to avoid data set inconsistencies. Another
important limitation is related to our small sample size (i.e.,
23 students). To mitigate this limitation, we use a robust
statistical method capable of accurately analyzing data from
small samples (i.e., PLS-PM) . However, we highlight the
importance of replicating the experiment with larger samples
to provide a greater results generalization, and we are sure that
this paper would serve as an excellent basis for such future
C. Ways forward
Based on our results, it is possible to provide some rec-
ommendations for future studies. Comparing our results with
results of other studies , , , we can conclude that
the relationship of each ﬂow experience dimension with the
different data logs can be changed according to the context
(i.e., with the type of educational system or even with the type
of data analysis). Therefore, we recommend that future stud-
ies conduct similar studies in different types of educational
environments (e.g., MOOCs, ITS, educational games, and
others). In the same way, we also recommend analyzing the
data through different techniques (e.g., qualitative analysis,
data mining and machine learning techniques), seeking to
carry out the experiments with larger samples, to increase the
generalization of the results.
In our study, we considered the entry model as a set of
different variables (i.e., data logs). However, we do not model
each data log individually with each ﬂow experience dimen-
sion. Thus, we recommend that future studies can model the
relationship of each data log (individually) with the students’
ﬂow experience dimensions. In our study, our input data
(data logs) were deﬁned based on the study by Oliveira et
al.  However, not all data from the theoretical model
was considered in our study. Thus, we recommend that
future studies also include the other data logs proposed
in the theoretical model. Last but not least, we recommend
that further research improve the inputs by analysing
at whether gamiﬁcation data (e.g., number of points,
badges, and leader-boards (alone)) affects students’ ﬂow
V. C ONCLUDING REMARKS
Modeling students’ ﬂow experience in gamiﬁed educational
systems is a contemporary challenge. In this study, we used
PLS-PM to model the ﬂow experience of the students through
their data logs in a gamiﬁed educational system. Our results
demonstrate a correlation between the interaction logs and
four ﬂow experience dimensions, moving towards modeling
the students’ ﬂow experience in educational games using data
logs. Future research is planned to replicate this experiment
with larger numbers of participants, using new data analysis
methods, such as data mining and machine learning.
Previous studies of this project have been published:
Oliveira et al.  conducted a systematic literature review
about Flow Theory and Educational Technologies; Oliveira
 presented the project overview; Oliveira et al. 
proposed a theoretical model relating students’ data logs and
their ﬂow experience in educational systems; and Oliveira et
al.  conducted a qualitative study analysing students’ data
logs and their ﬂow experience in an educational systems.
 A. Bozkurt, E. Akg¨
Ozbek, and O. Zawacki-Richter, “Trends and
patterns in massive open online courses: Review and content analysis of
research on moocs (2008-2015),” International Review of Research in
Open and Distributed Learning: IRRODL, vol. 18, no. 5, pp. 118–147,
 E. Mousavinasab, N. Zarifsanaiey, S. R. Niakan Kalhori, M. Rakhshan,
L. Keikha, and M. Ghazi Saeedi, “Intelligent tutoring systems: a sys-
tematic review of characteristics, applications, and evaluation methods,”
Interactive Learning Environments, pp. 1–22, 2018.
 J. Koivisto and J. Hamari, “The rise of motivational information systems:
A review of gamiﬁcation research,” International Journal of Information
Management, vol. 45, pp. 191–210, 2019.
 C. Jack and S. Higgins, “Embedding educational technologies in early
years education,” Research in learning technology, vol. 27, 2019.
 I. Buil, S. Catal´
an, and E. Mart´
ınez, “The inﬂuence of ﬂow on learning
outcomes: An empirical study on the use of clickers,” British Journal
of Educational Technology, vol. 50, no. 1, pp. 428–439, 2019.
 S. Deterding, D. Dixon, R. Khaled, and L. Nacke, “From game design
elements to gamefulness: deﬁning gamiﬁcation,” in Proceedings of the
15th international academic MindTrek conference: Envisioning future
media environments. ACM, 2011, pp. 9–15.
 C. Hursen and C. Bas, “Use of gamiﬁcation applications in science
education.” International Journal of emerging technologies in Learning,
vol. 14, no. 1, 2019.
 S. Bai, K. F. Hew, and B. Huang, “Is gamiﬁcation “bullshit”? evidence
from a meta-analysis and synthesis of qualitative data in educational
contexts,” Educational Research Review, p. 100322, 2020.
 L. Shi, A. I. Cristea, A. M. Toda, and W. Oliveira, “Exploring navigation
styles in a futurelearn mooc,” in International Conference on Intelligent
Tutoring Systems. Springer, 2020, pp. 45–55.
 W. Oliveira, A. Toda, P. Palomino, L. Rodrigues, S. Isotani, and
L. Shi, “Towards automatic ﬂow experience identiﬁcation in educational
systems: A theory-driven approach,” in Extended Abstracts of the
Annual Symposium on Computer-Human Interaction in Play Companion
Extended Abstracts. ACM, 2019, pp. 581–588.
 L. Shi, A. I. Cristea, A. M. Toda, and W. Oliveira, “Social engagement
versus learning engagement an exploratory study of futurelearn learners,”
in 2019 IEEE 14th International Conference on Intelligent Systems and
Knowledge Engineering (ISKE). IEEE, 2019, pp. 476–483.
 ——, “Revealing the hidden patterns: A comparative study on proﬁling
subpopulations of mooc students,” International Conference on Infor-
mation Systems Development, 2020.
 M. Csikszentmihalyi, Finding ﬂow: The psychology of engagement with
everyday life. Basic Books, 1997.
 J. F. Hair Jr, G. T. M. Hult, C. Ringle, and M. Sarstedt, A primer on
partial least squares structural equation modeling (PLS-SEM). Sage
 W. Oliveira, I. I. Bittencourt, S. Isotani, D. Dermeval, L. B. Marques, and
I. F. Silveira, “Flow theory to promote learning in educational systems: Is
it really relevant?” Brazilian Journal of Computers in Education, vol. 26,
no. 02, p. 29, 2018.
 A. Perttula, K. Kiili, A. Lindstedt, and P. Tuomi, “Flow experience
in game based learning–a systematic literature review,” International
Journal of Serious Games, vol. 4, no. 1, pp. 57–72, 2017.
 W. Oliveira, O. Pastushenko, L. Rodruigues, A. M. Toda, P. T. Palomino,
J. Hamari, and S. Isotani, “Does gamiﬁcation affect ﬂow experience? a
systematic literature review,” in GamiFIN Conference 2021: Proceedings
of the 5th International GamiFIN Conference, 2021, pp. 1–10.
 P.-M. Lee, S.-Y. Jheng, and T.-C. Hsiao, “Towards automatically detect-
ing whether student is in ﬂow,” in International Conference on Intelligent
Tutoring Systems. Springer, 2014, pp. 11–18.
 G. C. Challco, F. R. Andrade, S. S. Borges, I. I. Bittencourt, and
S. Isotani, “Toward a uniﬁed modeling of learner’s growth process and
ﬂow theory,” Journal of Educational Technology & Society, vol. 19,
no. 2, 2016.
 F. De Kock, The Neuropsychological Measure (EEG) of Flow Under
Conditions of Peak Performance. Unisa, 2014. [Online]. Available:
 W. Oliveira, L. Rodrigues, A. Toda, P. Palomino, L. Shi, and S. Isotani,
“Towards automatic ﬂow experience identiﬁcation in educational sys-
tems: A qualitative study,” in Brazilian Symposium on Computers in
Education, vol. 31, 2020.
 Y. C. Semerci and D. Goularas, “Evaluation of students’ ﬂow state in
an e-learning environment through activity and performance using deep
learning techniques,” Journal of Educational Computing Research,p.
 V. Dhar, “Data science and prediction,” Communications of the ACM,
vol. 56, no. 12, pp. 64–73, 2013.
 J. L´
opez-Zambrano, J. A. Lara, and C. Romero, “Towards portability of
models for predicting students’ ﬁnal performance in university courses
starting from moodle logs,” Applied Sciences, vol. 10, no. 1, p. 354,
 O. Pastushenko, T. Hruˇ
ska, and J. Zendulka, “Increasing students’
motivation by using virtual learning environments based on gamiﬁcation
mechanics: Implementation and evaluation of gamiﬁed assignments for
students,” in Proceedings of the Sixth International Conference on
Technological Ecosystems for Enhancing Multiculturality, 2018, pp.
 O. Pastushenko, W. Oliveira, S. Isotani, and T. Hruˇ
ska, “A methodology
for multimodal learning analytics and ﬂow experience identiﬁcation
within gamiﬁed assignments,” in Extended Abstracts of the 2020 CHI
Conference on Human Factors in Computing Systems, 2020, pp. 1–9.
 S. A. Jackson and R. C. Eklund, “Assessing ﬂow in physical activity:
The ﬂow state scale–2 and dispositional ﬂow scale–2,” Journal of Sport
and Exercise Psychology, vol. 24, no. 2, pp. 133–150, 2002.
 J. Hamari and J. Koivisto, “Measuring ﬂow in gamiﬁcation: Disposi-
tional ﬂow scale-2,” Computers in Human Behavior, vol. 40, pp. 133–
 S. Jackson, B. Eklund, and A. Martin, “The ﬂow manual the manual for
the ﬂow scales manual. sampler set,” Mind, pp. 1–85, 2011.
 R. Orji, G. F. Tondello, and L. E. Nacke, “Personalizing persuasive
strategies in gameful systems to gamiﬁcation user types,” in Proceedings
of the 2018 CHI Conference on Human Factors in Computing Systems,
2018, pp. 1–14.
 C. Wohlin, P. Runeson, M. H¨
ost, M. C. Ohlsson, B. Regnell, and
en, Experimentation in software engineering. Springer Science
& Business Media, 2012.
 J. Henseler, C. M. Ringle, and R. R. Sinkovics, “The use of partial least
squares path modeling in international marketing,” in New challenges to
international marketing. Emerald Group Publishing Limited, 2009.
 W. Oliveira, “Towards automatic ﬂow experience identiﬁcation in edu-
cational systems: A human-computer interaction approach,” in Extended
Abstracts of the Annual Symposium on Computer-Human Interaction in
Play Companion Extended Abstracts, 2019, pp. 41–46.