Available via license: CC BY-SA 4.0
Content may be subject to copyright.
Indonesian Journal of Computer Science
ISSN 2549-7286 (online)
Jln. Khatib Sulaiman Dalam No. 1, Padang, Indonesia
Website: ijcs.stmikindonesia.ac.id | E-mail: ijcs@stmikindonesia.ac.id
Attribution-ShareAlike 4.0 International License Vol. 13, No. 2, Ed. 2024 | page 3627
The Effect of Virtual Reality Gaming on Developing Computational Thinking Skills
Sukirman1,2, Laili Farhana Md Ibharim1, Che Soh Said1, Budi Murtiyasa2
sukirman@ums.ac.id, laili@meta.upsi.edu.my, chesoh@meta.upsi.edu.my,
budi.murtiyasa@ums.ac.id
1Faculty of Computing and Meta-Technology, Universiti Pendidikan Sultan Idris, Perak, Malaysia
2Faculty of Teacher Training and Education, Universitas Muhammadiyah Surakarta, Indonesia
Article Information
Abstract
Submitted : 17 Mar 2024
Reviewed : 2 Apr 2024
Accepted : 30 Apr 2024
In the digital age, where programming prowess is increasingly crucial, the
enhancement of Computational Thinking (CT) skills becomes essential. This
study ventures into the scarcely explored domain of leveraging game-based
learning (GBL) within virtual reality (VR) settings to bolster CT skills.
Specifically, it introduces "CT Saber," a VR game inspired by the popular
"Beat Saber," tailored to cultivate CT competencies. Employing a Design and
Development Research (DDR) methodology across five stages—analysis,
design, development, implementation, and evaluation—this investigation
assessed the game's impact on 37 computer science students (25 male, 12
female) aged 21-24. A quasi-experimental design with pretest-posttest
evaluation was utilized, revealing significant enhancements in CT skills post-
intervention (Z = -4.496, p < 0.05), as analyzed through Wilcoxon Signed-
Rank tests. The findings underscore the VR game's efficacy in CT skill
development, suggesting a promising direction for integrating VR
technologies in programming education.
Keywords
computational thinking,
CT saber, game-based
learning, virtual reality
ISSN 2549-7286 (online)
Indonesian Journal of Computer Science Vol. 13, No. 2, Ed. 2024 | page 3628
A. Introduction
Digital technology has become an integral part of our lives, playing a crucial
role in solving a myriad of problems. In today's world, virtually all activities
involve the use of digital technology, underscoring its pervasive influence on our
daily routines. This technology is primarily developed by software engineers or
individuals who have programming skills, highlighting the importance of technical
expertise in the digital age. One of the skills that bolster programming skills is
Computational Thinking (CT) [1]. CT enables individuals to solve problems, design
systems, and understand human behavior by drawing on the concepts
fundamental to computer science. This skill set is not only vital for software
engineers but also even enhances the ability to think logically and systematically
for all work backgrounds, making it a critical asset in the development and
application of digital technology. Therefore, many researchers stated that CT is
believed to be a basic competency that should be developed by twenty-first-
century students [2], [3].
CT is described as thought processes involved in formulating problems and
the solutions represented in an effective form to be carried out [4]. The term of CT
originated from a procedural concept proposed by Papert in 1980[5], then its
definition, teaching, learning, and evaluation were discussed by various scholars
and researchers [6]. Wing (2006) defined CT as a kind of analytical thinking which
includes elements of problem-solving, system design, and understanding human
behavior based on the concepts of computer science [7]. Afterwards, Wing (2014)
emphasizes that CT is not only about problem-solving, but also problem
formulation [8]. Even, Wing (2006) stated that CT is equal to other skills like
reading, writing, or arithmetic that should be empowered by all students, not only
computer science students. Additionally, the International Society for Technology
in Education (ISTE) and Computer Science Teachers Association (CSTA) believe
that CT is essential to improve the achievement level of students, preparing them
for global competitiveness and blending academics into real life [9].
Educational systems in many countries realized the importance of CT so that
they integrated CT or subjects related to programming and computer science into
compulsory education or just an additional. However, the name used is various due
to the term of CT have not well-established, such as "digital competence" in
Sweden, "computing" in England, or "computer science" in the USA [10]. Portugal
made a policy that computer science and programming are compulsory for all
pupils in primary and secondary schools since 2017 [11]. Indonesia ministry of
education and culture (Kemdikbud) has issued a regulation to apply computer
science in middle school with the name of “Informatika” with the core is CT [12]
[13]. It means that the CT has internationally acknowledged and must be adopted
to prepare a better generation. Because the facts showed that this 21st century
society is massively computerized and a country with better computer science has
a more advanced level.
Teaching CT can be conducted through several approaches, such as visual
block programming [14], tangible programming[15], and games. Visual block
programming that potential for teaching CT are App Inventor, Alice, Construct 2,
Kodu, Blockly, and the most popular one is Scratch [1]. Game-based learning (GBL)
is one of other strategies that many researchers employed for learning CT. GBL
ISSN 2549-7286 (online)
Indonesian Journal of Computer Science Vol. 13, No. 2, Ed. 2024 | page 3629
refers to a strategy of utilizing games in the educational process to enhance
learning activities so that engage and motivate students to actively process
educational content and foster development process [16]. GBL allows students to
keep learn by interacting with a game and actively thinking while playing without
being aware of it, so that it may improve learning motivations, engagement, and
problem-solving. Noroozi et al., (2020) address that the GBL environments
supposed to aid users in gaining 21st century skills such as problem-solving,
decision making, critical thinking, analytical, critical, and argumentation [17].
The games used in the GBL approach can be set up in various platforms and
environments, such as computer desktops, web browsers, mobile applications,
augmented reality (AR), and virtual reality (VR). One innovative setting that may
enhance a student’s learning experience in a virtual environment to facilitate the
GBL is VR technology [18]. Educause Horizon’s report shows that one of the
emerging technologies and practices that is believed to have a significant impact
on the future of postsecondary teaching and learning is VR [19]. Rogers (2019)
also stated that VR is one learning aid for the 21st century that allows users to
retain more information and better apply what they have learned after using it
[20], [21]. One reason is that VR technologies can generate, visualize, and simulate
a virtual environment with 3D objects or artifacts and make users experience a
high degree of immersion so that users perceive that they are actually “there” [18].
Hence, it can be beneficial if VR technologies are utilized to foster CT skills through
GBL environment settings. However, the strategies to foster CT skills that harness
VR in GBL settings are still lacking. The majority of tools used for it are visual block
programming with Scratch [1]. This study aims to investigate the effect of a VR
game on the CT skills development.
B. Research Method
This study employs a design and development research (DDR) method to
achieve the objectives. DDR is the systematic study of design, development, and
evaluation processes to establish an empirical basis for creating instructional and
non-instructional products, tools, and new or enhanced models that govern their
development [22], [23]. This aligns with the professional suggestions in
instructional design and technology that facilitate learning and improve
performance by creating, using, and managing appropriate instructional and non-
instructional interventions.
DDR method can be categorized into two main types: (1) research on
products and tools and (2) research on design and development[23]. This study
suits the first type, research on products and tools, where the product that will be
developed is a software application of VR, which is considered a tool for learning.
The entire design and development process can be documented through a model of
analysis, design, development, implementation, and evaluation (ADDIE). Figure 1
shows the research design in the ADDIE model.
In the initial phase of the DDR utilizing the ADDIE model, the analysis stage is
crucial for laying the foundational framework of the educational intervention. This
stage encompasses a comprehensive analysis of needs, where an evaluation is
conducted to identify the specific requirements and challenges that the
intervention aims to address. Following this, learning objectives are defined,
ISSN 2549-7286 (online)
Indonesian Journal of Computer Science Vol. 13, No. 2, Ed. 2024 | page 3630
establishing clear, measurable goals that the educational tool intends to achieve in
terms of learner outcomes. Furthermore, determining the target audience is
another activity conducted in the analysis phase. These steps used to ensure that
the development of the educational tool is precisely aligned with the learners'
requirements and the educational goals.
Figure 1. Research design employed
Transitioning to the design phase, attention shifts towards the technical
assembly of the educational tool, specifically focusing on GBL interventions using
aVR game. In this context, game mechanics are thoughtfully devised to ensure
engaging, interactive, and pedagogically sound experiences that effectively
facilitate learning objectives. Table 1 shows the design of game mechanics
implemented in the developed game. Storyboards are crafted, outlining the visual
and narrative flow of the game, thereby serving as a blueprint for the development
process. Additionally, the design of game algorithms plays a critical role in this
phase, underpinning the logic, decision-making processes, and dynamic elements
within the game. These algorithms are essential for creating adaptive learning
environments that respond to the learner's actions and progress. Together, these
components of the design phase are instrumental in shaping a compelling and
educationally effective game-based learning tool, bridging the gap between
educational theory and the immersive experience of VR gaming.
Table 1. Design of game mechanics
Game Mechanics
Description
System creates a grid
map
The game system creates a grid map that contains obstacles, a heart icon,
and the locations of the start and finish points randomly.
System generates
cubes
The system generates cubes with random positions and flying towards the
player. The generated cubes contain an arrow with a certain direction.
Player makes a route
The game system permits players to make a route from the start to the
finish point by slashing a cube. The arrow direction embedded in the cube
determines the pathway.
Player slashing a cube
Cubes that are randomly generated by the system can be slashed by
players using lightsabers.
Scoring system
Players get a point when slashing a cube. The maximum score for each
level is 100.
Double score system
Players get a double point when the route created passes through a heart
icon.
Game accomplished
The game is accomplished when a player makes a route from start to finish
and then continues to the next level.
Leveling system
The game consists of six levels with different maps and obstacles at each
level.
ISSN 2549-7286 (online)
Indonesian Journal of Computer Science Vol. 13, No. 2, Ed. 2024 | page 3631
Game Mechanics
Description
Timer system
The timer is set based on the length of the audio clip used as the back
sound.
Game over
The game is over in two conditions, time is over or the route created by the
player is hitting an obstacle.
The development stage involves the creation of a prototype alongside the
necessary instruments for evaluation. This phase is characterized by the actual
construction of the educational tool, where theoretical designs and algorithms are
transformed into a tangible, interactive product. The development of a prototype is
a critical step, allowing for the preliminary testing and refinement of the
educational intervention. Concurrently, the development of instruments to
measure the effectiveness of the intervention, namely assessment tasks in pretest
and posttest forms. The instruments are used to evaluate the intervention’s impact
on learning outcomes.
In the implementation phase, users are engaged in an experimental setup
employing a quasi-experimental design, which includes both pretest and posttest
measures to ascertain the intervention's effectiveness. This design allows for the
examination of learning outcomes before and after the intervention, providing
insights into the educational tool's impact on the target audience's knowledge and
skills. Following this, the evaluation phase conducts a comprehensive analysis of
the collected data, utilizing statistical methods to assess the extent to which the
intervention meets the defined learning objectives. This stage is used to measure
the effectiveness of the VR game tool that has been developed and the
implemented learning approach, identifying areas for improvement, and informing
future iterations of the development process.
Table 2. Demographic information of participants involved
Info
n
%
Gender
Male
25
67.6
Female
12
42.4
Total
37
100
Age
19-20
19
51.4
21-22
16
43.2
23-24
2
5.4
Total
37
100
Have ever learned CT?
Yes
31
83.8
No
6
16.2
Total
37
100
Have ever used VR devices?
Yes
24
64.9
No
13
35.1
Total
37
100
Table 2 shows the demographic information for the participants in this study.
The gender distribution of the 37 participants is predominantly male, constituting
67.6% (n=25), while females make up 32.4% (n=12). In terms of age, the majority
ISSN 2549-7286 (online)
Indonesian Journal of Computer Science Vol. 13, No. 2, Ed. 2024 | page 3632
of participants are between the ages of 19 and 20, accounting for 51.4% (n=19) of
the sample. Those aged 21 to 22 represent 43.2% (n=16), and a smaller portion,
5.4% (n=2), are between the ages of 23 and 24. When they asked about CT, a
significant majority of participants, 83.8% (n=31), had previously learned about
CT, which is the subject matter of the VR game. In contrast, 16.2% (n=6) have not
had such an educational experience. Additionally, familiarity with the technology
used in the intervention is relatively high; 64.9% (n=24) of the participants have
used VR devices before, while 35.1% (n=13) have not, indicating a good level of
readiness in terms of engagement with the virtual environment used for the
developed VR game.
C. Result and Discussion
1. Game Development Features
The virtual reality (VR) game developed for this research, named "CT Saber,"
is engineered using the Unity game engine, specifically version 2020.2.7f1. The
software development kit (SDK) used was Oculus Integration version 34.0,
released in November 2021, then updated regularly when the notification came up.
This game is designed to be compatible exclusively with VR devices such as the
Oculus or Meta Quest and their variants, leveraging the immersive capabilities of
these platforms to facilitate a unique learning experience. The title "CT Saber"
amalgamates the concept of Computational Thinking (CT) with an homage to the
popular Oculus game, Beat Saber. Inheriting the core game mechanics from Beat
Saber, where players slice through floating cubes directed at them, CT Saber
introduces a novel educational twist. Beyond the visceral action of slicing cubes, CT
Saber compels players to engage in computational thinking by navigating the best
possible route to a designated endpoint on a grid map. This requires players to
strategize their movements from the starting point, circumventing obstacles to
reach the target destination successfully. This integration of computational
problem-solving into the gameplay mechanics not only enriches the player's
engagement with the game but also fosters the development of critical thinking
skills in a fun and interactive environment.
Figure 2 illustrates the gaming environment present within the VR game CT
Saber. The development of this environment is grounded in the design of game
mechanics outlined in Table 1, such as the system for creating a grid map filled
with obstacles, a heart icon, and designated start and finish locations. The
appearance of these elements is generated randomly to foster a diverse map
variety with each gameplay session. Additionally, the system incorporates a timer
to establish a maximum duration within which players are required to complete
the given mission. This design strategy not only challenges the players' strategic
planning and problem-solving skills, which are essential components of CT, but
also introduces an element of urgency to enhance engagement and simulate real-
world problem-solving scenarios. Through this intricate combination of
randomized map generation and time constraints, CT Saber aims to provide an
immersive and intellectually stimulating experience, pushing players to adapt their
strategies and decision-making processes continuously.
The VR game CT Saber is structured across six distinct levels, each featuring a
unique map design while maintaining consistent gameplay mechanics. As depicted
ISSN 2549-7286 (online)
Indonesian Journal of Computer Science Vol. 13, No. 2, Ed. 2024 | page 3633
in Figure 3, the game concludes with one of two possible outcomes: accomplished
or failed. Figure 3(a) illustrates the scenario in which the player successfully
completes the mission, signified by a smiling emoticon, indicating a successful
navigation through the game's challenges to reach the intended goal. Conversely, a
failed attempt is represented by a sad emoticon, as shown in Figure 3(b), signaling
the player's inability to meet the game's goals.
Figure 2. Game environment in the CT Saber
Upon the completion of each game session, a pop-up menu appears,
providing a comprehensive summary of the player's performance. This summary
includes the score achieved, the number of steps taken, the optimal route that
could have been followed, the highest score attained, and various other selectable
options. Notably, if the player fails to achieve the level's target, the option to
proceed to the next level is withheld. Instead, the player is presented with a choice
to either retry the same level or return to the main menu. This mechanism ensures
that players are adequately challenged to develop and apply effective strategies,
reinforcing the game's educational objective to enhance computational thinking
skills through engaging and iterative gameplay experiences.
Among the six existing levels of the game, two quizzes are included, adopted
from the Bebras Challenge available on bebras.or.id. These quizzes serve a dual
purpose: to embed an evaluative component within the gameplay to assess the
players' CT skills and to provide a seamless integration of assessment within the
learning trajectory. The positioning of quizzes is deliberately placed after the
completion of levels two and four, serving as checkpoints that gauge the
ISSN 2549-7286 (online)
Indonesian Journal of Computer Science Vol. 13, No. 2, Ed. 2024 | page 3634
progression of CT abilities developed through interaction with the game. By
utilizing questions from a recognized platform like the Bebras Challenge, the game
ensures that the evaluation of CT capabilities is both standardized and reflective of
an internationally benchmarked understanding of these skills. This approach not
only enriches the gaming experience with an educational assessment but also
aligns the player's in-game progress with tangible learning outcomes in CT.
(a) (b)
Figure 3. The Game ends in two conditions: (a) accomplished, and (b) failed
2. Effectiveness of the Game
The effectiveness of the CT Saber VR game as a tool for learning CT was
statistically evaluated using SPSS software. The first step in the data analysis
process involved assessing the distribution of the data to determine normality.
This preliminary examination is critical as it dictates the appropriate statistical
methods for subsequent analysis. If the data were found to be normally
distributed, parametric tests could be applied with greater confidence in their
assumptions. However, in the presence of non-normal data, alternative non-
parametric methods would be warranted.
Table 3 presents the results of normality tests conducted on the scores from
a pretest and posttest using the SPSS statistical package. The Kolmogorov-Smirnov
and Shapiro-Wilk tests are both utilized to evaluate the distribution of the data
against a normal distribution. For the pretest scores, the Kolmogorov-Smirnov
statistic is 0.164 with a significance level (p-value) of 0.014, and the Shapiro-Wilk
statistic is 0.929 with a significance level of 0.02. In the case of the posttest scores,
the Kolmogorov-Smirnov statistic is 0.228 with a significance level effectively at 0,
while the Shapiro-Wilk statistic is 0.906 with a significance level of 0.004. The
degrees of freedom (df) for both tests are 37, corresponding to the sample size.
Both tests' significance levels for pretest and posttest scores are below the
common alpha level of 0.05, suggesting that the scores do not follow a normal
distribution. The Lilliefors Significance Correction indicates that a correction has
been applied to the Kolmogorov-Smirnov test, enhancing its accuracy for small
ISSN 2549-7286 (online)
Indonesian Journal of Computer Science Vol. 13, No. 2, Ed. 2024 | page 3635
sample sizes. These results imply that non-parametric statistical techniques are
more appropriate for further analysis of the CT Saber VR game's effectiveness in
learning CT.
Table 3. Normality test
Categories
Kolmogorov-Smirnova
Shapiro-Wilk
Statistic
df
Sig.
Statistic
df
Sig.
Scores
Pretest
.164
37
.014
.929
37
.020
Posttest
.228
37
.000
.906
37
.004
a. Lilliefors Significance Correction
Given the non-normal distribution of data, the Wilcoxon Signed-Rank Test
was utilized as the non-parametric method for statistical analysis. Table 4 presents
the ranked data from the Wilcoxon Signed-Rank Test, showing the distribution of
differences between post-test and pretest scores. The table indicates that 2
participants had negative ranks (mean rank = 7.50, sum of ranks = 15.00),
signifying that their posttest scores were lower than their pretest scores.
Conversely, 28 participants exhibited positive ranks (mean rank = 16.07, sum of
ranks = 450.00), indicating higher posttest scores relative to their pretest scores.
The presence of 7 ties suggests that for these participants, there was no change
between their pretest and posttest scores. The total number of participants
included in this analysis is 37.
Table 4. Ranked data from the Wilcoxon signed-rank test
N
Mean Rank
Sum of Ranks
Posttest - Pretest
Negative Ranks
2a
7.50
15.00
Positive Ranks
28b
16.07
450.00
Ties
7c
Total
37
a. Posttest < Pretest
b. Posttest > Pretest
c. Posttest = Pretest
Table 5 details the test statistics from the Wilcoxon Signed-Rank Test applied
to the posttest-pretest score differences. A Z-score of -4.496, with an asymptotic
significance (2-tailed) of 0, reflects a highly significant difference in scores. The
significance level of 0 indicates that the probability of this result occurring by
chance is extremely low, thus confirming the presence of a statistically significant
change in scores after the intervention. This result was based on the negative
ranks, as indicated by the notation 'b' in the table, which signifies that the analysis
was influenced by the participants who scored lower on the posttest than on the
pretest. The evidence provided by Table 5 supports the conclusion that the
intervention, presumably the CT Saber VR game, had a significant impact on the
participants' Computational Thinking skills.
Table 5. Wilcoxon signed-rank test results
Posttest - Pretest
Z
-4.496b
Asymp. Sig. (2-tailed)
.000
a. Wilcoxon Signed Ranks Test
b. Based on negative ranks.
ISSN 2549-7286 (online)
Indonesian Journal of Computer Science Vol. 13, No. 2, Ed. 2024 | page 3636
3. Discussion
This section discusses the implications and significance of the observed
statistical outcomes derived from deploying the CT Saber VR game as an
educational tool for fostering CT skills. The Wilcoxon Signed-Rank Test results
elucidate a substantial difference in CT competencies as evidenced by pretest and
posttest evaluations, indicating the potential efficacy of the VR intervention. This
segment examines the data to interpret the impact of the game mechanics and
immersive VR environment on the participants' CT abilities. We align our findings
with the broader narrative of digital learning, drawing on the theoretical
underpinnings of CT education and the empirical evidence from prior research in
the field. The focus is to contextualize the significance of these findings in
contributing to the evolving landscape of educational technologies and
instructional methods, particularly in CT skills development.
The findings from the present study provide evidence for the efficacy of the
CT Saber VR game as a pedagogical tool in enhancing CT skills. The statistically
significant improvement in post-test scores suggests that immersive virtual reality
environments can be conducive to learning and skill development, in line with
Mayer's Cognitive Theory of Multimedia Learning, which posits that rich
multimedia environments can enhance learning by providing multiple channels for
information processing [24].
The implications of these results are multifaceted. Firstly, the study
contribute to the burgeoning field of educational technology by supporting the
integration of VR gaming as an effective instructional strategy, especially in
disciplines that require abstract and systematic thinking. Secondly, these findings
hold practical significance for educators and curriculum developers seeking
innovative methods to teach complex cognitive skills. The ability of CT Saber to
engage learners in an active and experiential learning process showcases the
potential of VR to facilitate higher-order thinking skills.
Underlying these implications is the theoretical mechanism supported by the
Cognitive Load Theory, which argues that learning is optimized when cognitive
resources are efficiently used [25]. The CT Saber game's mechanics, requiring
players to strategize and make rapid decisions, may have facilitated cognitive
schema formation, thereby fostering CT skills without overloading the learners'
cognitive capacities. Furthermore, the constructivist learning theory, which
emphasizes learning as an active, constructive process, is echoed in the game's
design, which allows learners to construct knowledge through interactive
problem-solving tasks, reinforcing the connection between educational theory and
the observed outcomes of the study.
While yielding insightful results regarding the efficacy of the CT Saber VR
game in fostering CT skills, the present study has limitations. One of the primary
constraints is the sample size, which, although adequate for initial explorations,
may not fully represent the diverse populations that could benefit from such
educational interventions. Additionally, the study’s scope is limited by the singular
use of the VR platform, which may not account for the variance in technological
familiarity among different demographic groups. This focus on a specific VR device
ISSN 2549-7286 (online)
Indonesian Journal of Computer Science Vol. 13, No. 2, Ed. 2024 | page 3637
also narrows the generalizability of the findings to similar environments and user
experiences.
Further research is suggested to expand the demographic inclusivity of the
sample, potentially providing a broader understanding of the CT Saber VR game’s
impact across various age groups, educational backgrounds, and technological
proficiencies. Future studies could also benefit from a longitudinal design,
assessing the retention of CT skills over time post-intervention, which was beyond
the scope of the current study. Moreover, comparing the VR-based learning
approach with traditional or alternative educational methods could yield richer
insights into the optimal contexts and conditions for teaching CT concepts. Finally,
qualitative data, such as participant feedback and observational notes, would
complement the quantitative data, offering a more nuanced perspective on the
user experience and the learning process within the VR environment.
D. Conclusion
The investigation into the CT Saber VR game's impact on CT abilities has
culminated in several noteworthy conclusions. The statistical analysis, conducted
through the Wilcoxon Signed-Rank Test due to the non-normal distribution of the
data, has indicated a significant effect of the game on participants' CT skills. This
suggests that engaging with the VR environment of CT Saber can effectively
enhance certain aspects of CT. The incorporation of elements from the Bebras
Challenge into the game's design is particularly promising, as it appears to
contribute to the observed improvement in CT abilities. However, it is important to
acknowledge the study's limitations, including its sample size and the use of a
single type of VR device, which may influence the extent to which these findings
can be generalized.
In light of these results, it can be inferred that VR technology has the potential
to serve as a valuable tool in the realm of educational interventions aimed at
developing CT skills. CT Saber demonstrates the viability of using game-based
learning to not only engage learners but also to deliver educational content in a
manner that is both effective and enjoyable. Future research should continue to
explore this avenue, expanding on the diversity of participants and examining
long-term learning outcomes to fully ascertain the educational value of VR in
fostering essential 21st-century skills.
E. Acknowledgment
Thanks to the Rector of Universitas Muhammadiyah Surakarta (UMS) and the
Dean of the Faculty of Teacher Training and Education UMS for supporting this
study. The study is fully funded by Universitas Muhammadiyah Surakarta. This
study is a part Doctor of Philosophy program in Information Technology
Education, Faculty of Computing & Meta-Technology, Universiti Pendidikan Sultan
Idris (UPSI), Malaysia.
F. References
[1] L. Zhang and J. Nouri, “A systematic review of learning computational
thinking through Scratch in K-9,” Comput. Educ., vol. 141, p. 103607, 2019,
ISSN 2549-7286 (online)
Indonesian Journal of Computer Science Vol. 13, No. 2, Ed. 2024 | page 3638
doi: https://doi.org/10.1016/j.compedu.2019.103607. [Online]. Available:
http://www.sciencedirect.com/science/article/pii/S0360131519301605
[2] M. Jou, P.-C. Chen, and J. Wang, “The developmental characteristics of
computational thinking and its relationship with technical skills: taking the
department of engineering as an example,” Interact. Learn. Environ., vol. 31,
no. 6, pp. 3380–3395, Aug. 2023, doi: 10.1080/10494820.2021.1928236.
[Online]. Available: https://doi.org/10.1080/10494820.2021.1928236
[3] J. Nouri, L. Zhang, L. Mannila, and E. Norén, “Development of computational
thinking, digital competence and 21st century skills when learning
programming in K-9,” Educ. Inq., vol. 11, no. 1, pp. 1–17, Jan. 2020, doi:
10.1080/20004508.2019.1627844. [Online]. Available:
https://doi.org/10.1080/20004508.2019.1627844
[4] J. Cuny, L. Snyder, and J. M. Wing, “Demystifying computational thinking for
non-computer scientists,” Unpubl. Manuscr. progress, Ref. http//www. cs. C.
edu/~ CompThink/resources/TheLinkWing. pdf, 2010.
[5] S. Papert, Mindstorms: Children, computers, and powerful ideas. Basic Books,
Inc., 1980.
[6] T.-C. Hsu, S.-C. Chang, and Y.-T. Hung, “How to learn and how to teach
computational thinking: Suggestions based on a review of the literature,”
Comput. Educ., vol. 126, pp. 296–310, 2018, doi:
https://doi.org/10.1016/j.compedu.2018.07.004. [Online]. Available:
http://www.sciencedirect.com/science/article/pii/S0360131518301799
[7] J. M. Wing, “Computational Thinking,” Commun. ACM, vol. 49, no. 3, pp. 33–
35, Mar. 2006, doi: 10.1145/1118178.1118215. [Online]. Available:
http://doi.acm.org/10.1145/1118178.1118215
[8] J. Wing, “Computational Thinking Benefits Society,” 2014. [Online].
Available: http://socialissues.cs.toronto.edu/index.html. [Accessed: Jan. 21,
2020]
[9] S. Carolyn, “Computational thinking for all,” International Society for
Technology in Education (ISTE), 2014. [Online]. Available:
https://www.iste.org/explore/Solutions/Computational-thinking-for-all.
[Accessed: Dec. 20, 2019]
[10] F. Heintz, L. Mannila, and T. Färnqvist, “A review of models for introducing
computational thinking, computer science and computing in K-12
education,” in 2016 IEEE Frontiers in Education Conference (FIE), Oct. 2016,
pp. 1–9, doi: 10.1109/FIE.2016.7757410.
[11] P. João, D. Nuno, S. F. Fábio, and P. Ana, “A Cross-analysis of Block-based and
Visual Programming Apps with Computer Science Student-Teachers,” Educ.
Sci., vol. 9, no. 3, 2019, doi: 10.3390/educsci9030181. [Online]. Available:
https://www.mdpi.com/2227-7102/9/3/181
[12] D. Kemdikbud, Kesiapan Sekolah Menengah Pertama dan Sekolah Menengah
Atas dalam Menerapkan Informatika sebagai Mata Pelajaran pada Tahun
2019/2020. Indonesia, 2019.
[13] S. SUKIRMAN, L. F. M. IBHARIM, C. S. SAID, and B. MURTIYASA, “A Strategy of
Learning Computational Thinking through Game Based in Virtual Reality:
Systematic Review and Conceptual Framework,” Informatics Educ., vol. 21,
no. 1, pp. 179–200, Jun. 2022, doi: 10.15388/infedu.2022.07. [Online].
ISSN 2549-7286 (online)
Indonesian Journal of Computer Science Vol. 13, No. 2, Ed. 2024 | page 3639
Available: https://infedu.vu.lt/doi/10.15388/infedu.2022.07
[14] Sukirman, D. A. Pramudita, A. Afiyanto, and Utaminingsih, “Block-Based
Visual Programming as a Tool for Learning the Concepts of Programming for
Novices,” Int. J. Inf. Educ. Technol., vol. 12, no. 5, pp. 365–371, 2022, doi:
10.18178/ijiet.2022.12.5.1628.
[15] S. Trilles and C. Granell, “Advancing preuniversity students’ computational
thinking skills through an educational project based on tangible elements
and virtual block-based programming,” Comput. Appl. Eng. Educ., vol. 28, no.
6, pp. 1490–1502, Nov. 2020, doi: https://doi.org/10.1002/cae.22319.
[Online]. Available: https://doi.org/10.1002/cae.22319
[16] V. Trajkovik, T. Malinovski, T. Vasileva-Stojanovska, and M. Vasileva,
“Traditional games in elementary school: Relationships of student’s
personality traits, motivation and experience with learning outcomes,” PLoS
One, vol. 13, no. 8, p. e0202172, Aug. 2018 [Online]. Available:
https://doi.org/10.1371/journal.pone.0202172
[17] O. Noroozi, H. Dehghanzadeh, and E. Talaee, “A systematic review on the
impacts of game-based learning on argumentation skills,” Entertain. Comput.,
vol. 35, p. 100369, 2020, doi:
https://doi.org/10.1016/j.entcom.2020.100369. [Online]. Available:
http://www.sciencedirect.com/science/article/pii/S187595212030077X
[18] J. Radianti, T. A. Majchrzak, J. Fromm, and I. Wohlgenannt, “A systematic
review of immersive virtual reality applications for higher education: Design
elements, lessons learned, and research agenda,” Comput. Educ., vol. 147, p.
103778, 2020, doi: https://doi.org/10.1016/j.compedu.2019.103778.
[Online]. Available:
http://www.sciencedirect.com/science/article/pii/S0360131519303276
[19] M. Brown et al., “2020 Educause Horizon Report Teaching and Learning
Edition,” Louisville, CO, 2020 [Online]. Available:
https://www.learntechlib.org/p/215670
[20] S. Rogers, “Virtual Reality: The Learning Aid Of The 21st Century,” Forbes,
2019. [Online]. Available:
https://www.forbes.com/sites/solrogers/2019/03/15/virtual-reality-the-
learning-aid-of-the-21st-century/#2423860a139b. [Accessed: Jan. 23, 2020]
[21] E. Krokos, C. Plaisant, and A. Varshney, “Virtual memory palaces: immersion
aids recall,” Virtual Real., vol. 23, no. 1, pp. 1–15, 2019, doi: 10.1007/s10055-
018-0346-3. [Online]. Available: https://doi.org/10.1007/s10055-018-
0346-3
[22] R. C. Richey and J. D. Klein, Design and Development Research: Methods,
Strategies, and Issues. Mahwah, New Jersey: Lawrence Erlbaum Associates,
2007.
[23] R. C. Richey and J. D. Klein, “Design and Development Research BT -
Handbook of Research on Educational Communications and Technology,” J.
M. Spector, M. D. Merrill, J. Elen, and M. J. Bishop, Eds. New York, NY:
Springer New York, 2014, pp. 141–150 [Online]. Available:
https://doi.org/10.1007/978-1-4614-3185-5_12
[24] R. E. Mayer, “The Past, Present, and Future of the Cognitive Theory of
Multimedia Learning,” Educ. Psychol. Rev., vol. 36, no. 1, p. 8, 2024, doi:
ISSN 2549-7286 (online)
Indonesian Journal of Computer Science Vol. 13, No. 2, Ed. 2024 | page 3640
10.1007/s10648-023-09842-1. [Online]. Available:
https://doi.org/10.1007/s10648-023-09842-1
[25] J. Sweller, “The Development of Cognitive Load Theory: Replication Crises
and Incorporation of Other Theories Can Lead to Theory Expansion,” Educ.
Psychol. Rev., vol. 35, no. 4, p. 95, 2023, doi: 10.1007/s10648-023-09817-2.
[Online]. Available: https://doi.org/10.1007/s10648-023-09817-2
