Content uploaded by Mladjan Jovanovic
Author content
All content in this area was uploaded by Mladjan Jovanovic on Jul 06, 2024
Content may be subject to copyright.
96
SINTEZA 2024
INTERNATIONAL SCIENTIFIC CONFERENCE ON INFORMATION TECHNOLOGY, COMPUTER SCIENCE, AND DATA SCIENCE
Sinteza 2023
submit your manuscript | sinteza.singidunum.ac.rs DOI: 10.15308/Sinteza-2024-
96-102
EXPLORING DECISION-MAKING IN SERIOUS GAMES VS.
TRADITIONAL SURVEYS: COMPARATIVE STUDY OF MEDIUM
EFFECTS ON RISK ASSESSMENT
Abstract:
This study evaluates the impact of experimental mediums — serious games
and traditional online surveys — on decision-making processes, risk assessment
and the risky choice framing effect bias. Conducted with 77 participants from
the University of Belgrade, the experiment utilized a serious game developed in
Unity Engine and a standard text-based survey, presented in a counterbalanced
experimental design to assess participant responses in different mediums. The
primary aims were to compare how these mediums affect decision-making
times and responses and to test the validity of serious games for cognitive
research. Response times and decision patterns were analysed using paired
T-tests and ANOVA, revealing no significant differences between the two
mediums. These results suggest that serious games provide an experiential depth
comparable to traditional surveys, maintaining consistent decision-making
outcomes. The study underscores the potential of serious games as a robust
platform for psychological research, capable of simulating decision-making
environments while preserving the integrity of experimental conditions.
Future research should focus on enhancing game realism and participant
engagement to possibly uncover more subtle distinctions in decision-making
behaviour across these mediums. This research confirms the suitability of
serious games for exploring complex cognitive processes, setting a foundation
for their broader application in scientific studies.
Keywords:
Decision making, Human-computer interaction, Serious games, Risky choice
framing effect.
INTRODUCTION
Technological developments in the last decades have introduced new
mediums of human-computer interaction. One of the notable mediums
that has gained popularity since its early development is computer video
games. In the current digital age video games, although primarily devel-
oped and distributed for the purpose of entertainment, oer an interac-
tive experience towards a clear goal, based on a set of agreed rules and
constraints, oen accompanied by challenges and constant feedback. [1]
is tight feedback loop, usually immediate and common to both regu-
lar and video games, oers a unique experience to the players in which
they challenge themselves to overcome certain goals. One way of using
this rich medium for non-entertainment purposes is by implementing
INFORMATION TECHNOLOGY SESSION
Sara Knežević1*,
[0009-0006-3698-287X]
Kaja Damnjanović2,
[0000-0002-9254-1263]
Mlađan Jovanović1
[0000-0003-2355-9424]
1Singidunum University,
Belgrade, Serbia
2Laboratory for Experimental Psychology,
Department of Psychology,
Faculty of Philosophy,
University of Belgrade,
Belgrade, Serbia
Correspondence:
Sara Knežević
e-mail:
lost.func@gmail.com
97
Sinteza 2024
submit your manuscript |
sinteza.singidunum.ac.rs
Information Technology Session
SINTEZA 2024
INTERNATIONAL SCIENTIFIC CONFERENCE ON INFORMATION TECHNOLOGY, COMPUTER SCIENCE, AND DATA SCIENCE
pedagogy as subordinate to the story, described by Mi-
chael Zyda as: “Serious game: a mental contest, played
with a computer in accordance with specic rules, that
uses entertainment to further government or corporate
training, education, health, public policy, and strategic
communication objectives.” [2] A systematic literature
review done by Rissanen et al. [3] establishes the role
of serious games in knowledge and skill acquisition, as
well as behaviour modication [4]. Such cognitive or
educational objectives, embedded within entertaining
gameplay and game design, create grounds for simulat-
ing complex decision-making scenarios.
Although primarily used in education and corpo-
rate training [5], [6], [7], serious games have potential
in other elds as well. e one explored in this paper
relates to cognitive science and the intricacies of risky
decision-making processes and the risky choice framing
eect bias. e goal is to compare standard text-only
online surveys, traditionally employed in cognitive psy-
chology “on paper”, and the implementation of the same
survey in a virtual environment in the form of a seri-
ous game. By conducting an experiment in which par-
ticipants take a basic decision-making questionnaire in
both mediums, we compared the validity of conducting
experiments through serious games. One of the main
reasons why we chose to use a serious game medium to
conduct the survey is because of the engagement aspect
of computer video games. Providing well-designed lev-
els, serious games can oer a dynamic platform for cre-
ating a more immersive environments to explore risky
decision making. Level design is essential for creating
immersive experiences that integrate game mechanics
with the storytelling and the visual environment, foster-
ing greater emotional and cognitive engagement with
the player than simple text on a screen. [8]
e game design principle in serious games oers a
platform to explore the risky choice framing eect and
prospect theory within the context of decision-making.
Prospect theory, developed by Kahneman and Tversky,
suggests that individuals evaluate potential losses and
gains dierently, leading to the dependence of the deci-
sions on how choices are presented or "framed". [9] Ex-
amples of positive and negative framing, along with their
certain and uncertain or risky choices, are shown in Table
1. e reversal in the preference of risk due to the dier-
ent descriptions of the same choices is dubbed as risky
choice framing. By developing game levels that simulate
decision-making scenarios with dierent framing and
providing players with immediate feedback on their de-
cisions (i.e. losing health points or gaining gold coins),
serious games provide an immersive environment to
observe and understand these cognitive biases in action.
is level design approach enables players to experience
the consequences of their choices in a controlled setting,
illustrating the impact of loss aversion and the inuence
of positive or negative framing on decision-making pro-
cesses. We propose that serious games, through purposely
designed levels, can be an eective tool for studying the
risky choice framing eect, decision-making processes,
and other cognitive biases practically and engagingly. is
study aims to make the rst step towards that idea by ex-
ploring if serious games are as valid as online surveys for
conducting decision-making questionnaires.
2. METHOD
2.1. GAME DEVELOPMENT AND DESIGN
e serious video game was developed in Unity
Engine [10], and is an upgraded version of the pilot
game [11], with the goal of presenting participants with
seven decision-making scenarios involving risk, requir-
ing choosing between certain and risky choices. e
game aimed to create a step towards a more immersive
experience, given that players would face immediate
consequences of their choices within the simulated 3D
fantasy environment. e complete game interface was
in Serbian and included a consent form in the beginning
for data collection, including response times, question
responses, age, gender, and educational level.
Table 1. Examples of positive and negative framing, and their corresponding certain and risky choices.
Certain choice Risky choice
Positive Framing
Guaranteed receipt of €5000, enhancing your
nancial stability and enabling potential
investments.
50% chance of receiving €10 000, doubling your
nancial gain, and signicantly boosting your
investment opportunities.
Negative Framing
Immediate reduction of nancial uncertainties
with only €5000, possibly insucient for larger
plans or emergencies.
50% chance of receiving nothing, potentially
missing out on nancial growth and necessary
funding.
98
Sinteza 2024
submit your manuscript |
sinteza.singidunum.ac.rs
Information Technology Session
SINTEZA 2024
INTERNATIONAL SCIENTIFIC CONFERENCE ON INFORMATION TECHNOLOGY, COMPUTER SCIENCE, AND DATA SCIENCE
Central to the game’s mechanism was a decision-
making questionnaire, reviewed and approved by cog-
nitive scientists, made up of seven tasks (questions)
presented by non-playable characters. ese tasks are
created to t the narrative of the game, as the tasks them-
selves tell the game story. e player’s (participant’s)
character is a traveling adventurer in this fantasy world,
and the choices they make immediately aect them, and
the consequences are provided visually, audibly, and
concerning the story. In this way, we personalized the
reference point for gains or losses. is personalized
approach aimed to bring the risk assessment closer to
the player, in comparison to the usual way of completing
these questionnaires – by reading questions and trying
to imagine the hypothetical scenario in their mind.
e game’s environment is thematically consist-
ent, extending to the dialogue and interactions, which
are designed to immerse the player in scenarios that
enhance the relevance of the tasks to the player's char-
acter. is game design choice aimed to contextualise
the decision and its consequence. e game’s archi-
tecture is strict in its task ordering and sequence. For
instance, completing the third task allows for initiating
the fourth task. is decision was made to make sure
that the online and gamied versions show the ques-
tions in the same sequence, as well as to ensure no logi-
cal fallacies are created within the story (e.g. interacting
with the doctor before you are ill). is progression is
crucial in illustrating the consequences of the player’s
decisions, which are reinforced by in-game metrics like
health points and gold coins, which alter based on the
player’s choices.
rough this design and structured implementation,
the game serves as a dynamic platform for exploring
decision-making processes and the framing eect. Situ-
ating theoretical concepts in a vivid, interactive envi-
ronment, can help illustrate how framing can inuence
decisions. is approach could theoretically not only en-
gage players on a deeper level but also provide insights
into human cognition and decision-making behaviours.
2.2.DESIGNING DECISION-MAKING SCENARIOS
e questionnaire consists of seven questions (tasks).
Each question was developed with both a positive and
negative framing, and the nal version of the question-
naire in both mediums included the following framings,
in question order: negative (N), positive (P), negative
(N), positive (P), negative (N), negative (N), and positive
(P). is was done to strategically inuence the player's
decision-making process. Each question oers a choice
between a certain choice and a risky one, regardless of
the framing, challenging players to weigh their decisions
within the context provided and embodying the essence
of risk assessment in human cognition.
e nal version of the questionnaire, with its
xed sequence of framed questions, serves as a direct
investigation into the framing eect's inuence on de-
cision-making, providing a novel comparison point to
conventional survey techniques. By integrating simple
game mechanics and narrative elements, this game aims
to validate the eectiveness of game-based simulations
as a viable and innovative method for conducting psy-
chological research on decision-making under risk. e
questionnaire was the same across both mediums.
However, the decisions that the players make di-
rectly aect their in-game character in terms of health
points or the amount of gold coins they have – making
their decisions in a similar vein to the standard risky
choice framing eect introductory questions regarding
economics or health.
Risk decision questions are comprised of two key
components: surface and deep structure. Deep structure
relates to the question itself presenting the situation in
which the decision maker is placed, the certain choice,
and the risky choice in probabilistic terms, while sur-
face structure represents anything that does not aect
the meaning of the deep structure, such as the risk type,
whether it is monetary, health-based, etc. [12]
e assumption then follows that the phrasing of
the question, as well as the phrasing and framing of the
available responses, will aect the decision-makers feel-
ing about the problem at hand and inuence their re-
lationship with risk by placing them into a position of
loss or gain. e expectation, in accordance to the pros-
pect theory, is that if the person is placed in a situation
of loss, they will respond in a way that will move their
reference point towards gain, making them risk-prone,
while placing them in a situation of gain will inuence
them to pick the choice that keeps the reference point
in place or towards more gain, making them risk-averse.
Emotions play a signicant role in decision-making and
that is why the framing eect is so eective, as it directly
aects choice based on what emotions are evoked within
the person responding to the questions.
99
Sinteza 2024
submit your manuscript |
sinteza.singidunum.ac.rs
Information Technology Session
SINTEZA 2024
INTERNATIONAL SCIENTIFIC CONFERENCE ON INFORMATION TECHNOLOGY, COMPUTER SCIENCE, AND DATA SCIENCE
2.3. DATA COLLECTION
In this study, the participants, all students at the Uni-
versity of Belgrade, were randomly assigned into two
groups that alternately responded to the questionnaire
(serious game and online questionnaire), with a dier-
ence of 7 days between conducting the tests. On the rst
day (phase 1), one group completed the questionnaire
by playing the serious game, and through the written
online form on the seventh day (phase 2). e other
group completed the questionnaires in the opposite or-
der. is design allowed each participant to experience
both forms of the questionnaire, maintaining identical
questions across both platforms to ensure consistency
in data collection. Both mediums provided unlimited
response time and mandatorily required responses to
continue to the next question. While the questions were
identical, the serious game provided the participants
with immediate feedback on the player’s decisions in
contrast to the survey where participants simply pro-
ceeded to the next question without any feedback. e
recorded data was the response (choice made) and re-
sponse time, as well as demographic info in the begin-
ning (gender, education level, age), preceded by a con-
sent form. e online survey was conducted using the
SoSciSurvey platform [13], and the game was distributed
as an executable le, ensuring ease of access and partici-
pation. In further analysis, the following variables are
taken into consideration: reaction time and response
choice as dependent variables, and type of medium as
the independent variable.
3. RESULTS
In total, 77 participants completed the survey across
both mediums – 77 participants across both phases
completed the gamied survey (90.28% women, mean
age 19.9 SD +/- 3.28), 72 participants in both phases
for online survey (83.12% women, mean age 20.38 SD
+/- 5.11). All participants completed the survey in both
mediums. Phase one included 35 participants responded
to the online survey, while 39 participants responded to
the gamied survey. Phase 2 included 37 participants in
the online survey, and 38 participants in the gamied
survey.
e average response time across both phases for the
online surveys was 31.15 seconds SD 6.16, and for the
gamied survey it was 29.53 seconds STD 5.28. Average
response times calculated across both phases for each
medium per question are shown in Figure 1. A single-
tailed paired T-test performed on the response speeds
PER QUESTION across both mediums and both phases
yields a value of 0.23 (p > 0.05), which is statistically not
signicant and shows that there is no signicant dier-
ence in response time speed for online and gamied sur-
veys. An ANOVA was conducted to compare response
times between participants taking an online survey and
those engaged with a gamied survey. e between-
groups sum of squares (SS) is 9.09, with 1 degree of
freedom (df), resulting in a mean square (MS) of 9.09.
Figure 1. Average response time for each question across both groups and both phases.
100
Sinteza 2024
submit your manuscript |
sinteza.singidunum.ac.rs
Information Technology Session
SINTEZA 2024
INTERNATIONAL SCIENTIFIC CONFERENCE ON INFORMATION TECHNOLOGY, COMPUTER SCIENCE, AND DATA SCIENCE
e within-group SS is considerably larger at 395.16
with 12 df, giving an MS of 32.93. e F-statistic, calcu-
lated by dividing the between-groups MS by the within-
groups MS, is 0.28, indicating the ratio of variance be-
tween the groups to the variance within the groups. e
critical F-value at an assumed common signicance level
of 0.05 for 1 and 12 degrees of freedom is 4.75. Since
the calculated F-value is much lower than the critical
F-value, and the p-value is 0.68 (p > 0.05), the results
fail to reject the null hypothesis. is suggests that there
is no signicant dierence in response times between
the online survey and gamied survey participants. e
ANOVA indicates that the gamied survey is just as
valid as the online survey in terms of response times.
is supports the hypothesis that gamied surveys can
be an eective alternative to traditional online surveys
for collecting response time data.
For all of the 7 questions, across both phases and
both mediums, participants had the choice between a
certain and risky choice, regardless of framing. We ex-
plored if there exists any signicant dierence between
the number of times the participants picked certain or
risky choices across both mediums and both phases. In
the gamied survey, for both phases, the total number
of risky responses was 240 (44.53% of all responses). e
individual question response distribution is shown in
Figure 2. In phase 1, the number of risky responses was
124 (45.42% of all phase 1 gamied survey responses). In
phase 2, the number of risky responses was 116 (43.6%
of all phase 2 gamied survey responses).
In the online survey, for both phases, the total num-
ber of risky responses was 218 (43.25% of all responses).
e individual question response distribution is shown
in Figure 3. In phase 1, the number of risky responses
was 105 (42.86% of all phase 1 online survey respons-
es). In phase 2, the number of risky responses was 113
(43.63% of all phase 2 online survey responses).
e chi-square statistic calculated for the sum across
phases of certain and risky responses across both me-
diums is 0.1714, and the associated p-value is 0.68. e
p-value (p > 0.05) indicates that the dierence in the
counts between the "Certain" and "Risky" responses for
both mediums is not statistically signicant. is sug-
gests that regardless of medium, participants decision
making process was consistent across two mediums,
that is that they responded in a similar pattern in terms
of choosing risky or certain choices in our survey.
Figure 2. Gamied survey, distribution of risky and certain responses for both groups and phases.
101
Sinteza 2024
submit your manuscript |
sinteza.singidunum.ac.rs
Information Technology Session
SINTEZA 2024
INTERNATIONAL SCIENTIFIC CONFERENCE ON INFORMATION TECHNOLOGY, COMPUTER SCIENCE, AND DATA SCIENCE
4. DISCUSSION
e immersive qualities of the video game do not
substantially alter the fundamental aspects of decision-
making processes as they relate to risk preference, as
suggested by the consistent patterns of participants’ re-
sponses in two mediums, in the simulated environment
and the online questionnaire. is outcome supports
the hypothesis that decision-making mechanisms are
robust across various presentation mediums. However,
it also highlights the necessity for further advancements
in simulation technology and game design to increase
the realism and engagement of such environments,
potentially uncovering nuanced dierences in future
research, most importantly considering engagement
as related to behaviour or cognition. [14] Because the
lack of signicant dierences in decision outcomes be-
tween the two environments, this research underscores
the potential of video games as a medium for examin-
ing decision-making processes. e immersive nature of
video games, which can simulate real-world experiences,
hold promise for advancing our understanding of how
individuals make decisions under risk.
Developing and enhancing the realism and en-
gagement of such environments could uncover more
nuanced dierences in decision-making processes in
future studies. e potential of video games not just as
a medium for posing questions but to observe decision-
making through gameplay dynamics itself also poses
a task for future research. For example, future serious
games might feature decision-making scenarios that do
not provide explicit instructions. Engaging respondents
more deeply in the simulation could lead to a greater
sense of responsibility for their decisions. Creating more
intricate and clearly dened questionnaires for study-
ing the framing eect, along with developing highly im-
mersive games, possibly in virtual reality (VR), could
enhance the study outcomes in terms of immersion and
engagement. However, our present ndings indicate a
promising beginning and demonstrate that games are
an eective method for conducting decision-making
experiments related to the framing eect, although there
is potential for further renement.
5. CONCLUSION
Our research demonstrates the consistency of
decision-making processes across two distinct mediums
and underscores the untapped potential of video games
for cognitive science research. Future work should aim
to enhance participant involvement and engagement in
simulations, which could provide deeper insights into
how people evaluate risks and make decisions. is may
reveal new angles on behaviour and cognition within
simulated settings. As we advance simulation technology
and game design, video games could not only serve to
explore decision-making psychology but also as a potent
instrument for inuencing it.
Figure 3. Online survey, distribution of risky and certain responses for both groups and phases.
102
Sinteza 2024
submit your manuscript |
sinteza.singidunum.ac.rs
Information Technology Session
SINTEZA 2024
INTERNATIONAL SCIENTIFIC CONFERENCE ON INFORMATION TECHNOLOGY, COMPUTER SCIENCE, AND DATA SCIENCE
6. REFERENCES
[1] P. Wouters, C. Van Nimwegen, H. Van Oosten-
dorp, and E. D. Van Der Spek, “A meta-analysis
of the cognitive and motivational eects of serious
games.,” J. Educ. Psychol., vol. 105, no. 2, pp. 249–
265, May 2013, doi: 10.1037/a0031311.
[2] M. Zyda, “From visual simulation to virtual reality
to games,” Computer, vol. 38, no. 9, pp. 25–32, Sep.
2005, doi: 10.1109/MC.2005.297.
[3] M. Rissanen, L. Metso, K. Elfvengren, and T. Sink-
konen, “Serious Games for Decision-Making Pro-
cesses: A Systematic Literature Review,” in Engi-
neering Assets and Public Infrastructures in the Age
of Digitalization, J. P. Liyanage, J. Amadi-Echendu,
and J. Mathew, Eds., Cham: Springer International
Publishing, 2020, pp. 330–338. doi: 10.1007/978-3-
030-48021-9_37.
[4] E. A. Boyle et al., “An update to the systematic
literature review of empirical evidence of the im-
pacts and outcomes of computer games and serious
games,” Comput. Educ., vol. 94, pp. 178–192, 2016,
doi: 10.1016/j.compedu.2015.11.003.
[5] S. Flood, N. A. Cradock-Henry, P. Blackett, and P.
Edwards, “Adaptive and interactive climate futures:
systematic review of ‘serious games’ for engagement
and decision-making,” Environ. Res. Lett., vol. 13,
no. 6, p. 063005, Jun. 2018, doi: 10.1088/1748-9326/
aac1c6.
[6] I. A. Ștefan, J. B. Hauge, F. Hasse, and A. Ștefan,
“Using Serious Games and Simulations for Teach-
ing Co-Operative Decision-making,” Procedia
Comput. Sci., vol. 162, pp. 745–753, Jan. 2019, doi:
10.1016/j.procs.2019.12.046.
[7] F. Bellotti, R. Berta, and A. D. Gloria, “Designing
Eective Serious Games: Opportunities and Chal-
lenges for Research,” Int. J. Emerg. Technol. Learn.
IJET, vol. 5, pp. 22–35, Nov. 2010, doi: 10.3991/ijet.
v5iSI3.1500.
[8] C. W. Totten, An Architectural Approach to Level
Design. New York: A K Peters/CRC Press, 2018. doi:
10.1201/b21989.
[9] D. Kahneman and A. Tversky, “Prospect eory: An
Analysis of Decision under Risk,” Econometrica, vol.
47, no. 2, pp. 263–291, 1979, doi: 10.2307/1914185.
[10] J. K. Haas, “A history of the unity game engine.”
2014.
[11] S. Knežević and M. Jovanović, “Simulating a ques-
tionnaire on the framing eects in decision-making
processes in a serious game.” arXiv, Oct. 28, 2021.
doi: 10.48550/arXiv.2110.15011.
[12] K. Damnjanovic and V. Gvozdenovic, “Inuence of
the probability level on the framing eect,” Psihol.
Teme, vol. 25, no. 3, pp. 405–429, 2016.
[13] D. J. Leiner, “SoSci Survey.” 2024. [Online]. Avail-
able: https://www.soscisurvey.de
[14] G. Hookham and K. Nesbitt, “A Systematic Review
of the Denition and Measurement of Engagement
in Serious Games,” in Proceedings of the Australa-
sian Computer Science Week Multiconference, in
ACSW ’19. New York, NY, USA: Association for
Computing Machinery, Jan. 2019, pp. 1–10. doi:
10.1145/3290688.3290747.
