Conference PaperPDF Available

Teaching Responsible Conduct of Research Through an Interactive Storytelling Game

Authors:
  • mPath AI
  • Midjourney

Abstract and Figures

Concepts utilizing applied ethics, such as responsible conduct of research (RCR), can prove difficult to teach due to the complexity of problems faced by researchers and the many underlying perspectives involved in such dilemmas. To address this issue, we created Academical, a choice-based interactive storytelling game for RCR education that enables players to experience a story from multiple perspectives. In this paper, we describe the design rationale of Academical, and present results from an initial pilot study comparing it with traditional web-based educational materials from an existing RCR course. The preliminary results highlight that utilizing a choice-based interactive story game may prove more effective for RCR education, with significantly higher engagement and comparable or better scores for tests of RCR topics.
Content may be subject to copyright.
CHI 2020 Late-Breaking Work
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
Teaching Responsible Conduct of
Research Through an Interactive
Storytelling Game
Edward F. Melcer
University of California, Santa
Cruz
Santa Cruz, CA 95064, USA
eddie.melcer@ucsc.edu
James Ryan
BBN Technologies
St. Louis Park, MN 55416, USA
jryan@bbn.com
Nick Junius
Max Kreminski
University of California, Santa
Cruz
Santa Cruz, CA 95064, USA
njunius@ucsc.edu
mkremins@ucsc.edu
Dietrich Squinkifer
Independent Artist
Montreal, QC H3W 1R1, Canada
hey@squinky.me
Brent Hill
University of Utah
Salt Lake City, UT 84112
brent.hill@hsc.utah.edu
Noah Wardrip-Fruin
University of California, Santa
Cruz
Santa Cruz, CA 95064, USA
nwf@soe.ucsc.edu
Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
CHI ’20 Extended Abstracts, April 25–30, 2020, Honolulu, HI, USA.
© 2020 Copyright is held by the author/owner(s).
ACM ISBN 978-1-4503-6819-3/20/04.
DOI: https://doi.org/10.1145/3334480.3382973
Abstract
Concepts utilizing applied ethics, such as responsible con-
duct of research (RCR), can prove difficult to teach due to
the complexity of problems faced by researchers and the
many underlying perspectives involved in such dilemmas.
To address this issue, we created Academical, a choice-
based interactive storytelling game for RCR education that
enables players to experience a story from multiple per-
spectives. In this paper, we describe the design rationale of
Academical, and present results from an initial pilot study
comparing it with traditional web-based educational mate-
rials from an existing RCR course. The preliminary results
highlight that utilizing a choice-based interactive story game
may prove more effective for RCR education, with signifi-
cantly higher engagement and comparable or better scores
for tests of RCR topics.
Author Keywords
Choice-based; Role-playing; Interactive Storytelling; Narra-
tive Game; Educational Game; RCR.
CCS Concepts
Human-centered computing Human computer inter-
action (HCI); User studies;
LBW034, Page 1
CHI 2020 Late-Breaking Work
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
Introduction
Topics such as the responsible conduct of research (RCR)
are difficult to teach due to the complexity of applied ethics
and ethical decision-making [2], the need for moral rea-
soning [39], and the lack of existing educational tools that
are motivating and foster critical thinking [12]. While past
work has attempted to address these issues through alter-
native learning approaches such as group mentoring [48]
and role-playing [3, 40], these issues have still remained
largely unaddressed—resulting in ill-defined content, for-
mat, and goals, as well as minimal evidence for effective-
ness [11]. Conversely, in the context of educational games,
choice-based interactive storytelling is a popular format for
narrative videogames [7, 38, 28], and this format has been
shown to increase engagement/motivation as well as learn-
ing outcomes for rote STEM topics [35, 46, 49].
We hypothesized that the choice-based, role-playing nature
of interactive storytelling games could also be employed to
improve student engagement and learning outcomes within
ethically complex topics (such as RCR education), which
require learners to understand a variety of perspectives and
perform ethical decision-making. As a result, we created
Academical—a choice-based interactive storytelling game
for RCR education that allows players to experience a story
from multiple perspectives. In this paper, we discuss the de-
sign of Academical, and provide results from an initial pilot
study comparing engagement and learning outcomes of our
web-based game with traditional web-based educational
materials from a RCR course at the University of Utah.
Background
Choice-based Interactive Storytelling
Though it is attested as far back as the sixteenth century
[36, 27], choice-based interactive storytelling was made
famous by the Choose Your Own Adventure book series
[34, 37] and is now most prominent as a popular format
for narrative videogames [7, 38, 28]. In this format, players
navigate a plot graph [47] by making decisions (typically on
behalf of a character) at branching points in the narrative
(e.g., see Figure 2). Research in this area has typically con-
cerned the history [7, 27, 38, 36], analysis [22, 28, 23], or
generation [10, 24, 32] of works in the choice-based format.
Interactive Storytelling and Learning
Interactive storytelling has substantial potential for educa-
tion and games [5, 45]. Specifically, narrative/storytelling
is an important element that can be added to educational
games in order to maintain and increase students’ moti-
vation [4, 6, 31, 35], with some suggesting that integration
of a good story into an educational game will determine
its success or failure [8]. Interactive storytelling has been
incorporated into a number of educational games focus-
ing on topics such as history [4, 42], STEM [5, 46, 49], and
bullying [1, 44]. However, the majority of research on edu-
cational interactive storytelling games has focused on adap-
tivity [9, 16], interactivity [42, 49], emergent narrative [1],
and the game creation process [4, 43]. As a result, there is
surprisingly little work examining the impact of an interac-
tive storytelling approach on learning outcomes (exceptions
being [35, 44, 46, 49]), especially for topics with ethically
complex concepts that require a variety of perspectives.
Responsible Conduct of Research
Although students generally know that they should report
data honestly and cite sources accurately, they might not
know specific standards or obligations of RCR—such as
criteria for co-authorship and maintaining confidentiality
of manuscripts reviewed for publication [40]. The impor-
tance of RCR is such that many major funding agencies,
such as the National Institutes of Health (NIH) and National
Science Foundation (NSF), explicitly require researchers
LBW034, Page 2
CHI 2020 Late-Breaking Work
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
Figure 1: A choice point from Academical’s final scenario, “Fallen Angel Y2K.” In this scene, the player controls a busy professor whose
graduate student suspects that a postdoc in the lab has fabricated research results. The two highlighted text blocks represent dialogue options
between which the player must select.
supported by their grants to receive RCR training [30, 33].
Currently, the NIH provides a guideline of nine core RCR
topics: 1) conflict of interest, 2) human and animal subjects,
3) mentoring, 4) collaboration, 5) peer review, 6) data man-
agement, 7) research misconduct, 8) authorship and pub-
lication, and 9) scientists and society [13]. Past research
on RCR education has ranged from issues teaching ethi-
cal theories underlying RCR [2] and identifying metacog-
nitive reasoning strategies that facilitate ethical decision-
making [17] to the use of group mentoring [48] and role-
playing [3, 40] for improved efficacy. However, there is still
a notable engagement issue within current RCR education,
and a critical need for a variety of tools to improve discus-
sion, engagement, and critical thinking [12]. As a result,
an interactive storytelling approach may prove effective for
increasing motivation and fostering deeper critical thinking.
Design of Academical
Academical is a work of choice-based interactive story-
telling [22, 23, 18] that was developed using the Twine au-
thoring framework [7, 38]. Figure 1 shows a screenshot
taken during gameplay, which occurs in a web browser. The
LBW034, Page 3
CHI 2020 Late-Breaking Work
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
Figure 2: Plot graphs for two of
Academical’s playable scenarios,
visualized in the Twine authoring
environment. Each node in these
graphs is a Twine “passage“ (story
unit), some of which are player
choice points that link to other
passages. As the game
progresses, the scenarios become
more complex—of the two
scenarios shown here, the bottom
one comes later in the game.
game comprises nine playable scenarios, each pertain-
ing to a specific issue in RCR [13]. These scenarios are
adapted (with permission) from a series of existing educa-
tional RCR role-playing prompts [3, 40].
Each playable scenario in Academical centers on a conver-
sation between two stakeholders in the RCR issue at hand,
one of whom is controlled by the player—in the sense that
they select dialogue options for that character. By virtue of
these choices, the player will ultimately reach one of several
possible endings, a subset of which represent successful
navigations of the situation. Upon reaching a good ending
for one character, the player unlocks the other interlocutor
and may replay the scenario from that person’s viewpoint.
In turn, reaching a good ending for the second character in
a given scenario unlocks the next scenario, and the game
concludes upon completion of the final scenario. Gener-
ally, the scenarios become more complex (and difficult to
navigate) as the game proceeds, as Figure 2 illustrates.
At the outset of the project, it was decided that the format of
choice-based interactive storytelling—which allows a player
to experience a story from multiple perspectives, and to re-
play scenes to see how different actions play out—would
demonstrate the complicated nature of RCR to students
in a compelling way. In adapting the role-playing prompts,
we sought to show how seemingly obvious answers around
questions of research ethics can be complicated by factors
such as power dynamics and marginalized identities and
experiences. Instead of cleanly delineating right and wrong
answers, Academical showcases complexity and uncer-
tainty to provoke questions around how courses of action
could have unexpected consequences. In turn, while all
successful paths through the game’s scenarios represent
the player character acting responsibly, not all of the situa-
tions reach clear resolutions. Specifically, many scenarios
feature paths that appear to represent obvious solutions,
but ultimately lead to bad outcomes. Through replaying and
selecting new options, the player explores the social con-
cerns encompassing a given RCR scenario, which may
lead to a richer understanding of the ethical complications
that one can encounter while conducting research.
Pilot Study
RCR is a complicated topic to teach that requires under-
standing a variety of perspectives and dilemmas that impact
research ethics [14, 41]. As a result, we wanted to evalu-
ate whether a choice-based interactive storytelling design,
such as the one employed in Academical, could prove more
effective than traditional approaches for teaching ethically
complex topics. We hypothesized that the choice-based,
role-playing nature of Academical—which is specifically de-
signed to highlight how research ethics can be complicated
by many factors such as power dynamics and marginalized
identities—would be 1) more engaging and 2) as effective
as traditional RCR educational materials. In order to ex-
plore these hypotheses, we conducted a between-subjects
pilot study comparing our choice-based interactive story-
telling game approach with existing web-based educational
materials from an RCR course at the University of Utah.
The pilot study consisted of two conditions: 1) a group that
read through two modules of the web-based educational
RCR materials covering peer review and authorship; and
2) a group that played two chapters/levels of Academical
covering peer review and authorship content.
Procedure
Participants were told that the study was to explore differ-
ent approaches to RCR education, and that they would ei-
ther play a game or read materials teaching selected RCR
concepts. They then completed an online survey collecting
demographic information (age, prior gaming experience,
LBW034, Page 4
CHI 2020 Late-Breaking Work
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
RCR Peer Review Questions
1. "According to the study
materials, peer reviewers
are asked to make
judgements about the
quality of a proposed or
completed project. This
certainly includes all
EXCEPT the following:"
(Multiple Choice)
2. "If you can figure out
the authors of a paper
you are peer reviewing
after conflicts of interest
are disclosed, should you
still review the paper?"
(Yes/No)
3. "There is no simple
solution to the problem of
bias in peer review.
However, researchers can
lessen the impact of bias
by writing transparent
reviews."
(True/False)
Table 1: The post-test RCR peer
review quiz questions. Questions
were taken from an existing RCR
course at the University of Utah.
prior RCR experience, and so forth). Upon completing the
survey, participants were randomly assigned to one of the
two conditions (web materials or Academical). After com-
pleting the RCR training for peer review and authorship,
participants were then asked to complete a post-test that
assessed their engagement with the training material and
knowledge of peer review and authorship RCR concepts.
Participants
A convenience sample of 28 university graduate and un-
dergraduate students—the target populations for RCR
training—were recruited for the study (age: µ=24.8, σ=7.6).
There were 10 female, 14 male, and 3 non-binary partici-
pants, with 1 declining to disclose gender. During the study,
participants were randomly assigned to one of the two con-
ditions: web materials (14 total; 3 female, 2 non-binary, 8
male, 1 decline to answer) and Academical game (14 total;
7 female, 1 non-binary, 6 male). None of the participants
reported prior RCR training within the past 2 years.
Measures
Temple Presence Inventory, Engagement Subscale
Engagement is an critical aspect of the learning process [15],
drastically influencing a learner’s motivation to continue in-
teracting with a system and the educational content [29]. In
order to assess participant engagement with the two edu-
cational RCR tools employed, we utilized the Engagement
subscale of the Temple Presence Inventory (TPI) [19]. The
TPI is an instrument that has been validated for use with
games [20], and measuring game engagement [21].
Peer Review and Authorship RCR Quizzes
To assess and compare how effective the two RCR tools
were for teaching peer review and authorship concepts, we
utilized two quizzes from an existing online RCR course at
the University of Utah. Each quiz consists of three ques-
tions around a respective topic, and each question is either
Post-Test Results
Measures
Web
µ σ Game
µ σ Sig
p
ES
r
TPI Engagement
Peer Review Test
23.4
2.14
9
0.77
30.1
2.93
6.1
0.27
.029
.001
.4
.56
Authorship Test 2.36 0.75 2 0.79 .23 N/A
Table 2: Post-test results for the TPI Engagement subscale, Peer
Review test, and Authorship test. The table contains mean scores,
standard deviations, t-test scores for significance, and effect
size—which is medium to large for significant differences.
true/false, yes/no, or multiple choice (see Tables 1 and 3).
Results
Prior Knowledge and Experience
According to a series of independent samples t-tests, par-
ticipants in the two conditions did not differ with respect to
age, prior game experience, or prior interactive story ex-
perience (all p values >= .12). Similarly, no participants re-
ported prior RCR training in the past 2 years. Therefore,
we can assume that participants in both groups had similar
prior RCR, game, and interactive story experience.
Engagement with RCR Training Tools
We first examine participant engagement between the dif-
ferent RCR educational tools. In order to analyze differ-
ences between the web materials and Academical game
conditions, we used an independent samples t-test. The
first row of Table 2 shows descriptive statistics for scores
on the TPI Engagement subscale, as well as significant
differences and effect sizes. Results found a significant dif-
ference in favor of Academical increasing participant en-
gagement (p = .029, r = .4), suggesting that a choice-based
LBW034, Page 5
CHI 2020 Late-Breaking Work
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
RCR Authorship Questions
1. "When should
authorship for a paper be
discussed?"
(Multiple Choice)
2. "Which of the following
is NOT considered a
contribution to a paper?"
(Multiple Choice)
3. "There is disagreement
over whether authorship
should be limited to
individuals who
contributed to all phases
of a publication or
whether individuals
who made more limited
contributions deserve
authorship credit."
(True/False)
Table 3: The post-test RCR
authorship quiz questions.
Questions were taken from an
existing RCR course at the
University of Utah.
interactive story game is a more engaging experience for
RCR training than traditional web reading materials.
RCR Learning Outcomes
To better understand participants’ learning outcomes, we
analyzed post-test scores on the RCR peer review and au-
thorship quizzes. Descriptive statistics, statistical signifi-
cance, and effect sizes for the two measures are shown in
the bottom two rows of Table 2. A series of independent
samples t-tests showed that participants in the Academical
condition scored significantly higher on the peer review test
(p = .001, r = .56) and comparable to the web materials for
the authorship test, with no statistically significant difference
between scores (p = .23). This suggests that, in terms of
short-term learning, a choice-based interactive story ap-
proach can be more effective or comparable to traditional
educational RCR materials for certain RCR topics.
Discussion
Results from our pilot study highlight the potential of choice-
based interactive storytelling games for improving student
engagement and learning outcomes within RCR education.
First, the independent samples t-test for the TPI Engage-
ment subscale showed that Academical was significantly
more engaging than traditional web-based RCR educational
materials (p=.029, r=.4). This confirmed our first hypothe-
sis, and also falls in line with existing claims [4, 6, 16, 31,
42] and findings [35, 44, 49] that interactive storytelling de-
signs can improve learner engagement and motivation. Ad-
ditionally, we further extend these findings to illustrate that
interactive storytelling games can also increase motivation
when learning more ethically complex topics—beyond the
current rote STEM [35, 49] and history [4] examples.
Second, we found that short term learning outcomes for
RCR education were comparable (authorship; p=.23) or
significantly better for Academical (peer review; p=.001,
r=.56). This serves to extend current findings on the learn-
ing outcomes of educational interactive storytelling games [35,
44, 46, 49] by providing evidence for the efficacy of such
games in teaching concepts that require ethical decision-
making and multiple perspectives. Furthermore, given that
a choice-based interactive storytelling design approach is
both more engaging than traditional RCR materials and
equally/more effective for learning outcomes, Academical
is ultimately a useful tool to address the engagement and
critical thinking needs of current RCR education [12].
Conclusion, Limitations, and Future Work
In this paper we described the design of Academical, a
choice-based interactive storytelling game for RCR edu-
cation that enables players to experience a story from mul-
tiple perspectives. We also presented results from an ini-
tial pilot study comparing Academical with traditional web-
based RCR educational materials. The pilot study results
highlighted that a choice-based interactive story game de-
sign may prove to be an effective RCR education tool, with
significantly higher engagement and comparable or better
scores for tests of RCR topics. However, the sample size
is notably small for sufficiently evaluating the efficacy of an
educational game [25]. Therefore, future work will need to
further validate these results with a larger sample size to
fully prove the hypotheses, e.g., [26]. A longitudinal study
also needs to be done to examine long term learning out-
comes and improvements to RCR practices over time. Fur-
thermore, prior work has demonstrated that traditional edu-
cation approaches may increase students’ RCR knowledge,
but not their moral reasoning [39]. Given that Academical
enables a more direct exploration of ethics-based decisions,
future work should also explore if a choice-based interactive
story design can have a more substantial impact on learn-
ers’ moral reasoning.
LBW034, Page 6
CHI 2020 Late-Breaking Work
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
REFERENCES
[1] Ruth S Aylett, Sandy Louchart, Joao Dias, Ana Paiva,
and Marco Vala. 2005. FearNot!–an experiment in
emergent narrative. In International Workshop on
Intelligent Virtual Agents. Springer, 305–316.
[2] Mathieu Bouville. 2008. On using ethical theories to
teach engineering ethics. Science and Engineering
Ethics 14, 1 (2008), 111–120.
[3] Bradley J Brummel, CK Gunsalus, Kerri L Anderson,
and Michael C Loui. 2010. Development of role-play
scenarios for teaching responsible conduct of
research. Science and Engineering Ethics 16, 3
(2010), 573–589.
[4] Dimitrios Christopoulos, Pavlos Mavridis, Anthousis
Andreadis, and John N Karigiannis. 2011. Using
Virtual Environments to Tell the Story:" The Battle of
Thermopylae". In 2011 Third International Conference
on Games and Virtual Worlds for Serious Applications.
IEEE, 84–91.
[5] Polina Danilicheva, Stanislav Klimenko, Yury Baturin,
and Alexander Serebrov. 2009. Education in virtual
worlds: Virtual storytelling. In 2009 International
Conference on CyberWorlds. IEEE, 333–338.
[6] Michele D Dickey. 2006. Game design narrative for
learning: Appropriating adventure game design
narrative devices and techniques for the design of
interactive learning environments. Educational
Technology Research and Development 54, 3 (2006),
245–263.
[7] Jane Friedhoff. 2013. Untangling Twine: A Platform
Study. In Proc. DiGRA.
[8] Stefan Göbel, André de Carvalho Rodrigues, Florian
Mehm, and Ralf Steinmetz. 2009. Narrative
game-based learning objects for story-based digital
educational games. narrative 14 (2009), 16.
[9] Stefan Göbel and Florian Mehm. 2013. Personalized,
adaptive digital educational games using narrative
game-based learning objects. In Serious Games and
Virtual Worlds in Education, Professional
Development, and Healthcare. IGI Global, 74–84.
[10] Matthew Guzdial, Brent Harrison, Boyang Li, and Mark
Riedl. 2015. Crowdsourcing Open Interactive
Narrative.. In Proc. Foundations of Digital Games.
[11] Michael Kalichman. 2013. A brief history of RCR
education. Accountability in research 20, 5-6 (2013),
380–394.
[12] Michael Kalichman. 2014. Rescuing responsible
conduct of research (RCR) education. Accountability in
research 21, 1 (2014), 68–83.
[13] Michael Kalichman. 2016. Responsible Conduct of
Research Education (What, Why, and Does It Work?).
Academic medicine: journal of the Association of
American Medical Colleges 91, 12 (2016), e10.
[14] Michael W Kalichman and Dena K Plemmons. 2007.
Reported goals for responsible conduct of research
courses. Academic Medicine 82, 9 (2007), 846–852.
[15] Greg Kearsley and Ben Shneiderman. 1998.
Engagement theory: A framework for
technology-based teaching and learning. Educational
technology 38, 5 (1998), 20–23.
LBW034, Page 7
CHI 2020 Late-Breaking Work
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
[16] Michael D Kickmeier-Rust, Stefan Göbel, and Dietrich
Albert. 2008. 80Days: Melding adaptive educational
technology and adaptive and interactive storytelling in
digital educational games. In Proceedings of the First
International Workshop on Story-Telling and
Educational Games (STEG’08). 8.
[17] Vykinta Kligyte, Richard T Marcy, Sydney T Sevier,
Elaine S Godfrey, and Michael D Mumford. 2008. A
qualitative approach to responsible conduct of
research (RCR) training development: Identification of
metacognitive strategies. Science and Engineering
Ethics 14, 1 (2008), 3–31.
[18] Hartmut Koenitz, Gabriele Ferri, Mads Haahr, Di˘
gdem
Sezen, and Tonguç ˙
Ibrahim Sezen. 2015. Interactive
digital narrative: history, theory and practice.
Routledge.
[19] Matthew Lombard, Theresa B Ditton, and Lisa
Weinstein. 2009. Measuring presence: the temple
presence inventory. In Proceedings of the 12th Annual
International Workshop on Presence. 1–15.
[20] Matthew Lombard, Lisa Weinstein, and Theresa
Ditton. 2011. Measuring telepresence: The validity of
the Temple Presence Inventory (TPI) in a gaming
context. In ISPR 2011: The International Society for
Presence Research Annual Conference. Edinburgh.
[21] Rosa Mikeal Martey, Kate Kenski, James Folkestad,
Laurie Feldman, Elana Gordis, Adrienne Shaw,
Jennifer Stromer-Galley, Ben Clegg, Hui Zhang,
Nissim Kaufman, and others. 2014. Measuring game
engagement: multiple methods and construct
complexity. Simulation & Gaming 45, 4-5 (2014),
528–547.
[22] Peter Mawhorter, Michael Mateas, Noah
Wardrip-Fruin, and Arnav Jhala. 2014. Towards a
theory of choice poetics. In Proc. Foundations of
Digital Games.
[23] Peter Mawhorter, Carmen Zegura, Alex Gray, Arnav
Jhala, Michael Mateas, and Noah Wardrip-Fruin. 2018.
Choice poetics by example. In Arts, Vol. 7.
Multidisciplinary Digital Publishing Institute, 47.
[24] Peter Andrew Mawhorter. 2016. Artificial intelligence
as a tool for understanding narrative choices. Ph.D.
Dissertation. UC Santa Cruz.
[25] Edward F Melcer, Victoria Hollis, and Katherine
Isbister. 2017. Tangibles vs. mouse in educational
programming games: Influences on enjoyment and
self-beliefs. In Proceedings of the 2017 CHI
conference extended abstracts on human factors in
computing systems. 1901–1908.
[26] Edward F Melcer and Katherine Isbister. 2018. Bots &
(Main) frames: exploring the impact of tangible blocks
and collaborative play in an educational programming
game. In Proceedings of the 2018 CHI Conference on
Human Factors in Computing Systems. 1–14.
[27] Brian Moriarty. 2015. I Sing the Story Electric.
PRACTICE. (2015).
http://ludix.com/moriarty/electric.html
Accessed Jan 2 2020.
[28] John Thomas Murray. 2018. Telltale Hearts: Encoding
Cinematic Choice-based Adventure Games. Ph.D.
Dissertation. UC Santa Cruz.
[29] Heather L O’Brien and Elaine G Toms. 2008. What is
user engagement? A conceptual framework for
defining user engagement with technology. Journal of
the American society for Information Science and
Technology 59, 6 (2008), 938–955.
LBW034, Page 8
CHI 2020 Late-Breaking Work
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
[30] National Institutes of Health and others. 1992.
Reminder and update: requirement for instruction in
the responsible conduct of research in national
research service award institutional training grants.
NIH Guide for Grants and Contracts 21, 43 (1992).
[31] Natalia Padilla-Zea, Francisco L Gutiérrez, José Rafael
López-Arcos, Ana Abad-Arranz, and Patricia
Paderewski. 2014. Modeling storytelling to be used in
educational video games. Computers in Human
Behavior 31 (2014), 461–474.
[32] Nathan Partlan, Elin Carstensdottir, Erica Kleinman,
Sam Snodgrass, Casper Harteveld, Gillian Smith,
Camillia Matuk, Steven C Sutherland, and Magy Seif
El-Nasr. 2019. Evaluation of an
automatically-constructed graph-based representation
for interactive narrative. In Proc. Foundations of Digital
Games. ACM.
[33] SH Plimpton. 2009. NSF’s Implementation of Section
7009 of the America COMPETES Act. Fed. Regist 74,
160 (2009), 42126–42128.
[34] Jake Rossen. 2014. A Brief History of “Choose Your
Own Adventure”. Mental Floss (Apr. 10 2014).
[35] Jonathan P Rowe, Lucy R Shores, Bradford W Mott,
and James C Lester. 2011. Integrating learning,
problem solving, and engagement in
narrative-centered learning environments. International
Journal of Artificial Intelligence in Education 21, 1-2
(2011), 115–133.
[36] James Ryan. In press. A Garden, A Forking Path:
Interactive Branching Narrative in The Lady of May
(1578). In Proc. 1st Workshop on the History of
Expressive Systems.
[37] Anastasia Salter. 2014. What is your quest?: from
adventure games to interactive books. University of
Iowa Press.
[38] Anastasia Salter. 2016. Playing at empathy:
Representing and experiencing emotional growth
through Twine games. In Proc. International
Conference on Serious Games and Applications for
Health. IEEE, 1–8.
[39] Karen B Schmaling and Arthur W Blume. 2009. Ethics
instruction increases graduate students’ responsible
conduct of research knowledge but not moral
reasoning. Accountability in Research 16, 5 (2009),
268–283.
[40] Stephanie N Seiler, Bradley J Brummel, Kerri L
Anderson, Kyoung Jin Kim, Serena Wee, CK
Gunsalus, and Michael C Loui. 2011. Outcomes
assessment of role-play scenarios for teaching
responsible conduct of research. Accountability in
Research 18, 4 (2011), 217–246.
[41] Adil E Shamoo and David B Resnik. 2009.
Responsible conduct of research. Oxford University
Press.
[42] Qiulian Song, Ling He, and Xiaoqiang Hu. 2012. To
improve the interactivity of the history educational
games with digital interactive storytelling. Physics
Procedia 33 (2012), 1798–1802.
[43] Ulrike Spierling. 2008. ’Killer Phrases’: Design Steps
for a Game with Digital Role-Playing Agents. In
Transactions on edutainment I. Springer, 150–161.
LBW034, Page 9
CHI 2020 Late-Breaking Work
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
[44] Scott Watson, Natalie Vannini, Megan Davis, Sarah
Woods, Marc Hall, Lynne Hall, and Kerstin
Dautenhahn. 2007. FearNot! an anti-bullying
intervention: Evaluation of an interactive virtual
learning environment. Artificial Intelligence and
Simulation of Behaviour (AISB), April 24 (2007).
[45] Sebastian A Weiß and Wolfgang Müller. 2008. The
potential of interactive digital storytelling for the
creation of educational computer games. In
International Conference on Technologies for
E-Learning and Digital Entertainment. Springer,
475–486.
[46] Jui-Feng Weng, Hsiu-lien Kuo, and Shian-Shyong
Tseng. 2011. Interactive storytelling for elementary
school nature science education. In 2011 IEEE 11th
International Conference on Advanced Learning
Technologies. IEEE, 336–338.
[47] Peter Weyhrauch. 1997. Guiding Interactive Fiction.
Ph.D. Dissertation. Carnegie Mellon University.
[48] Caroline Whitebeck. 2001. Group mentoring to foster
the responsible conduct of research. Science and
Engineering Ethics 7, 4 (2001), 541–558.
[49] Lei Zhang, Doug A Bowman, and Caroline N Jones.
2019. Exploring Effects of Interactivity on Learning
with Interactive Storytelling in Immersive Virtual
Reality. In 2019 11th International Conference on
Virtual Worlds and Games for Serious Applications
(VS-Games). IEEE, 1–8.
LBW034, Page 10
... In designing module content, we drew on the seven-principle framework for video lessons in online learning (Ou et al. 2019), as well as recent work on using choice-based interactive games for RCR education (Grasse et al. 2021;Melcer et al. 2020aMelcer et al. , 2020b to provide active learning opportunities. Although peer interaction-based activities (e.g., group discussion, "pair and share, " etc.) are often preferred active learning tasks, particularly for RCR courses given the complexity of the topics covered (Kalichman 2014), by definition any activity that engages students in reading, writing, discussing, or solving problems while requiring higher-order thinking (e.g., analysis, synthesis, and evaluation) is considered "active learning" (Bonwell and Eison 1991). ...
... They receive instant feedback on their selections ("adaptive feedback": Ou et al. 2019, p. 87) and are offered "tips" or additional information through pop-ups at various points (illustrated in Figure 5). Burgeoning evidence on the use of interactive (e.g., "game-based") activities suggests that such multimedia activities are highly engaging and effective for mastery of skills (Grasse et al. 2021;Melcer et al. 2020aMelcer et al. , 2020b. Given that our goals for the module are to (1) increase students' confidence in their ability to justify their decisions and beliefs about authorship, and (2) increase their ability to identify potentially unethical authorship practices, the simulation was an ideal instructional tool. ...
Article
Full-text available
In this paper, we discuss our approach to developing engaging course content linked to distinct learning outcomes on the topic of academic authorship. Academic authorship is a critical element of research culture and responsible conduct of research (RCR) courses. Drawing on instructional design methods, our online course aims to stimulate critical thinking about ethical authorship practices and to help students develop skills for resolving authorship-related conflicts. The course is scaffolded to facilitate engagement by tying video and podcast-style media, a choice-based interactive simulation, and traditional case study activities, to specific learning objectives. We describe how each instructional technique enhances student engagement and learning for the paired objective, and the benefits of the holistic course design. Finally, we illustrate how RCR instructors can incorporate our materials in full or in part into their own course curricula and encourage others to use our course development approach when designing their own courses.
... Customizability, developmental alignment, and the ability to captivate are critical considerations in the development of interactive games intended for preschoolers (Danilina and Harder, 2020;Melcer et al., 2020). The significance of clear and concise guidelines, logical progression, and uncomplicated regulations is underscored by J. Kostolny et al. (2021). ...
... The use of interactive games not only improves the quality of the educational process but also creates conditions for organising remote communication between teachers of the group, professionals and parents. As a result, it can be mentioned that interactive games put parents in an active position in the development of the child's mental abilities; increase the willingness of parents to interact with the teacher; clarify the expectations and ideas of parents concerning preschool education; deepen and enrich the interaction of teachers and parents; develop a media library for preschool games (Danilina and Harder, 2020;Melcer et al., 2020). The multimedia educational game has replaced the conventional educational games. ...
Article
Full-text available
The objective of this study is to evaluate the capacity of interactive didactic games to augment cognitive development in preschool-aged children and to establish a methodical approach for incorporating these games into early childhood education curricula. The research employs analysis and synthesis methodologies, in addition to induction, deduction, and modelling, to deconstruct and reassemble learning process components cognitively. Interactive games foster cognitive development through the provision of immediate feedback and stimulate learning through entertainment. Nevertheless, commercially accessible games frequently fail to provide suitable educational material. Educators can generate individualised games by employing multimedia tools by a three-part structure for lessons: preparatory, main, and concluding. These games should feature explicit guidelines, captivating narratives, succinct action sequences, specified instructional objectives, and interactivity that is suitable for the age group involved. Scientific accuracy, developmental appropriateness, language comprehension, intuitive navigation, and research/problem-solving abilities are among the proposed selection criteria. Benefits observed in comparison to conventional instruction include increased retention, sustained attention, and problem-solving incentives. By judiciously incorporating sanitary regulations, interactive didactics have the potential to augment child cognition, foster parent-teacher cooperation, and elevate educator prestige. Capitalising on children’s innate fascination with computers for educational purposes establishes a fundamental basis for scholastic achievement. Technology cannot, however, substitute for teacher-student interaction.
... While textbooks provide theoretical knowledge, they often fail to provide the practical experience of designing experiments for human subjects, collecting data from human subjects, data cleaning and other research methods. However, recently interactive narratives have become a favored tool in classrooms [4,8] for teaching experimental research. Interactive narratives encourage students to design experiments within an engaging storyline. ...
... Several authors have incorporated interactive narratives in their classrooms to teach experimental research and research methods. Melcer et al. [8] introduced a game called 'Academical' to instruct students about the 'Responsible Conduct of Research' (RCR), a pivotal component of experimental research. Harteveld et al. [5] discussed methodologies where creating interactive narratives aids in crafting gamified surveys. ...
... The TPI is an instrument that has been validated for use with games [46], and measuring game engagement [49]. Importantly, it has previously been used to assess readers' engagement with an interactive narrative themed around a serious topic [53]. More specifically, it was used to demonstrate significantly different levels of engagement between linear and interactive narrative content, and it has been shown to positively correlate with a change in readers' attitude [27]. ...
Article
Full-text available
Imposter syndrome (IS) is a mental state that can severely inhibit a person's productivity and well-being. Although IS is extremely common (especially in high achievers, e.g., academics), few studies offer effective modes of support and recovery. In response, we introduce the application of relatable narrative as a way to help people overcome IS. We present a randomized group comparison study exploring the impact of various design factors -- specifically, choice-based interactivity and character relatability -- on our story's capacity to engage and impact 107 STEM undergraduates. Almost all participants enthusiastically endorsed the story as a valuable learning experience, and a majority reported positive impacts to mood and feelings of relatedness to peers. We discuss 1) how our specific digital story design can support positive changes in mental health at both individual and institutional levels and 2) how our findings can inform efficient design of interactive narrative content for educational applications.
... An increase in the number of correct responses was exhibited by participants in the postactivity survey for the "why" and "how" questions but not for "when, " possibly due to participants' awareness of the timing of elephant movement before the activity. These findings are consistent with previous studies by Melcer et al. (2020) and Panjaitan (2016), which showed that participants scored higher on postactivity surveys after being exposed to CYOA stories. CYOA activities provide appropriate contextual frameworks that help learners correct misconceptions and may improve the long-term retention of new knowledge. ...
Article
Full-text available
Choose‐your‐own‐adventure (CYOA) narratives offer immersive experiences that can effectively convey complex conservation concepts, fostering empathy and critical thinking, particularly in addressing issues like human–elephant conflict. Despite their potential, there is limited research on the use of CYOA activities in conservation education. In this study, we created an interactive story centered on elephant conservation, drawing from existing research to distill scientific concepts into engaging narratives and utilizing various modes of delivery (YouTube and live performances) to reach diverse audiences. We conducted postactivity surveys to assess variations in audience perception, learning, and conservation engagement intentions in relation to sociodemographic factors, activity type, and messages encountered. We then modeled the relationships between ordinal responses and explanatory variables using cumulative ordinal mixed models (N = 398). For the YouTube version, we also considered sociodemographic factors (YouTube n = 53, non‐YouTube n = 47). Our findings indicate a preference for live performances over online YouTube activity, with participants gaining knowledge about elephant conservation from both formats. For the YouTube activity, participants strongly agreed that the activity allowed engagement with the character and topic. They also expressed a higher likelihood of participating in several conservation actions, relative to a control group. While CYOA storytelling shows promise for conservation education, challenges remain in simplifying scientific language, assessing its impact on comprehension of complex issues, standardizing outcomes, and effectively communicating knowledge. Further research is recommended to adapt this approach, making it applicable to various audiences and domains beyond conservation.
... This accessible aspect of interactive narratives has made them highly popular with men, women, and even novices to games [12], as well as for educational/training purposes both commercially and in academia [11,18,46,50]. Furthermore, the role-playing nature of interactive narratives has been shown to be highly effective at improving player attitudes [24,25], motivation [23], knowledge [45], and skills [44]. This makes the use of interactive narrative games an ideal approach for training resiliency as it easily and intuitively enables learners to role-play through various stereotype and social identity threat scenarios. ...
Chapter
Full-text available
Stereotype threat and social identity threat are social phenomena that adversely affect underrepresented groups within STEM (i.e., women and people of color). While there are existing programs and techniques for training resiliency against these threats, the use of biofeedback and serious games may prove useful to enhance the efficacy and engagement of such training. In this paper, we present the work in progress on our interactive narrative biofeedback game (Resilient IN) that utilizes resonant frequency heart rate variability to train player resilience to stereotype and social identity threat as they move through a mock interview at a tech company within the game. Specifically, we discuss the design of the game in detail—focusing on how specific elements of the design draw from existing literature to evoke and train resilience during play, as well as design and validation of the game narrative/script with individuals in the technology and engineering industry. Finally, we provide future directions for the work, such as upcoming studies to validate the game’s efficacy in evoking and training resiliency to different kinds of threats.KeywordsSocial identity threatStereotype threatSerious gamesHeart rate variabilityBiofeedback
... Our first study was a randomized group comparison study (N = 14 for each group) that showed that our game is equal or better than traditional web-based training materials at teaching students RCR knowledge and moral reasoning skills Melcer et al. (2020a). Notably, this experiment also revealed that the game was significantly more engaging than the web-based training materials Melcer et al. (2020b). Our second experiment-a within-subjects comparison study (N = 60)-showed that the game can significantly improve attitudes about RCR for students who have never received RCR training Grasse et al. (2021). ...
Article
Full-text available
Choice-based interactive storytelling games such as Academical, our responsible conduct of research training game, show great promise as a novel way of providing efficacious ethics training. However, much work remains to determine what factors of such games contribute to their advantages over traditional text-based training tools, especially if we hope to further improve their enjoyment, engagement and efficacy. In this article, we present a case study exploring how the motivational factors of Self-Determination Theory (SDT) underlie players’ perceived most and least enjoyable experiences arising from the design of Academical. Specifically, we discuss how certain elements of Academical’s design influence different SDT factors and subsequently player experience, as well as how such elements can be changed to further improve the game. Furthermore, our work highlights potential limitations of existing conceptualizations for the relatedness factor of SDT—discussing ways that it can be extended to properly understand player enjoyment within single-player educational interactive narrative games.
Conference Paper
Full-text available
The premise that "good" games embody sound pedagogy in their designs, even if incorporation was not deliberate, suggests that commercial entertainment games may also hold surprising educational potential. However, there is limited research exploring the potential learning experiences that entertainment games can provide, as well as how such unintended experiences could influence players' everyday lives. In this paper, we present an exploratory study surveying thirteen university students to understand their perceived learning experiences from entertainment games, how they applied these experiences to their lives, and why they believed the experiences were personally impactful. We found that participants believed they learned (1) practical skills of collaboration and planning, and (2) a wide range of everyday knowledge and educational content. Additionally, we found all reported experiences were relevant and meaningful to players' lives outside of the game. Lastly, we utilize findings to inform the design of games beyond entertainment, identifying potential areas for improved educational game design.
Article
Full-text available
Teknologi multimedia merupakan teknologi yang telah digunakan pada hampir seluruh sektor di era digital. Untuk menghasilkan artefak-artefak multimedia yang berkualitas, salah satunya diperlukan keahlian dalam bidang desain. MAN 4 Sleman memiliki kelas keterampilan Desain Komunikasi Visual (DKV) yang mengajarkan siswa untuk membuat berbagai desain digital, seperti desain stiker, mug, dan lain-lain. Hal ini sejalan dengan keilmuan kelompok klaster Multimedia dan Visi Komputer di Program Studi Informatika - Program Sarjana, Universitas Islam Indonesia. Keterkaitan ini mendorong klaster Multimedia dan Visi Komputer untuk turut mendukung kelas keterampilan DKV di MAN 4 Sleman dalam mengembangkan kreativitas melalui kegiatan pengabdian masyarakat. Kegiatan pengabdian kepada masyarakat ini dilakukan dalam bentuk pelatihan yang terdiri dari 3 tahapan, yaitu pengenalan teknologi multimedia, pengembangan aset multimedia, dan pemanfaatan aset multimedia. Hasil akhir dari pelatihan tersebut adalah sebuah aplikasi gim storytelling. Hasil evaluasi melalui testimoni peserta dan guru menunjukkan bahwa kegiatan pelatihan ini berjalan dengan menyenangkan dan memberikan pengalaman baru bagi siswa. Kreativitas kelompok dan individual diasah melalui sesi kelompok (penyusunan alur cerita) dan sesi individual (pengembangan aset multimedia).
Article
Background: The current pandemic has led to unprecedented scenarios worldwide. In this context, educational institutions had to move abruptly from in-person to online classes, having to be flexible and innovative, one of the main concerns being the semester-end examinations in practical modules. Objective: This study aimed to explore nursing students’ perceptions of the use of a serious game-like model in their final online objective structured clinical examination (OSCE). Design: An exploratory phenomenological study was conducted during the COVID-19 pandemic, in particular during June 2020. Settings: This study took place at the University of Almeria with nursing students enrolled in a clinical placement module. Participants: Eighty-nine fourth-year nursing students took part in this study. Methods: Following the educational principles of the OSCE, a virtual OSCE was designed, using the Genially platform, an online platform for creating interactive content, to set up an assessment as a story-telling game-like model. A total of 5 focus groups (FGs) were conducted afterwards via the GoogleMeet platform. In addition, 30 semi-structured interviews were performed via the same platform. The data was analysed using a content analysis approach and supported by the ATLAS.ti 8.4 software. Results: Our finding revealed 2 main themes and 4 sub-themes. The two main themes were (i) generating emotions and feelings in times of virtuality; including emotions and feelings experienced by students during their online assessment process, and (ii) online assessment: a potential alternative to educational barriers; describing the variability of traditional modality, its implications for learning and the acquisition of competences. Conclusions: Serious game-like models, such as the story-telling game proposed, as part of their online OSCE assessment appear to be an appropriate alternative assessment method for face-to-face approaches. This paper adds new evidence on the use of innovative and state-of-art resources as part of nursing OSCE assessments in a new reality for most students and teachers. Serious game-like models in online OSCE may empower students and help them to remove perceived barriers in face-to-face assessments.
Conference Paper
Full-text available
Interactivity and player experience are inextricably entwined with the creation of compelling narratives for interactive digital media. Narrative shapes and buttresses many such experiences, and therefore designers must construct compelling narrative arcs while carefully considering the effects of interaction on both the story and the player. As the narrative becomes more structurally complex, due to choice-based branching and other player actions, designers need to employ commensurately capable models and visualizations to keep track of that growing complexity. However, previous models of interactive narrative have failed to fully capture interactive elements with automated, operationalized visualizations. In this paper, we describe an algorithm for automated construction of a framework-driven, graph-based representation of interactive narrative. This representation more fully and transparently models structural and interactive features of the narrative than did prior approaches. We present an initial evaluation of this representation, based on modified cognitive walkthroughs performed by interactive narrative design and research experts from our research team, and we describe the takeaways for future improvement on interactive narrative modeling and analysis.
Conference Paper
Full-text available
Immersive virtual reality (VR) holds great potential for learning, but it is unclear how VR experiences should be designed to maximize learning potential. In this study, we explored how the level of interactivity in an educational VR storytelling experience for immunology learning affects a user's learning gains. We created three versions of the VR experience with low (system automates as many actions as possible), medium (a combination of system automation and user-controlled actions), and high (as many user-controlled actions as possible) levels of interactivity. We hypothesized that too much or too little interactivity would result in smaller learning gains than a medium level of interactivity. Although data from pre and post-tests showed no significant difference in students' learning gains due to interactivity level, questionnaire and interview data suggest that interactivity in the experience significantly affects students' engagement in learning, attention, and focus on learning material. Participants also perceived that they could learn better and more effectively in a VR experience with a higher level of interactivity.
Article
Full-text available
Choice poetics is a formalist framework that seeks to concretely describe the impacts choices have on player experiences within narrative games. Developed in part to support algorithmic generation of narrative choices, the theory includes a detailed analytical framework for understanding the impressions choice structures make by analyzing the relationships among options, outcomes, and player goals. The theory also emphasizes the need to account for players’ various modes of engagement, which vary both during play and between players. In this work, we illustrate the non-computational application of choice poetics to the analysis of two different games to further develop the theory and make it more accessible to others. We focus first on using choice poetics to examine the central repeated choice in “Undertale,” and show how it can be used to contrast two different player types that will approach a choice differently. Finally, we give an example of fine-grained analysis using a choice from the game “Papers, Please,” which breaks down options and their outcomes to illustrate exactly how the choice pushes players towards complicity via the introduction of uncertainty. Through all of these examples, we hope to show the usefulness of choice poetics as a framework for understanding narrative choices, and to demonstrate concretely how one could productively apply it to choices “in the wild.”
Conference Paper
Full-text available
While recent work has begun to evaluate the efficacy of educational programming games, many common design decisions in these games (e.g., single player gameplay using touchpad or mouse) have not been explored for learning outcomes. For instance, alternative design approaches such as collaborative play and embodied interaction with tangibles may also provide important benefits to learners. To better understand how these design decisions impact learning and related factors, we created an educational programming game that allows for systematically varying input method and mode of play. In this paper, we describe design rationale for mouse and tangible versions of our game, and report a 2x2 factorial experiment comparing efficacy of mouse and tangible input methods with individual and collaborative modes of play. Results indicate tangibles have a greater positive impact on learning, situational interest, enjoyment, and programming self-beliefs. We also found collaborative play helps further reduce programming anxiety over individual play.
Conference Paper
Full-text available
Computer Science (CS) and related skills such as programming and Computational Thinking (CT) have recently become topics of global interest, with a large number of programming games created to engage and educate players. However, there has been relatively limited work to assess 1) the efficacy of such games with respect to critical educational factors such as enjoyment and programming self-beliefs; and 2) whether there are advantages to alternative, physically embodied design approaches (e.g., tangibles as input). To better explore the benefits of a tangible approach, we built and tested two versions of an educational programming game that were identical in design except for the form of interaction (tangible programming blocks vs. mouse input). After testing 34 participants, results showed that while both game versions were successful at improving programming self-beliefs, the tangible version corresponded to higher self-reports of player enjoyment. Overall, this paper presents a comparison between the efficacy of tangible and mouse design approaches for improving key learning factors in educational programming games.
Conference Paper
While many games use physical health as a metric and ask the player to become invested in “health” as a resource, emotional health and experiences of trauma, depression, stress, and anxiety are less frequently explored. Yet games represent a powerful space for exploring emotional conditions and allowing players to experiment with choices in the face of difficult circumstances. The Twine platform, a free open-source tool for making primarily text-based games, offers a compelling system for representations of this kind. Close play of games made with this system demonstrates that what is essential for emotional representation is not player agency: instead, it is the lack of choice that is most strongly resonant.