Conference PaperPDF Available

Teaching Responsible Conduct of Research Through an Interactive Storytelling Game

Authors:

Abstract and Figures

Concepts utilizing applied ethics, such as responsible conduct of research (RCR), can prove difficult to teach due to the complexity of problems faced by researchers and the many underlying perspectives involved in such dilemmas. To address this issue, we created Academical, a choice-based interactive storytelling game for RCR education that enables players to experience a story from multiple perspectives. In this paper, we describe the design rationale of Academical, and present results from an initial pilot study comparing it with traditional web-based educational materials from an existing RCR course. The preliminary results highlight that utilizing a choice-based interactive story game may prove more effective for RCR education, with significantly higher engagement and comparable or better scores for tests of RCR topics.
Content may be subject to copyright.
CHI 2020 Late-Breaking Work
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
Teaching Responsible Conduct of
Research Through an Interactive
Storytelling Game
Edward F. Melcer
University of California, Santa
Cruz
Santa Cruz, CA 95064, USA
eddie.melcer@ucsc.edu
James Ryan
BBN Technologies
St. Louis Park, MN 55416, USA
jryan@bbn.com
Nick Junius
Max Kreminski
University of California, Santa
Cruz
Santa Cruz, CA 95064, USA
njunius@ucsc.edu
mkremins@ucsc.edu
Dietrich Squinkifer
Independent Artist
Montreal, QC H3W 1R1, Canada
hey@squinky.me
Brent Hill
University of Utah
Salt Lake City, UT 84112
brent.hill@hsc.utah.edu
Noah Wardrip-Fruin
University of California, Santa
Cruz
Santa Cruz, CA 95064, USA
nwf@soe.ucsc.edu
Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
CHI ’20 Extended Abstracts, April 25–30, 2020, Honolulu, HI, USA.
© 2020 Copyright is held by the author/owner(s).
ACM ISBN 978-1-4503-6819-3/20/04.
DOI: https://doi.org/10.1145/3334480.3382973
Abstract
Concepts utilizing applied ethics, such as responsible con-
duct of research (RCR), can prove difficult to teach due to
the complexity of problems faced by researchers and the
many underlying perspectives involved in such dilemmas.
To address this issue, we created Academical, a choice-
based interactive storytelling game for RCR education that
enables players to experience a story from multiple per-
spectives. In this paper, we describe the design rationale of
Academical, and present results from an initial pilot study
comparing it with traditional web-based educational mate-
rials from an existing RCR course. The preliminary results
highlight that utilizing a choice-based interactive story game
may prove more effective for RCR education, with signifi-
cantly higher engagement and comparable or better scores
for tests of RCR topics.
Author Keywords
Choice-based; Role-playing; Interactive Storytelling; Narra-
tive Game; Educational Game; RCR.
CCS Concepts
Human-centered computing Human computer inter-
action (HCI); User studies;
LBW034, Page 1
CHI 2020 Late-Breaking Work
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
Introduction
Topics such as the responsible conduct of research (RCR)
are difficult to teach due to the complexity of applied ethics
and ethical decision-making [2], the need for moral rea-
soning [39], and the lack of existing educational tools that
are motivating and foster critical thinking [12]. While past
work has attempted to address these issues through alter-
native learning approaches such as group mentoring [48]
and role-playing [3, 40], these issues have still remained
largely unaddressed—resulting in ill-defined content, for-
mat, and goals, as well as minimal evidence for effective-
ness [11]. Conversely, in the context of educational games,
choice-based interactive storytelling is a popular format for
narrative videogames [7, 38, 28], and this format has been
shown to increase engagement/motivation as well as learn-
ing outcomes for rote STEM topics [35, 46, 49].
We hypothesized that the choice-based, role-playing nature
of interactive storytelling games could also be employed to
improve student engagement and learning outcomes within
ethically complex topics (such as RCR education), which
require learners to understand a variety of perspectives and
perform ethical decision-making. As a result, we created
Academical—a choice-based interactive storytelling game
for RCR education that allows players to experience a story
from multiple perspectives. In this paper, we discuss the de-
sign of Academical, and provide results from an initial pilot
study comparing engagement and learning outcomes of our
web-based game with traditional web-based educational
materials from a RCR course at the University of Utah.
Background
Choice-based Interactive Storytelling
Though it is attested as far back as the sixteenth century
[36, 27], choice-based interactive storytelling was made
famous by the Choose Your Own Adventure book series
[34, 37] and is now most prominent as a popular format
for narrative videogames [7, 38, 28]. In this format, players
navigate a plot graph [47] by making decisions (typically on
behalf of a character) at branching points in the narrative
(e.g., see Figure 2). Research in this area has typically con-
cerned the history [7, 27, 38, 36], analysis [22, 28, 23], or
generation [10, 24, 32] of works in the choice-based format.
Interactive Storytelling and Learning
Interactive storytelling has substantial potential for educa-
tion and games [5, 45]. Specifically, narrative/storytelling
is an important element that can be added to educational
games in order to maintain and increase students’ moti-
vation [4, 6, 31, 35], with some suggesting that integration
of a good story into an educational game will determine
its success or failure [8]. Interactive storytelling has been
incorporated into a number of educational games focus-
ing on topics such as history [4, 42], STEM [5, 46, 49], and
bullying [1, 44]. However, the majority of research on edu-
cational interactive storytelling games has focused on adap-
tivity [9, 16], interactivity [42, 49], emergent narrative [1],
and the game creation process [4, 43]. As a result, there is
surprisingly little work examining the impact of an interac-
tive storytelling approach on learning outcomes (exceptions
being [35, 44, 46, 49]), especially for topics with ethically
complex concepts that require a variety of perspectives.
Responsible Conduct of Research
Although students generally know that they should report
data honestly and cite sources accurately, they might not
know specific standards or obligations of RCR—such as
criteria for co-authorship and maintaining confidentiality
of manuscripts reviewed for publication [40]. The impor-
tance of RCR is such that many major funding agencies,
such as the National Institutes of Health (NIH) and National
Science Foundation (NSF), explicitly require researchers
LBW034, Page 2
CHI 2020 Late-Breaking Work
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
Figure 1: A choice point from Academical’s final scenario, “Fallen Angel Y2K.” In this scene, the player controls a busy professor whose
graduate student suspects that a postdoc in the lab has fabricated research results. The two highlighted text blocks represent dialogue options
between which the player must select.
supported by their grants to receive RCR training [30, 33].
Currently, the NIH provides a guideline of nine core RCR
topics: 1) conflict of interest, 2) human and animal subjects,
3) mentoring, 4) collaboration, 5) peer review, 6) data man-
agement, 7) research misconduct, 8) authorship and pub-
lication, and 9) scientists and society [13]. Past research
on RCR education has ranged from issues teaching ethi-
cal theories underlying RCR [2] and identifying metacog-
nitive reasoning strategies that facilitate ethical decision-
making [17] to the use of group mentoring [48] and role-
playing [3, 40] for improved efficacy. However, there is still
a notable engagement issue within current RCR education,
and a critical need for a variety of tools to improve discus-
sion, engagement, and critical thinking [12]. As a result,
an interactive storytelling approach may prove effective for
increasing motivation and fostering deeper critical thinking.
Design of Academical
Academical is a work of choice-based interactive story-
telling [22, 23, 18] that was developed using the Twine au-
thoring framework [7, 38]. Figure 1 shows a screenshot
taken during gameplay, which occurs in a web browser. The
LBW034, Page 3
CHI 2020 Late-Breaking Work
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
Figure 2: Plot graphs for two of
Academical’s playable scenarios,
visualized in the Twine authoring
environment. Each node in these
graphs is a Twine “passage“ (story
unit), some of which are player
choice points that link to other
passages. As the game
progresses, the scenarios become
more complex—of the two
scenarios shown here, the bottom
one comes later in the game.
game comprises nine playable scenarios, each pertain-
ing to a specific issue in RCR [13]. These scenarios are
adapted (with permission) from a series of existing educa-
tional RCR role-playing prompts [3, 40].
Each playable scenario in Academical centers on a conver-
sation between two stakeholders in the RCR issue at hand,
one of whom is controlled by the player—in the sense that
they select dialogue options for that character. By virtue of
these choices, the player will ultimately reach one of several
possible endings, a subset of which represent successful
navigations of the situation. Upon reaching a good ending
for one character, the player unlocks the other interlocutor
and may replay the scenario from that person’s viewpoint.
In turn, reaching a good ending for the second character in
a given scenario unlocks the next scenario, and the game
concludes upon completion of the final scenario. Gener-
ally, the scenarios become more complex (and difficult to
navigate) as the game proceeds, as Figure 2 illustrates.
At the outset of the project, it was decided that the format of
choice-based interactive storytelling—which allows a player
to experience a story from multiple perspectives, and to re-
play scenes to see how different actions play out—would
demonstrate the complicated nature of RCR to students
in a compelling way. In adapting the role-playing prompts,
we sought to show how seemingly obvious answers around
questions of research ethics can be complicated by factors
such as power dynamics and marginalized identities and
experiences. Instead of cleanly delineating right and wrong
answers, Academical showcases complexity and uncer-
tainty to provoke questions around how courses of action
could have unexpected consequences. In turn, while all
successful paths through the game’s scenarios represent
the player character acting responsibly, not all of the situa-
tions reach clear resolutions. Specifically, many scenarios
feature paths that appear to represent obvious solutions,
but ultimately lead to bad outcomes. Through replaying and
selecting new options, the player explores the social con-
cerns encompassing a given RCR scenario, which may
lead to a richer understanding of the ethical complications
that one can encounter while conducting research.
Pilot Study
RCR is a complicated topic to teach that requires under-
standing a variety of perspectives and dilemmas that impact
research ethics [14, 41]. As a result, we wanted to evalu-
ate whether a choice-based interactive storytelling design,
such as the one employed in Academical, could prove more
effective than traditional approaches for teaching ethically
complex topics. We hypothesized that the choice-based,
role-playing nature of Academical—which is specifically de-
signed to highlight how research ethics can be complicated
by many factors such as power dynamics and marginalized
identities—would be 1) more engaging and 2) as effective
as traditional RCR educational materials. In order to ex-
plore these hypotheses, we conducted a between-subjects
pilot study comparing our choice-based interactive story-
telling game approach with existing web-based educational
materials from an RCR course at the University of Utah.
The pilot study consisted of two conditions: 1) a group that
read through two modules of the web-based educational
RCR materials covering peer review and authorship; and
2) a group that played two chapters/levels of Academical
covering peer review and authorship content.
Procedure
Participants were told that the study was to explore differ-
ent approaches to RCR education, and that they would ei-
ther play a game or read materials teaching selected RCR
concepts. They then completed an online survey collecting
demographic information (age, prior gaming experience,
LBW034, Page 4
CHI 2020 Late-Breaking Work
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
RCR Peer Review Questions
1. "According to the study
materials, peer reviewers
are asked to make
judgements about the
quality of a proposed or
completed project. This
certainly includes all
EXCEPT the following:"
(Multiple Choice)
2. "If you can figure out
the authors of a paper
you are peer reviewing
after conflicts of interest
are disclosed, should you
still review the paper?"
(Yes/No)
3. "There is no simple
solution to the problem of
bias in peer review.
However, researchers can
lessen the impact of bias
by writing transparent
reviews."
(True/False)
Table 1: The post-test RCR peer
review quiz questions. Questions
were taken from an existing RCR
course at the University of Utah.
prior RCR experience, and so forth). Upon completing the
survey, participants were randomly assigned to one of the
two conditions (web materials or Academical). After com-
pleting the RCR training for peer review and authorship,
participants were then asked to complete a post-test that
assessed their engagement with the training material and
knowledge of peer review and authorship RCR concepts.
Participants
A convenience sample of 28 university graduate and un-
dergraduate students—the target populations for RCR
training—were recruited for the study (age: µ=24.8, σ=7.6).
There were 10 female, 14 male, and 3 non-binary partici-
pants, with 1 declining to disclose gender. During the study,
participants were randomly assigned to one of the two con-
ditions: web materials (14 total; 3 female, 2 non-binary, 8
male, 1 decline to answer) and Academical game (14 total;
7 female, 1 non-binary, 6 male). None of the participants
reported prior RCR training within the past 2 years.
Measures
Temple Presence Inventory, Engagement Subscale
Engagement is an critical aspect of the learning process [15],
drastically influencing a learner’s motivation to continue in-
teracting with a system and the educational content [29]. In
order to assess participant engagement with the two edu-
cational RCR tools employed, we utilized the Engagement
subscale of the Temple Presence Inventory (TPI) [19]. The
TPI is an instrument that has been validated for use with
games [20], and measuring game engagement [21].
Peer Review and Authorship RCR Quizzes
To assess and compare how effective the two RCR tools
were for teaching peer review and authorship concepts, we
utilized two quizzes from an existing online RCR course at
the University of Utah. Each quiz consists of three ques-
tions around a respective topic, and each question is either
Post-Test Results
Measures
Web
µ σ Game
µ σ Sig
p
ES
r
TPI Engagement
Peer Review Test
23.4
2.14
9
0.77
30.1
2.93
6.1
0.27
.029
.001
.4
.56
Authorship Test 2.36 0.75 2 0.79 .23 N/A
Table 2: Post-test results for the TPI Engagement subscale, Peer
Review test, and Authorship test. The table contains mean scores,
standard deviations, t-test scores for significance, and effect
size—which is medium to large for significant differences.
true/false, yes/no, or multiple choice (see Tables 1 and 3).
Results
Prior Knowledge and Experience
According to a series of independent samples t-tests, par-
ticipants in the two conditions did not differ with respect to
age, prior game experience, or prior interactive story ex-
perience (all p values >= .12). Similarly, no participants re-
ported prior RCR training in the past 2 years. Therefore,
we can assume that participants in both groups had similar
prior RCR, game, and interactive story experience.
Engagement with RCR Training Tools
We first examine participant engagement between the dif-
ferent RCR educational tools. In order to analyze differ-
ences between the web materials and Academical game
conditions, we used an independent samples t-test. The
first row of Table 2 shows descriptive statistics for scores
on the TPI Engagement subscale, as well as significant
differences and effect sizes. Results found a significant dif-
ference in favor of Academical increasing participant en-
gagement (p = .029, r = .4), suggesting that a choice-based
LBW034, Page 5
CHI 2020 Late-Breaking Work
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
RCR Authorship Questions
1. "When should
authorship for a paper be
discussed?"
(Multiple Choice)
2. "Which of the following
is NOT considered a
contribution to a paper?"
(Multiple Choice)
3. "There is disagreement
over whether authorship
should be limited to
individuals who
contributed to all phases
of a publication or
whether individuals
who made more limited
contributions deserve
authorship credit."
(True/False)
Table 3: The post-test RCR
authorship quiz questions.
Questions were taken from an
existing RCR course at the
University of Utah.
interactive story game is a more engaging experience for
RCR training than traditional web reading materials.
RCR Learning Outcomes
To better understand participants’ learning outcomes, we
analyzed post-test scores on the RCR peer review and au-
thorship quizzes. Descriptive statistics, statistical signifi-
cance, and effect sizes for the two measures are shown in
the bottom two rows of Table 2. A series of independent
samples t-tests showed that participants in the Academical
condition scored significantly higher on the peer review test
(p = .001, r = .56) and comparable to the web materials for
the authorship test, with no statistically significant difference
between scores (p = .23). This suggests that, in terms of
short-term learning, a choice-based interactive story ap-
proach can be more effective or comparable to traditional
educational RCR materials for certain RCR topics.
Discussion
Results from our pilot study highlight the potential of choice-
based interactive storytelling games for improving student
engagement and learning outcomes within RCR education.
First, the independent samples t-test for the TPI Engage-
ment subscale showed that Academical was significantly
more engaging than traditional web-based RCR educational
materials (p=.029, r=.4). This confirmed our first hypothe-
sis, and also falls in line with existing claims [4, 6, 16, 31,
42] and findings [35, 44, 49] that interactive storytelling de-
signs can improve learner engagement and motivation. Ad-
ditionally, we further extend these findings to illustrate that
interactive storytelling games can also increase motivation
when learning more ethically complex topics—beyond the
current rote STEM [35, 49] and history [4] examples.
Second, we found that short term learning outcomes for
RCR education were comparable (authorship; p=.23) or
significantly better for Academical (peer review; p=.001,
r=.56). This serves to extend current findings on the learn-
ing outcomes of educational interactive storytelling games [35,
44, 46, 49] by providing evidence for the efficacy of such
games in teaching concepts that require ethical decision-
making and multiple perspectives. Furthermore, given that
a choice-based interactive storytelling design approach is
both more engaging than traditional RCR materials and
equally/more effective for learning outcomes, Academical
is ultimately a useful tool to address the engagement and
critical thinking needs of current RCR education [12].
Conclusion, Limitations, and Future Work
In this paper we described the design of Academical, a
choice-based interactive storytelling game for RCR edu-
cation that enables players to experience a story from mul-
tiple perspectives. We also presented results from an ini-
tial pilot study comparing Academical with traditional web-
based RCR educational materials. The pilot study results
highlighted that a choice-based interactive story game de-
sign may prove to be an effective RCR education tool, with
significantly higher engagement and comparable or better
scores for tests of RCR topics. However, the sample size
is notably small for sufficiently evaluating the efficacy of an
educational game [25]. Therefore, future work will need to
further validate these results with a larger sample size to
fully prove the hypotheses, e.g., [26]. A longitudinal study
also needs to be done to examine long term learning out-
comes and improvements to RCR practices over time. Fur-
thermore, prior work has demonstrated that traditional edu-
cation approaches may increase students’ RCR knowledge,
but not their moral reasoning [39]. Given that Academical
enables a more direct exploration of ethics-based decisions,
future work should also explore if a choice-based interactive
story design can have a more substantial impact on learn-
ers’ moral reasoning.
LBW034, Page 6
CHI 2020 Late-Breaking Work
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
REFERENCES
[1] Ruth S Aylett, Sandy Louchart, Joao Dias, Ana Paiva,
and Marco Vala. 2005. FearNot!–an experiment in
emergent narrative. In International Workshop on
Intelligent Virtual Agents. Springer, 305–316.
[2] Mathieu Bouville. 2008. On using ethical theories to
teach engineering ethics. Science and Engineering
Ethics 14, 1 (2008), 111–120.
[3] Bradley J Brummel, CK Gunsalus, Kerri L Anderson,
and Michael C Loui. 2010. Development of role-play
scenarios for teaching responsible conduct of
research. Science and Engineering Ethics 16, 3
(2010), 573–589.
[4] Dimitrios Christopoulos, Pavlos Mavridis, Anthousis
Andreadis, and John N Karigiannis. 2011. Using
Virtual Environments to Tell the Story:" The Battle of
Thermopylae". In 2011 Third International Conference
on Games and Virtual Worlds for Serious Applications.
IEEE, 84–91.
[5] Polina Danilicheva, Stanislav Klimenko, Yury Baturin,
and Alexander Serebrov. 2009. Education in virtual
worlds: Virtual storytelling. In 2009 International
Conference on CyberWorlds. IEEE, 333–338.
[6] Michele D Dickey. 2006. Game design narrative for
learning: Appropriating adventure game design
narrative devices and techniques for the design of
interactive learning environments. Educational
Technology Research and Development 54, 3 (2006),
245–263.
[7] Jane Friedhoff. 2013. Untangling Twine: A Platform
Study. In Proc. DiGRA.
[8] Stefan Göbel, André de Carvalho Rodrigues, Florian
Mehm, and Ralf Steinmetz. 2009. Narrative
game-based learning objects for story-based digital
educational games. narrative 14 (2009), 16.
[9] Stefan Göbel and Florian Mehm. 2013. Personalized,
adaptive digital educational games using narrative
game-based learning objects. In Serious Games and
Virtual Worlds in Education, Professional
Development, and Healthcare. IGI Global, 74–84.
[10] Matthew Guzdial, Brent Harrison, Boyang Li, and Mark
Riedl. 2015. Crowdsourcing Open Interactive
Narrative.. In Proc. Foundations of Digital Games.
[11] Michael Kalichman. 2013. A brief history of RCR
education. Accountability in research 20, 5-6 (2013),
380–394.
[12] Michael Kalichman. 2014. Rescuing responsible
conduct of research (RCR) education. Accountability in
research 21, 1 (2014), 68–83.
[13] Michael Kalichman. 2016. Responsible Conduct of
Research Education (What, Why, and Does It Work?).
Academic medicine: journal of the Association of
American Medical Colleges 91, 12 (2016), e10.
[14] Michael W Kalichman and Dena K Plemmons. 2007.
Reported goals for responsible conduct of research
courses. Academic Medicine 82, 9 (2007), 846–852.
[15] Greg Kearsley and Ben Shneiderman. 1998.
Engagement theory: A framework for
technology-based teaching and learning. Educational
technology 38, 5 (1998), 20–23.
LBW034, Page 7
CHI 2020 Late-Breaking Work
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
[16] Michael D Kickmeier-Rust, Stefan Göbel, and Dietrich
Albert. 2008. 80Days: Melding adaptive educational
technology and adaptive and interactive storytelling in
digital educational games. In Proceedings of the First
International Workshop on Story-Telling and
Educational Games (STEG’08). 8.
[17] Vykinta Kligyte, Richard T Marcy, Sydney T Sevier,
Elaine S Godfrey, and Michael D Mumford. 2008. A
qualitative approach to responsible conduct of
research (RCR) training development: Identification of
metacognitive strategies. Science and Engineering
Ethics 14, 1 (2008), 3–31.
[18] Hartmut Koenitz, Gabriele Ferri, Mads Haahr, Di˘
gdem
Sezen, and Tonguç ˙
Ibrahim Sezen. 2015. Interactive
digital narrative: history, theory and practice.
Routledge.
[19] Matthew Lombard, Theresa B Ditton, and Lisa
Weinstein. 2009. Measuring presence: the temple
presence inventory. In Proceedings of the 12th Annual
International Workshop on Presence. 1–15.
[20] Matthew Lombard, Lisa Weinstein, and Theresa
Ditton. 2011. Measuring telepresence: The validity of
the Temple Presence Inventory (TPI) in a gaming
context. In ISPR 2011: The International Society for
Presence Research Annual Conference. Edinburgh.
[21] Rosa Mikeal Martey, Kate Kenski, James Folkestad,
Laurie Feldman, Elana Gordis, Adrienne Shaw,
Jennifer Stromer-Galley, Ben Clegg, Hui Zhang,
Nissim Kaufman, and others. 2014. Measuring game
engagement: multiple methods and construct
complexity. Simulation & Gaming 45, 4-5 (2014),
528–547.
[22] Peter Mawhorter, Michael Mateas, Noah
Wardrip-Fruin, and Arnav Jhala. 2014. Towards a
theory of choice poetics. In Proc. Foundations of
Digital Games.
[23] Peter Mawhorter, Carmen Zegura, Alex Gray, Arnav
Jhala, Michael Mateas, and Noah Wardrip-Fruin. 2018.
Choice poetics by example. In Arts, Vol. 7.
Multidisciplinary Digital Publishing Institute, 47.
[24] Peter Andrew Mawhorter. 2016. Artificial intelligence
as a tool for understanding narrative choices. Ph.D.
Dissertation. UC Santa Cruz.
[25] Edward F Melcer, Victoria Hollis, and Katherine
Isbister. 2017. Tangibles vs. mouse in educational
programming games: Influences on enjoyment and
self-beliefs. In Proceedings of the 2017 CHI
conference extended abstracts on human factors in
computing systems. 1901–1908.
[26] Edward F Melcer and Katherine Isbister. 2018. Bots &
(Main) frames: exploring the impact of tangible blocks
and collaborative play in an educational programming
game. In Proceedings of the 2018 CHI Conference on
Human Factors in Computing Systems. 1–14.
[27] Brian Moriarty. 2015. I Sing the Story Electric.
PRACTICE. (2015).
http://ludix.com/moriarty/electric.html
Accessed Jan 2 2020.
[28] John Thomas Murray. 2018. Telltale Hearts: Encoding
Cinematic Choice-based Adventure Games. Ph.D.
Dissertation. UC Santa Cruz.
[29] Heather L O’Brien and Elaine G Toms. 2008. What is
user engagement? A conceptual framework for
defining user engagement with technology. Journal of
the American society for Information Science and
Technology 59, 6 (2008), 938–955.
LBW034, Page 8
CHI 2020 Late-Breaking Work
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
[30] National Institutes of Health and others. 1992.
Reminder and update: requirement for instruction in
the responsible conduct of research in national
research service award institutional training grants.
NIH Guide for Grants and Contracts 21, 43 (1992).
[31] Natalia Padilla-Zea, Francisco L Gutiérrez, José Rafael
López-Arcos, Ana Abad-Arranz, and Patricia
Paderewski. 2014. Modeling storytelling to be used in
educational video games. Computers in Human
Behavior 31 (2014), 461–474.
[32] Nathan Partlan, Elin Carstensdottir, Erica Kleinman,
Sam Snodgrass, Casper Harteveld, Gillian Smith,
Camillia Matuk, Steven C Sutherland, and Magy Seif
El-Nasr. 2019. Evaluation of an
automatically-constructed graph-based representation
for interactive narrative. In Proc. Foundations of Digital
Games. ACM.
[33] SH Plimpton. 2009. NSF’s Implementation of Section
7009 of the America COMPETES Act. Fed. Regist 74,
160 (2009), 42126–42128.
[34] Jake Rossen. 2014. A Brief History of “Choose Your
Own Adventure”. Mental Floss (Apr. 10 2014).
[35] Jonathan P Rowe, Lucy R Shores, Bradford W Mott,
and James C Lester. 2011. Integrating learning,
problem solving, and engagement in
narrative-centered learning environments. International
Journal of Artificial Intelligence in Education 21, 1-2
(2011), 115–133.
[36] James Ryan. In press. A Garden, A Forking Path:
Interactive Branching Narrative in The Lady of May
(1578). In Proc. 1st Workshop on the History of
Expressive Systems.
[37] Anastasia Salter. 2014. What is your quest?: from
adventure games to interactive books. University of
Iowa Press.
[38] Anastasia Salter. 2016. Playing at empathy:
Representing and experiencing emotional growth
through Twine games. In Proc. International
Conference on Serious Games and Applications for
Health. IEEE, 1–8.
[39] Karen B Schmaling and Arthur W Blume. 2009. Ethics
instruction increases graduate students’ responsible
conduct of research knowledge but not moral
reasoning. Accountability in Research 16, 5 (2009),
268–283.
[40] Stephanie N Seiler, Bradley J Brummel, Kerri L
Anderson, Kyoung Jin Kim, Serena Wee, CK
Gunsalus, and Michael C Loui. 2011. Outcomes
assessment of role-play scenarios for teaching
responsible conduct of research. Accountability in
Research 18, 4 (2011), 217–246.
[41] Adil E Shamoo and David B Resnik. 2009.
Responsible conduct of research. Oxford University
Press.
[42] Qiulian Song, Ling He, and Xiaoqiang Hu. 2012. To
improve the interactivity of the history educational
games with digital interactive storytelling. Physics
Procedia 33 (2012), 1798–1802.
[43] Ulrike Spierling. 2008. ’Killer Phrases’: Design Steps
for a Game with Digital Role-Playing Agents. In
Transactions on edutainment I. Springer, 150–161.
LBW034, Page 9
CHI 2020 Late-Breaking Work
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
[44] Scott Watson, Natalie Vannini, Megan Davis, Sarah
Woods, Marc Hall, Lynne Hall, and Kerstin
Dautenhahn. 2007. FearNot! an anti-bullying
intervention: Evaluation of an interactive virtual
learning environment. Artificial Intelligence and
Simulation of Behaviour (AISB), April 24 (2007).
[45] Sebastian A Weiß and Wolfgang Müller. 2008. The
potential of interactive digital storytelling for the
creation of educational computer games. In
International Conference on Technologies for
E-Learning and Digital Entertainment. Springer,
475–486.
[46] Jui-Feng Weng, Hsiu-lien Kuo, and Shian-Shyong
Tseng. 2011. Interactive storytelling for elementary
school nature science education. In 2011 IEEE 11th
International Conference on Advanced Learning
Technologies. IEEE, 336–338.
[47] Peter Weyhrauch. 1997. Guiding Interactive Fiction.
Ph.D. Dissertation. Carnegie Mellon University.
[48] Caroline Whitebeck. 2001. Group mentoring to foster
the responsible conduct of research. Science and
Engineering Ethics 7, 4 (2001), 541–558.
[49] Lei Zhang, Doug A Bowman, and Caroline N Jones.
2019. Exploring Effects of Interactivity on Learning
with Interactive Storytelling in Immersive Virtual
Reality. In 2019 11th International Conference on
Virtual Worlds and Games for Serious Applications
(VS-Games). IEEE, 1–8.
LBW034, Page 10
... With a growing dependence at younger ages on such technologies comes a growing need to teach technology and media literacy, especially with respect to internet cookies. Specifically, internet cookies are widely misunderstood, and most individuals (from children to adult) do not fully understand how they work or what they even do [21]. This is especially problematic for adolescents and young adults since they are particularly susceptible to the erosion of privacy due to social media and internet search engines [8,30]. ...
... One tool which has been demonstrated to effectively enhance learning outcomes of knowledge and ethical issues are serious games, especially ones that incorporate an interactive narrative [2,20]. While there are serious games that teach important related topics such as responsible conduct of research [20,21], social anxiety [26], and ethics [9], to name a few, to our knowledge there has not been a game created to teach concepts focusing on internet cookies. In this demonstration paper, we discuss the design of Cookie Mania-a serious game created to develop knowledge of internet cookies and underlying ethical issues through an interactive narrative. ...
... From the beginning, we wanted to teach the learning outcomes through an interactive narrative game. Prior work has suggested that effective ways to enable educational games to address learning outcomes is the incorporation of a convincing and interactive narrative [21], scaffolding and proximal growth for development [19], and also to appeal to players from different ethnic, social, and individual backgrounds [25]. The gameplay loop for Cookie Mania draws from this, providing a linear story that spans 5 different "months" (learning outcomes) with branching paths that change various dialogue as well as the ending. ...
Conference Paper
Full-text available
Internet cookies are data storage tools that collect information from a platform’s users to inform various aspects of the platform such as marketing and recommendations systems. Notably, there is a lack of understanding around what internet cookies are and how they work, which spans from adolescents to adults. This can lead to major misunderstandings of how consenting information works in online mediums, and results in negative outcomes and effects for users by default. To help increase understanding of internet cookies and address these issues, we have developed a serious game called Cookie Mania. Our game aims to develop knowledge of internet cookies and related ethical issues through an interactive narrative, as well as foster additional engagement through cookie-focused minigames. In this demonstration paper, we discuss the design and development of Cookie Mania—highlighting how the various design choices target learning objectives for internet cookies and their underlying ethical issues.
... Development of an RCR training method (especially one that is digitally automated) that can address this issue would be a valuable tool to supplement existing training standards. Our previous research showed that Academical, an interactive choice-based narrative game, was overall a more effective tool than traditional web-based training for engaging university students and teaching them RCR knowledge and moral reasoning skills [38,39]. Importantly, that study was the first to empirically demonstrate the value of an interactive narrative game for training moral reasoning skills. ...
... In a prior study, we hypothesized that the choice-based, role-playing nature of interactive storytelling games could be employed to improve student engagement as well as knowledge and moral reasoning skills important for RCR. This prediction inspired the development of Academical, a choice-based interactive narrative video game (accessible from any web browser) that presents a set of nine stories addressing the nine core RCR topics identified by the NIHsee Figure 1 for a story sample and [38,39] for a more in-depth description of the game design and rationale. In this single-player game, learners role-play through both perspectives in an ethical dilemma, challenging them to make a series of difficult choices in order to reach an optimal solution. ...
... In this single-player game, learners role-play through both perspectives in an ethical dilemma, challenging them to make a series of difficult choices in order to reach an optimal solution. We initially conducted a randomized group comparison study showing that Academical is overall more effective than traditional web-based RCR training tools at engaging students and teaching them knowledge [39] as well as skills important for RCR such as moral reasoning [38]. Our successful results, combined with existing evidence that role-playing activities are useful for improving socio-affective learning outcomes, indicate that Academical would likely also be able to train improvements in players' attitudes about RCR. ...
Conference Paper
Full-text available
Responsible conduct of research (RCR) is an essential skill for all researchers to develop, but training scientists to behave ethically is complex because it requires addressing both cognitive (i.e., conceptual knowledge and moral reasoning skills) and socio-affective (i.e., attitudes) learning outcomes. Currently, both classroom- and web-based forms of RCR training struggle to address these distinct types of learning outcomes simultaneously. In this paper, we present a study providing initial evidence that playing a single brief session of Academical, a choice-based interactive narrative game, can significantly improve players’ attitudes about RCR. We further demonstrate the relationship between engagement with the game and resulting attitudes. Combined with our previous work showing Academical’s advantages over traditional RCR training for teaching cognitive learning outcomes, this study’s results highlight that utilizing a choice-based interactive story game is a uniquely effective way to holistically address RCR learning outcomes that drive ethical research behavior.
... Our first study was a randomized group comparison study (N = 14 for each group) that showed that our game is equal or better than traditional web-based training materials at teaching students RCR knowledge and moral reasoning skills Melcer et al. (2020a). Notably, this experiment also revealed that the game was significantly more engaging than the web-based training materials Melcer et al. (2020b). Our second experiment-a within-subjects comparison study (N = 60)-showed that the game can significantly improve attitudes about RCR for students who have never received RCR training Grasse et al. (2021). ...
Article
Full-text available
Choice-based interactive storytelling games such as Academical, our responsible conduct of research training game, show great promise as a novel way of providing efficacious ethics training. However, much work remains to determine what factors of such games contribute to their advantages over traditional text-based training tools, especially if we hope to further improve their enjoyment, engagement and efficacy. In this article, we present a case study exploring how the motivational factors of Self-Determination Theory (SDT) underlie players’ perceived most and least enjoyable experiences arising from the design of Academical. Specifically, we discuss how certain elements of Academical’s design influence different SDT factors and subsequently player experience, as well as how such elements can be changed to further improve the game. Furthermore, our work highlights potential limitations of existing conceptualizations for the relatedness factor of SDT—discussing ways that it can be extended to properly understand player enjoyment within single-player educational interactive narrative games.
... Participants first read and wrote responses to the short story about peer review, then answered the same three questions for the other scenario involving authorship. After completion of the study, two of the authors scored these answers using the behaviorally anchored rating scale (BARS) method (see Melcer et al. 2020a andSeiler et al. 2011) for more information on the authors' coding procedure. ...
Chapter
Full-text available
Responsible conduct of research (RCR) is an essential skill for all researchers to develop, but training scientists to behave ethically is complex because it requires addressing both cognitive (e.g., conceptual knowledge and moral reasoning skills) and socio-affective (e.g., attitudes) learning outcomes. Both classroom- and web-based forms of RCR training struggle to address these distinct types of learning outcomes simultaneously. This chapter presents a pair of experiments providing initial evidence that playing a single brief session of Academical, a choice-based interactive narrative game, has positive effects on all three key RCR learning outcomes. Our results highlight that utilizing a choice-based interactive storytelling game is a uniquely effective way to holistically address RCR learning outcomes that drive ethical research behaviors.
... For example, Fisher and Schoemann (2018) address ethical considerations of dark tourism and specifically settings where virtual reality and interactive storytelling is used in a real-world location with dark history. In addition, Melcer et al. (2020) have used interactive storytelling application to teach ethics. ...
Chapter
When used in its best way, interactive storytelling has the power to create unique, adaptive and unforgettable stories for an interactor. However, the same mechanisms that are able to create elements of surprise and joy can be used to construct ethically questionable and even malevolent experiences. As the creators are often held responsible for their work and their consequences, whether these are intentionally done or not, ethical consideration should be taken into account as early as possible. This chapter seeks to line out possible threats and their ethical consequences in the design and implementation of interactive stories. To approach the ethical issues at hand, we divide interactive storytelling into four key elements—designer, platform, storyworld and interactor—and review separately the ethical considerations related to them.
... First, VNs only show one snippet of text at a time as the player interacts with the game [31], the snippet being dependent on player action or choices and even shaping future interaction. Further, while VNs share many traits with choose-your-own adventure books [10]-a legacy shared with most modern interactive fiction and storytelling design in video games [34,62,64]-VNs often incorporate more involved mechanical and constraint based gameplay to impact the progression of the story and the relevance/complexity of player choice [15]. ...
Article
Full-text available
Visual Novel (VN) is a widely recognizable genre of narrative-focused games that has grown in popularity over the past decade. Surprisingly, despite being so widely recognizable, there is not a singular definition to help guide the design and analysis of such games---with academic definitions and implementations ranging from "interactive textbooks" to "adventure games with multi-ending stories". In this paper, we present a unified definition of VNs drawn from an analysis of 30 prior academic definitions. We also examined 54 existing VNs to further refine our definition and employ a deeper analysis of the interactivity within VNs. We highlight key features of VNs that arise in our definition, and discuss resulting implications for the design of VNs. This work is relevant for narrative game designers and researchers, affording a more unified structure and clearer guidelines to identify, analyze, and design future VN games.
... Interactive stories are imbued with unique properties that allow them to leverage the emotions of their audience [25]. The emotional impact of interactive stories has even been wielded as a means to change user behavior [1,7,3,15,26]. Thus, it is useful to capture players' emotions while they are playing interactive stories to be able to, for example, evaluate the effectiveness of the story for changing user behaviors, drive the path of a nonlinear narrative [16,21], or maintain a certain emotional experience intended by the author [10]. ...
Conference Paper
Full-text available
Capturing the player’s emotions in an interactive story can serve many purposes, such as to understand player response to a story or to alter the story’s path. The concept of diegesis defines the boundary between the story world and the real world, and has been used to explore different ways of conceptualizing user interface (UI) elements in interactive media. This paper explores methods of capturing players’ emotions during an interactive story and the concept of diegesis. Specifically, we posit that whether emotion capture exists in a diegetic versus a non-diegetic space can affect its accuracy when capturing player emotion. We developed two interactive stories with two emotion capture methods, one using a diegetic approach, and the other using a non-diegetic approach. We conducted a study with 64 participants to compare the two emotion capture approaches. Our results show that a diegetic approach leads to a better story experience, but that a non-diegetic approach leads to the player’s emotion being captured more accurately. We discuss the implications of the study results for integrating emotion capture in the design of interactive stories.
... There has been a number of works suggesting the potential for interactive narrative/storytelling in improving education [34,59,87,104,123], and prior work has found a substantial positive impact from adding narrative/storytelling to educational games in order to maintain or increase students' motivation [30,39,102,110]. However, there is still a surprisingly limited amount of work evaluating the efficacy of such games on learning outcomes, especially when compared to other genres of educational games [24,65,91], with the few notable exceptions being [80,89,92,110,122,124,131]. The majority of work on educational interactive narrative has instead focused primarily on adaptivity [52,66,77,97,109], interactivity [116,131], player and knowledge modeling [55,83,108], narrative planning and generation [54,105,121,133], emergent narrative [18], player experience [69], and the game creation process [30,117]. ...
Conference Paper
Full-text available
Interactive narratives are widely used to frame and contextualize education in games. However, the specifics of how their designs aid the learning process and outcomes remains relatively unexplored. To better understand this space, a study was conducted that focused on one sub-genre of interactive narrative, Visual Novels. Specifically, in this paper we conducted a survey of thirty-one existing educational Visual Novels, analyzing design elements that fostered learning and delivered educational content. The resulting taxonomy consists of five key dimensions for educational design and teaching strategies within Visual Novels: 1) Teaching Through Choice, 2) Teaching Through Scripted Sequences, 3) Teaching Through Mini-games, 4) Teaching Through Exploration and 5) Non-interactive Teaching. These dimensions demonstrate that there are a number of design considerations for supporting learning through Visual Novels. This work has implications for designers of educational games by classifying the different designs a Visual Novel can employ to teach-ultimately informing how to better present educational subject matter in interactive narrative games.
... being [37,53,67,70,73]), especially for topics such as RCR with ethically complex concepts that require a variety of perspectives. ...
Conference Paper
Full-text available
Concepts utilizing applied ethics, such as responsible conduct of research (RCR), can prove difficult to teach due to the complexity of problems faced by researchers and the many underlying perspectives involved in such dilemmas. To address this issue, we created Academical, a choice-based interactive storytelling game for RCR education that enables players to experience a story from multiple perspectives. In this paper, we describe the design rationale of Academical, and present results from an initial study comparing it with traditional web-based educational materials from an existing university RCR course. The results highlight that utilizing a choice-based interactive story game is more effective for RCR education, with learners developing significantly higher engagement, stronger overall moral reasoning skills, and better knowledge scores for certain RCR topics.
Article
Background: The current pandemic has led to unprecedented scenarios worldwide. In this context, educational institutions had to move abruptly from in-person to online classes, having to be flexible and innovative, one of the main concerns being the semester-end examinations in practical modules. Objective: This study aimed to explore nursing students’ perceptions of the use of a serious game-like model in their final online objective structured clinical examination (OSCE). Design: An exploratory phenomenological study was conducted during the COVID-19 pandemic, in particular during June 2020. Settings: This study took place at the University of Almeria with nursing students enrolled in a clinical placement module. Participants: Eighty-nine fourth-year nursing students took part in this study. Methods: Following the educational principles of the OSCE, a virtual OSCE was designed, using the Genially platform, an online platform for creating interactive content, to set up an assessment as a story-telling game-like model. A total of 5 focus groups (FGs) were conducted afterwards via the GoogleMeet platform. In addition, 30 semi-structured interviews were performed via the same platform. The data was analysed using a content analysis approach and supported by the ATLAS.ti 8.4 software. Results: Our finding revealed 2 main themes and 4 sub-themes. The two main themes were (i) generating emotions and feelings in times of virtuality; including emotions and feelings experienced by students during their online assessment process, and (ii) online assessment: a potential alternative to educational barriers; describing the variability of traditional modality, its implications for learning and the acquisition of competences. Conclusions: Serious game-like models, such as the story-telling game proposed, as part of their online OSCE assessment appear to be an appropriate alternative assessment method for face-to-face approaches. This paper adds new evidence on the use of innovative and state-of-art resources as part of nursing OSCE assessments in a new reality for most students and teachers. Serious game-like models in online OSCE may empower students and help them to remove perceived barriers in face-to-face assessments.
Conference Paper
Full-text available
Interactivity and player experience are inextricably entwined with the creation of compelling narratives for interactive digital media. Narrative shapes and buttresses many such experiences, and therefore designers must construct compelling narrative arcs while carefully considering the effects of interaction on both the story and the player. As the narrative becomes more structurally complex, due to choice-based branching and other player actions, designers need to employ commensurately capable models and visualizations to keep track of that growing complexity. However, previous models of interactive narrative have failed to fully capture interactive elements with automated, operationalized visualizations. In this paper, we describe an algorithm for automated construction of a framework-driven, graph-based representation of interactive narrative. This representation more fully and transparently models structural and interactive features of the narrative than did prior approaches. We present an initial evaluation of this representation, based on modified cognitive walkthroughs performed by interactive narrative design and research experts from our research team, and we describe the takeaways for future improvement on interactive narrative modeling and analysis.
Conference Paper
Full-text available
Immersive virtual reality (VR) holds great potential for learning, but it is unclear how VR experiences should be designed to maximize learning potential. In this study, we explored how the level of interactivity in an educational VR storytelling experience for immunology learning affects a user's learning gains. We created three versions of the VR experience with low (system automates as many actions as possible), medium (a combination of system automation and user-controlled actions), and high (as many user-controlled actions as possible) levels of interactivity. We hypothesized that too much or too little interactivity would result in smaller learning gains than a medium level of interactivity. Although data from pre and post-tests showed no significant difference in students' learning gains due to interactivity level, questionnaire and interview data suggest that interactivity in the experience significantly affects students' engagement in learning, attention, and focus on learning material. Participants also perceived that they could learn better and more effectively in a VR experience with a higher level of interactivity.
Article
Full-text available
Choice poetics is a formalist framework that seeks to concretely describe the impacts choices have on player experiences within narrative games. Developed in part to support algorithmic generation of narrative choices, the theory includes a detailed analytical framework for understanding the impressions choice structures make by analyzing the relationships among options, outcomes, and player goals. The theory also emphasizes the need to account for players’ various modes of engagement, which vary both during play and between players. In this work, we illustrate the non-computational application of choice poetics to the analysis of two different games to further develop the theory and make it more accessible to others. We focus first on using choice poetics to examine the central repeated choice in “Undertale,” and show how it can be used to contrast two different player types that will approach a choice differently. Finally, we give an example of fine-grained analysis using a choice from the game “Papers, Please,” which breaks down options and their outcomes to illustrate exactly how the choice pushes players towards complicity via the introduction of uncertainty. Through all of these examples, we hope to show the usefulness of choice poetics as a framework for understanding narrative choices, and to demonstrate concretely how one could productively apply it to choices “in the wild.”
Conference Paper
Full-text available
While recent work has begun to evaluate the efficacy of educational programming games, many common design decisions in these games (e.g., single player gameplay using touchpad or mouse) have not been explored for learning outcomes. For instance, alternative design approaches such as collaborative play and embodied interaction with tangibles may also provide important benefits to learners. To better understand how these design decisions impact learning and related factors, we created an educational programming game that allows for systematically varying input method and mode of play. In this paper, we describe design rationale for mouse and tangible versions of our game, and report a 2x2 factorial experiment comparing efficacy of mouse and tangible input methods with individual and collaborative modes of play. Results indicate tangibles have a greater positive impact on learning, situational interest, enjoyment, and programming self-beliefs. We also found collaborative play helps further reduce programming anxiety over individual play.
Conference Paper
Full-text available
Computer Science (CS) and related skills such as programming and Computational Thinking (CT) have recently become topics of global interest, with a large number of programming games created to engage and educate players. However, there has been relatively limited work to assess 1) the efficacy of such games with respect to critical educational factors such as enjoyment and programming self-beliefs; and 2) whether there are advantages to alternative, physically embodied design approaches (e.g., tangibles as input). To better explore the benefits of a tangible approach, we built and tested two versions of an educational programming game that were identical in design except for the form of interaction (tangible programming blocks vs. mouse input). After testing 34 participants, results showed that while both game versions were successful at improving programming self-beliefs, the tangible version corresponded to higher self-reports of player enjoyment. Overall, this paper presents a comparison between the efficacy of tangible and mouse design approaches for improving key learning factors in educational programming games.
Conference Paper
While many games use physical health as a metric and ask the player to become invested in “health” as a resource, emotional health and experiences of trauma, depression, stress, and anxiety are less frequently explored. Yet games represent a powerful space for exploring emotional conditions and allowing players to experiment with choices in the face of difficult circumstances. The Twine platform, a free open-source tool for making primarily text-based games, offers a compelling system for representations of this kind. Close play of games made with this system demonstrates that what is essential for emotional representation is not player agency: instead, it is the lack of choice that is most strongly resonant.
Article
Storytelling and gaming approaches are used as motivational instruments for suspenseful, engaging learning. This chapter describes the concept of Narrative Game-Based Learning Objects (NGLOB), providing a model of how to combine these different axes (narration, gaming, and learning) and how to apply it within personalized, adaptive Digital Educational Games (DEG). From a research perspective, this results in one of the main technical challenges of Serious Games (SG): personalization and adaptation. Here, the central question might be summarized with "How does one create and control a game during play considering the game context and characteristics of individual users or user groups?" This question and the use of NGLOBs are illustrated through the example of "Save the Earth" for teaching and learning geography.