Conference PaperPDF Available

Getting Academical: A Choice-Based Interactive Storytelling Game for Teaching Responsible Conduct of Research

Conference Paper

Getting Academical: A Choice-Based Interactive Storytelling Game for Teaching Responsible Conduct of Research

Abstract and Figures

Concepts utilizing applied ethics, such as responsible conduct of research (RCR), can prove difficult to teach due to the complexity of problems faced by researchers and the many underlying perspectives involved in such dilemmas. To address this issue, we created Academical, a choice-based interactive storytelling game for RCR education that enables players to experience a story from multiple perspectives. In this paper, we describe the design rationale of Academical, and present results from an initial study comparing it with traditional web-based educational materials from an existing university RCR course. The results highlight that utilizing a choice-based interactive story game is more effective for RCR education, with learners developing significantly higher engagement, stronger overall moral reasoning skills, and better knowledge scores for certain RCR topics.
Content may be subject to copyright.
Geing Academical: A Choice-Based Interactive Storytelling
Game for Teaching Responsible Conduct of Research
Edward F. Melcer
University of California, Santa Cruz
Santa Cruz, CA
eddie.melcer@ucsc.edu
Katelyn M. Grasse
University of California, Santa Cruz
Santa Cruz, CA
katy@ucsc.edu
James Ryan
Carleton College
Northeld, MN
jryan@carleton.edu
Nick Junius
University of California, Santa Cruz
Santa Cruz, CA
njunius@ucsc.edu
Max Kreminski
University of California, Santa Cruz
Santa Cruz, CA
mkremins@ucsc.edu
Dietrich Squinkifer
Independent Artist
Montreal, QC, Canada
hey@squinky.me
Brent Hill
University of Utah
Salt Lake City, UT
brent.hill@hsc.utah.edu
Noah Wardrip-Fruin
University of California, Santa Cruz
Santa Cruz, CA
nwardrip@ucsc.edu
ABSTRACT
Concepts utilizing applied ethics, such as responsible conduct of
research (RCR), can prove dicult to teach due to the complexity
of problems faced by researchers and the many underlying perspec-
tives involved in such dilemmas. To address this issue, we created
Academical, a choice-based interactive storytelling game for RCR
education that enables players to experience a story from multi-
ple perspectives. In this paper, we describe the design rationale of
Academical, and present results from an initial study comparing it
with traditional web-based educational materials from an existing
university RCR course. The results highlight that utilizing a choice-
based interactive story game is more eective for RCR education,
with learners developing signicantly higher engagement, stronger
overall moral reasoning skills, and better knowledge scores for
certain RCR topics.
CCS CONCEPTS
Human-centered computing;
KEYWORDS
choice-based, role-playing, interactive storytelling, narrative game,
educational game, responsible conduct of research, ethics
ACM Reference Format:
Edward F. Melcer,Katelyn M. Grasse, James Ryan, Nick Junius, Max Kreminski,
Dietrich Squinkifer, Brent Hill, and Noah Wardrip-Fruin. 2020. Getting Aca-
demical: A Choice-Based Interactive Storytelling Game for Teaching Respon-
sible Conduct of Research. In International Conference on the Foundations of
Digital Games (FDG ’20), September 15–18, 2020, Bugibba, Malta. ACM, New
York, NY, USA, 12 pages. https://doi.org/10.1145/3402942.3403005
Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
FDG ’20, September 15–18, 2020, Bugibba, Malta
©2020 Copyright held by the owner/author(s).
ACM ISBN 978-1-4503-8807-8/20/09.
https://doi.org/10.1145/3402942.3403005
1 INTRODUCTION
Topics such as the responsible conduct of research (RCR) are dif-
cult to teach due to the complexity of applied ethics and ethical
decision-making [
3
], the need for moral reasoning [
58
], and the
lack of existing educational tools that are motivating and foster crit-
ical thinking [
19
]. While past work has attempted to address these
issues through alternative learning approaches such as group men-
toring [
72
] and role-playing [
5
,
59
], these issues have still remained
largely unaddressed—resulting in ill-dened content, format, and
goals, as well as minimal evidence for eectiveness [
18
]. Conversely,
in the context of educational games, choice-based interactive sto-
rytelling is a popular format for narrative videogames [
12
,
40
,
56
].
There have even been educational interactive narratives designed
specically to teach issues related to ethics [
17
], although they have
yet to be evaluated for eectiveness. Interactive storytelling (and
educational games in general [
23
,
34
,
35
]) have also been shown to
increase engagement/motivation and learning for more rote top-
ics with clearly dened answers and educational outcomes, such
as in the areas of STEM [
53
,
70
,
73
]. However, past work has not
fully examined the capabilities of choice-based interactive story-
telling games in teaching more ambiguous concepts such as moral
reasoning and ethical decision-making.
RCR in particular is an important concept that warrants study
of and improvement to existing training tools. This is because it
comprises fundamental ethical topics that inform all aspects of the
research process, which can also be further complicated by many
factors such as power dynamics and marginalized identities. As a
result, RCR requires understanding a variety of perspectives and
dilemmas that impact underlying research ethics [
21
,
60
]. Addition-
ally, current educational RCR tools suer from a notable lack of
user engagement and motivation when learning the material [19].
Interactive storytelling games may be particularly eective for
addressing the above issues with RCR education. Specically, we
hypothesized that the choice-based, role-playing nature of inter-
active storytelling games could also be employed to improve stu-
dent engagement, learning outcomes, and moral reasoning within
FDG ’20, September 15–18, 2020, Bugibba, Malta Melcer et al.
ethically complex topics such as RCR education—which requires
learners to understand a variety of perspectives and perform ethical
decision-making. As a result, we created Academical, a choice-based
interactive storytelling game for RCR education that allows play-
ers to experience a story from multiple perspectives. In this paper,
we discuss the design of Academical, and provide results from an
initial study comparing engagement and learning outcomes of our
web-based game with traditional web-based educational materials
from an existing RCR course at the University of Utah. We conclude
with a discussion of the results and their implications for the usage
of choice-base interactive storytelling games for teaching ethics
knowledge, moral reasoning skills, RCR, and improving the overall
experience of educational role-playing.
2 BACKGROUND
In this section, we provide background information on our project,
with an emphasis on choice-based interactive storytelling and its
use in learning materials. We also discuss RCR, the subject area
for which Academical serves as an educational resource, and past
research exploring RCR education.
2.1 Choice-based Interactive Storytelling
Though it is attested as far back as the sixteenth century [
38
,
54
], choice-based interactive storytelling was made famous by the
Choose Your Own Adventure book series [
51
,
55
] and is now most
prominent as a popular format for narrative videogames [
12
,
40
,
56
].
For instance, the various titles developed by Telltale Games, e.g.,
[
64
,
65
]. In this format, players navigate a plot graph [
71
] by making
decisions (typically on behalf of a character) at branching points
in the narrative (see Figure 1for an excerpt from the plot graph
for Academical). Research in this area has typically concerned the
history [
12
,
38
,
54
,
56
], analysis [
31
,
32
,
40
], or procedural genera-
tion [
15
,
33
,
45
] of works in the choice-based format. Of particular
relevance to our study here is prior work that has argued for the
format’s power in terms of evoking empathy [
4
,
56
,
57
],
1
providing
therapeutic benets [
9
,
63
], and enabling learning experiences, the
latter of which we discuss next in a dedicated section.
2.2 Interactive Storytelling and Learning
Interactive storytelling has substantial potential for education and
games [
6
,
8
,
36
,
41
,
69
]. Specically, narrative/storytelling is an im-
portant element that can be incorporated into educational games in
order to maintain and increase students’ motivation [
7
,
10
,
44
,
53
],
with some suggesting that integration of a good story into an edu-
cational game will determine its success or failure [
13
]. Interactive
storytelling has been incorporated into a number of educational
games focusing on topics such as history [
7
,
61
], STEM [
8
,
70
,
73
],
and bullying [
2
,
67
]. However, the majority of research on edu-
cational interactive storytelling games has focused on adaptiv-
ity [
14
,
24
], interactivity [
61
,
73
], emergent narrative [
2
], player and
knowledge modeling [
29
,
52
], narrative planning and generation
[
16
,
50
,
66
,
74
], and the game creation process itself [
7
,
62
]. As a
result, there is surprisingly little work evaluating the impact of an
interactive storytelling approach on learning outcomes (exceptions
1Though see [49] for a critique of this notion.
Figure 1: Plot graphs for two of Academical’s playable
scenarios, visualized in the Twine authoring environment.
Each node in these graphs is a Twine “passage“ (story unit),
some of which are player choice points that link to other pas-
sages. As the game progresses, the scenarios become more
complex—of the two scenarios shown here, the one on the
right comes later in the game.
being [
37
,
53
,
67
,
70
,
73
]), especially for topics such as RCR with
ethically complex concepts that require a variety of perspectives.
2.3 Responsible Conduct of Research
Although students generally know that they should report data
honestly and cite sources accurately, they might not know specic
standards or obligations of RCR—such as criteria for co-authorship
and maintaining the condentiality of manuscripts reviewed for
publication [
48
,
59
]. The importance of RCR is such that many
major funding agencies, such as the National Institutes of Health
(NIH) and National Science Foundation (NSF), explicitly require re-
searchers supported by their grants to receive RCR training [
43
,
47
].
Currently, the NIH provides a guideline of nine core RCR topics [
20
]:
1) conict of interest, 2) human and animal subjects, 3) mentoring, 4)
collaboration, 5) peer review, 6) data management, 7) research miscon-
duct, 8) authorship and publication, and 9) scientists and society. Past
research on RCR education has ranged from issues teaching ethical
theories underlying RCR [
3
] and identifying metacognitive reason-
ing strategies that facilitate ethical decision-making [
25
,
39
] to the
use of group mentoring [
72
] and role-playing [
5
,
59
] for improved
training ecacy. However, there is still a notable engagement issue
within current RCR education, and a critical need for a variety of
tools to improve discussion, engagement, and critical thinking [
19
].
As a result, an interactive storytelling approach may prove eective
for increasing motivation and fostering deeper critical thinking.
3 ACADEMICAL
Academical is a work of choice-based interactive storytelling [
26
,
31
,
32
] that was created using the Twine authoring framework [
12
,
56
].
The game comprises nine playable scenarios, each pertaining to
a specic topic in RCR [
20
]. These scenarios are adapted (with
Geing Academical: A Choice-Based Interactive Storytelling Game for Teaching Responsible Conduct of Research FDG ’20, September 15–18, 2020, Bugibba, Malta
Figure 2: A choice point from Academical’s nal scenario, “Fallen Angel Y2K.” In this scene, the player controls a busy professor
whose graduate student suspects that a postdoc in the lab has fabricated research results. The two highlighted text blocks
represent dialogue options between which the player must select. To complete the scenario, the player must also navigate the
situation responsibly while acting as the graduate student.
permission) from a series of existing educational RCR role-playing
prompts [
5
,
59
]. Figure 2shows a screenshot taken during gameplay,
which occurs in a web browser.
Each playable scenario in Academical centers on a conversation
between two stakeholders in the RCR issue at hand, one of whom
is controlled by the player—in the sense that they select dialogue
options for that character. By virtue of these choices, the player
will ultimately reach one of several possible endings, a subset of
which represent successful navigation of the situation. Upon reach-
ing a good ending for the rst character, the player then unlocks
the other interlocutor and replays the scenario from that person’s
viewpoint. In turn, reaching a good ending for the second character
in a given scenario unlocks the next scenario/RCR topic. The game
concludes upon completion of the nal scenario. Generally, the
scenarios become more complex (and dicult to navigate) as the
game proceeds, as Figure 1illustrates.
At the outset of the project, we decided that the format of choice-
based interactive storytelling—which allows a player to experience
a story from multiple perspectives and replay scenes to see how
dierent actions play out—would demonstrate the complicated
nature of RCR to students in a compelling way. In adapting the
role-playing prompts, we sought to show how seemingly obvious
answers around questions of research ethics can be complicated
by factors such as power dynamics and marginalized identities
and experiences. Instead of cleanly delineating right and wrong
answers, Academical showcases complexity and uncertainty to
provoke questions around how courses of action could have unex-
pected consequences. In turn, while all successful paths through
the game’s scenarios represent the player character acting respon-
sibly, not all of the situations reach clear resolutions. Specically,
many scenarios feature paths that appear to represent obvious so-
lutions, but ultimately lead to bad outcomes. Through replaying
and selecting new options, the player explores the social concerns
encompassed in a given RCR scenario, which will lead to a richer
understanding of the ethical complications that one can encounter
while conducting research as well as aid future moral reasoning.
4 METHODOLOGY
RCR is a complicated topic to teach that requires understanding a va-
riety of perspectives and dilemmas that impact research ethics [
21
,
60
]. As a result, we wanted to evaluate whether a choice-based
interactive storytelling design, such as the one employed in Aca-
demical, could prove more eective than traditional approaches
for teaching ethically complex topics. We hypothesized that the
choice-based, role-playing nature of Academical—which is specif-
ically designed to highlight how research ethics can be compli-
cated by many factors such as power dynamics and marginalized
identities—would be 1) more engaging, 2) as eective as traditional
RCR educational materials at developing knowledge of RCR con-
cepts, and 3) result in stronger moral reasoning skills. In order to
explore these hypotheses, we conducted a between-subjects study
comparing our choice-based interactive storytelling game approach
with web-based educational materials from an existing RCR course
FDG ’20, September 15–18, 2020, Bugibba, Malta Melcer et al.
Figure 3: An excerpt from the traditional web-based educational materials used in this study. As is common with current
educational RCR tools, the material is more heavily focused on historical context and case studies than Academical. These
materials were borrowed from an existing university RCR course.
(see Figure 3). The study consisted of two conditions: 1) a group
that read through two modules of the web-based educational RCR
materials covering peer review and authorship; and 2) a group
that played two chapters of Academical covering peer review and
authorship content.
4.1 Procedure
Participants were told that the study was to explore dierent ap-
proaches to RCR education, and that they would either play a game
or read materials teaching selected RCR concepts. They then com-
pleted an online survey collecting demographic information (age,
prior gaming experience, prior RCR experience, and so forth). Upon
completing the survey, participants were randomly assigned to one
of the two conditions (web materials or Academical). After complet-
ing the RCR training for peer review and authorship, participants
then completed a post-test that assessed their 1) engagement with
the training material, 2) quantitative knowledge of peer review and
authorship RCR concepts and 3) qualitative moral reasoning skills
for these same concepts. All participants completed the same topics
in the same order for both the training and testing phases.
4.2 Participants
A convenience sample of 28 university graduate and undergraduate
students—the standard target populations for RCR training—were
recruited for the study (age:
µ
=24.8,
σ
=7.6). There were 10 female,
14 male, and 3 non-binary participants, with 1 declining to disclose
gender. During the study, participants were randomly assigned to
one of the two conditions: web materials (
14 total
; 3 female, 2 non-
binary, 8 male, 1 decline to answer) and Academical game (
14 total
;
7 female, 1 non-binary, 6 male). None of the participants reported
prior RCR training within the past 2 years.
4.3 Measures
4.3.1 Temple Presence Inventory, Engagement Subscale. Engage-
ment is an critical aspect of the learning process [
22
], drastically
inuencing a learner’s motivation to continue interacting with a
system and the educational content [
42
]. In order to assess partici-
pant engagement with the two educational RCR tools employed, we
utilized the Engagement subscale of the Temple Presence Inventory
(TPI) [
27
]. The TPI is an instrument that has been validated for use
with games [28] and measuring game engagement [30].
4.3.2 Peer Review and Authorship RCR izzes. To assess and com-
pare how eective the two RCR tools were for teaching knowledge
of peer review and authorship concepts, we utilized two quizzes
from the existing online RCR course at the University of Utah.
Each quiz consists of three questions around a respective topic, and
each question is either true/false, yes/no, or multiple choice (see
Appendix A).
4.3.3 alitative Assessment of Moral Reasoning. To assess and
compare how eective the two RCR tools were for teaching moral
reasoning skills, we utilized qualitative test materials from a pre-
vious study that evaluated the eect of role-play on RCR learning
outcomes [
59
]. These test materials included two RCR-themed short
stories obtained from the Online Ethics Center for Engineering and
Research (OEC; https://www.onlineethics.org, Appendix B) and
three short answer questions that the previous study designed to
characterize a student’s ability to 1) analyze a moral problem, 2)
consider the viewpoints of all individuals involved, and 3) propose
solutions and anticipate their possible short- and long-term conse-
quences. Participants rst read and wrote responses to the short
story about peer review, then answered the same three questions
for the other scenario involving authorship. After completion of
the study, two of the authors scored these answers using the be-
haviorally anchored rating scale (BARS) method (see Figure 4). The
coders initially used the same rubric described in the previous study
to separately evaluate all answers, then compared results to assess
score distributions and inter-rater reliability. Similar to the previous
study, it was necessary to relax some grading criteria for questions
that rarely received "ideal" answers. Using these updated rubrics,
Geing Academical: A Choice-Based Interactive Storytelling Game for Teaching Responsible Conduct of Research FDG ’20, September 15–18, 2020, Bugibba, Malta
Identify Issues Representative Response
1: Indicates that there is no problem, or states that there
is a simple disagreement amongst the parties.
"Mike and Lisa are not clear about the partnership."
3: Misses some of the moral issues present in the case.
Primarily restates the issues as presented in the case
without naming the issue or mentioning specic stan-
dards.
"The main issue in this scenario is that not everyone who worked on the experiment is getting
the credit they deserve. Mike was convinced by his adviser that he should take the credit because
it would further his career."
5: Accurately identies and names most or all of the
moral issues present in the case. If applicable, mentions
relevant standards.
"Mike failed to make a more meaningful impact with his paper because he decided to submit the
paper as sole author. Although one might argue that being both the designer and experimenter
of a paper is more prestigious, it is not worth sacricing your vision and purpose by removing
the experiment which gives it validation among the scientic community. In addition, he tried
to take credit for the work that Lisa did for their project, which is denitely unacceptable."
Describe Viewpoints Representative Response
1: Primarily restates the behaviors of the parties in-
volved as they are given in the case; states that there
is no excuse for the behavior of one or more of the
parties.
"Slater was asked to review a manuscript from competitor’s lab, he thinks he could be objective
and shared the manuscript with his student."
3: Explains at least two viewpoints. However, the focus
is either primarily on the interest of only one of the
parties involved, or the student indicates that the par-
ties involved are entitled to their opinions but that one
perspective is “more correct” than other perspectives
without providing justication.
"Slater stands to benet by sabotaging the competitor’s work, but both Slater and Parker could
possibly tarnish their reputations if this is exposed in the science world. The authors who
submitted the manuscript that was rejected are just being completely screwed over."
5: Presents a balanced view from the perspective of
several involved parties. States the dierent attitudes,
values, and possible motives of the parties without
making unfounded assumptions about intent.
"The rst viewpoint is from the professor’s perspective; he thinks that he can review the paper
objectively despite the circumstances. The second viewpoint is from the grad student, whose
professor put them in a compromising position. The third viewpoint is from the authors of the
paper who received a reject review from a competing lab that also took a tip from their paper.
The fourth is from the Journal of Cool Results that thought they were getting an objective review
from the professor, but really received a biased reject."
Propose Solutions Representative Response
1: Solution is to ignore the problem, to interfere or “go
behind someone’s back”, or act immediately without
considering whether this is the best course of action.
Student does not mention, or devalues, the undesirable
consequences of the chosen solution.
"Unfortunately, this is unavoidable. Slater and Parker were aware of the rules of conduct for peer
reviewing, and they chose to subvert them. Any sense of competition will incite this kind of
behavior. However, given that peer reviews often summon multiple people to provide feedback,
I think that the quality of a work will be recognized by the majority."
3: Solution is practical, but incomplete or vaguely
formulated. Student understands some of the conse-
quences of the proposed solution but does not propose
strategies for minimizing these consequences.
"I believe Slater and Parker should withdraw their statement of the manuscript since it is biased,
and either credit or not use the solution found by the competition’s research. Not using the
solution may not be that simple, but if they do then they need to credit where they found the
idea from. Additionally, they should refrain from responding to research that is bias on their end
in the future. There was clear conict of interest, and it should be addressed instead of agreeing
to do the research."
5: Solution is practical and directly addresses the issues
at hand. Solution aims to optimize the outcomes of
all parties involved and to maintain relationships and
reputations. Solution adopts standard best practices
and does not violate ethical standards. Student under-
stands the consequences of the solution and mentions
strategies for minimizing negative consequences.
"Prof. Slater should write back to the Journal of Cool Results with his feedback, along with a
description of his situation regarding his current work and the conict of interest. Prof. Slater
and Ms. Parker might want to contact the author directly for permission to use the original
author’s work and discuss credit in their paper. When Prof. Slater and Ms. Parker publish their
results, they should mention the original author as the person who came up with the technique.
The Journal of Cool Results might nd Prof. Slater to be unprofessional/unethical, leading to a
stain on his image. If he mentioned "sharing of the paper with Ms. Parker" with the Journal, he
might be barred from reviewing papers any further, and increased scrutiny in their current work.
The original author might want more credit than what Prof. Slater and Ms. Parker want to share,
according to original author’s perception of the contribution of his technique in their work."
Figure 4: Initial BARS rubric for scoring qualitative answers and representative responses. The left column is taken directly
from [59] while the right column provides representative responses from our study participants. The nal rubric was applied
similarly to both of the RCR topics.
FDG ’20, September 15–18, 2020, Bugibba, Malta Melcer et al.
Table 1: Post-test results for the TPI Engagement sub-
scale, Peer Review test, and Authorship test. The table con-
tains mean scores, standard deviations, t-test and Wilcoxon
rank sum scores for signicance, and eect size—which is
medium to large for signicant dierences.
Quantitative Test Results
Web Game Sig ES
Measures µ σ µ σ p d r
TPI Engagement 23.4 9 30.1 6.1 .029 .87 .4
Peer Review Test 2.14 0.77 2.93 0.27 .002 1.4 .56
Authorship Test 2.36 0.75 2 0.79 .23 -.47 -.23
the coders again separately scored all answers and then met to
discuss rationale for any discrepancies. In the end, the scores for
each of the six questions had good inter-rater reliability, with ac-
ceptable levels of percent agreement (ranging .82-1) and Cohen’s
kappa values (ranging .72-1). Final scores for the few unresolved
ratings were calculated as the average of the two coders’ scores.
5 RESULTS
In this section, we provide the results of our study in terms of
participant prior knowledge and experience, as well as dierences
between the two conditions with regard to engagement with the
materials and learning outcomes.
5.1 Prior Knowledge and Experience
According to a series of independent samples t-tests, participants
in the two conditions did not dier with respect to age, prior game
experience, or prior interactive story experience (all p values >= .12).
Similarly, no participants reported prior RCR training in the past 2
years. Therefore, we can assume that participants in both groups
had similar prior RCR, game, and interactive story experience.
5.2 Engagement with RCR Training Tools
We rst examine participant engagement between the dierent RCR
educational tools. In order to analyze dierences between the web
materials and Academical game conditions, we used an independent
samples t-test. The rst row of Table 1shows descriptive statistics
for scores on the TPI Engagement subscale, as well as signicant
dierences and eect sizes. Results found a signicant dierence in
favor of Academical increasing participant engagement (p = .029,
r = .4), suggesting that a choice-based interactive story game is a
more engaging experience for RCR training than traditional web
reading materials.
5.3 RCR Learning Outcomes
5.3.1 Peer Review and Authorship RCR izzes. To better under-
stand participants’ knowledge of RCR concepts, we analyzed post-
test scores on the RCR peer review and authorship quizzes (see
Figure 5, left). Descriptive statistics, statistical signicance, and
eect sizes for the two measures are shown in the bottom two rows
of Table 1. A series of Wilcoxon rank sum tests showed that partic-
ipants in the Academical condition scored signicantly higher on
Table 2: Post-test results for the qualitative assessment of
moral reasoning. The table contains mean scores, standard
deviations, Wilcoxon rank sum test scores for signicance,
and eect size—which is medium to large for signicant dif-
ferences.
Qualitative Test Results
Web Game Sig ES
Measures µ σ µ σ p d r
Identify Issues 6.93 1.9 8.57 1.6 .023 .92 .42
Describe Viewpoints 4.71 2.8 7.36 2.5 .016 .99 .44
Propose Solutions 4.71 2.3 7.14 2.3 .015 1.1 .47
Total Score 16.4 5.7 23.1 4.7 .004 1.3 .54
the peer review test (p = .002, r = .56) and neither signicantly better
or worse than the web materials for the authorship test (n.s., p =
.23). This suggests that, in terms of short-term learning, a choice-
based interactive story approach is more eective than traditional
educational materials for developing knowledge of certain RCR
topics.
5.3.2 alitative Assessment of Moral Reasoning. To better under-
stand participants’ moral reasoning skills, we analyzed a series of
qualitative responses they wrote evaluating multiple aspects of two
scenarios addressing either peer review or authorship concepts
(see Appendix B). Descriptive statistics, statistical signicance, and
eect sizes for these measures are shown in Table 2. A series of
Wilcoxon rank sum tests showed that participants in the Academi-
cal group scored signicantly higher overall on the qualitative tests
of moral reasoning (total score: p = .004, r = .54). Combining the
scores across the two scenarios revealed that these participants had
similarly signicant improvements for all three aspects of moral
reasoning (Issues: p = .023, r = .42; Viewpoints: p = .016, r = .44;
Solutions: p = .015, r = .47). A series of independent-samples t-tests
similarly highlighted that the Academical group also demonstrated
better moral reasoning skills all together for both scenarios (Peer
Review: p = .015, r = .44; Authorship: p = .0028, r = .53; see Figure 5,
right). These results indicate that, in terms of short-term learning,
a choice-based interactive story approach is more eective than tra-
ditional educational RCR materials for developing moral reasoning
skills necessary to properly employ RCR.
6 DISCUSSION
The results from this study suggest that a choice-based interactive
story game design is eective as an RCR education tool, with learn-
ers developing signicantly higher engagement, stronger overall
moral reasoning skills, and better knowledge scores for certain RCR
topics with neither signicantly better or worse scores for others.
Results from our study highlight the potential of choice-based in-
teractive storytelling games for improving student engagement and
learning outcomes within RCR education as a whole. We discuss
our results in more detail below.
Geing Academical: A Choice-Based Interactive Storytelling Game for Teaching Responsible Conduct of Research FDG ’20, September 15–18, 2020, Bugibba, Malta
PR A
1
2
3
4
5
BARS Score
* *
PR A
0
1
2
3
Quiz Score
*
*
Game
Web
Figure 5: Post-test results for the peer review (PR) and au-
thorship (A) scenarios. Left: The Academical group (n =
14, shown in blue) demonstrated signicantly better knowl-
edge scores for PR, and no statistical dierences on knowl-
edge scores for A. Right: The Academical group also demon-
strated better moral reasoning skills for both scenarios. Di-
amonds represent group average scores and error bars indi-
cate SD. Signicance was determined by Wilcoxon rank sum
tests and Wilcoxon signed rank tests where appropriate and
is noted as *p<.05.
6.1 Engagement with RCR Training
An independent samples t-test for the TPI Engagement subscale
showed that Academical was signicantly more engaging than tra-
ditional web-based RCR educational materials. This conrmed our
rst hypothesis, and also falls in line with existing claims [
7
,
10
,
24
,
44
,
61
] and ndings [
53
,
67
,
73
] that interactive storytelling designs
can improve learner engagement and motivation. Additionally, we
further extend these ndings to illustrate that interactive story-
telling games can also increase motivation when learning more
ethically complex and ambiguous content—beyond the generally
rote material covered in existing STEM [
53
,
73
] and history [
7
]
examples.
6.2 RCR Learning Outcomes
Our study also identied that short term quantitative learning
outcomes for knowledge of RCR concepts in Academical was neither
signicantly better or worse for Authorship, and was signicantly
better for Peer Review. This serves to extend current ndings on the
learning outcomes of educational interactive storytelling games [
17
,
53
,
67
,
70
,
73
] by providing evidence for the ecacy of such games in
teaching knowledge of RCR concepts and ethical decision-making.
This conrmed, and even outperformed, our second hypothesis that
interactive storytelling games would be as eective as traditional
educational materials at developing knowledge of RCR concepts.
Additionally, and perhaps most importantly, we found that the
Academical group performed signicantly better overall on qualita-
tive tests assessing moral reasoning skills. We also found a consis-
tent signicant increase for the Academical group’s performance
across all three questions addressing the dierent aspects of moral
reasoning. As a whole, this suggests that a choice-based interactive
narrative approach also better prepares students to navigate the
key aspects of moral dilemmas and ethical decision-making that are
common in research. These medium to large eect sizes (Table 2)
indicate that Academical may also provide a substantial improve-
ment over existing live action immersive role-play techniques for
improving moral reasoning and knowledge of RCR concepts [
59
].
However, this needs to be further veried through additional stud-
ies. Overall, these results are very encouraging considering that
various other (non role-playing) educational methods that have
been shown to improve knowledge of RCR concepts often report ei-
ther comparatively weak benets, no eect, or even harm to moral
reasoning skills [
1
,
11
,
46
,
58
]. Conversely, Academical appears to
have a signicant impact on improving both players’ knowledge of
RCR concepts and their moral reasoning skills—providing a marked
improvement over most existing RCR training tools.
These positive outcomes are also particularly impressive and
interesting considering that the traditional web-training course
was designed to teach the specic knowledge tested in this study’s
quantitative quizzes. In comparison, Academical immersed players
in moral dilemmas that did not explicitly provide instruction about
correct moral behavior or RCR concepts, yet they almost always
performed better than the web-trained group on the same tests. Fu-
ture studies are required to determine why Academical is seemingly
able to provide these strong benets.
6.3 Relative Diculty of RCR Topics
Dierent RCR topics will vary in their perceived complexity, moral
ambiguity, and professional relevance. Therefore, applying the same
pedagogical methods to widely dierent subject matter is not guar-
anteed to be equally eective at teaching those topics [
39
,
68
].
Comparing test results between topics can help educators better un-
derstand what information is being taught most eectively. In this
study, all participants were trained and tested exclusively on two
common yet distinct RCR concepts, peer review and authorship. We
showed that the Academical group signicantly outperformed the
web-trained group for both knowledge and moral reasoning skills
related to peer review content, demonstrating that the game was
the superior tool for teaching that topic. In comparison, while the
Academical group also did signicantly better than the web-trained
group on qualitative tests of moral reasoning related to authorship,
playing the game did not provide a similar boost to knowledge
of the subject. This result suggests that Academical participants
may have generally performed better on tests about peer review,
but struggled as much as the traditional RCR educational approach
when learning concepts related to authorship—which could indicate
dierences in pedagogical ecacy. In order to further explore these
results and any potential dierences in pedagogical ecacy, we con-
ducted a secondary analysis comparing participants’ performance
between the two RCR topics.
For the quantitative results assessing RCR knowledge, we used
non-parametric signed rank tests to analyze within-subject quiz
results for each training group. We found that the Academical group
had signicantly better scores for knowledge of peer review than
for authorship (p = .0007, r = .62), while the web-trained group’s
results were comparable between the two topics (p = 0.47, r = .14).
Considering that the Academical group’s peer review knowledge
scores were also signicantly better than those of the web-trained
group (see Table 1and Figure 5, left), this suggests that Academical
FDG ’20, September 15–18, 2020, Bugibba, Malta Melcer et al.
was more eective at teaching peer review material compared to
the authorship content.
For the qualitative results assessing moral reasoning skills, our
prior independent-samples t-tests revealed signicantly higher
overall moral reasoning scores for the Academical group in both
scenarios (see Table 2and Figure 5, right). However, this result does
not indicate if participants performed better or worse on one topic
over another within the Academical or web groups. Therefore, in
order to explore any dierences in the pedagogical ecacy of devel-
oping moral reasoning skills for dierent scenarios, we conducted
a paired-sample t-test across all participants and compared their
scores between the two RCR topics. The aggregated scores showed
that everyone had signicantly better moral reasoning skills for
the peer review scenario than the authorship scenario (All partic-
ipants: p = .012, r = .21). Performing paired t-tests at the group
level also found that each group individually showed a similar but
not signicant trend towards better overall scores on peer review
than authorship (Academical: p = 0.098, r = .22; Web: p = 0.068, r =
.26). These results suggest that teaching moral reasoning skills with
respect to the RCR topic of authorship is harder to teach than peer
review overall—regardless of which educational RCR tool was used.
However, because everyone read and responded to the two test sce-
narios in the same order (with authorship last), it is still somewhat
unclear whether these overall dierences in moral reasoning skill
between the two RCR concepts are due to pedagogical ecacy or
simply performance fatigue.
Considered together, the results of this secondary analysis sup-
port the idea that Academical is more eective at teaching the
tested peer review material over the authorship content, and that
authorship content is substantially more dicult to teach in general
regardless of the tool. Future work is required to more explicitly
explore this and the relative diculty of all other RCR topics. Over-
all, these observations at least indicate that the quantitative and
qualitative test measures used for this study are sensitive enough
to detect signicant dierences in performance across topics after
training.
6.4 Role-Playing
Given that a choice-based interactive storytelling design approach is
both more engaging than traditional RCR materials and equally/more
eective for both quantitative and qualitative learning outcomes,
Academical is ultimately a useful tool to address the engagement
and critical thinking needs of current RCR education [
19
]. Studies
have shown that live action interactive role-play can help students
practice moral reasoning skills, but when compared to playing a
computer game, it is a relatively resource-intensive activity in terms
of the time and energy needed to facilitate and evaluate the train-
ing process. Furthermore, role-playing with others in the physical
world can be an uncomfortable experience for some people, poten-
tially compromising the learning experience [
59
]. In comparison,
Academical is an engaging single-player role-playing experience
that carries no social pressure, allowing students to explore multi-
ple perspectives at their own pace. Furthermore, its digital nature
means that all students can play through the same training scenar-
ios with the same dialogue options, and consequently their learning
progress and progression through the stories can be tracked far
more easily. Critically, the improved convenience of using Academ-
ical for ethical training has the potential to reach a far broader
audience than live action role-playing, as well as enable larger and
more controlled studies of its eects on RCR learning outcomes.
7 STUDY LIMITATIONS AND FUTURE WORK
Despite Academical’s encouraging eect on engagement and RCR
learning outcomes, one notable limitation of this study is the small
sample size of participants. Additionally, the training and test pro-
cedures were not randomized, making it somewhat more dicult
to explain dierences in performance between topics. Furthermore,
this study only measured short-term learning resulting from a single
session of training. Overall, these positive results are quite valuable
given the relatively poor state of current RCR education [
19
,
20
],
but further work is needed to assess long-term skill retention and
engagement. Another potential disadvantage is that, since we did
not test an untrained group of participants, we could not report
how much of an eect RCR training in general had on our learning
outcomes. Finally, improvements in the above learning outcomes
do not necessarily lead to better attitudes or moral behavior [
48
],
therefore the impact of Academical on such factors needs to be
explored in future studies as well.
Specically, future work will include longitudinal studies that
measure long-term learning outcomes and improvements to RCR
practices over time. We plan to achieve this by embedding the
game content into relevant university courses. Future studies will
also examine whether improvements in RCR learning outcomes
from training with Academical can generalize to untrained content.
We are also interested in determining which design aspects of
Academical best contribute to learning, and similarly want to better
understand how dierent player types can aect engagement and
learning outcomes. Lastly, future studies will include an additional
assessment to determine how dierent training methods aect a
player’s attitude about the importance of moral conduct in research.
8 CONCLUSION
In this paper we described the design of Academical, a choice-based
interactive storytelling game for RCR education that enables players
to experience a story from multiple perspectives. We also presented
results from an initial study comparing Academical with traditional
web-based educational materials from an existing university RCR
course. The initial study results highlighted that a choice-based
interactive story game design is eective for an RCR education
tool—with signicantly higher engagement, better scores overall
for qualitative tests of moral reasoning skills, and signicantly
better scores for some quantitative tests of RCR knowledge and
neither signicantly better or worse scores for others.
ACKNOWLEDGMENTS
We would like to thank Jim Moore and the UCSC Division of Grad-
uate Studies for sponsoring the development and evaluation of
Academical. We would also like to thank the many UCSC under-
graduate students that assisted with various aspects of the game’s
development: Janel Catajoy, Aislynn Cetera, Lisa Durand, Yani Mo-
hamad Fauzi, Trevor Holoch, Adesh Kumar, Merita Lundstrom,
Jacinda Ni, Jinah Noh, David Nguyen, Jared Ono, Silvia Ordonez,
Geing Academical: A Choice-Based Interactive Storytelling Game for Teaching Responsible Conduct of Research FDG ’20, September 15–18, 2020, Bugibba, Malta
Tiany Phan, Emily Rodriguez, Thomas Ruiz, Thovatey Tep, and
Reshma Zachariah. Furthermore, we would like to thank the Uni-
versity of Utah for kindly providing us with access to their RCR
course materials and assessments for this study. Finally, we also
thank Gene Amberg, C. K. Gunsalus, Sylvie Khan, and Michael
Loui of the University of Illinois, both for allowing us to adapt their
materials to create this game and for providing feedback on an early
prototype.
REFERENCES
[1]
Alison L. Antes, Xiaoqian Wang, Michael D. Mumford, Ryan P. Brown, Shane
Connelly, and Lynn D. Devenport. 2010. Evaluating the eects that existing
instruction on responsible conduct of research has on ethical decision making.
Academic Medicine 85, 3 (2010), 519–26.
[2]
Ruth S Aylett, Sandy Louchart, Joao Dias, Ana Paiva, and Marco Vala. 2005.
FearNot!–an experiment in emergent narrative. In International Workshop on
Intelligent Virtual Agents. Springer, 305–316.
[3]
Mathieu Bouville. 2008. On using ethical theories to teach engineering ethics.
Science and Engineering Ethics 14, 1 (2008), 111–120.
[4]
Tharrenos Bratitsis. 2016. A digital storytelling approach for fostering empathy
towards autistic children: Lessons learned. In Proc. International Conference on
Software Development and Technologies for Enhancing Accessibility and Fighting
Info-exclusion. 301–308.
[5]
Bradley J Brummel, CK Gunsalus, Kerri L Anderson, and Michael C Loui. 2010.
Development of role-play scenarios for teaching responsible conduct of research.
Science and Engineering Ethics 16, 3 (2010), 573–589.
[6]
Janelynn Camingue, Edward F. Melcer, and Elin Carstensdottir. 2020. A (Vi-
sual) Novel Route to Learning: A Taxonomy of Educational Visual Novels. In
Proceedings of the 15th International Conference on the Foundations of Digital
Games.
[7]
Dimitrios Christopoulos, Pavlos Mavridis, Anthousis Andreadis, and John N
Karigiannis. 2011. Using Virtual Environments to Tell the Story:" The Battle
of Thermopylae". In 2011 Third International Conference on Games and Virtual
Worlds for Serious Applications. IEEE, 84–91.
[8]
Polina Danilicheva, Stanislav Klimenko, Yury Baturin, and Alexander Serebrov.
2009. Education in virtual worlds: Virtual storytelling. In 2009 International
Conference on CyberWorlds. IEEE, 333–338.
[9]
Lucas Pfeier Salomâo Dias, Jorge Luis Victoria Barbosa, and Henrique Dam-
asceno Vianna. 2018. Gamication and serious games in depression care: A
systematic mapping study. Telematics and Informatics 35, 1 (2018), 213–224.
[10]
Michele D Dickey. 2006. Game design narrative for learning: Appropriating
adventure game design narrative devices and techniques for the design of inter-
active learning environments. Educational Technology Research and Development
54, 3 (2006), 245–263.
[11]
James M. DuBois, Jerey M. Dueker, Emily E. Anderson, and Jean Campbell. 2008.
The development and assessment of an NIH-funded research ethics training
program. Academic Medicine 83, 6 (2008), 596–603.
[12] Jane Friedho. 2013. Untangling Twine: A Platform Study. In Proc. DiGRA.
[13] Stefan Göbel, André de Carvalho Rodrigues, Florian Mehm, and Ralf Steinmetz.
2009. Narrative game-based learning objects for story-based digital educational
games. narrative 14 (2009), 16.
[14]
Stefan Göbel and Florian Mehm. 2013. Personalized, adaptive digital educational
games using narrative game-based learning objects. In Serious Games and Virtual
Worlds in Education, Professional Development, and Healthcare. IGI Global, 74–84.
[15]
Matthew Guzdial, Brent Harrison, Boyang Li, and Mark Riedl. 2015. Crowdsourc-
ing Open Interactive Narrative.. In Proc. Foundations of Digital Games.
[16]
Rania Hodhod, Paul Cairns, and Daniel Kudenko. 2011. Innovative integrated
architecture for educational games: challenges and merits. In Transactions on
edutainment V. Springer, 1–34.
[17]
Rania Hodhod, Daniel Kudenko, and Paul Cairns. 2009. AEINS: Adaptive Edu-
cational Interactive Narrative System to Teach Ethics. In AIED 2009: 14 th Inter-
national Conference on Articial Intelligence in Education Workshops Proceedings.
79.
[18]
Michael Kalichman. 2013. A brief history of RCR education. Accountability in
Research 20, 5-6 (2013), 380–394.
[19]
Michael Kalichman. 2014. Rescuing responsible conduct of research (RCR) edu-
cation. Accountability in research 21, 1 (2014), 68–83.
[20]
Michael Kalichman. 2016. Responsible Conduct of Research Education (What,
Why, and Does It Work?). Academic medicine: journal of the Association of
American Medical Colleges 91, 12 (2016), e10.
[21]
Michael W Kalichman and Dena K Plemmons. 2007. Reported goals for responsi-
ble conduct of research courses. Academic Medicine 82, 9 (2007), 846–852.
[22]
Greg Kearsley and Ben Shneiderman. 1998. Engagement theory: A framework
for technology-based teaching and learning. Educational technology 38, 5 (1998),
20–23.
[23]
Oleksandra Keehl and Edward Melcer. 2019. Radical tunes: exploring the impact
of music on memorization of stroke order in logographic writing systems. In
Proceedings of the 14th International Conference on the Foundations of Digital
Games. 1–6.
[24]
Michael D Kickmeier-Rust, Stefan Göbel, and Dietrich Albert. 2008. 80Days: Meld-
ing adaptive educational technology and adaptive and interactive storytelling in
digital educational games. In Proceedings of the First International Workshop on
Story-Telling and Educational Games (STEG’08). 8.
[25]
Vykinta Kligyte, Richard T Marcy, Sydney T Sevier, Elaine S Godfrey, and
Michael D Mumford. 2008. A qualitative approach to responsible conduct of
research (RCR) training development: Identication of metacognitive strategies.
Science and Engineering Ethics 14, 1 (2008), 3–31.
[26]
Hartmut Koenitz, Gabriele Ferri, Mads Haahr, Diğdem Sezen, and Tonguç İbrahim
Sezen. 2015. Interactive digital narrative: history, theory and practice. Routledge.
[27]
Matthew Lombard, Theresa B Ditton, and Lisa Weinstein. 2009. Measuring
presence: the temple presence inventory. In Proceedings of the 12th Annual Inter-
national Workshop on Presence. 1–15.
[28]
Matthew Lombard, Lisa Weinstein, and Theresa Ditton. 2011. Measuring telep-
resence: The validity of the Temple Presence Inventory (TPI) in a gaming context.
In ISPR 2011: The International Society for Presence Research Annual Conference.
Edinburgh.
[29]
Brian Magerko. 2007. Evaluating preemptive story direction in the interactive
drama architecture. Journal of Game Development 2, 3 (2007), 25–52.
[30]
Rosa Mikeal Martey, Kate Kenski, James Folkestad, Laurie Feldman, Elana Gordis,
Adrienne Shaw, Jennifer Stromer-Galley, Ben Clegg, Hui Zhang, Nissim Kauf-
man, et al
.
2014. Measuring game engagement: multiple methods and construct
complexity. Simulation & Gaming 45, 4-5 (2014), 528–547.
[31]
Peter Mawhorter, Michael Mateas, Noah Wardrip-Fruin, and Arnav Jhala. 2014.
Towards a theory of choice poetics. In Proc. Foundations of Digital Games.
[32]
Peter Mawhorter, Carmen Zegura, Alex Gray, Arnav Jhala, Michael Mateas, and
Noah Wardrip-Fruin. 2018. Choice poetics by example. In Arts, Vol. 7. Multidisci-
plinary Digital Publishing Institute, 47.
[33]
Peter Andrew Mawhorter. 2016. Articial intelligence as a tool for understanding
narrative choices. Ph.D. Dissertation. UC Santa Cruz.
[34]
Edward F Melcer, Victoria Hollis, and Katherine Isbister. 2017. Tangibles vs.
mouse in educational programming games: Inuences on enjoyment and self-
beliefs. In Proceedings of the 2017 CHI conference extended abstracts on human
factors in computing systems. 1901–1908.
[35]
Edward F Melcer and Katherine Isbister. 2018. Bots & (Main) frames: exploring the
impact of tangible blocks and collaborative play in an educational programming
game. In Proceedings of the 2018 CHI Conference on Human Factors in Computing
Systems. 1–14.
[36]
Edward F. Melcer,Truong-Huy D. Nguyen, Zhengxing Chen, Alessandro Canossa,
Magy Seif El-Nasr, and Katherine Isbister. 2015. Games Research Today: Analyz-
ing the Academic Landscape 2000-2014. In Proceedings of the 10th International
Conference on the Foundations of Digital Games.
[37]
Edward F. Melcer, James Ryan, Nick Junius, Max Kreminski, Dietrich Squinkifer,
Brent Hill, and Noah Wardrip-Fruin. 2020. Teaching Responsible Conduct of
Research Through an Interactive Storytelling Game. In Extended Abstracts of the
2020 CHI Conference on Human Factors in Computing Systems.
[38]
Brian Moriarty. 2015. I Sing the Story Electric. PRACTICE. http://ludix.com/
moriarty/electric.html Accessed Jan 2 2020.
[39]
Michael D. Mumford, Shane Connelly, Ryan P. Brown, Stephen T. Murphy, Ja-
son H. Hill, Alison L. Antes, Ethan P. Waples, and Lynn D. Devenport. 2008. A
sensemaking approach to ethics training for scientists: Preliminary evidence of
training eectiveness. Ethics and Behavior 18, 4 (2008), 315–339.
[40]
John Thomas Murray. 2018. Telltale Hearts: Encoding Cinematic Choice-based
Adventure Games. Ph.D. Dissertation. UC Santa Cruz.
[41]
Truong-Huy D Nguyen, Edward Melcer, Alessandro Canossa, Katherine Isbister,
and Magy Seif El-Nasr. 2018. Seagull: A bird’s-eye view of the evolution of
technical games research. Entertainment computing 26 (2018), 88–104.
[42]
Heather L O’Brien and Elaine G Toms. 2008. What is user engagement? A
conceptual framework for dening user engagement with technology. Journal of
the American society for Information Science and Technology 59, 6 (2008), 938–955.
[43]
National Institutes of Health et al
.
1992. Reminder and update: requirement for
instruction in the responsible conduct of research in national research service
award institutional training grants. NIH Guide for Grants and Contracts 21, 43
(1992).
[44]
Natalia Padilla-Zea, Francisco L Gutiérrez, José Rafael López-Arcos, Ana Abad-
Arranz, and Patricia Paderewski. 2014. Modeling storytelling to be used in
educational video games. Computers in Human Behavior 31 (2014), 461–474.
[45]
Nathan Partlan, Elin Carstensdottir, Erica Kleinman, Sam Snodgrass, Casper
Harteveld, Gillian Smith, Camillia Matuk, Steven C Sutherland, and Magy Seif
El-Nasr. 2019. Evaluation of an automatically-constructed graph-based represen-
tation for interactive narrative. In Proc. Foundations of Digital Games. ACM.
FDG ’20, September 15–18, 2020, Bugibba, Malta Melcer et al.
[46]
Dena K. Plemmons, Suzanne A. Brody, and Michael W. Kalichman. 2006. Stu-
dent perceptions of the eectiveness of education in the responsible conduct of
research. Science and Engineering Ethics 12, 3 (2006), 571–82.
[47]
SH Plimpton. 2009. NSF’s Implementation of Section 7009 of the America COM-
PETES Act. Fed. Regist 74, 160 (2009), 42126–42128.
[48]
Sean T. Powell, Matthew A. Allison, and Michael W. Kalichman. 2007. Eective-
ness of a responsible conduct of research course - a preliminary study. Science
and Engineering Ethics 13, 2 (2007), 249–64.
[49]
Teddy Pozo. 2018. Queer games after empathy: Feminism and haptic game design
aesthetics from consent to cuteness to the radically soft. Game Studies 18, 3
(2018).
[50]
Mark O Riedl, Andrew Stern, Don Dini, and Jason Alderman. 2008. Dynamic
experience management in virtual worlds for entertainment, education, and
training. International Transactions on Systems Science and Applications, Special
Issue on Agent Based Systems for Human Learning 4, 2 (2008), 23–42.
[51]
Jake Rossen. 2014. A Brief History of “Choose Your Own Adventure”. Mental
Floss (Apr. 10 2014).
[52]
Jonathan P Rowe and James C Lester. 2010. Modeling user knowledge with dy-
namic Bayesian networks in interactive narrative environments. In Sixth Articial
Intelligence and Interactive Digital Entertainment Conference.
[53]
Jonathan P Rowe, Lucy R Shores, Bradford W Mott, and James C Lester. 2011.
Integrating learning, problem solving, and engagement in narrative-centered
learning environments. International Journal of Articial Intelligence in Education
21, 1-2 (2011), 115–133.
[54]
James Ryan. In press. A Garden, A Forking Path: Interactive Branching Narrative
in The Lady of May (1578). In Proc. 1st Workshop on the History of Expressive
Systems.
[55]
Anastasia Salter. 2014. What is your quest?: from adventure games to interactive
books. University of Iowa Press.
[56]
Anastasia Salter. 2016. Playing at empathy: Representing and experiencing
emotional growth through Twine games. In Proc. International Conference on
Serious Games and Applications for Health. IEEE, 1–8.
[57]
Ben Samuel, Jacob Garbe, Adam Summerville, Jill Denner, Sarah Harmon, Gina
Lepore, Chris Martens, Noah Wardrip-Fruin, and Michael Mateas. 2017. Leverag-
ing procedural narrative and gameplay to address controversial topics. In Proc.
International Conference on Computational Creativity.
[58]
Karen B Schmaling and Arthur W Blume. 2009. Ethics instruction increases
graduate students’ responsible conduct of research knowledge but not moral
reasoning. Accountability in Research 16, 5 (2009), 268–283.
[59]
Stephanie N Seiler, Bradley J Brummel, Kerri L Anderson, Kyoung Jin Kim, Serena
Wee, CK Gunsalus, and Michael C Loui. 2011. Outcomes assessment of role-play
scenarios for teaching responsible conduct of research. Accountability in Research
18, 4 (2011), 217–246.
[60]
Adil E Shamoo and David B Resnik. 2009. Responsible conduct of research. Oxford
University Press.
[61]
Qiulian Song, Ling He, and Xiaoqiang Hu. 2012. To improve the interactivity
of the history educational games with digital interactive storytelling. Physics
Procedia 33 (2012), 1798–1802.
[62]
Ulrike Spierling. 2008. ’Killer Phrases’: Design Steps for a Game with Digital
Role-Playing Agents. In Transactions on edutainment I. Springer, 150–161.
[63]
Katryna Starks, Dakoda Barker, and Alayna Cole. 2016. Using Twine as a Ther-
apeutic Writing Tool for Creating Serious Games. In Proc. Joint International
Conference on Serious Games. Springer, 89–103.
[64] Telltale Games. 2012. The Walking Dead. Telltale Games.
[65] Telltale Games. 2013. The Wolf Among Us. Telltale Games.
[66]
Pengcheng Wang, Jonathan Rowe, Bradford Mott, and James Lester. 2016. De-
composing drama management in educational interactive narrative: A modular
reinforcement learning approach. In International Conference on Interactive Digital
Storytelling. Springer, 270–282.
[67]
Scott Watson, Natalie Vannini, Megan Davis, Sarah Woods, Marc Hall, Lynne
Hall, and Kerstin Dautenhahn. 2007. FearNot! an anti-bullying intervention:
Evaluation of an interactive virtual learning environment. Articial Intelligence
and Simulation of Behaviour (AISB), April 24 (2007).
[68]
Logan L. Watts, Kelsey E. Medeiros, Tyler J. Mulhearn, Logan M. Steele, Shane
Connelly, and Michael D. Mumford. 2017. Are ethics training programs improv-
ing? A meta-analytic review of the past and present ethics instruction in the
sciences. Ethics and Behavior 27, 5 (2017), 351–384.
[69]
Sebastian A Weiß and Wolfgang Müller. 2008. The potential of interactive digital
storytelling for the creation of educational computer games. In International
Conference on Technologies for E-Learning and Digital Entertainment. Springer,
475–486.
[70]
Jui-Feng Weng, Hsiu-lien Kuo, and Shian-Shyong Tseng. 2011. Interactive sto-
rytelling for elementary school nature science education. In 2011 IEEE 11th
International Conference on Advanced Learning Technologies. IEEE, 336–338.
[71]
Peter Weyhrauch. 1997. Guiding Interactive Fiction. Ph.D. Dissertation. Carnegie
Mellon University.
[72]
Caroline Whitbeck. 2001. Group mentoring to foster the responsible conduct of
research. Science and Engineering Ethics 7, 4 (2001), 541–558.
[73]
Lei Zhang, Doug A Bowman, and Caroline N Jones. 2019. Exploring Eects
of Interactivity on Learning with Interactive Storytelling in Immersive Virtual
Reality. In 2019 11th International Conference on Virtual Worlds and Games for
Serious Applications (VS-Games). IEEE, 1–8.
[74]
Alexander Zook, Stephen Lee-Urban, Mark O Riedl, Heather K Holden, Robert A
Sottilare, and Keith W Brawner. 2012. Automated scenario generation: toward
tailored and optimized military training in virtual environments. In Proceedings
of the international conference on the foundations of digital games. 164–171.
A STUDY MEASURES
A.1 Peer Review RCR Quiz
The post-test RCR peer review quiz questions. Questions were taken
from an existing RCR course at the University of Utah:
(1)
"According to the study materials, peer reviewers are asked
to make judgements about the quality of a proposed or com-
pleted project. This certainly includes all EXCEPT the fol-
lowing:"
(Multiple Choice)
(a)
Making sure the conclusions are supported by the evidence
presented.
(b)
Checking calculations and/or conrming the logic of im-
portant arguments.
(c)
Assessing whether the research methods are appropriate.
(d)
Conrming that the relevant literature has been consulted
and cited.
(e) Verifying the qualications of graduate students.
(2)
"If you can gure out the authors of a paper you are peer
reviewing after conicts of interest are disclosed, should you
still review the paper"
(Yes/No)
(3)
"There is no simple solution to the problem of bias in peer
review. However, researchers can lessen the impact of bias
by writing transparent reviews."
(True/False)
A.2 Authorship RCR Quiz
The post-test RCR authorship quiz questions. Questions were taken
from an existing RCR course at the University of Utah:
(1) "When should authorship for a paper be discussed?"
(Multiple Choice)
(a) Just before submitting the paper.
(b) Before starting the paper.
(c) Throughout the process of working on the paper.
(2)
"Which of the following is NOT considered a contribution
to a paper?"
Geing Academical: A Choice-Based Interactive Storytelling Game for Teaching Responsible Conduct of Research FDG ’20, September 15–18, 2020, Bugibba, Malta
(Multiple Choice)
(a) Drafting and editing manuscripts.
(b) Developing methodology.
(c) Dening problems.
(d) Presenting or interpreting theories or ideas.
(e) Being the leading researcher in the eld.
(3)
"There is disagreement over whether authorship should be
limited to individuals who contributed to all phases of a
publication or whether individuals who made more limited
contributions deserve authorship credit."
(True/False)
A.3 Qualitative Moral Reasoning Assessment
The post-test RCR qualitative moral reasoning questions. Questions
were phrased to match the content of the three scoring categories
from the RCR BARS rubric described in [59]:
(1)
"What are the issues in this scenario? Please write a para-
graph."
(2)
"Describe the various viewpoints in this scenario. Please
write a paragraph."
(3)
"What would you propose as a solution? What problems
might occur in resolving these issues? Please write a para-
graph."
B STORIES FOR QUALITATIVE ASSESSMENT
B.1 Peer Review Scenario
Professor John Slater is supervising a research project conducted
by Alice Parker, a graduate student in Slater’s lab. Parker is trouble-
shooting a protein purication protocol; she wants to use the pro-
tocol to purify a recombinant form of a mammalian protein growth
factor expressed in bacteria. Parker needs the puried protein to
complete the nal experiment required to prove her experimental
model. Parker and Slater intend to submit a manuscript based on
this model to The Journal of Cool Results.
While Parker is trouble-shooting the protocol, The Journal of
Cool Results sends Slater a manuscript to review; he is asked to
return the manuscript with his comments and recommendation for
publication. The manuscript turns out to be from a competitor’s
lab, and the title indicates that the work closely resembles the work
Parker and Slater intend to publish.
Slater considers the situation. He decides that he can be objective
in his review, and he proceeds to read and evaluate the manuscript.
After his initial review, he asks Parker for her comments on the
manuscript, as the work falls within her eld of expertise. Slater
and Parker agree that the data are not convincing and that the
paper should not be accepted for publication. Slater returns the
manuscript to the editor of The Journal of Cool Results, with his
recommendation that it not be accepted for publication.
After reviewing the manuscript, Slater and Parker note that the
authors use a recombinant form of the protein growth factor that
they puried from yeast using a novel technique. Slater suggests
that Parker apply this technique to her purication protocol. The
revised protocol works well, and Parker is able to complete the nal
experiment.
Source: To Review or Not - Reviewing the Competition.
https://www.onlineethics.org/Resources/gradres/gradresv2/review.
aspx
B.2 Authorship Scenario
Mike is a bright, young post-doc working in a big research group
in the physics department at Bambuka University. His life-long
career goal is to conduct research in a leading research university
as a professor. During one of his job interviews, he had a discus-
sion about a particular problem in his eld of expertise with the
interviewer. In the course of the interview, he was not able to satis-
factorily prove his point, because his theoretical arguments did not
convince the interviewer. Upon returning to his lab, Mike decides
to pursue the matter further and conduct an experiment to verify
his argument. Mike’s experimental background is not sucient to
obtain the desired results.
Lisa, Mike’s friend, is a fourth-year graduate student working
on her PhD in the same lab. She volunteers to help Mike with the
experiment. Lisa is a talented experimentalist, and she successfully
completes the experiment. Mike sends the results to the interviewer,
thereby proving his point.
While working on this small experiment, Mike gets an idea
for an interesting study, which, if done correctly, could yield a
good publication in an important journal. But Mike is discouraged,
because he knows he cannot handle the complicated experiment
alone. Lisa encourages him to proceed with the idea and promises
to design and complete the experimental aspect of the project. Mike
agrees and while he is working on the theory, Lisa designs and
builds the experiment. Mike is very excited about his theoretical
results and shares them with his adviser. The professor likes Mike’s
ideas and tells him that it is time for Mike to get his name noticed
in the scientic community. He encourages Mike to publish the
results in a famous journal. The adviser also suggests that it would
be better for Mike’s career if he publishes the work in a single
author paper. He says, "You worked on it exclusively, and it would
be a wonderful opportunity to write a paper by yourself. It will be a
stronger paper if you could validate your theory with experimental
data."
Mike likes the suggestion. He doesn’t mention that Lisa has
already done a signicant amount of work on the project. He tells
Lisa that his adviser recommended his publishing results in a single
author paper, and says, "I really think that this would help my career,
plus that’s what our adviser wants. How cheated will you feel if
I publish this paper alone using all the data that your experiment
provides?"
Lisa and Mike are good friends, and she feels obligated to help
him. Even though Lisa is disappointed, she tells Mike to do whatever
he feels is right. Mike decides to submit the paper as the sole author.
FDG ’20, September 15–18, 2020, Bugibba, Malta Melcer et al.
After this conversation, Lisa stops working on the experiment
and Mike takes over. He did not design the experiment; therefore
he cannot manage to get it to work and does not make any progress.
Lisa does not oer any more help, and Mike doesn’t ask her for
any. Finally, Mike decides to submit the paper without the experi-
mental part. It will be an interesting theoretical investigation, but
it will not have the scientic impact that it could have had with the
experimental validation.
Source: A Single Author Paper.
https://www.onlineethics.org/Resources/gradres/gradresv4/single.
aspx
... The convergences of technology and interactivity into narratives enrich the storytelling deliverances to the viewer [1]. The prospects of interactive storytelling as the potential medium for promoting rich content in the creative industry should be taken into considerations by the industry player and the academia [2]- [3]. Minimal research has been investigated on evolution trends of interactive storytelling globally. ...
... Interestingly, broad application of interactive storytelling can be seen in entertainment, infotainment and edutainment sectors globally [2,3,6]. Besides this, several medium of interactivity are identified by recent researchers for the interactive storytelling namely the Virtual Reality (VR) storytelling that utilizes virtual environments (VE), interactive 3D applications and Interactive web document arises that promotes user engagement to the topics through innovative storytelling approaches [6]- [8]. ...
Article
The explosions of new media led the studies of interactive storytelling in the area of creative industry. The field of interactive storytelling is identified as the key prospect areas of the study. Hence future research directions are call upon for the growth benefits of the research field. This study aims to assess the evolution of publication trends in interactive storytelling between the year of 1996 to 2020 through a bibliometric analysis. A sample of 795 studies from the SCOPUS database were analysed via the VOSviewer and Harzing’s Perish or Publish software tools to distinguish research activity on interactive storytelling. The identification of the dominant articles and authors are traced based on the event of the citations, publications, its location and network. The growth on the related publication somewhat slow from 1996 until it starts picking up in 2007 with an average of 36 publications a year since then. The highest number of publications is observed in 2011, with a total of 71 documents (8.93%). The subject of Computer Science is majorly depicted on the studies of Interactive Storytelling (53.5%), followed with Mathematics (24.53%). Meanwhile, the subjects of Engineering (7.30%), Social Sciences (6.47 %), and Arts and Humanities (3.99%) contributing to the total publications of Interactive storytelling. Furthermore, computer science and mathematics subject are the most represented for the studies of Interactive Storytelling to explain the complexity and technicality aspects of the scientific narratives with the compelling features of the interactivity.
... This accessible aspect of interactive narratives has made them highly popular with men, women, and even novices to games [12], as well as for educational/training purposes both commercially and in academia [11,18,46,50]. Furthermore, the role-playing nature of interactive narratives has been shown to be highly effective at improving player attitudes [24,25], motivation [23], knowledge [45], and skills [44]. This makes the use of interactive narrative games an ideal approach for training resiliency as it easily and intuitively enables learners to role-play through various stereotype and social identity threat scenarios. ...
Chapter
Full-text available
Stereotype threat and social identity threat are social phenomena that adversely affect underrepresented groups within STEM (i.e., women and people of color). While there are existing programs and techniques for training resiliency against these threats, the use of biofeedback and serious games may prove useful to enhance the efficacy and engagement of such training. In this paper, we present the work in progress on our interactive narrative biofeedback game (Resilient IN) that utilizes resonant frequency heart rate variability to train player resilience to stereotype and social identity threat as they move through a mock interview at a tech company within the game. Specifically, we discuss the design of the game in detail—focusing on how specific elements of the design draw from existing literature to evoke and train resilience during play, as well as design and validation of the game narrative/script with individuals in the technology and engineering industry. Finally, we provide future directions for the work, such as upcoming studies to validate the game’s efficacy in evoking and training resiliency to different kinds of threats.KeywordsSocial identity threatStereotype threatSerious gamesHeart rate variabilityBiofeedback
... An advantage of using games for education is interactivity. Digital video games have been used in educational settings for the purpose of learning [9], training [10], assessment [11], and experience [12]. Using games for education promotes interactivity, engagement and increases motivation [13]. ...
Conference Paper
Full-text available
This research describes the development of an adaptive virtual assistant in an immersive virtual reality (VR) serious game aimed at teaching engineering students manufacturing concepts. For undergraduate manufacturing education, students need to learn product design and manufacturing systems that require well-coordinated analysis of requirements and hands-on practices in complex manufacturing assembly lines. While it is often not feasible and practical for students to participate in real factory environments, simulations are created to offer a flexible alternative of digital learning. With the advancements in immersive technologies, VR opens new opportunities for teaching and learning manufacturing, and enables remote learning from any physical location. In this research, we describe the elements of a serious game built using the Unity game engine with VR technology that allows students to practice the concept of craft production. Prior research has shown that adapting learning material to suit individual student needs increases motivation and student successes. While learning remotely using an immersive virtual environment, a student is often working in an independent manner. Seeking help often requires the student to leave the virtual environment and break immersion. In this research, we propose an adaptive virtual assistant in the game environment to support the student learning process. By tracking student actions in the game environment and building a model of the student using reinforcement learning, the virtual assistant can learn and adapt to the student's preference in the types of assistance to provide. We show the adaptation of the virtual assistant through simulated experiments of typical student preferences.
... We have conducted two studies evaluating the efficacy of Academical for teaching RCR learning outcomes Grasse et al. (2022). Our first study was a randomized group comparison study (N = 14 for each group) that showed that our game is equal or better than traditional web-based training materials at teaching students RCR knowledge and moral reasoning skills Melcer et al. (2020a). Notably, this experiment also revealed that the game was significantly more engaging than the web-based training materials Melcer et al. (2020b). ...
Article
Full-text available
Choice-based interactive storytelling games such as Academical, our responsible conduct of research training game, show great promise as a novel way of providing efficacious ethics training. However, much work remains to determine what factors of such games contribute to their advantages over traditional text-based training tools, especially if we hope to further improve their enjoyment, engagement and efficacy. In this article, we present a case study exploring how the motivational factors of Self-Determination Theory (SDT) underlie players’ perceived most and least enjoyable experiences arising from the design of Academical. Specifically, we discuss how certain elements of Academical’s design influence different SDT factors and subsequently player experience, as well as how such elements can be changed to further improve the game. Furthermore, our work highlights potential limitations of existing conceptualizations for the relatedness factor of SDT—discussing ways that it can be extended to properly understand player enjoyment within single-player educational interactive narrative games.
... Interactive story-telling games, for instance, have a meaningful learning potential as an essential part that can be added to maintain and increase students' motivation and content retention (Habes et al., 2020;Young et al., 2015). Stories with a choice-based format allow players to experience the plot from multiple perspectives, prompting critical thinking and exploring complex topics such as moral or ethical choices (Melcer et al., 2020). ...
Article
Background: The current pandemic has led to unprecedented scenarios worldwide. In this context, educational institutions had to move abruptly from in-person to online classes, having to be flexible and innovative, one of the main concerns being the semester-end examinations in practical modules. Objective: This study aimed to explore nursing students’ perceptions of the use of a serious game-like model in their final online objective structured clinical examination (OSCE). Design: An exploratory phenomenological study was conducted during the COVID-19 pandemic, in particular during June 2020. Settings: This study took place at the University of Almeria with nursing students enrolled in a clinical placement module. Participants: Eighty-nine fourth-year nursing students took part in this study. Methods: Following the educational principles of the OSCE, a virtual OSCE was designed, using the Genially platform, an online platform for creating interactive content, to set up an assessment as a story-telling game-like model. A total of 5 focus groups (FGs) were conducted afterwards via the GoogleMeet platform. In addition, 30 semi-structured interviews were performed via the same platform. The data was analysed using a content analysis approach and supported by the ATLAS.ti 8.4 software. Results: Our finding revealed 2 main themes and 4 sub-themes. The two main themes were (i) generating emotions and feelings in times of virtuality; including emotions and feelings experienced by students during their online assessment process, and (ii) online assessment: a potential alternative to educational barriers; describing the variability of traditional modality, its implications for learning and the acquisition of competences. Conclusions: Serious game-like models, such as the story-telling game proposed, as part of their online OSCE assessment appear to be an appropriate alternative assessment method for face-to-face approaches. This paper adds new evidence on the use of innovative and state-of-art resources as part of nursing OSCE assessments in a new reality for most students and teachers. Serious game-like models in online OSCE may empower students and help them to remove perceived barriers in face-to-face assessments.
... Furthermore, it is essential to point out that SMGs are rarely centered on supporting the development of MS. We are aware of only one study that successfully developed a game to promote moral sensitivity and knowledge in the context of responsible conduct of research (Melcer et al., 2020). ...
Article
Full-text available
Serious games have emerged as a promising new form of education and training. Even though the benefits of serious games for education are undisputed, there is still a further need for research on the efficacy of such games. The main goal of our research is to examine the effectiveness of a serious moral game—uFin: The Challenge—that was designed to promote moral sensitivity in business, a precondition of ethical decision-making and behavior and a core moral competency of moral intelligence. A second goal is to examine the role of metacognitive prompting and prosocial nudging in influencing learning effectiveness. Participants (N = 345) took part in an experimental game-based intervention study and completed a pre- and post-test questionnaire assessing moral sensitivity. The analyses of both questionnaire and game data suggest that merely playing this game is effective in promoting moral sensitivity. Neither self-reflection nor exposure to prosocial nudges, however, were determined to be factors that improve learning effectiveness. In contrast, those interventions even decreased the learning outcome in some cases. Overall, findings demonstrate the potential for game-based learning in the moral domain. An important avenue for future research is to examine others ways of increasing the effectiveness of the game.
... First, VNs only show one snippet of text at a time as the player interacts with the game [31], the snippet being dependent on player action or choices and even shaping future interaction. Further, while VNs share many traits with choose-your-own adventure books [10]-a legacy shared with most modern interactive fiction and storytelling design in video games [34,62,64]-VNs often incorporate more involved mechanical and constraint based gameplay to impact the progression of the story and the relevance/complexity of player choice [15]. ...
Article
Full-text available
Visual Novel (VN) is a widely recognizable genre of narrative-focused games that has grown in popularity over the past decade. Surprisingly, despite being so widely recognizable, there is not a singular definition to help guide the design and analysis of such games---with academic definitions and implementations ranging from "interactive textbooks" to "adventure games with multi-ending stories". In this paper, we present a unified definition of VNs drawn from an analysis of 30 prior academic definitions. We also examined 54 existing VNs to further refine our definition and employ a deeper analysis of the interactivity within VNs. We highlight key features of VNs that arise in our definition, and discuss resulting implications for the design of VNs. This work is relevant for narrative game designers and researchers, affording a more unified structure and clearer guidelines to identify, analyze, and design future VN games.
... In general, case studies, role plays and other constructivist approaches have been shown to be especially effective techniques for research ethics instruction (Freeman et al. 2014;Atkinson 2008;Todd et al. 2017;Tammeleht et al. 2019). Recently, researchers have also demonstrated that choice-based interactive simulation games provide another constructivist approach with potential to reach wide audiences (e.g., Schrier 2015; Katsarov et al. 2020;Melcer et al. 2020;Grasse et al. 2021;Tanner et al. 2021). As Yarborough and Hunter (2013) and others discuss, the method of delivery affects level of student engagement, and the academic community within which a student is embedded will also affect student socialization with respect to research ethics. ...
Article
Worldwide, undergraduate science and pre-medical students are encouraged to participate in authentic active learning lab work and undergraduate research experiences. Unfortunately, these experiences rarely include training in science or research ethics. Although several governmental and scientific organizations have called for increased training in responsible research conduct, relatively few studies report on the effectiveness of different pedagogical approaches. Too often science ethics socialization and training is limited to conversations with individual mentors. This paper describes how viewing an interactive theatrical presentation of several research misconduct scenarios was associated with an increase in first-year students’ self-assessed understanding of the topics addressed: proper treatment of data images, respect for animal protocols, authorship considerations, and plagiarism issues. There was no decrease in self-reported responsible conduct of research (RCR) knowledge for students surveyed 10 weeks, as compared to 2 weeks, after the science ethics presentations. RCR test question scores showed only a slight decrease in correct answers from 2 to 10 weeks. Theatrical presentation is an inexpensive yet engaging approach that provides students with a chance to actively consider the importance of RCR and the complexities of contexts surrounding ethics decisions before starting a research career.
Conference Paper
Full-text available
The premise that "good" games embody sound pedagogy in their designs, even if incorporation was not deliberate, suggests that commercial entertainment games may also hold surprising educational potential. However, there is limited research exploring the potential learning experiences that entertainment games can provide, as well as how such unintended experiences could influence players' everyday lives. In this paper, we present an exploratory study surveying thirteen university students to understand their perceived learning experiences from entertainment games, how they applied these experiences to their lives, and why they believed the experiences were personally impactful. We found that participants believed they learned (1) practical skills of collaboration and planning, and (2) a wide range of everyday knowledge and educational content. Additionally, we found all reported experiences were relevant and meaningful to players' lives outside of the game. Lastly, we utilize findings to inform the design of games beyond entertainment, identifying potential areas for improved educational game design.
Conference Paper
Full-text available
Interactive narratives are widely used to frame and contextualize education in games. However, the specifics of how their designs aid the learning process and outcomes remains relatively unexplored. To better understand this space, a study was conducted that focused on one sub-genre of interactive narrative, Visual Novels. Specifically, in this paper we conducted a survey of thirty-one existing educational Visual Novels, analyzing design elements that fostered learning and delivered educational content. The resulting taxonomy consists of five key dimensions for educational design and teaching strategies within Visual Novels: 1) Teaching Through Choice, 2) Teaching Through Scripted Sequences, 3) Teaching Through Mini-games, 4) Teaching Through Exploration and 5) Non-interactive Teaching. These dimensions demonstrate that there are a number of design considerations for supporting learning through Visual Novels. This work has implications for designers of educational games by classifying the different designs a Visual Novel can employ to teach-ultimately informing how to better present educational subject matter in interactive narrative games.
Conference Paper
Full-text available
Concepts utilizing applied ethics, such as responsible conduct of research (RCR), can prove difficult to teach due to the complexity of problems faced by researchers and the many underlying perspectives involved in such dilemmas. To address this issue, we created Academical, a choice-based interactive storytelling game for RCR education that enables players to experience a story from multiple perspectives. In this paper, we describe the design rationale of Academical, and present results from an initial pilot study comparing it with traditional web-based educational materials from an existing RCR course. The preliminary results highlight that utilizing a choice-based interactive story game may prove more effective for RCR education, with significantly higher engagement and comparable or better scores for tests of RCR topics.
Conference Paper
Full-text available
Interactivity and player experience are inextricably entwined with the creation of compelling narratives for interactive digital media. Narrative shapes and buttresses many such experiences, and therefore designers must construct compelling narrative arcs while carefully considering the effects of interaction on both the story and the player. As the narrative becomes more structurally complex, due to choice-based branching and other player actions, designers need to employ commensurately capable models and visualizations to keep track of that growing complexity. However, previous models of interactive narrative have failed to fully capture interactive elements with automated, operationalized visualizations. In this paper, we describe an algorithm for automated construction of a framework-driven, graph-based representation of interactive narrative. This representation more fully and transparently models structural and interactive features of the narrative than did prior approaches. We present an initial evaluation of this representation, based on modified cognitive walkthroughs performed by interactive narrative design and research experts from our research team, and we describe the takeaways for future improvement on interactive narrative modeling and analysis.
Conference Paper
Full-text available
Music and rhythm are powerful tools that can be employed to enhance learning and memory. While games are commonly utilized to aid in second language acquisition, few have explored the implications of sound on learner's ability to draw and remember logographic characters (such as those in Chinese hanzi, Japanese kanji and Korean hanja). We created Radical Tunes, a kanji drawing music game, to explore the impacts of incorporating music on players' ability to retain meaning and stoke order of several kanji. In this paper, we describe the design rationale for Radical Tunes, and present results from a pilot study comparing a music focused version of the game with one that uses non-musical sound effects. Results show that while both conditions improved players' short-term ability to remember/draw kanji, there were no significant differences in improvement between the conditions. However, the use of music did improve immersion-an important factor related to learning. This work has implications for designers of second language acquisition games, and how they can incorporate rhythm and music into their games to increase player engagement.
Conference Paper
Full-text available
Immersive virtual reality (VR) holds great potential for learning, but it is unclear how VR experiences should be designed to maximize learning potential. In this study, we explored how the level of interactivity in an educational VR storytelling experience for immunology learning affects a user's learning gains. We created three versions of the VR experience with low (system automates as many actions as possible), medium (a combination of system automation and user-controlled actions), and high (as many user-controlled actions as possible) levels of interactivity. We hypothesized that too much or too little interactivity would result in smaller learning gains than a medium level of interactivity. Although data from pre and post-tests showed no significant difference in students' learning gains due to interactivity level, questionnaire and interview data suggest that interactivity in the experience significantly affects students' engagement in learning, attention, and focus on learning material. Participants also perceived that they could learn better and more effectively in a VR experience with a higher level of interactivity.
Article
Full-text available
Choice poetics is a formalist framework that seeks to concretely describe the impacts choices have on player experiences within narrative games. Developed in part to support algorithmic generation of narrative choices, the theory includes a detailed analytical framework for understanding the impressions choice structures make by analyzing the relationships among options, outcomes, and player goals. The theory also emphasizes the need to account for players’ various modes of engagement, which vary both during play and between players. In this work, we illustrate the non-computational application of choice poetics to the analysis of two different games to further develop the theory and make it more accessible to others. We focus first on using choice poetics to examine the central repeated choice in “Undertale,” and show how it can be used to contrast two different player types that will approach a choice differently. Finally, we give an example of fine-grained analysis using a choice from the game “Papers, Please,” which breaks down options and their outcomes to illustrate exactly how the choice pushes players towards complicity via the introduction of uncertainty. Through all of these examples, we hope to show the usefulness of choice poetics as a framework for understanding narrative choices, and to demonstrate concretely how one could productively apply it to choices “in the wild.”
Conference Paper
Full-text available
While recent work has begun to evaluate the efficacy of educational programming games, many common design decisions in these games (e.g., single player gameplay using touchpad or mouse) have not been explored for learning outcomes. For instance, alternative design approaches such as collaborative play and embodied interaction with tangibles may also provide important benefits to learners. To better understand how these design decisions impact learning and related factors, we created an educational programming game that allows for systematically varying input method and mode of play. In this paper, we describe design rationale for mouse and tangible versions of our game, and report a 2x2 factorial experiment comparing efficacy of mouse and tangible input methods with individual and collaborative modes of play. Results indicate tangibles have a greater positive impact on learning, situational interest, enjoyment, and programming self-beliefs. We also found collaborative play helps further reduce programming anxiety over individual play.
Article
Within Entertainment Computing, games research has grown to be its own area, with numerous publication venues dedicated to it. As this area evolves, it is fruitful to examine its overall development—which subcommunities and research interests were present from the start, which have come and gone, and which are currently active—to better understand the research community as a whole and where it may proceed. In this paper, we present a data-driven analysis and interactive visualization tool to shed light on how technical domains within the games research field have evolved from 2000 - 2013, based on publication data from over 8,000 articles collected from 48 games research venues, including Entertainment Computing, FDG, AIIDE, and DiGRA. The approach we present is descriptive. We first used data mining algorithms to group related papers into clusters of similar research topics and evolve these clusters over time. We then designed an interactive visualization system, named Seagull, comprised of Sankey diagrams that allow us to interactively visualize and examine the transition and coalescing of different clusters across time. We present our descriptive analysis in this paper and also contribute the visualization interface to allow other researchers to examine the data and develop their own analysis.