PosterPDF Available

Synergies between research and game design: Reflections on interactive narrative experiments by student game designers

Authors:

Abstract and Figures

One way to prepare research-literate citizens is to engage them in designing research studies. However, this feat is challenging and has many practical barriers. We present StudyCrafter, a new platform that makes human behavior research more accessible by using game design and gameplay as a framework for research design and participation. In a qualitative exploration of students’ StudyCrafter projects within a graduate classroom implementation, we found that those projects that demonstrated well-executed principles of game design also showed qualities of robust research study design. We discuss the potential instructional value of using game design to inform research design, and the importance of accessible tools and environments. These findings can inform supports for developing students’ research design and literacy.
Content may be subject to copyright.
Findings'
Projects*ranged*in*foci*(effect*of*interpersonal*relations*on*resource*management,*influence*of*context*on*motivation,*impact*of*marketing*strategies*on*purchasing*behavior,*implicit*bias,*effect*of*NPCs*on*player*behavior,*etc.)*
Students*showed*diverse*abilities*in*each*of*game*design*and*research*design.*
An*extreme*focus*on*game*design*appeared*to*downplay*the*research*design,*and*vice*versa.*
METHODS'
Synergies Between Research and Game Design: Reflections on Interactive Narrative Experiments
by Student Game Designers Matuk, C., Sutherland, S., Althoff, W., Snodgrass, S., Partlan, N., Smith, G., Seif El-Nasr, M. & Harteveld, C.
STUDYCRAFTER'
RATIONALE'
Game'design'as'an'opportunity'to'support'research'literacy'
Learning*to*design*research*experiments*is*one*way*to*cultivate*citizens’*general*research*literacy,*
but*presents*barriers*[1]:*
Conceptually,*students*struggle*with*operationalizing*variables,*manipulating*independent*
variables,*and*controlling*for*threats*to*internal*validity.*
Practically,0studying*human*behavior*in*authentic*situations*is*difficult*in*both*laboratory*and*
real-life*settings,*and*impractical*within*course*settings.*
Principles*of*good*game*design*align*with*those*of*good*research*design:*
Good'games'compel*players*to*participate*by*immersing*them*in*believable*narratives,*and*by*
presenting*consequential*choices.*
Robust'research'study'designs'compel*participants*to*behave*in*ways*that*are*consistent*with*
the*context,*in*spite*of*it*being*simulated;*they*show*experimental'and*mundane'realism.*
*
OBJECTIVES'
We*observe*students*using*an*online*platform*to*create*and*run*experiments*into*human*behavior.*
We*explore*the*potential*for*using*game*design*as*a*framework*to*guide*students’*research*study*
design.*
Research'questions'
What*principles*of*research*and*game*design*are*apparent*in*students’*projects?*
In*what*ways*are*students’*research*study*designs*strengthened*by*their*use*of*game*design*
principles?*
CONCLUSIONS'
Game*design*offers*a*useful*framework*for*research*study*design,*but*balancing*their*execution*is*an*
ongoing*challenge*for*students.*
Findings*contribute*to*research*on*constructionist*gaming*[2]*and*to*the*growing*interest*in*using*games*
and*interactive*fiction*to*study*human*behavior*[3,4].*
Future*work*might*ask:*
How*do*different*levels*of*game*design*expertise*affect*students’*research*design*abilities?*
What*research-relevant*attitudes,*concepts,*and*skills*might*game*design*promote*in*students?*
What*instructor*and/or*system-automated*support*might*students*require*to*excel*in*using*games*as*
a*framework*for*learning*to*design*research*studies?*
*
*
Features'of'robust'
StudyCrafter'designs'
Example'from'
students’'projects'
Non-example'from'
students’'projects'
RESEARCH'DESIGN'
Mundane'
realism*People*and*
contexts*investigated*are*
true*to*reality,*such*that*
findings*are*likely*to*
generalize*to*real-world*
situations.*
Space0Exploration*uses*
dark*character*
silhouettes*to*
represent*the*
characters*of*the*
participant*and*a*
partner*selected*by*
the*participant.*This*
avoided*stereotyping*
the*characters*(space*
explorers)*as*
participants*could*
project*their*own*
mental*image*onto*
the*silhouettes.*
Buy0Courses*asks*players*
to*look*at*various*course*
bundle*packages*and*
make*the*best*decision*for*
a*character*named*Tom*
who*wants*to*take*a*
number*of*courses*in*
history.*However,*the*user*
is*not*provided*with*any*
other*information*and*the*
resulting*decision*may*
therefore*be*arbitrary*or*
biased*by*assumptions*
that*participants*make.**
Experimental'
realism'Goals,*
tasks,*and*choices*for*
which*there*are*
experienced*
consequences,*creating*
situations*in*which*
participation*feels*
meaningful.*
Click0Money*shows*
participants*a*
leaderboard*(although*
non-functioning)*to*
track*the*number*of*
clicks*they*perform,*
which*affects*the*
outcome*of*the*game.*
*
In*Wormhaven*players*are*
part*of*an*apocalyptic*
multiplayer*game*and*are*
asked*to*trade*different*
items*of*value*with*
another*player*(which*is*
randomized*in*terms*of*
experience).*Neither*the*
purpose*of*the*exchange*
nor*the*value*of*the*items*
are*communicated,*
making*this*exchange*
arbitrary.**
GAME'DESIGN'
Player'
Agency'Clear*goals,*
choices*with*feedback,*and*
performance*incentives*
give*players*the*sense*that*
their*actions*have*
consequences,*and*are*of*
their*own*volition.*
Yet0Another0RPG0
shows*participants*the*
damage*that*their*
character*takes*as*an*
outcome*of*their*
decisions.*This*can*
then*be*used*to*
inform*decisions*
about*what*upgrades*
to*select.*
In*Robot0Wars*players*
have*to*defeat*elitist*
robots.*While*the*robots*
are*creatively*put*
together,*and*the*
narrative*makes*
intelligent*use*of*humor,*
essentially*all*the*player*
does*is*to*click*through*a*
story*with*only*a*few*
actual*decisions.**
Compelling'
Narrative'
Immersion*is*created*
through*plot*and*character*
archetypes,*humor*and*
playfulness,*and*inclusive*
representations*that*
reflect*players*and*their*
realities,*and*that*together*
create*a*sense*of*being*
part*of*a*world*and*an*
unfolding*story.*
An0Unusual0Situation*
places*participants*in*
the*role*of*a*character,*
whose*situation*
unfolds*through*a*
mixture*of*narration*
and*character*
dialogue.*This*project*
also*intersperses*
scenes*with*text-only*
screens*to*convey*the*
sense*of*time*passing*
between*events.*
Additionally,*An*
Unusual*Situation*
transitions*between*
1st*person*and*3rd*
person*narrative*at*
times,*which*may*
influence*how*
participants*interpret*
the*situations*in*the*
game.**
Colors0and0Shapes*
displays*a*series*of*
screens*on*which*players*
are*asked*to*choose*which*
of*two*shapes*they*think*
is*best*associated*with*
each*color.*It*provides*no*
narrative*or*characters,*
nor*does*it*offer*any*
context*outside*of*the*
description*of*the*
experiment.*In*this*sense,*
it*resembles*a*typical*
laboratory*study,*which*
attempts*to*isolate*
variables*under*study.*
REFERENCES'
1.Woolley*JS,*Deal*AM,*Green*J,*Hathenbruck*F,*Kurtz*SA,*Park*TKH,*et*al.*
Undergraduate*students*demonstrate*common*false*scientific*reasoning*strategies.*
Thinking*Skills*and*Creativity.*2018;27:*101–113.*doi:10.1016/j.tsc.2017.12.004*
2.Kafai*YB,*Burke*Q.*Constructionist*Gaming:*Understanding*the*Benefits*of*Making*
Games*for*Learning.*Educ*Psychol.*2015;50:*313–334.*doi:
10.1080/00461520.2015.1124022*
3.Bainbridge*WS.*The*scientific*research*potential*of*virtual*worlds.*Science.*
2007;317:*472–476.*
4.Calvillo-Gámez*E,*Gow*J,*Cairns*P.*Introduction*to*special*issue:*Video*games*as*
research*instruments.*Entertain*Comput.*2011;2:*1–2.*
**
Week' Activity'
1*
Instructor*demonstrates*
StudyCrafter.*
Students*submit*project*
proposals*and*form*teams*based*
on*study*themes.*
2* Project*work*in*and*out*of*class*
Receive*instructor*feedback*
3* Students*recruit*participants*
Start*data*collection*
4* Analyze*data,*write*up*findings*
Conduct*peer*review*
5* Present*projects*
Submit*final*reports*
Participants'&'Context'
21*students*(13*m,*8*f)*enrolled*in*graduate-level*course*on*
user*research*methods*for*game*design,*at*an*eastern*US*
university.*
Students*ranged*in*academic*backgrounds,*and*in*abilities*in*
programming,*language,*and*familiarity*with*basic*research.*
Students*completed*a*5-week*module*on*research*design,*in*
which*they*used*StudyCrafter*to*design*and*conduct*their*
own*research*studies.*
Data'&'Analysis'
Data*included*students’*playable*projects*and*scripts,*
accompanying*final*reports*and*the*instructor’s*course*
observations.*
Project*features*were*organized*into*a*matrix*based*on,*for*
example,*research*foci,*context*of*study*(game*or*real-life),*
and*study*design.*
Themes*around*research*design*(mundane*&*experimental*
realism)*and*game*design*(narrative*&*agency)*were*refined*
through*consensus*coding.*
Projects*were*ranked*along*these*thematic*dimensions.*
Studycrafter.com*
A*free,*online*platform*with*tools*to*visually*script*games*and*interactive*stories,*called*Projects.*
An*Asset0Library0offers*a*choice*of*pre-designed*characters,*objects,*and*backgrounds*that*users*
arrange*within*a*Layout.*
The*Layout*is*a*visual*of*one*or*more*Scenes.*
An*Editor0allows*users*to*script*what*will*occur*in*each*scene*layout,*and*under*what*conditions.*
Links*to*StudyCrafter*projects*can*be*distributed*to,*and*played*by*participants.*
Data*from*players’*interactions*with*the*projects*are*logged*for*project*creators*to*analyze.*
Schedule'
Limited'research'design,'limited'game'design'
Course'Selection'Frenzy:'Does*scarcity*make*a*
resource*more*desirable?*
*
*
*
*
*
*
*
'
'
'
'
Between-subjects'design:'Participants*either*select*from*among*two*courses*with*
the*same*or*different*seat*capacities.*
*
Hypothesis:*Participants*will*choose*the*course*with*less*capacity*more*frequently,*
and*take*less*time*to*decide*on*this.*
*
Critique:'
The*limited*narrative*context*allowed*isolation*of*the*factor*of*interest*(seat*
capacity)*
The*lack*of*contextual*factors*that*typically*affect*course*choices*(e.g.,*credit*
status,*degree*requirements,*schedule,*instructor)*lowered*mundane*and*
experimental*realism.*
Funding'for'this'research'
was'provided'by'the'
National'Science'
Foundation,'an'IUSE'under'
award'numbers'
1736065,*1736185,1736056*
Robust'research'design,'limited'game'design'
Colors'and'Shapes:'Do*people*associate*color*saturation*with*
shape?*
*
*
*
*
'
'
Within-subjects'design:'Participants*view*a*series*of*colors*of*varying*saturation,*and*choose*either*the*
geometric*or*the*irregular*shape*that*they*feel*best*corresponds*to*it.*
*
Hypothesis:*Participants*will*associate*saturated*colors*with*geometric*shapes,*and*less*saturated*
colors*with*irregular*shapes.*
*
Critique:'
High*in*experimental*realism,*low*in*mundane*realism.*
A*lack*of*features*of*interactive*fiction*and*game*design*placed*this*project*closest*to*the*canonical*
lab-based*study.*
Robust'research'design,'robust'game'design''
An'Unusual'Situation:'Does0the0Halo0Effect0exist0in0
a0game0setting?*
*
*
*
*
*
*
*
'
'
'
'
'
'
'
'
'
'
'
'
'
'
'
Between-subjects'design:'Participants*re-encounter*a*long-time*acquaintance,*Bill.*
In*one*condition,*Bill*reveals*that*he’s*the*one*behind*the*player’s*unemployment.*
In*another*condition,*Bill*attributes*his*success*to*the*player.*
*
Hypothesis:*Participants*who*encounter*the*morally*ambiguous*Bill*vs.*the*virtuous*
Bill*will*be*more*likely*to*later*punch*the*stranger*in*the*bar.*
*
Critique:'
Mundane*and*experimental*realism*were*strengthened*by*the*effective*use*of*
game*features*(e.g.,*narrative*closure,*dialogue,*pacing,*player*agency).*
Robust'research'design,'limited'game'design'
Spider'Lab:'How*does*a*fixed*vs.*random*reward*system*affect*
players’*desire*to*continue*playing?*
*
*
*
'
'
'
'
'
'
'
'
'
'
'
Between-subjects'design:'Participants*complete*rounds*of*spider*killing*to*gain*enough*“knowledge”*
points*to*inform*the*creation*of*a*pesticide.*In*one*condition,*knowledge*gained*after*each*round*is*
fixed.*In*another,*it*is*random.*After*a*number*of*rounds,*participants*are*given*the*choice*to*continue*
killing*spiders*to*gain*enough*“knowledge”*to*cure*cancer.*
*
Hypothesis:*Participants*will*be*more*likely*to*continue*playing*within*a*fixed*reward*system.*
*
Critique:*Sophisticated*game*features*(player*agency,*interactivity)*but*weak*execution*(e.g.,*grammar,*
pacing)*reduced*the*sense*of*player*immersion*and*lowered*the*study’s’*mundane*realism.**
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
American education is failing to fill the growing demand for science, technology, engineering, and mathematics (STEM) graduates. The lack of critical reasoning skills may be a causal factor in student attrition from STEM majors. Our objective in this study was to discover and describe common false strategies used by undergraduate students during the scientific reasoning process. Each of these false strategies is described, with accompanying examples from student responses, to illustrate the thinking patterns. We defined targeted areas for instruction that can lead to better performance, greater academic self-confidence, and increased retention in STEM degrees. Understanding how students think through problems and where they are making mistakes facilitates the creation of specialized programs to correct these false reasoning strategies and increase the scientific reasoning ability of students.
Conference Paper
Full-text available
Having students create their own experiments is an important way to teach science inquiry and research methods. Additionally, having individuals create experiments to replicate previous research creates a better understanding of the concepts derived from previous findings. Replication is often difficult due to limited access to the necessary tools. However, with the increased prevalence of digital games and the ease with which new games can be modified for research, games offer the promise that individuals will be able to recreate experiments previously difficult to replicate. In this paper we discuss a new digital game, Mad Science, created specifically for novice users with no programming skills to create their own experiments and to recruit and collect data from participants. Our findings suggest that replication of previous findings can be accomplished using this platform and will be a valuable tool for teaching concepts and methods from the decision science literature. Games as research method, games as instructional method, replication
Conference Paper
Full-text available
In this paper, we explore the topic of games as research methods, which has received little attention and has much promise. More specifically, our aim is to provide insight into what design considerations are made for creating game-based social experiments. In order to achieve this, we discuss our experiences of translating a traditional experiment focused on the decoy effect into a game-based experiment. Our preliminary results suggest that monetary incentives with no real-world consequences can affect behavior even when time delays with real-world consequences were a competing factor. This has implications for how research can be gamified.
Conference Paper
Full-text available
Game-like environments are increasingly used for conducting research due to the affordances that such environments offer. However, the problem remains that such environments treat their users equally. In order to address this, personalization is necessary. In this paper we discuss the need to personalize gamified research environments to motivate participation by illustrating a playful platform called Mad Science, which is being developed to allow users to create social and behavioral studies. This discussion is both informed by the platform's affordances and use thus far as well as existing theories on player motivation, and contributes to theory-informed approaches to (gamified) personalization technologies.
Conference Paper
Full-text available
Playful environments are increasingly being used for conducting research. This makes a game platform for authoring research studies and teaching about how to conduct research a necessary progression. In this paper, we discuss Mad Science, a playful platform that is being created to allow users to create behavioral experiments. We discuss iterations of the authoring tools, including lessons learned, and the need for AI assistance to guide and teach users.
Conference Paper
Full-text available
Designing games requires a complex sequence of planning and executing actions. This paper suggests that game design requires computational thinking, and discusses two methods for analyzing computational thinking in games designed by students in the visual programming language Scratch. We present how these two analyses produce different narratives of computational thinking for our case studies, and reflect on how we plan to move forward with our larger analysis.
Article
Full-text available
There has been considerable interest in examining the educational potential of playing video games. One crucial element, however, has traditionally been left out of these discussions—namely, children's learning through making their own games. In this article, we review and synthesize 55 studies from the last decade on making games and learning. We found that the majority of studies focused on teaching coding and academic content through game making, and that few studies explicitly examined the roles of collaboration and identity in the game making process. We argue that future discussions of serious gaming ought to be more inclusive of constructionist approaches to realize the full potential of serious gaming. Making games, we contend, not only more genuinely introduces children to a range of technical skills but also better connects them to each other, addressing the persistent issues of access and diversity present in traditional digital gaming cultures. http://www.tandfonline.com/doi/abs/10.1080/00461520.2015.1124022
Article
Full-text available
This chapter recounts the game-making resurgence in the twenty-first century. It focuses on how various tools and communities for game making have been utilized thus far by constructionists to heighten children's learning in and around schools. The chapter examines how the value of game making for learning has shifted over the years in terms of educational outcomes. It provides an overview of the educational software currently available to youth and identifies the salient elements of what qualifies as an effective game design tool. Low floors, high ceilings, and wide walls offer some visage of a metaphorical house in terms of what game-making tools should look like. But here a fourth component to the consideration of computational construction kits is presented: New Windows. The chapter concludes by looking forward to the future of game making and how the various tools and communities are increasingly becoming intertwined.
Article
Although the vignette method is widely used in psychology, it is often implemented without the key feature of iterative decision making that can affect the eventual outcome of the vignettes. This Tutorial provides an explanation of how to use Twine, an interactive narrative platform, to create vignettes with iterative decision making. Twine is an especially useful tool for experiments involving branching narratives, spatial navigation, and resource allocation. We provide code for creating exemplar experiments in social and cognitive psychology, as well as behavioral economics, and explain how to integrate Twine projects with survey-management platforms, such as Qualtrics. After following this Tutorial, researchers will be able to use Twine in their experiments to update the classic vignette method by incorporating iterative decision-making tasks.
Article
Interactive narrative is a form of digital interactive experience in which users create or influence a dramatic storyline through their actions. The goal of an interactive narrative system is to immerse users in a virtual world such that they believe that they are an integral part of an unfolding story and that their actions can significantly alter the direction or outcome of the story. In this article we review the ways in which artificial intelligence can be brought to bear on the creation of interactive narrative systems. We lay out the landscape of about 20 years of interactive narrative research and explore the successes as well as open research questions pertaining to the novel use of computational narrative intelligence in the pursuit of entertainment, education, and training. Copyright © 2013, Association for the Advancement of Artificial Intelligence.