ArticlePDF Available

Toward effective game-based social skills tutoring for children: An evaluation of a social adventure game

Authors:

Abstract

This paper describes a study of a prototype of a novel game-based intelligent tutor that teaches children positive social skills. The results provide considerable support for the po-tential value of this game as a social skills training tool, de-spite the comparatively brief play-through duration of the prototype. Key to the initial success is a development frame-work that fostered deep collaboration and rapid prototyping between the subject matter experts and game designers.
Toward Effective Game-Based Social Skills Tutoring for
Children: An Evaluation of a Social Adventure Game
James M. Thomas
Digital Games Research Center
Liquid Narrative Research Group
North Carolina State University
jmthoma5@ncsu.edu
Melissa E. DeRosier, Ph.D.
3-C Institute For Social Development
1901 North Harrison Avenue, Suite 200
Cary, North Carolina 27513
derosier@3cisd.com
ABSTRACT
This paper describes a study of a prototype of a novel game-
based intelligent tutor that teaches children positive social
skills. The results provide considerable support for the po-
tential value of this game as a social skills training tool, de-
spite the comparatively brief play-through duration of the
prototype. Key to the initial success is a development frame-
work that fostered deep collaboration and rapid prototyping
between the subject matter experts and game designers.
Keywords
Social Skills Learning, Digital Games, Intelligent Tutoring,
Interactive Narrative, Data-Driven Design
1. INTRODUCTION
This paper describes the design, development, and eval-
uation of a prototype game for the intelligent tutoring of
social skills to children. The game is called “Adventures
Aboard the S.S.GRIN”, as its curriculum is based on Social
Skills GRoup INtervention (S.S.GRIN) [10], a scientifically
validated, skill-based social skills training intervention that
combines social learning and cognitive-behavioral techniques
to build children’s social skills and social relationships. Sta-
tistically significant correlations were found between chil-
dren’s performance within the game and their assessed social
skills competencies, even with exposure to the game limited
to less than twenty minutes.
This research is significant in that it establishes the useful-
ness of applying Intelligent Tutoring Systems techniques to
children’s social skill development. Furthermore, it shows
that such tutoring can be effective within a narrative-rich
point-and-click adventure game. Some of the success of
the game can be ascribed to the thoughtful collaboration
among computer scientists, graphic artists, and psycholo-
gists throughout the design and development process. The
conceptual distance between these disciplines underscores
the significance of highlighting the aspects of software de-
sign and architecture that facilitated fruitful collaboration.
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, to
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.
FDG 2010, June 19-21, Monterey, CA, USA
Copyright 2010 ACM ACM 978-1-60558-937-4/10/06 ...$10.00.
2. BACKGROUND
2.1 Social Skills and Peer Relationships
Children’s relationships with other children are critically
important for their sense of well-being and adjustment [31,
24]. From the time children enter school, peers take on an
increasingly meaningful and influential role, becoming key
providers of support, advice, companionship, and affirma-
tion as children mature through the elementary school years
and into adolescence [18]. Positive social skills and rela-
tionships promote more positive emotional, behavioral, and
academic functioning. Positive peer relations also operate
as a protective factor, reducing the likelihood of negative
outcomes in the face of stressful life events, such as poverty
[23, 25, 31].
In contrast, social problems increase children’s risk of poor
adjustment across all areas of functioning. Children who
experience peer problems tend to exhibit a wide variety of
concurrent behavioral (e.g., disruptive, aggressive), psycho-
logical (e.g., low self-esteem, lonely, depressed), and aca-
demic (e.g., poor school performance, absenteeism) difficul-
ties. The developmental psychopathology literature under-
scores the insidious, damaging influence of peer problems
which place children at heightened risk for development of
numerous negative outcomes, including depression [4], anxi-
ety disorders [14], suicide [7], delinquency and antisocial be-
havior [5, 15], substance abuse [22, 34], educational under-
achievement [16, 37], and other mental health difficulties
[31].
Children can experience a variety of social difficulties within
the peer group. Three common categories of peer problems
are: Rejection (active dislike, avoidance, exclusion of a child
by the majority peer group), Victimization (bullying, as-
sault, intimidation, teasing, humiliation, rumor spreading),
and Social Isolation (lack of stable, close positive friend-
ships). As children experience social failure, many become
increasingly negative about themselves and anxious in so-
cial situations. They may withdraw, becoming more socially
isolated, or they may act out their anger and frustration on
others. When children are rejected from the normative peer
group, they tend to affiliate with other children also on the
out skirts. When antisocial peers affiliate with one another
more exclusively, negative behaviors and attitudes are com-
pounded. In effect, a vicious cycle may develop in which
poor social skills, social anxiety, and negative attitudes re-
inforce one another over time [24].
While the particular adjustment difficulties a child ex-
hibits vary depending on the specific peer problems expe-
rienced, as well as child and contextual factors, the research
literature indicates that the quality of children’s peer rela-
tionships directly influences their academic, behavioral, and
emotional adjustment. This influence is not simply an ar-
tifact of other associated behaviors, such as aggressiveness,
nor environmental factors, such as poverty, but rather peer
problems contribute to the development of future problems
with unique and independent prediction. As peer problems
become more chronic or severe, children’s risk for negative
outcomes increases [11]. A key to preventing the devel-
opment of more serious maladjustment is intervening with
problematic social behaviors before they become chronic and
intractable [20, 31]. Early intervention is critical in the pre-
vention of severe social skill deficits and entrenched negative
peer interaction patterns [8, 30].
2.2 Social Skills Training for Children
A variety of social skills training programs, with differ-
ing sets of skills and teaching formats, have been developed
over the years [2, 20]. Social skills training may be directed
towards children more generally as in school-based univer-
sal programs through which all children participate in the
intervention regardless of social skills level. Other social
skills training interventions are delivered in a small group
therapeutic setting and focus on children who are identified
as experiencing one or more social problems (i.e., indicated
programs). While the research literature clearly supports
the use of social skills training, those programs (universal
or indicated) that integrate behavioral (what to do or not
to do in a social situation), cognitive (assumptions, eval-
uating consequences, goal setting), and emotional (feelings
about/during a social situation) social skills have been found
to be most effective. This more integrative approach to
social skills training helps children think through a given
social situation, consider alternative social problem-solving
approaches, and evaluate the success of a selected approach
for achieving a social goal. In other words, those interven-
tions that build children’s capacity to actively engage in so-
cial problem-solving have the most significant and lasting
impact on the quality of their peer relations.
S.S.GRIN was developed and repeatedly tested, starting
with one elementary school in 1993 and progressing to lon-
gitudinal research within 45 elementary schools across the
NC Wake County Public School System. Research supports
the effectiveness of S.S.GRIN [9, 12]. For example, in a
study of third graders, children with specific peer problems
(high peer dislike, victimization by peers) and/or high so-
cial anxiety were identified and randomly assigned to Treat-
ment (n=187) or no-treatment Control (n=194). Trained
school counselors administered S.S.GRIN in the school set-
ting. Children who participated in S.S.GRIN showed im-
provement in peer liking, enhanced self-efficacy for deal-
ing with social situations, lower social anxiety, and greater
declines in aggressive behavior problems compared to con-
trols [9]. At one-year follow-up, all pre-post benefits were
still present and additional benefits were evident, including
lower victimization by peers, fewer negative nominations by
peers, and more positive outcome expectancy for social sit-
uations [12]. Overall, results support the effectiveness of
S.S.GRIN not only for enhancing children’s social relation-
ships, but also for improving social-cognitive, behavioral,
and emotional functioning, both at immediate and one-year
follow-up.
2.3 Intelligent Tutoring Systems
With continuous development spanning the past twenty
years, Intelligent Tutoring Systems (ITS) have advanced to
the point where many research-based systems exceed the
teaching effectiveness of traditional classroom-based instruc-
tion [36]. Of particular relevance are those ITS systems that
leverage the game paradigm to help guide exploratory learn-
ing [17, 6, 29, 3, 1]. Contemporaneous with this success,
commercial digital game technologies have shown remark-
able advancement in graphics and human-computer inter-
action, especially in terms of training. Gamers expect to
be taught how to use a new game not by reading a printed
manual, but rather through sophisticated in-game instruc-
tion. The games industry is now more lucrative than the
film industry, and that commercial clout has driven the de-
velopment of innovative in-game training methods. Signif-
icant independent voices [19, 32] have noted the potential
for extending these training techniques to cover academic
subjects.
However, bridging the disparate approaches to learner
guidance between games and ITS has proven to be diffi-
cult. Straightforward attempts to bolt-on learning content
to a game or game play to a learning system are notori-
ously disappointing. Knowledgeable cynics have noted that
in the collaborative efforts to date, excessive focus on learn-
ing tends to “suck the fun out” of games, to which the rejoin-
der is offered that a game design focus often “sucks out the
learning”[33]. More successful are approaches that integrate
learning content with what the game design community de-
scribes as the “core mechanic” of a game.
The ISTS game leverages the medium of interactive nar-
rative [27, 26] to tightly integrate the game mechanic with
the learning content [35]. In the case of ISTS, the learning
content of social skills is particularly well expressed through
narrative examples. However, adapting interactive narra-
tive for exploratory, constructive learning demands that user
autonomy be sufficient to support exploration of specific
“hypothesis-generation-testing cycles” [29]. A key feature
of the ISTS is to support and guide the student in making
multiple attempts at solving the same social problems.
Consistent with many successful ITS designs, the ISTS
required a thorough and thoughtful sequencing of the so-
cial challenges, action choices, hints and consequences in the
game. Each logical branch in this structure required collab-
oration with the mental health professionals. Unlike systems
in which unpredictable, emergent behaviors [3] could place
unpredictable learning demands on the student, the ISTS
provides a strictly deterministic set of internally consistent
learning paths, that are dynamically selected based on stu-
dent performance.
2.4 The Potential Benefits of Socially-Intelligent
Tutoring Systems
A key opportunity for computer-based social learning is
to provide a safe environment in which students can explore
and practice newly learned social strategies. Unlike in the
child’s actual social environment, mistakes made in explor-
ing social relationships in the virtual world will not become
persistent impediments to future growth. Thus the learn-
ing principle of the “Psychosocial Moratorium” articulated
by Gee [19] takes center stage with ISTS. This principle
states that one of the great benefits of games is that they
allow learners to take risks in a space where real-world con-
sequences are lowered. Given the high costs of social mis-
takes, it is especially useful for children to have a safe place
to practice.
3. INNOVATIVE GAME FEATURES
3.1 Overview
The S.S.GRIN intelligent social tutoring system (ISTS)
invites the child to participate in a social narrative through
an in-game avatar. The ISTS leverages the point-and-click
adventure game paradigm to guide the student through a
structured narrative of recruiting and maintaining friends
while coping with other, unfriendly characters. The develop-
ment was split into two components: a general-purpose en-
gine that encapsulated the point-and-click game paradigm,
and a separate component containing the game content spe-
cific to ISTS. Both components were written in ActionScript
3.0 which underlies Adobe
R
Flash
R
.
3.2 Game Mechanics
Figure 1: Avatar Customization Interface
Before social training begins, the child customizes the ap-
pearance of the avatar which will be their visual represen-
tation in the game, using an interface depicted in Figure
1. Being able to customize the avatar’s gender and appear-
ance enhances children’s identification with their character,
which in turn increases their engagement and learning [9,
51]. Children can select from a diverse set of fun and natu-
ralistic options through a quick and easy selection process.
To communicate with other characters, the child chooses
from a menu of dialogue and behavioral options, as shown
in Figure 2. As the child moves the cursor over each option
in the menu, the avatar’s facial expression changes to reflect
the emotional intent of that communication or action. This
novel interactive element helps children recognize visual cues
associated with non-verbal communication, a key area of so-
cial skill development. Avatar facial expressions are built on
the work of Dr. Paul Ekman and colleagues who identified
facial muscular changes associated with distinct emotional
states [13], as shown in Figure 3.
3.3 Calibration and Adaptation
Figure 2: Dialogue With Facial Expression
Figure 3: Ekman-Based Facial Expression
The geography of the game is structured so children must
master the basics of avatar navigation and dialogue selec-
tion described above before moving into the part of the game
where the social problem solving (SPS) challenges are posed.
The first scene represents a common, yet moderately difficult
social situation where animated wharf rats attempt to bully
the avatar. During this interaction, each menu presents the
child with a choice of one of three responses: to withdraw
(passive response), to escalate the conflict (aggressive re-
sponse), or to manage the situation in a nonaggressive, yet
assertive manner (assertive response). Over the course of
three exchanges, the child’s responses are tracked by the
software and scored according to type (passive, aggressive,
or assertive) and sequence. Because subsequent choices by
the child are based on more social information for the sit-
uation (i.e., how the rats responded to the child’s prior di-
alogue choice), children’s first response to the rats is given
less weight than later responses when determining the child’s
entry difficulty level for subsequent social problem solving
tasks. For example, a child who continues to escalate the
conflict demonstrates lower social reasoning than a child who
adapts from aggressive or passive to more assertive responses
over the course of the interaction.
The ISTS prototype focuses on a primary skill of the
S.S.GRIN curriculum, known as “Stop & Think” or impulse
control. Impulse control is a basic social skill that is centrally
important to many other social skills, such as cooperation
and communication [28].
The S.S.GRIN intervention stresses the value of using Stop
& Think across a variety of situations. During in-person
group therapy sessions, children practice Stop & Think via
role-playing of interpersonal situations (e.g., initiating a con-
versation, resolving a conflict, compromising) and through
behaviorally focused problem-solving exercises (e.g., work-
ing through a logic puzzle, following instructions to ensure
accurate responses). In this way, S.S.GRIN demonstrates
the importance of inhibiting impulsive responses more broadly
and provides practice in the application of impulse control
specifically for social situations. Therefore, the ISTS in-
cludes different types of SPS tasks requiring impulse control
for both behavioral tasks and social interactions.
For example, to rescue the first crew member (Stop &
Think Girl), the child has to successfully negotiate with an-
other character, the “fishing kid”, to borrow a jet ski. How-
ever, the scene includes elements designed to interfere with
impulse control and encourage the child to impulsively take
the jet ski without asking. These elements are more or less
difficult depending on the child’s prior performance ranging
from (a) the easiest level where the fishing kid initiates con-
versation to signal that the child needs to talk with him first
to (b) the moderate level where the fishing kid no longer ini-
tiates conversation and a key is prominently displayed as an
added temptation to take the jet ski to (c) the highest dif-
ficulty where an element of “peer” pressure is added with a
monkey actively encouraging the avatar to ignore the fishing
kid and take the jet ski. Performance on this task is based
on antisocial behavioral responses towards the fishing kid
(e.g., attempts to take the jet ski without permission) and
the quality of the child’s negotiation skills (i.e., demanding
vs. cooperative).
A more behaviorally focused SPS task is a puzzle where
the child must pay attention to details and carefully fol-
low instructions to convince a monkey to let down a bridge
along the way to get Stop & Think Girl. This scene pulls
for impulsive responding with several red herrings (e.g., cart
of bananas, large “Push for Bridge” button) and uncommon
twists to the solution (e.g., the monkey does not like ba-
nanas). Performance is based on the order, number, and
timing of impulsive responses.
A robust content and scoring algorithm, based on theory
and research in children’s social development [31], was used
to classify the Social Reasoning (SR) performance level rep-
resented by the child’s behavior during the ISTS (i.e., high,
medium, low). SR levels are calculated for overall perfor-
mance as well as performance for each SPS task.
3.4 Game Engine Responsibilities
The engine tracks and renders the avatar location, direc-
tional orientation, locomotion, facial expression, and hand/
body position. A “click-to-move” paradigm using a modified
A* algorithm is used to plan the path the avatar will walk
from one spot to another. This algorithm restricts the path
to each scene’s geometrically-defined “walkable area” and
avoids collisions between the avatar and objects in the scene.
The engine enables a flexible map that represents the con-
nection paths between independent scenes or rooms. Graph-
ics rendering includes shadowing and perspective changes in-
tegrated to alter the avatar’s size and shading depending on
its location. The avatar moves in front of or behind objects
depending on its depth in scene. These are key elements that
distinguish 2.5D from standard two-dimensional Flash ani-
mations. The engine also provides the set of rules defining
the ways characters can interact with clickable, tools that
can be used, and communication modes with pedagogical
agents and other external characters.
The engine also features an extensive logging system that
tracks and records the child’s actions (choices, sequence) for
each scene and calculates relevant performance metrics for
social problem-solving. This data is used by the software
engine to adjust the difficulty level of subsequent scenes in
response to the child’s performance in prior scenes as well as
to modulate the amount of pedagogical assistance provided
for a given scene (e.g., when an S.S.GRIN crew member
should provide advice to the child). In this way, the ISTS
adapts training to the child’s skill level for more individ-
ualized learning. The data gathered through this logging
process also form the basis for positive reinforcement and
constructive feedback provided to the child during the course
of completing the ISTS (e.g., pedagogical comments, Travel
Log summary) as well as for the summary reports regarding
the child’s ISTS performance in targeted areas.
3.5 Evolution of a Data-driven Architecture
When the ISTS presents a child with a set of options for
dialogue and/or behaviors, unique paths emerge, branch-
ing from each possible option. For example, a series of
three choice points with three options each yields 27 unique
paths of dialogue/behavior. However, selecting options at
explicit choice points is only one factor that influences a
child’s course through a scene. The way a child interacts
with a scene (where they move, what they click on, and the
sequence of these actions) influences what is presented to
the child and the subsequent course of action. The scripting
system is used to define rules for how ISTS responses are
triggered by a child’s interaction pattern with a scene.
An especially useful ISTS software design decision was to
specify the dialogue, behavioral choices, facial expressions,
text, and system responses (e.g., reaction by external char-
acter) for each scene in editable XML (eXtensible Markup
Language) files that are then read into the ISTS game en-
gine at run-time. This allowed psychologists and other non-
software designers on the development team to easily create
and modify all these key aspects of game play without hav-
ing to work through a software designer.
The value of this data-driven approach extended beyond
typical software engineering and maintainability benchmarks.
More significantly, it allowed the mental health professionals
to quickly iterate through permutations of the dialogue and
action trees that formed the basis of the core mechanic of the
game without requiring any direct interaction with the soft-
ware designers. Because the XML files that contained these
“guts” of the program structure were fully visible and ed-
itable by the mental health professionals. Because the same
files were loaded on each new invocation of the ISTS, these
subject matter experts could immediately see the results of
their edits on the flow of the tutorial on each run.
4. EVALUATION
A total of 37 children and their parents participated in
an evaluation of the ISTS. Families were selected on a “first
come first served” basis after flyers were sent to area ele-
mentary schools and pediatricians’ offices. Parent partic-
ipants were primarily female (67%) with a bachelor’s de-
gree or other college experience (53%) and an average age
of 40 years. Children were primarily male (59%) with an
average age of 10 years. Families represented the full range
of socioeconomic status with a racial/ethnic distribution of
73% White (5% of Hispanic/Latino ethnicity), 22% African
American, and 5% Asian American. All families reported
having a computer with an Internet connection (98% with
high-speed) available to the child at home.
4.1 Methodology
Six groups of parent-child pairs attended a two-hour sched-
uled meeting at 3-C ISD. Each child was assigned a private
kiosk where s/he could interact with the ISTS on a computer
(all kiosks had identical set-ups) while the parent observed.
Trained research assistants led the groups through a semi-
structured three-part data collection process. First, parents
and children completed consent forms and demographic and
social skills questionnaires. Second, following the project
introduction and computer orientation, children were given
30 minutes of free time to interact with the ISTS. Children
were asked to go through the SPS tasks twice (i.e., click “Try
Again” at least once). Parents were asked to observe only,
without providing assistance. Trained observers monitored
and recorded children’s level of attention, areas of difficulty,
and responses to ISTS tasks. Computer logs tracked the
location and time of each mouse click so children’s inter-
activity could be mapped and examined (e.g., for software
usability). Third, children and parents separately completed
questionnaires evaluating the ISTS. Research assistants gave
instructions, read questions aloud for children, and assisted
with completion, as needed. Parents and children wrote
comments, suggestions, and criticisms at the bottom of each
questionnaire. Once questionnaire data were collected, the
PI led group discussions to gather additional comments and
suggestions from parents and children.
An important aspect of this study was to examine the
degree to which performance was related to external mea-
sures of children’s social skills and behavior. To this end,
parents and children separately completed the Social Skills
Rating Scale [21] prior to interacting with the ISTS. Each
item (55 for parents; 34 for children) of the SSRS includes a
behavioral description (e.g.,“I make friends easily”) to which
respondents indicate how often that description is true for
the child (0=Never; 1=Sometimes; or 2=Very Often). Fac-
tor analysis of the parent-report (P-R) form supported four
social behavior scales (Cooperative, Responsible, Assertive,
Self-control) and three problem behavior scales (Internaliz-
ing, Externalizing, Hyperactivity) each with good internal
consistency (mean α=.79). Factor analysis for the child-
report (C-R) form supported four social behavior scales (Friendly,
Polite, Communication, Impulse control) also with good in-
ternal consistency (mean α =.71).
4.2 Results
The first choices presented to a child using the ISTS are
the selection of the visual characteristics of the avatar that
represents the child throughout the game. As expected, chil-
dren tended to select avatars that matched their own ap-
pearance. A high correspondence (χ
2
(1)
= 33.03, p < .0001)
was found between the child’s gender and that of the se-
lected avatar. Also, there was also a high correspondence
between the child’s racial/ethnic group and their selection
of avatar skin tone (χ
2
(12)
= 32.94, p < .001). For exam-
ple, 100% of African American children chose one of the
two darker skin tones and 88% of White children chose one
of the two lighter skin tones. Children used the full range
of clothing color and hair style options with a strong ten-
dency to select options that reflected their own appearance.
During the work groups, children were observed looking at
their own clothing and then selecting colors for the avatar’s
clothes to match. During follow-up discussion, children re-
ported that being able to customize their avatar made the
ISTS more fun and interesting (e.g.,“I like seeing me on the
screen”). This inclination of the students to select avatars
that match their appearance is encouraging as it indicates a
willingness to identify with the avatar, which perhaps links
more strongly the behavioral choices made in the game to
choices the child would make in real life.
To test whether children’s social skills differed accord-
ing to how they performed on the first SPS task (bully-
ing by wharf rats), Analyses of Variance (ANOVAs) were
conducted with social reasoning (SR) level predicting SSRS
scale scores. Reported levels of polite behavior (e.g., “I ask
before using other people’s things.”), communication (e.g.,
“I tell others when I am upset with them.”), and impulse
control (e.g., “I ignore others when they tease me.”) were
significantly higher for children with higher SR levels. Par-
ents reported that children with high SR were significantly
higher on cooperative behavior (e.g.,“Volunteers to help”),
marginally higher on self-control (e.g., “Receives criticism
well”), and marginally lower on externalizing behavior (e.g.,
“Fights with others”) compared to lower SR levels. A sec-
ond set of ANOVAs examined whether social skills were re-
lated to a child’s initial reaction to the social challenge by
the wharf rats: aggressive (n=11), passive (n=9), assertive
(n=17). Assertive children’s self-ratings for polite behav-
ior and their parents’ ratings of responsible behavior (e.g.,
“Asks permission before using another’s property”) were sig-
nificantly higher than those for passive children. Parents
also reported aggressive children displayed significantly more
hyperactive and externalizing behavior and significantly less
self-control compared to assertive or passive children. A final
set of ANOVAs was conducted to examine SR levels across
different SPS scenes. As expected, children with lower SR
in one scene performed more poorly on subsequent impulse
control tasks (e.g., significantly more error clicks).
In addition to SPS performance metrics, three systemic
metrics were calculated based on logs generated over the
course of ISTS game play to gain a deep understanding of
the child’s overall patterns of engagement with the ISTS.
First, dialogue speed (DS) is the mean number of times
a child clicked to exit a dialogue window before the allot-
ted time. Second, processing time (PT) equals the amount
of time spent during SPS tasks minus the number of er-
ror clicks. Last, exploration (EX) is the amount of time a
child spent exploring the non-SPS areas of the ISTS. Cor-
relations between these systemic metrics and performance
metrics were calculated. Given the small sample size and
limited power, correlations at up to the .10 p-level were re-
ported. A modest negative correlation was found with DS
and performance (r =-.25, p<.10) indicating children who
clicked to exit a dialogue window prematurely tended to
perform poorly in impulse control. In contrast, PT was pos-
itively correlated with performance (r =.55, p<.001) on SPS
tasks suggesting amount of time spent processing an SPS
task without excessive or random clicking was associated
with higher problem-solving ability. A positive correlation
between EX and performance (r =.40, p<.01) was also found
suggesting that being more investigative may enhance social
interaction with characters in the ISTS.
Finally, correlations between the various ISTS performance
indices and SSRS scales were run to examine the interrela-
tions among children’s social skills levels and their SPS dur-
ing the ISTS (due to limited power, correlations greater than
.20 are reported). Overall, the pattern of correlations was
consistent with expectations. More appropriate responding
was associated with being more polite (r =.50, p<.001) and
responsible (r =.26, p<.10) and with higher communication
(r =.34, p<.05). Inhibiting impulsive responses during the
ISTS was associated with greater impulse control (r =.31,
p<.05) and lower hyperactivity (r =-.29,p<.10). Quickly
moving through dialogue with other characters (DS) was
associated with lower communication (r=-.35, p<.05), co-
operation (r =-.33, p<.05) and impulse control (r =-.32,
p<.05) as well as higher externalizing behavior problems (r
=.37, p<.05). Higher processing time (PT) was modestly
associated with a trend toward more internalizing behavior
problems (r =.24, p<.10). More exploration tended to be
associated with higher communication (r =.33, p<.05) and
friendliness (r =.25, p<.10) as well as lower hyperactive be-
havior (r =-.26, p<.10). Though the number of subjects
per cell was too low to test statistically, examination of the
various findings from these analyses across different genders
and racial/ethnic groups indicated that the patterns of re-
sults held consistently across different sub-groups.
These results demonstrated that performance on the SPS
tasks was related to children’s social skills and behavior, as
assessed through an independent standardized instrument,
and these inter-relations were meaningfully linked to the in-
tended content of the game (e.g., impulsive behavior prob-
lems associated with impulsive responses). Further, the
ways in which children interacted with the game (systemic
metrics) corresponded with their social reasoning on SPS
tasks as well as their social behavior with peers (as mea-
sured by the SSRS). Results also showed that the ISTS has
been designed in such a way that SPS tasks are sufficiently
difficult to challenge children (e.g., variance in response pat-
terns), repeated exposure to SPS tasks results in significant
learning, and difficulty level can be effectively modulated
to fit the child’s social reasoning level. Test results also
provide evidence of convergent validity for the social task
performance indices.
It is worth noting that the tested prototype represented
only a small portion of the entire S.S.GRIN curriculum. A
full game would have eight to ten times the content and
play-through time of the prototype. For us to find statisti-
cally significant relations with such a reduced set of learning
content is strong encouragement for the potential teaching
effectiveness of a more complete tutorial.
5. CONCLUSION
This paper described an adventure game that employes
intelligent tutoring techniques to teach children positive so-
cial skills. An empirical evaluation of the game provided
considerable support for the potential value of this game as
a social skills training tool, despite the comparatively brief
play-through duration of the prototype. This shows that
an integration of ITS and games may provide a useful plat-
form for teaching social skills, at least within a thoughtfully
scripted interactive narrative.
From a software design standpoint, this study shows the
value of encapsulating a game engine independent from game-
specific content, and the value of further encapsulating much
of the game logic as externally editable data.
Future work should investigate the learning effectiveness
of a full game, both with and without supporting classroom-
based instruction.
6. REFERENCES
[1] V. Aleven and K. Koedinger. An effective
metacognitive strategy: Learning by doing and
explaining with a computer-based Cognitive Tutor.
Cognitive Science, 26(2):147–179, 2002.
[2] S. Asher, J. Parker, and D. Walker. Distinguishing
friendship from acceptance: Implications for
intervention and assessment. The company they keep:
Friendship in childhood and adolescence, pages
366–405, 1996.
[3] R. Aylett, S. Louchart, J. Dias, A. Paiva, M. Vala,
S. Woods, and L. Hall. Unscripted narrative for
affectively driven characters. IEEE Comput. Graph.
Appl., 26(3):42–52, 2006.
[4] M. Boivin and S. Hymel. Peer experiences and social
self-perceptions: A sequential model. Developmental
Psychology, 33(1):135–145, 1997.
[5] M. Brendgen, F. Vitaro, and W. Bukowski. Affiliation
with delinquent friends: Contributions of parents,
self-esteem, delinquent behavior, and rejection by
peers. The Journal of Early Adolescence,
18(3):244–265, 1998.
[6] A. Bunt and C. Conati. Probabilistic Student
Modelling to Improve Exploratory Behaviour. User
Modeling and User-Adapted Interaction,
13(3):269–309, 2003.
[7] J. Carney. Bullied to death: Perceptions of peer abuse
and suicidal behaviour during adolescence. School
Psychology International, 21(2):213–223, 2000.
[8] J. Coie, K. Dodge, R. Terry, and V. Wright. The role
of aggression in peer relations: An analysis of
aggression episodes in boys’ play groups. Child
Development, 62(4):812–826, 1991.
[9] M. DeRosier. Building relationships and combating
bullying: Effectiveness of a school-based social skills
group intervention. Journal of Clinical Child and
Adolescent Psychology, 33(1):196–201, 2004.
[10] M. DeRosier and M. Gilliom. Effectiveness of a parent
training program for improving children
˜
Os social
behavior. Journal of Child and Family Studies,
16(5):660–670, 2007.
[11] M. DeRosier, J. Kupersmidt, and C. Patterson.
Children’s academic and behavioral adjustment as a
function of the chronicity and proximity of peer
rejection. Child Development, 65(6):1799–1813, 1994.
[12] M. DeRosier and S. Marcus. Building friendships and
combating bullying: Effectiveness of SS GRIN at
one-year follow-up. Journal of Clinical Child and
Adolescent Psychology, 34(1):140–150, 2005.
[13] P. Ekman and E. Rosenberg. What the face reveals:
Basic and applied studies of spontaneous expression
using the Facial Action Coding System (FACS).
Oxford University Press, USA, 2005.
[14] S. Erath, K. Flanagan, and K. Bierman. Social
anxiety and peer relations in early adolescence:
Behavioral and cognitive factors. Journal of abnormal
child psychology, 35(3):405–416, 2007.
[15] M. Feinberg, T. Ridenour, and M. Greenberg.
Aggregating indices of risk and protection for
adolescent behavior problems: The Communities That
Care Youth Survey. Journal of Adolescent Health,
40(6):506–513, 2007.
[16] C. Fleming, K. Haggerty, R. Catalano, T. Harachi,
J. Mazza, and D. Gruman. Do social and behavioral
characteristics targeted by preventive interventions
predict standardized test scores and grades? Journal
of School Health, 75(9):342, 2005.
[17] R. Freedman, C. Ros´e, M. Ringenberg, and
K. VanLehn. ITS tools for natural language dialogue:
A domain-independent parser and planner. Lecture
notes in computer science, pages 433–442, 2000.
[18] W. Furman and D. Buhrmester. Age and sex
differences in perceptions of networks of personal
relationships. Child development, 63(1):103–115, 1992.
[19] J. Gee. What video games have to teach us about
learning and literacy. Computers in Entertainment
(CIE), 1(1):20–20, 2003.
[20] M. Greenberg, C. Domitrovich, and B. Bumbarger.
The prevention of mental disorders in school-aged
children: Current state of the field. Prevention &
Treatment, 4(1):1–52, 2001.
[21] F. Gresham and S. Elliott. Social skills rating system.
Circle Pines, MN: American Guidance Service, 1990.
[22] J. Hawkins, R. Catalano, and J. Miller. Risk and
protective factors for alcohol and other drug problems
in adolescence and early adulthood: Implications for
substance abuse prevention. Psychological bulletin,
112(1):64–105, 1992.
[23] A. Kazdin. Effectiveness of psychotherapy with
children and adolescents. Journal of Consulting and
Clinical Psychology, 59(6):785–798, 1991.
[24] J. Kupersmidt and M. DeRosier. How peer problems
lead to negative outcomes: An integrative mediational
model. Children
˜
Os peer relations: From development
to intervention, pages 119–138, 2004.
[25] S. Luthar. Resilience and vulnerability: Adaptation in
the context of childhood adversities. Cambridge Univ
Pr, 2003.
[26] B. Magerko, J. Laird, M. Assanie, A. Kerfoot, and
D. Stokes. AI Characters and Directors for Interactive
Computer Games. Proceedings of the 2004 Innovative
Applications of Artificial Intelligence Conference,
1001:48109–2110, 2004.
[27] M. Mateas and A. Stern. Structuring Content in the
Fcade Interactive Drama Architecture. AIIDE, pages
93–98, 2005.
[28] K. Merrell and G. Gimpel. Social Skills of Children
and Adolescents: Conceptualization, Assessment,
Treatment. Lawrence Erlbaum Associates, Inc., 10
Industrial Ave., Mahwah, NJ 07430, 1998.
[29] B. Mott and J. Lester. U-director: a decision-theoretic
narrative planning architecture for storytelling
environments. AAMAS, pages 977–984, 2006.
[30] C. Odgers, T. Moffitt, J. Broadbent, N. Dickson,
R. Hancox, H. Harrington, R. Poulton, M. Sears,
W. Thomson, and A. Caspi. Female and male
antisocial trajectories: From childhood origins to adult
outcomes. Development and Psychopathology,
20(02):673–716, 2008.
[31] J. Parker, K. Rubin, S. Erath, J. WOJSLAWOWICZ,
and A. Buskirk. Peer relationships, child development,
and adjustment: A developmental psychopathology
perspective. Developmental Psychopathology: Theory
and method, page 419, 2006.
[32] M. Prensky. Digital game-based learning. Computers
in Entertainment (CIE), 1(1):21–21, 2003.
[33] M. Prensky and J.-C. Bowers. Serious games debate.
Serious Games Summit, October 2005.
[34] C. Spooner. Causes and correlates of adolescent drug
abuse and implications for treatment. Drug and
Alcohol Review, 18(4):453–475, 1999.
[35] J. Thomas and R. Young. Using Task-Based Modeling
to Generate Scaffolding in Narrative-Guided
Exploratory Learning Environments. Proceedings of
the 14th International Conference on Artificial
Intelligence in Education, 2009.
[36] K. VanLehn. The Behavior of Tutoring Systems.
International Journal of Artificial Intelligence in
Education, 16(3):227–265, 2006.
[37] L. Woodward and D. Fergusson. Childhood peer
relationship problems and later risks of educational
under-achievement and unemployment. The Journal of
Child Psychology and Psychiatry and Allied
Disciplines, 41(02):191–201, 2000.
... In contrast to earlier critical assessments such as, e.g., (Jantke, 2006a), (Seelhammer and Niegemann, 2009), and others, the authors feel encouraged by recent reports on successful serious games projects from (Chaffin and Barnes, 2010), (Cooper et al., 2010) and (Thomas and DeRosier, 2010) to (Jenson et al., 2011). ...
... The processes of human learning are rather involved and need a certain firm scientific foundation. Besides educational psychology, in general, and particular aspects like motivation (from (Malone, 1981) to (Hoffman and Nadelson, 2010;Paas et al., 2005)), mental effort (from (Salomon, 1983) to (Paas et al., 2005)), and cognitive load (from (Sweller, 1988) to (Kalyuga, 2009;Mayer and Moreno, 2003), there are many lessons learned from recent applications like (Callaghan et al., 2010;Chaffin and Barnes, 2010;Cooper et al., 2010;De Castell et al., 2007;Hawlitschek, 2009;Jenson et al., 2011;Shelton and Scoresby, 2011;Thomas and DeRosier, 2010). From the two present authors' constructivist point of view 2 , learning needs the learner's activity. ...
Conference Paper
Full-text available
The design and employment of digital games for serious purposes such as learning has several prerequisites. Designing a game that affects human players effectively requires the anticipation of particular human game playing experiences. Recent digital games taxonomies provide the WHAT and storyboarding is the technology for determining the HOW of planning the manifold of potential affective experiences in digital game playing. Game-based learning needs storyboarding and storyboarding needs concepts of digital games taxonomies. The appropriate consolidation of taxonomies and storyboarding results in explicit media didactics in context.
... Furthermore even when SGs claimed to contain "measurable evidence of training or learning," the evidence presented is quite limited (to simple test score, number of mission accomplished, or best time) because there has yet to be any standardized performance metrics for SGs [17]. A data-driven approach [30] and an evidence-centered design [24] are much better assessment methods that will foster real adoption of serious games [16]. This can be resolved by endowing game AI with adaptive and/or personalized behaviour. ...
Chapter
Serious games (SG) have the potential to become one of the most important future e-learning tools. The use of SG in education is a large deviation from the common education standards, which usually are based on mass systems of instruction, assessment, grading and reporting students’ knowledge and skills. SG encourage self‐directness and independency of student, thus providing a framework for self-learning activities. However, the benefits of using SG as a learning tool are maximized in a personalised and adaptive environment. Although it has been suggested in the past that SG can take advantage of Artificial Intelligence (AI) methods for automated adaptation to the learner, there is not so much research in the field.
... Examples: With the application proposed in [25] for the development of social skills in children, the player can choose the responses of the character of this adventure game. The choices for response include affective expressions. ...
Conference Paper
In analogy to the concept of usability, learnability and playability, the concept of Affectibility was conceived to inform the design process – in this case with affective aspects of interaction. In this paper we present a revised set of the Principles for the Design for Affectibility, together with practical examples of use and application. The objective is to support designers in the process of creating educational systems for children, considering aspects of affect.
... It is doubtful that any manager would agree to the view of 'clients as enemies' in business training, or for parents to approve of the tallying of kills as high-scores for classroom learning! It has been suggested that a data-driven approach[34]and an evidence-centered design[3,33]are much better assessment methods that will foster real adoption of serious games[36],[37]. Findings in this study suggest string similarity to be a viable performance assessment metric for serious games. ...
Conference Paper
Full-text available
The literature on human training performance has long attested to the behavioral differences between experts and novices, in which ‘competency’ is a demonstrable attribute based on a person’s course of action in problem solving. The advances in technology have made it possible to trace players’ actions and behaviors (as user-generated data) within an online serious gaming environment for performance assessment purposes. In this study, we introduce string similarity as a performance metric to identify likely-experts among a group of unknown performers (mixture of novices and experts) according to their in-game course of action in problem solving. Our findings indicate that string similarity is both viable and potentially useful as the first performance metric for Serious Games Analytics (SEGA).
... Even worse is the flawed-design of media comparison research that compares classrooms with access to serious games against those without (Clark, 1985;Hastings & Tracey, 2004). It has been suggested that a datadriven approach (Thomas & DeRosier, 2010) and an evidence-centered design (Shute et al., 2010) are much better assessment methods that will foster real adoption of serious games (Ifenthaler, Eseryel, & Ge, 2012;Loh, 2009). We encourage others to research and develop new performance metrics that can take advantage of user-generated data (e.g., play-learners' in-game actions and behaviors) for SEGA. ...
Article
Full-text available
The behavioral differences between expert and novice performance is a well-studied area in training literature. Advances in technology have made it possible to trace players’ actions and behaviors within an online gaming environment as user-generated data for performance assessment. In this study, we introduce the use of string similarity to differentiate likely-experts from a group of unknown performers (mixture of novices and experts) based on how similar their in-game actions are to that of experts. Our findings indicate that string similarity is viable as an empirical assessment method to differentiate likely-experts from novices and potentially useful as the first performance metric for Serious Games Analytics (SEGA).
Conference Paper
The rise of the computer as an “entertainment medium” has been achieved today only through computer games. But computer or video games have the potential and capability to function as “mediums of education” too. Can game-based learning provide learning experience and yet there is fun in changing behavior (assertive communication) for the individual? Game-based learning has been used to teach various skills to people with quite encouraging results. In this paper, a study was carried out to confirm the hypothesis that game-based learning can be a good platform to teach assertive communication delivering learning and fun because of its benefits and encouraging results from other research. A high-fidelity game-based learning prototype, ClickTalk was created for this purpose and it was evaluated with some interesting results.
Book
Childhood resilience is the phenomenon of positive adaptation despite significant life adversities. While interest in resilience has burgeoned in recent years, considerable uncertainty remains regarding what research has revealed about this phenomenon. Integrated in this book are contributions from leading scientists who have studied children's adjustment across risks common in contemporary society. Chapters in the first half of the book focus on risks emanating from the family, and in the second half, on risks stemming from the wider community. The concluding chapter integrates the evidence presented to determine considerations for future research, and directions for interventions and social policies.
Article
While we have known for centuries that facial expressions can reveal what people are thinking and feeling, it is only recently that the face has been studied scientifically for what it can tell us about internal states, social behavior, and psychopathology. Today's widely available, sophisticated measuring systems have allowed us to conduct a wealth of new research on facial behavior that has contributed enormously to our understanding of the relationship between facial expression and human psychology. The chapters in this volume present the state-of-the-art in this research. They address key topics and questions, such as the dynamic and morphological differences between voluntary and involuntary expressions, the relationship between what people show on their faces and what they say they feel, whether it is possible to use facial behavior to draw distinctions among psychiatric populations, and how far research on automating facial measurement has progressed. © 1997, 2005 by Oxford University Press, Inc. All rights reserved.
Article
Tutoring systems are described as having two loops. The outer loop executes once for each task, where a task usually consists of solving a complex, multi-step problem. The inner loop executes once for each step taken by the student in the solution of a task. The inner loop can give feedback and hints on each step. The inner loop can also assess the student's evolving competence and update a student model, which is used by the outer loop to select a next task that is appropriate for the student. For those who know little about tutoring systems, this description is meant as a demystifying introduction. For tutoring system experts, this description illustrates that although tutoring systems differ widely in their task domains, user interfaces, software structures, knowledge bases, etc., their behaviors are in fact quite similar.
Article
Examined in this study was whether a perceived lack of closeness with parents (a) would be related indirectly to early adolescents' association with delinquent friends and (b)would be mediated by a lack of self-esteem. The mediated relation, however, was expected to hold only for (a) delinquent early adolescents who might be attracted to delinquent friends because of the similarity of behavioral characteristics or (b) early adolescents who are rejected by the majority of their peers. Using a sample of 267 boys and girls, the results showed that self-esteem mediated the relation between perceived closeness with parents at Time 1 and affiliation with delinquent friends at Time 2 but only for those who were rejected by the majority of their peers.
Article
This study investigated the perceptions of adolescents concerning the relationship between chronic peer abuse and suicidal behaviour. The major thesis is that chronic peer abuse is an additional risk for adolescent suicidal behaviour. The results of the study upheld this contention and added a surprising component in that bystander perceptions supported victims' views.