Massive Multiplayer Online Gaming: A Research Framework for Military Training and Education

Full text

Technical Report 2005-1
Massive Multiplayer Online Gaming:
A Research Framework for Military Training and Education
Curtis J. Bonk
Indiana University
Consortium Research Fellows Program
Vanessa P. Dennen
Florida State University
March 2005
OFFICE OF THE UNDER SECRETARY OF DEFENSE
FOR PERSONNEL AND READINESS
Approved for public release; distribution is unlimited.

ADVANCED DISTRIBUTED LEARNING INITIATIVE
READINESS AND TRAINING, OFFICE OF THE UNDER SECRETARY
OF DEFENSE FOR PERSONNEL AND READINESS
ROBERT A. WISHER
DIRECTOR
Technical review by
Mark Oehlert, Booz Allen Hamilton, Inc.
Michael W. Freeman, Advanced Distributed Learning Co-Laboratory
NOTICES
The findings in this Technical Report are not to be construed as an official Department of
Defense position, unless so designated by other authorized documents.

i
REPORT DOCUMENTATION PAGE
1. REPORT DATE (dd-mm-yy)
31-January-2005
2. REPORT TYPE
Final
3. DATES COVERED (from. . . to)
January 2004 – September 2004
4. TITLE AND SUBTITLE Massive Multiplayer Online Gaming: A
Research Framework for Military Education and Training 5a. CONTRACT OR GRANT NUMBER
5b. PROGRAM ELEMENT NUMBER
0603769SE0
6. AUTHOR(S) Curtis J. Bonk (Indiana University) & Vanessa P.
Dennen (Florida State University)
5c. PROJECT NUMBER
5d. TASK NUMBER
5e. WORK UNIT NUMBER
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)
Advanced Distributed Learning Initiative
Office of the Under Secretary of Defense (Personnel & Readiness)
Readiness and Training, 4000 Defense Pentagon
Washington, DC 20301-4000
8. PERFORMING ORGANIZATION REPORT NUMBER
9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)
Office of the Under Secretary of Defense (Personnel & Readiness)
Readiness and Training , 4000 Defense Pentagon
Washington, DC 20301-4000
10. MONITOR ACRONYM
11. MONITOR REPORT NUMBER
12. DISTRIBUTION/AVAILABILITY STATEMENT
Approved for public release; distribution is unlimited.
13. SUPPLEMENTARY NOTES
14. ABSTRACT (Maximum 200 words)
Massive multiplayer online gaming, first popularized in the entertainment world, is now finding growing interest in education
and training environments. The military and business have noted the potential for simulation and gaming technology to develop
higher order thinking skills; in particular, they see potential in such areas as problem solving, metacognition, and decision
making. However, much of the research in this area lags behind the technological advances, focusing on user demographics,
attention spans, and perceptual skills, instead of addressing the impact these games might have on player’s analysis, decision
making, and reflection skills. In part, the current body of research represents the interests of the gaming industry, which is more
focused on exploiting any new technology to satisfy the attitudes, preferences, and expectations of its users, rather than the
interests of education and training. It also reflects the fact that this is an emerging area that suffers from limited research and
strategic planning. The report reviews the relevant research literature and proposes 15 primary experiments.
15. SUBJECT TERMS
training, education, learning, distributed learning, distance learning, online instruction, online games, multiplayer games
SECURITY CLASSIFICATION OF
19. LIMITATION OF
20. NUMBER
21. RESPONSIBLE PERSON
16. REPORT
Unclassified
18. ABSTRACT
Unclassified
18. THIS PAGE
Unclassified
ABSTRACT
Unclassified
OF PAGES
49
(Name and Telephone Number)
Dr. Robert A. Wisher
(703) 697-4992

iii
Technical Report
Massive Multiplayer Online Gaming:
A Research Framework for Military Training and
Education
Curtis J. Bonk
Indiana University
Consortium Research Fellows Program
Vanessa P. Dennen
Florida State University
March 2005
Office of the Under Secretary of Defense (Personnel and Readiness)
Readiness and Training Directorate
Advanced Distributed Learning (ADL) Initiative
Washington, DC
Approved for public release; distribution is unlimited.

v
FOREWORD
The goal of the Advanced Distributed Learning (ADL) Initiative is to provide the highest
quality education and training, tailored to individual needs, delivered cost effectively, anywhere
at anytime. Since its inception in 1997, ADL has sought to create interoperability of online
learning content through the Sharable Content Object Reference Model (SCORM) and to make
quality content accessible from commonly designed content object repositories.
The current report examines multiplayer online games as a future focus area for ADL.
The adoption of and interest generated by these games is truly incredible. From a review of the
research literature, we know that playing such games can lead to measurable learning outcomes.
How well these outcomes correspond to military training and education requirements needs to be
understood to make full use of the capability. Can they become part of a performance support
architecture? What are some of the key experiments that can help us understand the full range of
benefits to learners and organizations?
This report examines these and related issues in online games. Its focus is on learning
rather than the underlying technology. It is academic in nature, reflecting the findings in the
research literature. This report serves as a research foundation for experimentation on massive
multiplayer games within the military.
ROBERT A. WISHER
Director, Advanced Distributed Learning Initiative

vii
Massive Multiplayer Online Gaming:
A Research Framework for Military Training and Education
EXECUTIVE SUMMARY
Research Requirement:
The United States Military is undergoing significant change in the training of its
workforce. Massive multiplayer online gaming (MMOG) is one technology that offers unique
education, training, and performance support opportunities. While the research on MMOG is
scant, there is a need for a review of measurement methodologies related to MMOGs for adult
learners. In particular, this document notes trends in the use of games and simulations for
education and training purposes, common and preferred communication features, motivational
aspects of multiplayer games, and the results of preliminary research in this field. In addition, it
outlines completed and ongoing efforts to develop training games in a military context as well as
findings related to the transfer of performance in games to performance on occupational tasks.
As is evident in this document, there is a pressing need to know how problem solving and
decision-making skills are being measured in online gaming environments.
Procedure:
Database searches yielded less than a dozen research reports specifically on MMOGs.
Instead, most articles in this field related to press releases about trends in the industry. However,
there were a number of research studies on various games and simulations in the military as well
as education and business which inform this report. In reviewing these studies, significant gaps
in the literature were noted. A set of 15 primary and 18 secondary experiments were designed to
address most of the gaps that were identified in the review of the research literature.
Findings:
There are many interesting research questions to be asked in the field of MMOG. Among
the open questions or issues are those related to the benefits MMOGs when blending face-to-face
and online learning, the impact of a learning community on player performance within a
MMOG, variables that impact persistence of group members, the motivational aspects of
MMOGs, the impact of MMOGs on player decision-making, leadership, and problem solving
skills, the ability to influence MMOG role playing, and the prevalence and impact of online
gaming addiction. In response, 15 primary experiments were designed that address these areas
and many others.
Utilization of Findings:
The proposed experiments can be implemented in a number of ways. For example, the
experiments can be embedded in courses undergoing a transition from the classroom to a
distributed learning environment. In addition, these experiments can be applied to either
professional development education or specialized skill training throughout the military.

ix
MASSIVE MULTIPLAYER ONLINE GAMING: A RESEARCH FRAMEWORK
FOR MILITARY TRAINING AND EDUCATION
CONTENTS Page
REPORT DOCUMENTATION PAGE............................................................................... i
FOREWORD.....................................................................................................................iii
EXECUTIVE SUMMARY ..............................................................................................vii
1. Introduction......................................................................................................................1
1.1 Size and Growth of the Computer Gaming Industry.................................................1
1.2 Impact of Computer Gaming in Everyday Life.........................................................2
1.3 Differences between Games and Simulations............................................................2
1.4 Gaming, Technology, and Learning ..........................................................................3
1.5 Computer-Based Simulations in Education...............................................................4
2. Massive Multiplayer Online Games ................................................................................5
2.1 Massive Multiplayer Online Games Defined ............................................................5
2.2 Size and Growth of MMOGs.....................................................................................6
2.3 Features and Design of MMOGs...............................................................................7
2.4 Social Elements of MMOGs......................................................................................9
3. Gaming Developments in the Military ..........................................................................11
4. Gaming Research...........................................................................................................13
4.1 Related Research......................................................................................................18
4.2 MMOGs and Online Community ............................................................................20
5. Suggestions for Research...............................................................................................22
5.1 Fifteen Primary Experiments...................................................................................22
5.1.1 Research Study I. Impact of After Action Review (AAR) on MMOG
Performance and Decision Making Style .....................................................22
5.1.2 Research Study II. Impact of Addiction to MMOG. ............................................23
5.1.3 Research Study III. Sense of Community and Group Longevity or Persistence..25
5.1.4 Research Study IV. Describe the Apprenticeship Process in MMOG..................25
5.1.5 Research Study V. Game Authenticity and Constructivism.................................26
5.1.6 Research Study VI. Bandwidth Constraints and Differences...............................27
5.1.7 Research Study VII. Role Assignment: Achievers, Socializers,
Explorers, Killers...................................................................................................28
5.1.8 Research Study VIII. Cognitive Tools and MMOG Performance and Dialogue .29
5.1.9 Research Study IX. Collaboration and Virtual Teaming in MMOG: Co-located
and Distance Groups.....................................................................................30
5.1.10 Research Study X. Decision-making, Leadership, and Interpersonal Conflict in
MMOGs........................................................................................................31
5.1.11 Research Study XI. Learning from Mistakes and Learning Histories................32
5.1.12 Research Study XII. Learning Style and Game Selection..................................33
5.1.13 Research Study XIII. Game-based Motivation...................................................34
5.1.14 Research Study XIV. Problem Solving Processes and Types of Knowledge
Facilitated by MMOGs.................................................................................35
5.1.15 Research Study XV. Social Skills and Friendship Development.......................36

x
5.2 MMOG Assessment and Evaluation........................................................................36
6. Future Directions ...........................................................................................................37
7. References......................................................................................................................38

1
Massive Multiplayer Online Gaming:
A Research Framework for Military Training and Education
1. Introduction
The field of computer gaming has risen to near equal status with the film and music
industries in terms of revenue, customers, and employees (Kirriemuir, 2002).
Jayakanthan (2002) likens the influence of the computer gaming industry on youth today
to the influence of music, religion, and politics in previous decades. In fact, computer
games are so popular that blockbuster films are being made about them (e.g. Super Mario
Brothers). According to Snider (2004), launching a game costs roughly $10 million with
nearly half of that amount supporting development costs and the other half for marketing.
Costs will depend, however, on the size and location of the development team, the need
for highly skilled developers, the number of special effects, and number and types of
platforms it is to be designed for (Sawyer, 2002).
Certain skills gained and practiced by gamers in massive multiplayer online gaming
environments closely parallel those required by a military transforming itself to operating
under the concept of network centric warfare. The technologies and practice
methodologies employed in multiplayer games also hold great potential to provide
appropriate network centric warfare training environments.
Network centric warfare describes an environment where “geographically or
hierarchically dispersed, knowledgeable entities” are linked (Alberts, Garstka & Stein,
1999). Multiplayer games are built around the same concept of electronically linking
participants and displaying them together in the same virtual space. Virtual teams in both
network centric warfare and MMOGs create shared situational awareness through the
sharing of information and through goal-directed collaboration. They also cut across
hierarchical lines by empowering and expecting knowledgeable entities to “collaborate
with one another to achieve a degree of self-synchronization” (Alberts et al., 1999).
1.1 Size and Growth of the Computer Gaming Industry
The commercial computer gaming industry in the United States has been estimated at
anywhere from $7 billion (Herz & Macedonia, 2002) to $9 billion (Vaknin, 2002) which
rivals, and many now argue surpasses, Hollywood movie box office revenues of $8.1
billion (Snider, 2002). Global estimates in 2002 for computer gaming were $27 billion
and growing at a rate of 20 percent annually in the United States alone (Delaney, 2003).
Furthermore, the gaming industry’s yearly revenue in the U.S. is projected to soon pass
both the record industry, as well as the home video sales market (Snider, 2002).
Computer games have reached a fairly ubiquitous level of availability -- whether one is in
a hotel room, on an airplane, sitting at home reading the paper, or using a cell phone or
wristwatch, odds are that a computer game is present. Already the gaming industry
attracts more than 60 percent of the American population, with an average age of 28
(Kirriemuir, 2002). The numbers are higher for teenage boys, 75 percent of whom play

2
computer games (Lenhart, Rainie, & Lewis, 2001). The opportunities to play online
games are projected to soar during the coming decade.
1.2 Impact of Computer Gaming in Everyday Life
The widespread availability and popularity of gaming has led to questions about its
effects on our lives. Through survey and ethnographic research, the Pew Internet and
American Life Project (Jones, 2003) explored the impact of gaming and entertainment
technology on the lives of college students. According to this study, 70 percent of
college students played computer, video, or online games at least once, and 65 percent
reported being regular or occasional game players. From such data, it is clear that once
one has played a game, there is a high likelihood of a repeat occurrence.
In this age of multitasking, games are highly integrated into leisure activities such as
music or interacting with friends. Initial computer games tended to be single-player or
involved turn taking. Not surprisingly, many people stereotype computer games as
isolated or individual activities. However, Jones (2003) argues that computer gaming
should no longer be connoted as a solitary activity, but instead it should be considered a
social endeavor involving friends, more like traditional board games.
Nearly half of the students in Jones’ (2003) study admitted that gaming sometimes kept
them from their studies. However, these college students viewed their time spent with
these games as a positive aspect of their lives, in contrast to the more negative terms
often seen in the research literature. Across these findings, Jones noted that college
students selecting online gaming environments were doing so to take part in an extremely
social activity but such activity was just part of many other tasks.
1.3 Differences between Games and Simulations
Games and simulations are broadly used. Computer games include strategy games,
building simulations, quests and adventures, sports games, war games, puzzles,
educational games, and computer role playing games (Ju & Wagner, 1997). According to
Sawyer (2002), the advantages of simulations and games is that they: (1) can foster
strategic thinking and other learning benefits through repeated use; (2) utilize technology
available to the general public; (3) can intertwine both fictional and non-fictional events.
Simulations attempt to provide an environment that represents or mimics reality so as to
facilitate learning of the player or user. They often are built to engage the learner in
situations or events that would be too costly, difficult, or hazardous in the real world
(Gredler, 1996) or that may be deemed problematic for ethical reasons (e.g., frog
dissection). They tend to immerse the learner in complex, evolving situations and
relationships. In contrast, games attempt to foster skill development by providing
entertaining challenges for the player. As Prensky (2001) notes, the structural
components of games include fun, play, competition, goals, rules, and winning.

3
Gredler (1996) points to three key differences between simulations and games. First,
games consist of rules that describe allowable moves, may not reflect the real world, and
penalize for non-permissible actions. In contrast, simulations are based on relationships
among several variables that tend to change over time, and, nonetheless, reflect authentic
causal processes (Leemkuil, de Jong, & Ootes, 2000). Instead of attempting to win a
game, those in a simulation experience are engaged in a more serious and authentic
responsibilities. Finally, games tend to be linear, whereas simulations present different
problems, issues, or situations depending upon previous participant decisions. Gredler
(1996) also notes that certain games and simulations can also be differentiated by the
types of tasks that are established as well as the actions that tend to be rewarded.
Games are motivational, in part, due to their uncertain outcome and the focus on a goal or
challenge that the user needs to accomplish. Early work by Malone (1981) indicated that
the most important feature of a game was having a goal. According to Malone (1980),
the goals and challenges within games are important since they engage a player’s self-
esteem. Multiple game goals or different levels of goals provide incentives and
challenges for players once an initial goal is accomplished. Game players, therefore,
must develop skills and strategies in order to win or achieve a goal (Sawyer, 2002).
Unlike most formal training or education, in games there are multiple paths to success.
In addition to the importance of having a goal, players can affect the outcome of the
game; in effect, they have some control over their learning situation. In an educational
environment, then, such goal-based learning and sense of control deemphasizes or shifts
the role of the instructor. From this perspective, success in a computer game, can
positively impact a person’s self-esteem. Games are also designed to be interesting, fun,
and played multiple times (Sawyer, 2002). Often they are multiplayer, and, more
recently, massive multiplayer. Massive tends to mean that there can be hundreds,
thousands, or even millions of game players in the same world simultaneously.
1.4 Gaming, Technology, and Learning
Gaming is not an isolated phenomenon. Its growth has paralleled advances in information
and communication technologies, and both in turn have had a clear effect on learning. For
example, enrollments in online courses are surging and today's students – immersed in an
increasingly digital world – are seeking richer and more engaging learning experiences.
Amid the rising tide of expectations, instructors are exploring innovative ways to use
technology to foster interaction, collaboration, and excitement for learning.
Oblinger (2003) claims that the learners of the early part of the twenty-first century are
accustomed to ubiquitous computing environments where multitasking and staying
connected are the norms. She further notes that students who are active participants in
the information age want to engage in relevant meaningful tasks rather than just complete
worksheets and accumulate knowledge. For instance, many teenagers are opting to chat
online with their friends instead of watching television or talking on the telephone. In
schools as well as informal learning settings, they want to collect real world data and
manipulate it. Instead of knowing basic facts of a situation or where a place is located,

4
learners can simply “Google” it. Perhaps, most importantly, as they engage in new forms
of learning, they simultaneously expect their teachers to offer learning opportunities in
exciting and engaging formats where they have some control and responsibility over their
own learning (e.g., the Internet). They expect learning to be available on demand whether
it is through online agents or using supplemental materials their instructors have posted.
The authenticity of learning environments also is on the rise. As Jayakanthan (2002)
notes, when U.S. Embassies around the globe were threatened, game developers began
designing new levels or challenges within their games to deal with hostile takeover
training and other terrorist threats. These challenges were based on actual embassy floor
plans, creating a realistic game context and environment. In a way, these were
momentous changes since they shifted the power and control of game development from
the game developer camp to the end users (Riddell, 1997).
1.5 Computer-Based Simulations in Education
In response to this perceived need for authenticity, there has been an increase in the use
of simulations in both home and school environments. Simulations can help learners
understand how to run a business, build a town, or save an environment. Learners can
watch changes in processes, explore different assumptions in key military battles, and
create alternative worlds and histories (Rickard & Oblinger, 2004).
Aldrich (2002; Morrison & Aldrich, 2003) delineated different types of simulations. First,
there are branching simulations, which he likens to adventure books. A branching
simulation might take the player through a situation such as a mock customer encounter.
Here, the simulation will continue down different paths and display outcomes depending
on the decisions of the player or user. Business school spreadsheets are another type of
simulation. Such simulations depict the results of changes in different variables such as
pricing, inventory, and outsourcing, on corporate sales and profits. Third, Aldrich
discusses game-based simulations such as quiz shows which have been popularized by
gaming advocates such as Marc Prensky (2001). Fourth, product-based simulations have
been proliferating with the popularity of Macromedia’s Flash software. A virtual product
might show the user where and how a particular device works.
Each type of simulation has distinct advantages and disadvantages (Aldrich, 2002). For
instance, branching models are good for developing soft skills and providing consistent
assessment. In addition, they are highly adaptive. However, he noted that in a branching
simulation, there is a continuous feeling of assessment to a user rather than a meaningful,
self-directed experience. Each type of simulation has a downside. According to Aldrich,
for instance, while real actors in a video-based simulation foster a sense of credibility and
can display subtle aspects of specific skills needed, they are highly expensive to produce
and are nearly impossible to change once created. Similarly, Aldrich points out that
computer graphics-based simulations can help focus on specific skills or competencies
needed and might tap or foster user creativity and adaptive, higher-level thinking skills.

5
One recent example of a computer graphics-based simulation is the Virtual Leader from
SimuLearn (Aldrich, 2004; Morrison & Aldrich, 2003) which Aldrich, himself, helped
produce. In a nutshell, Virtual Leader simulates a business meeting and requires the
player to perform a number of social interaction tasks (e.g., introduce ideas, refocus the
group on ideas, take ideas off the table) with other computer generated characters in order
to be an effective leader. The player must negotiate through tensions raised, explicit and
implicit signs of power, and comments that have little to do with the meeting.
Clearly, there are many cognitive and metacognitive skills are embedded in playing the
Virtual Leader. As alluded to above, from a cognitive standpoint, players learn how to
recognize who has the power at particular points in meetings, when to reduce or raise
tension in a room, how to gain power as a leader, and how to recognize ideas that appear
to be hidden. From a more metacognitive perspective, players learn about goal setting
and formulating plans to reach those goals, executing one’s plans, obtaining feedback on
their actions, and reflecting on and updating one’s understanding of a situation after
obtaining feedback (Aldrich, 2004). With this focus, Aldrich is not simply promoting a
set of vital higher-order thinking skills; instead, he is promoting a cycle of learning.
Simulations are popular in other business situations and management processes. Data
files, video clips, and flowcharts might help learners understand inventory control,
customer order processing, and shipping schedules (Dessouky, Verma, Bailey, & Rickel,
2001). Higher-level outcomes might include learning how to make strategic planning
decisions such as how to market products and where to locate new plants.
Stand alone simulations such as SimTower, and SimCoaster are exciting, initial
indications of where this technology can lead learners. In those programs, feedback is
typically from the system or game since the games are played alone. However, those
using single player games can upload their scores or replays for others in the community
to provide comments. As examples of the solution strategies of experts in the gaming
community, such replays can help the problem solving skills and metacognitive processes
of the users who access them as well as those posting them. And, with the recent rise of
collaborative tools on the Web, it is standard practice for simulations to offer more than
standard system feedback in solitary learning environments.
2. Massive Multiplayer Online Games
2.1 Massive Multiplayer Online Games Defined
Massive multiplayer online games (MMOGs) allow users to interact with other users
around the globe in real-time. But what, specifically, are MMOGs? According to Squire
and Steinkuehler (in press):
Massive multiplayer online games are highly graphical 3-D videogames played
online, allowing individuals, through their self-created digital characters or
“avatars,” to interact not only with the gaming software (the designed

6
environment of the game and the computer-controlled characters within it) but
with other players’ avatars as well (p. 2)
They further point out that MMOGs are persistent worlds for both social and material
exchanges that maintain a loose structure through fantasy and rich stories or narratives.
Players in these worlds can pretty much do as they please; if the want to slay a dragon
and an ogre, raise someone from the dead, siege a city or a castle, hunt for food, trade
goods and services, or be the village idiot, they have the prerogative to do so as long as
they are within the social boundaries of the game (Kolbert, 2001; Squire & Steinkuehler,
in press). In Star Wars Galaxies, they can also swap stories and strategies at the virtual
rebel outpost or local cantina with friends and foes from across the globe. In Ultima
Online, such social rituals might also include pet shows, marriages, comedy nights,
amateur theatricals, and disco parties (Kolbert, 2001). Whether one is interested in being
a hero or villain, it is all possible in MMOGs. They embed escapist fantasies yet take on
an emerging social realism in the memberships within the communities that form and the
actions that unfold (Steinkuehler, 2004). Small wonder they are growing so fast.
2.2 Size and Growth of MMOGs
Even though not all games have been successful (e.g., The Sims Online, etc.), many
scholars and leaders in the gaming field suggest that MMOGs are where most gaming
activity will reside during the coming decade (Schatz, 2001). There is an expanding list
of these MMOGs, including Anarchy Online, EverQuest, Asherton’s Call 2, Ultima
Online, and Dark Age of Camelot. As computer processing speed continues to increase,
computer hard drives grow in capacity, available bandwidth expands, and hardware costs
simultaneously plummet, the demand for MMOGs will continue to rise.
The scale in which they are used is impressive. By 1998, Ultima Online had more than
14,000 simultaneous players on many nights (Kim 1998). According to Herz and
Macedonia (2002), by 2002, EverQuest hosted 350,000 players with over 100,000
players during peak times. More impressively, at about the same time, Lineage had 2.5
million subscribers (Vaknin, 2002). However, this is a cultural phenomenon as Lineage
is mainly a Korean game and is embedded in the culture with pc gaming cafes (or
“baangs”, see Borland & Kanellos, 2004) which all have extremely high capacity. High
scorers in this game are cultural heroes. As such Lineage is a highly social game which
in many ways is played face-to-face since your opponents surround you at the café.
While Ultima Online ranked somewhat lower at around a quarter of million subscribers
in 2001, its players were intensely involved in this game; logging an average of 13 hours
per week, which equates to more than 1 hundred and sixty million man hours per year.
Some individuals such as Woodcock (2004) have monthly reports on online gaming
subscribers. By January 2004, Woodcock’s (2004) monthly tracking of MMOG
subscribers indicated that EverQuest subscribers had increased to 430,000, though it had
peaked at 460,000 just six months earlier. By March 2004, Woodcock reports that Star
Wars Galaxies had about 275,000 registered users of which perhaps 250,000 were active
players. These numbers are particularly impressive given that this game had just been

7
released in June 2003. Even more impressive was Final Fantasy XI, released in North
America in October 2003, which had over 500,000 subscribers as of January 2004, just
three months following its’ launch date (Shwayder, 2004).
Not all data are on MMOG growth are so favorable. The highly publicized game “The
Sims Online,” has yet to break 100,000 subscribers, even though the company had
targeted 400,000 subscribers by the end of 2003. And Ultima Online remained at roughly
the 225,000 users it had three years earlier. In fact, in his most recent report, Woodcock
(2004) suggests that the fantasy MMOG market is becoming saturated with too many
games. Despite the potential market saturation, MMOGs are clearly a huge source of
consumer entertainment in the early years of the twenty-first century.
There also are a variety of motivators that attract players to MMOGs. Unlike single user
games in which the primary enticements to play are related to intrinsic motivation
(Malone, 1980; Malone & Lepper, 1987), multiplayer games are filled with extrinsic
motivators (e.g., competition, collaboration, recognition, and material goods; Chen, Shen,
Ou, & Liu, 1998). For example, players are attracted to games by their desire to compete
and collaborate, potential affiliations with a global network of players, their appetite for
peer acknowledgement, and the tools available for custom character and scene creation.
In effect, players are drawn into MMOGs by the possibility of being part of a community.
In many instances, the players, themselves, are part of constant modification and
improvement of the games. Though not a MMOG, The Sims (not to be confused with
The Sims Online) spurred a network of player Websites that showcased custom Sim
objects and characters. In fact, over 90 percent of this game is produced by the player
population (Herz & Macedonia, 2002). From a cognitive perspective, it is important to
note that The Sim players are deeply engaged in the design process. From a social
perspective, they also are involved in relationships with other players that are competitive
as well as cooperative and collegial. The persistent nature of these environments
intertwines all three types of relationships -- competitive, cooperative, and collegial --
thereby offering unique learning possibilities.
2.3 Features and Design of MMOGs
There are a variety of features that make MMOGs attractive. Players might achieve
particular milestones, access new weapons and tactics, take on increasingly challenging
opponents and obstacles, and assume different roles or character identities (Herz &
Macedonia, 2002). Herz and Macedonia further state that a multiplayer online game
might embed player chat, tournaments, places to challenge opponents, and tools to share
ideas and strategies. However, the relative impact of each feature is not fully understood.
In contrast to video games that revolve strictly around shooting and killing, in MMOGs
there is more character development, friendships, and partnerships. Players might
develop character roles and be better known for such roles than for their real world
occupations, thereby granting them a status that they may not have previously
experienced. They might form new friendships, build homes, shop, or reconstruct a

8
society after a major catastrophe (Knight, 2003). According to Herz and Macedonia
(2002), “On a basic level, this interaction is what drives the massively multiplayer online
world—the standard societal tensions that inform any city-sized population” (p. 8).
These online games are empowering to learners for many reasons. According to Gee
(2003b), there are many learning principles implicitly built within the more effective
games. Gee (2003a) argues that when playing a game, learning should be both frustrating
and life enhancing. For games where there are rewards for longevity, life-enhancing
events are crucial. Gee (2003b) also claims that they should foster a sense of co-design
since the changes the user makes impacts the system. The player cannot be passive, but
must be a participant. It is here that learners are involved in a problem space where
expertise is built over time, and challenges increase (Downes, 2004). In MMOGs,
information is available and used upon demand. The immediate application of such
knowledge elevates the learning process and excites the learners. Gee (2003b) presents
36 learning principles that should be considered and built into video games. Ten of these
principles, summarized in Table 1, are particularly pertinent to MMOGs.
Table 1
Ten principles to be considered when creating MMOGs (adapted from Gee, 2003b)
Principle Description
1. Achievement Principle Learners should be constantly rewarded, at
each level of game play and skill mastery.
2. Amplification of Input Principle Learners should get out of the experience
something greater than they put in.
3. Distributed Principle Learners should find growth and knowledge in
their interactions with other learners,
technology, context, objects, and tools.
4. Identity Principle Learners should have the ability to build a
virtual identity and to reflect on its relationship
to their real-world identity and their desired
identity.
5. Multiple Routes Principle There should be more than one way for
learners to progress, encouraging them to make
decisions and solve problems.
6. Practice Principle Learners should be able to spend a lot of time
practicing in an interesting setting.
7. Probing Principle Learners should be encouraged to engage in
cycles of action, hypothesis building, and
inquiry.
8. “Psychosocial Moratorium” Principle Learners should be able to take risks in
artificial environments where there is a lower
chance of real-world consequences.
9. “Regime of Competence” Principle Learners should be challenged to push beyond
their comfort or current ability zone, but not to
an extent that is unsafe or unattainable.
10. Self-knowledge Principle Learners should learn both about the learning
environment and themselves via the gaming
experience.

9
Based on extensive experience with games, Sellers (2002) suggested various principles
for group design and interactions in “massively multiplayer games” (p. 1). In particular,
he agued that it is the social bonds created by players that motivate them and keep them
returning to a game. Sellers calls the initial social bonds that players form “temporary
groups” (p. 4), noting that these interactions are low-commitment but enable players to
more immediately feel confident, start the game, and make deeper social connections
later in the game.
Listed below are some of the key features that Sellers (2002) recommends for building
social interaction in MMOGs:
• Ability to know who is available and participating in the game along with their
current location.
• Private, within-group communication, including ability to see others’ game
statistics.
• Allowing for different, complementary roles with individual abilities, mirroring
real-life complementary roles and boosting the functionality and desirability of
teaming.
• Providing rewards for good group play and group accomplishments rather than
focusing on the work of individual players.
• Develop permanent groups, such as guilds.
2.4 Social Elements of MMOGs
While there are communities of practice underlying stand-alone games, some argue that
there is an even stronger social foundation underlying MMOGs (Sellers, 2002).
Multiplayer games may motivate through interdependent roles and the social bonds that
are formed between players. In addition, they have specialized characters with their own
highly contextualized jargon. And they may belong to guilds, allegiances, or other forms
of associations through which their expertise is nurtured and promoted. According to
Sellers (2002), “the broader the set of functional roles your game supports, the denser its
gameplay and social web will be” (p. 7). Having unique talents or traits builds a sense of
expertise and pride. As indicated, the social support, challenge, feedback, and sense of
identity that players receive online in MMOG often provides psychological fulfillment
that they may not be receiving in real life (Schiesel, 2003).
There are a wide variety of emotions expressed and felt when playing MMOGs.
According to Baron (1999), much of the social and psychological foundations in
multiplayer games revolve around the sense of glory and shame that they promote. He
argues that the ever present record keeping and external audience has enormous power.
And because of the multiple roles of online game players—spectators, visitors,
participants, designers, etc.—the notion of audience is highly unique. Although new
players may experience public embarrassment as they fumble to learn the game, when
their skills and social networks begin to grow, they are likely to find opportunities for
unique rewards and recognitions. Still, Baron argued, players return primarily for their

10
personal development in the social fabric of the community for that game, not just for the
glory.
There are many roles embedded within MMOGs. Some of these roles can be extremely
functional (e.g., tailors, miners, carpenters), while others involve fantasy (e.g., wizards,
demons, ghosts; Sellers, 2002). In addition, there may be group roles embedded within
the game by the designer (e.g., a leader) as well as those that might emerge or be
designed by the players (e.g., explorers, trail guides, innkeepers, and docents). Those
first entering a MMOG, often referred to as “newbies,” need patient, generous, and
responsive guides. The strength and longevity of a game may depend on these people
since everyone is, in a sense, a newbie at some point. Of course, there must be some type
of rewards and recognition built into these systems that strengthen or enhance the
opportunities for such guidance or help giving behavior. It is vital, therefore, for MMOG
developers, evaluators, and users to think about how the skills of different characters
benefit everyone in the group.
Some highly committed gamers will contribute to or invest in the community by finding
and eliminating bugs or flaws (Rickard & Oblinger, 2004). In effect, they take some
ownership over the learning process. The respect and recognition that they receive from
the community for their work enhances that environment and fuels additional
participation and pride. Acknowledgement is an extremely powerful incentive. As
Rickard and Oblinger (2004) point out, “gaming—especially massive multiplayer
games—builds communities of gamers who will continue playing as long as they can” (p.
16). Given today’s increased focus on cultivating communities of practice in the business
world (Wenger, McDermott, & Snyder, 2002) as well as learning communities in
education, research on the apprenticeship process in these MMOGs may prove highly
interesting and informative.
Besides providing opportunities for apprenticeship, MMOGs typically are highly
complex worlds. In an interview, Gee argued that these worlds offer opportunities for
thinking about one’s decisions and reflecting on the impact of each of them (Johnson,
2003). In addition to such metacognitive monitoring skills, there are numerous learning
opportunities during game play such as perceiving patterns in events and uncovering
hidden relationships. Rickard and Oblinger (2004) note that the scenarios and plots that
one must navigate in games foster critical thinking and problem solving skills.
Teamwork is also crucial as a player’s success often will be dependent on the degree to
which he or she collaborated or worked effectively with other players in virtual teams.
In addition to these cognitive and metacognitive skills, self-directed or exploratory
environments, such as online games, offer a unique motivational atmosphere rich in fun,
feedback, incentives, novelty, and challenges. Many motivational scholars have pointed
out the importance of a supportive but challenging environment, choices, short-term
goals, and immediate feedback on performance (Pintrich & Schunk, 1996; Reeve, 1996;
Stipek, 1998). The authenticity and hands-on nature of these environments is another key
motivator. According to Randy Hinrich, Microsoft Research Group Research Manager
for Learning Science and Technology, “What we do is create environments where

11
students can practice what the teachers are telling them” (King, 2003, p. 3). In effect, the
ability to apply what has been learned is an important aspect of online games.
3. Gaming Developments in the Military
The Department of Defense (DOD) has placed an emphasis on restructuring the entire
process of training. As described in the DOD Training Transformation Implementation
Plan (Department of Defense, 2004)
The dramatic transformation of America’s strategic environment continues its
significant impact on our military forces and its demand for an equally dramatic
transformation in how we prepare forces for combat and noncombat operations …
we meed to transform the way we train. (p.2)
The DOD recognizes a need for new warfare skills in challenging and rapidly
developing missions (United States Joint Forces Command, 2002). Net-centric warfare,
an emerging theory of war, involves a cultural change in virtual relationships involving
many individuals and teams (Raduege, 2004). Power is gained from information, access,
and speed, all characteristics of multiplayer games. As military operations become more
integrated and networked, the importance of planning, decision making, and
collaboration skills become increasingly obvious. To improve effectiveness, there is also
an increased focus on sharing information between military planners and operators
(United States Joint Forces Command, 2002).
With this focus on emerging technologies, the military is clearly interested in exploring
the use of online collaborative games to train staff on the modern day intricacies of
combat and noncombat operations. At the same time, the increasing focus on a remote-
controlled agents has raised expectations and excitement for realistic simulations and
games -- especially MMOGs. The military is developing games that could host
thousands of networked players. In these games, players potentially could participate for
months or years in different roles and later reflect on the consequences of their decisions
and actions (Harmon, 2003). Debriefings or reflective processing of these games could
help the user understand the purpose of the game and generalize it to different situations
(Prensky, 2001). The immediate goal, of course, is to enhance decision making, problem
solving, and reflection skills in the context of a military operation.
To help in the effort to provide better coordinated and timelier training technology
strategies, the DOD created the Advanced Distributed Learning (ADL) Initiative as a
collaborative partnership between government, industry, and academia to design and use
interoperable learning environments (Wisher and Fletcher, 2004). For instance,
distributed collaboration tools can enhance communication and strategic decision-making
among participants. As part of these efforts, the Defense Acquisition University (DAU)
redesigned its curriculum to meet the demands of budget cutbacks, a growing workforce,
and a huge percent of retirements on the horizon (Schettler, 2003). The ADL Initiative is
to examine methods for learning management systems to link to simulations and online
games as a means to enrich learning environments and enhance learning outcomes.

12
Herz and Macedonia (2002) declare that simulations have been an integral part of U.S.
military training for decades, though such use has been growing somewhat separately
from the commercial gaming industry. They note that the military is finally exploiting
advances in the commercial simulation and gaming industry to advance its training and
education. The military has adopted commercial simulations and games for classroom
and unit training.
In further recognition of the importance of entertainment in training technologies such as
games and simulations, in August, 1999, the military created the Institute of Creative
Technology (ICT) in Marina Del Ray, California. The ICT works closely with a number
of departments of the University of Southern California to create active as opposed to
passive programs and systems. This partnership evolved from a realization that the
gaming industry customers of the past two decades may become the future recruits, and,
potentially, the officers of the military. Furthermore, the gaming industry is increasingly
linked to Hollywood movies, since most blockbuster films will have a coinciding video
game (Snider, 2004), or, in some case, vice versa. These connections can help the
military appeal to a wider population as a career opportunity.
A key part of this initiative is to utilize the motivational aspects of gaming entertainment,
while fostering education and training (Jayakanthan, 2002). For instance, there is a push
for more realistic simulated experiences and scenarios (including user ability to see, hear,
and touch in the online environment) and intelligent computer-generated characters to
foster learner engagement and challenge (The National Research Council Computer
Science & Telecommunications Board, 1997). As part of this effort, motivational
principles such as challenge, conflict, curiosity, control, and fantasy are built into these
systems. The challenge is not in learning to handle or fire a weapon, but in learning to
lead squads, units, or larger forces.
ICT is currently developing several Battle-Command games that test as well as foster
leadership and strategic decision making. O’Neil, Baker, and Fisher (2002) reviewed the
key features and evaluation specifications of three games that have been developed at
ICT, namely, Full Spectrum Command, Full Spectrum Warrior, and Full Spectrum
Command Board Game. These games might be used as supplemental course material or
they could be fully integrated into specific courses. In addition, they might be single user
or multi-user, short (15-20 minute) or long (1.5 hours), and easy (1 hour of reading plus 2
hours of practice) or hard to learn (4 hours of reading plus practice and homework). The
focus when playing these games may shift between planning events to situational
awareness to visualization to tactical decision making. Of course, each of these skills,
strategies, and competencies require different instructional approaches and forms of
assessment (O’Neil et al., 2002).
The U.S. Army, for instance, has distributed videogames at no cost as part of its
recruitment efforts. Millions of potential recruits have gone through “basic training” in
America’s Army. Soon millions more will likely be playing, Full Spectrum Warrior, a
military action game originally created by ICT in partnership with Pandemic Studios, on

13
their Xbox (Stevenson, 2003). Given the need for decision making, planning, and
collaboration, mentioned earlier, it is not surprising that Full Spectrum Warrior focuses
on quick decision making by a squad leader in both urban combat as well as in
peacekeeping missions. Instead of simply reacting to threats, the player must devise a
strategy or plan (DiMascio, 2004). While this game will never totally replace live
training, it offers ideal supplemental training. In fact, it has been used by the Afghan
National Army to fight alongside U.S. troops. In effect, the players must be able to
recognize real threats, have extensive organizational and analysis skills, and, generally,
be able to lead (Wired News, 2003). Decisions about a variety of military objectives
must be made rapidly in a tense and hostile environment.
Based on the success of America’s Army, the U.S. Army has entered into a unique
contract with There.com, an innovative company in the MMOG space. Together they
will create innovative training opportunities (Davis, 2003). Among the goals of this
project is the development of MMOGs where tens, if not hundreds of thousands of
soldiers can interact simultaneously. Defense officials are interested in developing
programs that differ somewhat from typical MMOGs. For instance, such new programs
might offer intrusive coaching to players, mission rehearsal for special operations, and
after-action reviews for warfighters.
Many other MMOG types of programs are projected for the near future. According to
Peck (2004), the DARWARS from the Defense Advanced Research Projects Agency is
under development for on-demand personal training in military situations. While still in
the planning stages, technologies built into DARWARS will include digital video of
human instructors, intelligent tutoring systems, avatars, and MMOGs. Embedded in the
DARWARS system, if built, will be after action reviews and team support resources. In
addition, this program would target participant metacognition and reflection skills. In
terms of research, DARWARS is based on the notion of continuously available wars for
training (Defense Advanced Research Projects Agency, Defense Sciences Office, 2004).
Thus, it may provide a unique longitudinal research environment and a testbed for group
interaction and community building in MMOGs.
4. Gaming Research
Facts are accumulating about the impact of the Internet on physical activity, productivity,
lifestyle, and psychological health. Studies exist on the timing and patterns of Internet
use. However, online gaming is an area which has seen scant research (Jones, 2003),
even less in MMOGs (Griffiths & Davies, 2002). Instead, the prevalent research is
centered on adolescent youth in single user settings. Whereas studies of gamers in stand
alone systems often foster sentiment that such individuals are socially withdrawn or
introverted, MMOGs may involve more social interaction skill and be more educationally
valuable than anticipated or promoted. In fact, the questions behind MMOGs may prove
more interesting and educationally beneficial to answer than research on traditional
computer-based games since players are forced to interact with each other.

14
Why is there such a dearth of research in this area? In part, the lack of research reflects
the nature and funding of MMOGs and related technologies. Most of the widely known
and promoted MMOGs are developed and supported by commercial companies. As
such, most research conducted on these games is likely to be internal and unpublished –
as well as focused on topics of interest to the company instead of vital research areas of
interest to other audiences. Publications that discuss topics like MMOG applications in
education tend to be either theoretical or anecdotal in nature rather than based on a
systematic design and data collection. For example, it has been said that educational
MMOGs are likely a good way to foster critical thinking skills, but the research to
support this assertion has not been conducted. MMOGs are a relatively new
phenomenon, another reason for the limited research.
There are controversies regarding who might offer the most insight into the gaming field-
-art/film/literary critics, anthropologists, sociologists, psychologists, learning scientists,
or educators. One area of research interest is how gaming impacts a learner’s skill
development. For instance, Green and Bavelier (2003) received wide attention for their
study of video-gaming on perceptual and motor skills. They designed a series of five
experiments that found video-gaming increases visual attention resources. Their work
demonstrated that increased visual resource capacity can result from just 10 days of
training with video-gaming. While finely tuned perceptual skills will not guarantee
higher-order thinking and leadership, individuals with increased ability to process and
react to information over time will have more resources to attend to any problems they
encounter. Such resources include the ability to detect new enemies as well as track
existing ones. Green and Bavelier suggest that future studies look at the impact of
speeded perceptual processes and enhanced task management capabilities.
Other research has examined process skills and strategic planning. For example, one
multi-player Web-based game, SCUDHunt, forces players to collaborate and share
information obtained from initial search phase results in order to locate and eliminate
SCUD launchers (ThoughtLink, 2002). A simple measure of decision quality used in this
research was the percent of strikes which were successful. These SCUDHunt
experiments investigated the quality of information, timeliness of information, command
methods (e.g., by direction, plan, or influence), knowledge of assets, mode of
communication on accuracy, shared situational awareness, and other subjective measures.
Not surprisingly, the availability of a communication tool, whether direct (e.g., text chat
or voice) or indirect (e.g., shared visualization), was a major contributor to the quality of
decisions (ThoughtLink, 2001). In addition, those who were recognized leaders received
the highest quality scores.
Beal and Christ (2004) explored the impact of the war game Full Spectrum Command
(FSC), mentioned earlier, on the decision-making ability of 54 officers enrolled in the
Infantry Captains Career Course. Half played FSC in addition to regular coursework,
while the other half acted as the control group. Beal and Christ were interested in the
impact of FSC on the decision-making style and decision-making performance of the
participants as well as their perceptions of the training game and sense of involvement
within it. The three decision-making styles analyzed were (1) analytic individuals who

15
tend to employ rational-thinking styles which focus on performance and training; (2)
intuitive individuals who tend to employ experiential thinking styles which are goal
oriented and impulsive; and (3) high regret-based emotional individuals who score higher
on personal harm avoidance, self-doubt, and depression.
Unfortunately, participants in this study scored high on the pretest on the more favored
analytic and intuitive styles. As a result, ceiling effects may have masked any positive
impact of FSC on participant decision making. In addition, participant cognitive abilities,
as measured by the Wonderlic Personnel Test, were not correlated with their decision-
making style. And while the participants gave favorable impressions to the game,
especially in terms of sense of challenge, realism, and personal involvement, they were
more focused on the technological controls than on strategic planning and mission
execution. Compounding the problems, the authors admitted to several key limitations in
the study, including limited time to play FCS and the absence of a measure of the
decision-making performance of game players. Still, there were some indicators that the
game provided valuable tactical experiences.
As indicated earlier, the U.S. Army has provided a PC-based game for potential new
recruits called “America’s Army.” The response to this game has been tremendous with
more than 3.4 million registered users of which more than 2 million have completed its’
Basic Training level (http://www.americasarmy.com). Interestingly, America’s Army
game players completed over a hundred million missions in just over one year.
A study of America’s Army with 20 new recruits and one ROTC cadet explored the
motivational and informational aspects of this highly popular first-person game
(Belanich, Sibley, & Orvis, 2004). In terms of information, the study participants
recalled procedures of the game better than factual information. Moreover, information
presented in a graphical and spoken format was better recalled by participants than print
information. Based on these results, Belanich et al. (2004) suggest that print information
not be eliminated but perhaps should be reduced during game play. This study also
revealed that information related to a player’s progress in the game was better recalled
than information that was not relevant to a player’s progression. In response to open-
ended questions, the participants indicated that the key factors that impacted learner
motivation related to player sense of challenge, game realism, opportunities to explore or
discover new information, and learner control. According to the researchers, all four
motivational variables should be considered in game development and use.
Recent survey research from the Human Computer Interaction Institute at Carnegie
Mellon University investigated the social aspects of MMOG both within and outside
participant gaming environments (Seay, Jerome, Lee, & Kraut, 2004). Over 1,800
players of EverQuest, Dark Ages of Camelot, Asherton’s Call, or Ultima Online
responded to Seay et al.’s 69 item survey. The results indicated that 90 percent of the
players were males who played MMOGs for 15-21 hour per week. Those who claimed to
be members of guilds were likely to spend more time playing online than those who did
not. Nearly 40 percent played, at least in part, for the social experience. However, when
asked to indicate the “main” reason for playing MMOGs, the primary responses were fun

16
(20 percent) and character growth (21 percent), while social contacts was listed as the
main reason for just 15 percent of respondents. On the negative side, addiction was
mentioned by 10 percent of respondents.
In similar research on MMOG players, Yee (2004) conducted a series of survey research
studies on EverQuest (EQ) and other MMOGs as part of his senior thesis at Haverford
College. During the year of his study, he collected more than 20,000 surveys from
roughly 4,000 individual participants. Most of his surveys where multiple choice and
consisted of 30-50 questions, requiring 5-10 minutes to complete. In order to gain
respect in the MMOG community and learn the relevant online gaming lingo, Yee
became an active player in EQ. His credibility enabled him to recruit thousands of
participants for his studies from well known Websites that catered to MMOG players
(Valenza, 2002).
In terms of whether MMOGs fostered or improved leadership and conflict resolution
skills, more than half of Yee’s respondents indicated that they learned mediation and
overall leadership skills and abilities such as reducing group conflicts and tensions
(2003). Nearly half indicated that they also learned persuasion skills and how to instill
loyalty or encourage and motivate group members. What appealed to MMOG players
was exploring fantasy worlds, enjoying social interactions, achieving goals, moving up
game levels, and completing quests. Less satisfying were feelings of power, killing angry
mobs, and crafting complex skills or learning a trade. Yee also explored the how long
players had played a game before they quit. Ultima Online was the longest at 28.1
months followed by EverQuest at 22.7 months. Such game playing longevity and
persistence indicates commitment and motivation for these multiplayer online games.
Recent research on MMOGs reveals how complex they are to study. Steinkuehler used
cognitive ethnography to investigate the cultural practices and consequences of a MMOG
called Lineage (Steinkuehler, 2003). In Lineage, a game that takes place during medieval
times, complete with elves, knights, magicians, princes, and princesses, there are blood
pledges or guilds which vie for different castles or kingdoms. Characters gain
reputations, wars break out over stories and rumors, special alliances are formed, and
countless stories are told. Her extensive description of this MMOG environment was
intended to help others understand what happens in different virtual gaming communities.
For instance, she was interested in how new learners are apprenticed into the community
through different activities, the role that different technology or cognitive tools play in
such an environment, and the contexts in which different learning community events
occur.
Steinkuehler’s ethnography was conducted through participant observation in order to be
immersed in the gaming experience. As Steinkuehler notes, a participant observation can
record and transcribe naturally occurring game-related activities and events, game related
communications (e.g., discussion board posts, e-mails, chat and instant messages, bulletin
board posts, e-mails, etc.), and all relevant documents of the online community (e.g.,
community-written player manuals, fan or e-commerce related Web sites, press releases,

17
white papers, technical reports, community-authored scenarios and game fictions,
company- and community-written player manuals and guidebooks, etc.).
In her study, discourse analysis was conducted on player discussion and chat transcripts.
Additionally, players were analyzed and categorized in terms of a community continuum
from newcomer, newbie, rookie, or legitimate peripheral participant (LPP) (per Lave &
Wenger, 1991) to status as a veteran, central participant, or expert. She notes that
through participation in valued practices or activities, one’s character or online identity
acquires pertinent knowledge of the game content (knowledge of terrain, enemies,
weapons, etc.), points system, goods available, social status, and social connections or
networks that move the player from the side-lines or fringes of the game to become a
more important or knowledgeable player within it. As this occurs, playing the game may
become an essential or pervasive part of one’s daily life. Not surprisingly, she found that
central participants or experts use terminology relevant to previous versions of a game,
whereas newcomers do not. Understanding any specialized terminology or specific
maneuvers of a game like Lineage is certainly related to one’s participation and role
within a community of practice.
Such research begs questions of how masters or experts of the game enculturate the new
players into it. How is their tacit knowledge shared with the rest of the community? Can
the process be described? Can training programs be created from it? As Steinkuehler
(2003, 2004) herself asks, what are the shared social spaces (Schrage, 1990) and material
practices that foster different ways of interacting, communicating, behaving, and valuing?
How might the skills learned in the MMOG transfer to the real world? And what is real
and what is one’s virtual identity? For instance, Steinkuehler noted that aspects of the
online world in Lineage can reemerge or spill over into the real world through telephone
calls, e-mail, and physical meetings.
Steinkuehler (2004) found that those who have mastered the social and material practices
of Lineage tend to scaffold or assist new gamers who lack sufficient knowledge and skill
to perform well (see Tharp, 1993; Tharp & Gallimore, 1988). Such scaffolded assistance
might be evident when modeling successful performance, sending key information on
how to navigate a difficult terrain, offering opportunities for practicing new skills, and
providing timely and situated feedback on one’s performance. Not too surprisingly, she
found the experts tend to grant newcomers increasing control of the learning situation
over time.
In the end, Steinkuehler cautions that the focus cannot be simply on the MMOG
technology or the curricular material that needs to be learned. The design of the system
must include the appropriate and emergent social structures or practices that will always
accompany the technology. In addition, she argues that without cognitive ethnography, it
would be extremely difficult to distinguish the types of practices, understandings, and
identities that MMOGs foster and equally hard to determine whether these are
productive, and, if so, portable to other environments. Of course, while some such as
Steinkuehler may point to the apprenticeship process within a game, extensive insight
might be revealed from exploring how games were used differently from the ways in

18
which game developers envisioned. Such work diverges from that which looks at the
social and cultural behavior of a consciously constructed culture shaped by certain
technological and commercial imperatives.
4.1 Related Research
Research conducted in related areas, such as small-group gaming environments, stand
alone computer games, and educational simulations, may be of use to developers,
researchers, and users of MMOGs. Such research is described below.
Research on learner preferences in personal computer games may provide useful tips for
designing MMOGs. For instance, Ju and Wagner (1997) conducted an analysis of
computer-based adventure games to determine (a) attributes of the games (b) variables
that make the games appealing, and (c) factors that influence a game’s ability to
contribute to the learning process. In particular, the authors were interested in how these
games might be used in managerial training situations. After revealing a long history of
using simulation-based games as part of managerial training, they categorized managerial
games in terms of the desired area of learning outcomes (e.g., problem solving games or
resource allocation games).
Through their analysis of many games, Ju and Wagner (1997) identified a model of
adventure games. The basic components of adventure games within this model included:
(1) Story, which sets forth the basic characteristics of the game by specifying (a)
story characteristics, (b) actors and actor characteristics, (c) resources, (d) tasks,
and (e) setting. It is through the interaction of these elements that the adventure
game takes its’ basic form.
(2) Development approach, which basically refers to how the specific details of the
game are created and whether actors and setting are virtual (computer-generated)
or real (video). This part of the model does not really affect the content or play of
the game.
(3) Implementation, which refers to the visual and technical aspects or interface of
the game. Implementation includes the user’s input options as well as the output
options such as scoring and feedback.
As alluded to in the research from Steinkuehler (2004) above, one area that would benefit
from additional study is how to scaffold learning in simulation environments (de Jong &
van Joolingen 1998). Simulations may contain conceptual models that focus on
principles, concepts, and facts, or operational models that target procedures and
experiential learning. In their detailed report, de Jong and van Joolingen cite a variety of
studies comparing simulation-based instruction to expository or instructor-led instruction,
noting varied results. For example, Bangert-Drowns, Kulik and Kulik (1985) found that
simulation based learning did not raise examination scores. However, de Jong and van

19
Joolingen note that the issue may not be the inherent effectiveness of simulations but
rather the challenges of using discovery learning. In terms of hypothesis generation, it is
quite possible that learners do not know what a hypothesis is, and, furthermore, those
who do may not be able to generate a hypothesis given the data found in a simulation.
In various studies (see Chinn & Brewer, 1993; Chambers et al., 1994) it has been shown
that learners often do a poor job of dealing with large amounts of anomalous data that
appear in a simulation. Once learners have a hypothesis, they may be likely to engage in a
phenomenon called confirmation bias in which they look to support rather than
disconfirm their working hypothesis. The dangers here are that learners may navigate the
learning environment seeing only what they want to see and, therefore, miss clues that
would guide them to a more fitting or appropriate hypothesis. Another challenge of using
discovery learning is the level of learner self-regulation. The presence of self-regulation
strategies prior to engaging in the discovery learning activity may well be a major
determinant of a learner’s potential success in that environment or learning activity.
Relating this back to MMOGs, which also may heavily rely on discovery learning when
used in an educational context, it is important to consider the guidelines or supports
learners may need to help ensure learning objectives are met.
Siemer (1995) recommends an evaluation methodology for intelligent gaming
simulations that target both internal and external evaluation. Within this context, internal
evaluation refers to the examination of the game’s architecture, what it does, and how it
reacts to user input. To this end, Siemer argues that one might draw upon Littman and
Soloway’s (1988) suggestion that intelligent tutoring systems be evaluated using analysis
of three key parts of the program:
(1) Knowledge level (does the program contain sufficient knowledge to meet learning
objectives?);
(2) Program process (how does the program work?); and
(3) Tutorial domain (what should the program do?).
External evaluation, per Siemer’s methodology, refers to the study of the game’s
usefulness to the learner in terms of both promoting learning and motivating the learner.
This same methodology might readily be applied to MMOGs, with internal evaluation
focusing on the game’s content and functioning and external evaluation focusing on the
learners’ interactions and resulting learning gains.
In a study of learner preferences regarding elements of computer-based games,
participant reactions to four commercial games – a strategy game, a simulation, a
shooting game, and an adventure game – were gathered (Amory, Naicker, Vincent, &
Adams, 1999). The researchers found that the simplest game was the most appealing (in
this case, the shooting game) and the most complex game (in this case, the simulation)
had the least appeal. In terms of learning, the adventure game was deemed best for
laying instructional foundations. While there were no differences in responses from male
and female subjects in this study, significant differences were found between racial
groups. Based on their results, Armory et al. suggest studying how one’s educational

20
background and socio-economic status might affect whether or not they have the requisite
psycho-social and cognitive skills for success within a particular game type. In addition,
they note that if learning goals are involved it might be important to ensure that learners
are adequately prepared for their interactions in the MMOG.
Sempsey and Johnston (2000) examined the social climate of 31 multi-user domains or
MUDs by comparing them to face-to-face (FTF) group dynamics. They found that MUDs
were more likely to foster self-expression, innovation, independence, and self-discovery.
In contrast, FTF groups scored higher in terms of order and organization as well as leader
control. No significant differences were found in terms of cohesion, leader support, task
orientation, and anger/aggression. This study suggests that online groups are perhaps
more likely to generate independent thinking and expression and less likely to be leader-
centered than FTF groups, although both are equally likely to band together and meet
their stated goals.
One concern that is sometimes expressed about entertainment-based gaming is the violent
nature of many games. Gaming stories often involve some type of war-based scenario or
require players to either defend themselves or attack others. The impact of these violent
interactions on the users’ demeanor is one area of concern. Of course, within a military
context this also is a real issue given the nature of training and resulting job tasks. In a
meta-analysis of 35 research reports involving more than 4,000 children and adults,
Anderson and Bushman (2001) found a positive and significant link between violent
video games and aggression. They concluded that violent videogames increase
physiological arousal and thoughts and feelings related to aggression or harming another
individual. They noted, however, that there is a need for more longitudinal research that
may help determine if repeated exposure to violent video games increased the long-term
nature of this aggression.
4.2 MMOGs and Online Community
Promoters of MMOG discuss the importance of persistent groups and worlds. It is
important to understand how persistent groups might facilitate or be facilitated by an
online community. There are many components to an online community (See Bonk,
Wisher, & Nigrelli, in press; Chao, 2001; Rovai, 2002; Schlager, Fusco, & Schank,
2002). Communities are collections of people who are bound together by some type of
goal or common purpose (Schwier, 1999). McLoughlin and Oliver (1999) contend that
in a community of learners, the participants know and value each other, discuss their
common concerns and experiences, lend support to each other, trust one another, share
aspects of control and responsibility in the learning environment, and feel safe taking
risks. Schwier (1999) points out that a learning community also needs:
(1) A leader to set the tone;
(2) Technology tools that facilitate task completion and interpersonal relationships
without getting in the way;
(3) A safe place to participate and communicate; and
(4) Opportunities for storytelling.

21
As indicated, there must be a purpose or reason for the initial activity or quest. The goal
or intent might be to problem solve, collaborate, share information, learn something new,
or accomplish some feat.
Learning communities have a number of common principles. Chao (2001) and Bonk et
al. (in press) note that online learning communities require membership, goals, purpose,
identity, shared knowledge, rituals, member participation or contributions, celebrations of
success or accomplishments, and, perhaps most importantly, trust. In a MMOG, such
communities might foster interaction and communication about the game as well as
resource sharing, negotiation and social construction of meaning, and expressions of
support and encouragement among players or students (Palloff & Pratt, 1999). Such an
electronic community, moreover, should have its own meeting or gathering place and
member roles as well as norms for resolving disputes.
The learning that takes place in a community-based environment, such as an educational
MMOG, needs not be focused solely on the content, tasks, and interactions embedded by
the game designers. Nevertheless, the tasks will likely drive the types of peer learning
opportunities that develop. When there are shared goals, idea negotiation, and rituals or
common practices, typically there are opportunities for members to work beside and learn
from more competent members (Barab & Duffy, 2000). This idea follows Lave and
Wenger’s (1991) concept of legitimate peripheral participation (LPP) in a community of
practice, and is consistent with Steinkuehler’s (2004) research findings, mentioned
earlier. New members of the community will likely look to more experienced members
for guidance, whether explicitly asking for help or more tacitly observing the actions of
those members.
In effect, when a community of practice and cognitive apprenticeship is formed, new
members have opportunities to observe and internalize some of the goals, practices, and
rituals of existing members. The expectation is that learners will both contribute to and
benefit from the community through their involvement. Over time, newcomers replace
the old timers (Brown, 2001), creating a cycle of life within the community.
Of course, the designers of these games and sellers of commercial goods related to it have
a vested interest in the forming of such communities. Their continuing existence is
dependent on certain roles being fulfilled and maintained. In fact, companies may pay
someone to act in certain community building or enhancing roles. Hence, the
development of a community of practice and associated apprenticeship process may not
be naturally occurring at all.
Aspects of community development are explored in some of the primary and secondary
research studies detailed in the following section. While the development of online
communities of learning is important for education, training, and performance support,
the above research review highlighted a number of additional gaps in existing MMOG
research that are directly or indirectly addressed in some of the studies suggested below.

22
5. Suggestions for Research
5.1 Fifteen Primary Experiments
O’Neil et al. (2002) point out that there is scant information about the impact of games on
adults. Given the state of educational research, Steinkuehler (2004) notes that it is ironic
that there is such a paucity of research on cognitive and cultural aspects of MMOGs. The
field is replete with publications and conferences related to virtual online communities
and online learning in general, and it has had a fascination for nearly two decades with
the impact of social interaction on learning. In addition to cognitive and sociocultural
questions and issues, there is a need for MMOG research in many other crucial areas
including the perceived authenticity of multiplayer online games, the processes and
principles one might learn or be exposed to when playing them, and the prior experiences
that might facilitate success. In an effort to illuminate areas that might be explored,
fifteen primary MMOG research studies are suggested below.
5.1.1 Research Study I. Impact of After Action Review (AAR) on MMOG
Performance and Decision Making Style
For a training game to be maximally effective there must be some type of after-action
review (AAR) (O’Neil et al., 2002). In essence, this AAR serves the purpose of
debriefing the MMOG experience, or connecting it to other training events. In this
proposed study, 100 individuals will play a MMOG for a period of three months. One
group will engage in extensive AARs after the three months and a second group will not.
The focus of the data collection will be on performance and decision-making style. After
the AAR, both the treatment and control groups will be divided into two groups, for a
total of four groups. Two groups (one receiving the AAR and one not receiving the AAR
treatment) will play the same MMOG and two groups will then play a different one for an
additional two-month period. Game performances (e.g., hits/misses, rounds expended,
and loss exchange ratios, etc.) across the four groups will be compared to determine if the
AAR experience enhanced the players’ decision-making processes (O’Neil et al., 2002).
Data also will be collected about any tactical lessons learned by the participants. When
combined, the tactical and the performance data should provide unique insights into the
player decision-making processes and results. Participants will also be given a decision-
making instrument to determine if there was any impact of the MMOGs and AARs on
decision-making style (e.g., analytical, intuitive, and regret-based emotional; Nygren,
2000). In an open-ended questionnaire, participants will be asked questions about how
they identified elements of the task, strategies that proved effective, and how the lessons
learned in the game might be applied to their particular job or within their organization
(Corbeil, 2003). There also will be individual interviews with six participants from each
group. Additionally, these participants will engage in focus group discussions about this
experience. The focus group discussion will emphasize the strategies that were used and
the results as well as what they might do differently next time.
Experiment: Explore impact of Impact of after action review (AAR) on MMOG
performance and decision making style

23
Independent variable: After-action review Dependent variable: Decision-making style
and performance and attitudes.
1. Group 1: AAR and same game
2. Group 2: AAR and different game
3. Group 3: No AAR and same game
4. Group 4: No AAR and different
game
• Participant attitudes
• Decision-making style
• Game performance
• Decision making tactics and strategies
Procedures:
• Record performance on game for all participants. Compare performance by treatment
group.
• Collect decision-making style information for all participants. Compare decision-
making style by treatment group.
• Survey of attitudes toward MMOG given to all participants.
• Collect information about decision making tactics and strategies with an open-ended
questionnaire.
• Six participants in each group will be interviewed.
• Six participants in each group will take part in focus group sessions.
• Online chat transcripts and bulletin board messages from game play will be analyzed.
Hypotheses:
1. Groups with AAR will perform better than group without AAR.
2. Group with AAR that continues on same game will perform highest.
3. Groups with AAR and who play a different game will exhibit the most change in
leadership style.
4. All groups will display a change in leadership style.
5. Groups with AAR will display more awareness of decision making tactics and
strategies.
6. Groups with AAR will have higher satisfaction than those without it.
5.1.2 Research Study II. Impact of Addiction to MMOG.
There are many reports about the potential addictive nature of computer games (Griffiths
& Davies, 2002). In fact, there are a host of social, psychological, and emotional
problems associated with overuse of the Internet (e.g., depression, deviant behaviors,
academic troubles, job burnout, over-involvement in online behaviors, unemployment,
etc.; Young, 2004). As the use of the Internet increases across aspects of life, such
addictions are bound to proliferate. In this study, different levels of MMOG will be
compared using an adapted version of the Diagnostic and Statistical Manual of Mental
Disorders (DSM-IV) addiction criteria for MMOG (Young, 2004). This study will
investigate whether over-involvement in MMOGs is related to problematic addictions.
For instance, does MMOG use negatively impact job performance? The questions below
are based on a list of Internet addiction items from Young (2004) (see
http://www.netaddiction.com/resources/internet_addiction_test.htm).

24
• Do you feel preoccupied with Massive Multiplayer Online Games (do you think
about previous gaming events and activities or anticipate the next online session)?
• Do you feel the need to use the Internet for MMOG with increasing amounts of
time in order to achieve satisfaction?
• Have you repeatedly made unsuccessful efforts to control, cut back, or stop
Internet use related to MMOG?
• Do you feel restless, moody, depressed, or irritable when attempting to cut down
or stop Internet use?
• Do you stay online longer than originally intended when playing MMOGs?
• Have you jeopardized or risked the loss of a significant relationship, job,
educational or career opportunity because of playing MMOGs?
• Have you lied to superiors, family members, therapists, or others to conceal the
extent of involvement in MMOGs?
• Do you use the MMOGs as a way of escaping from problems or of relieving a
dysphoric mood (for example, feelings of helplessness, guilt, anxiety,
depression)?
Additional questions might address the amount of time spent playing MMOGs compared
with other personal and professional activities (e.g., time with friends, volunteer work,
entertaining at home, club meetings, etc.).
Experiment: Investigate impact of addictive behaviors on time spent playing MMOGs
and job performance.
Independent variable: MMOG addictions. Dependent variable: Time spent in MMOG
environments, job performance.
1. Questionnaire related to MMOG
addictions. (Note: this study could
utilize regression or path analyses,
or it could be a correlational study).
• Job performance ratings by supervisor.
• Hours spent in MMOG per week.
Procedures:
• Complete a scale related to MMOG addictions.
• Collect self-report questionnaires related to time spent playing MMOG.
• Collect supervisor ratings of employee performance.
• Collect information relevant to signs of addictive behaviors (e.g., days called in sick,
depression, problem behaviors, criminal records, etc.).
Hypotheses:
1. Addiction to MMOGs will be negatively related to job performance.
2. Addiction to MMOGs will be positively related to hours spent playing online games.
3. Addiction to MMOGs will be highly correlated with escaping from problems,
irritability, moodiness, and feelings of helplessness.

25
5.1.3 Research Study III. Sense of Community and Group Longevity or
Persistence
Long-term associations with a group of online game players, or persistent groups, provide
a sense of belonging that Abraham Maslow emphasized as important to self-actualization.
Since social networks and bonding take place over extended periods of engagement, it is
important to determine the key attributes of successful groups. For example, do they
have features such as a purpose, strong leadership, gathering places and shared social
spaces, a unique vocabulary, member profiles, or a range of roles? Within the context of
MMOGs, what group-related factors motivate players to return? Do long term groups
have more of a sense of belonging than temporary groups (Sellers, 2002)? Do they
believe that they are part of something larger than themselves? Do they feel more of a
purpose or mission?
While research on online communities is increasingly more popular, there remain
numerous areas to investigate. How do the MMOG communities form and adjust over
time? What facilitates longevity of groups or guilds in MMOGs? This study will explore
online communities in several MMOGs. It will look at how communities form and are
maintained. Participants will complete a “sense of community” index or scale, which
will be correlated with group longevity.
Experiment: Does a sense of community lead to group longevity and sense of belonging?
Method
• Record group longevity.
• Collect data on sense of community and belonging to environment (use community
index or scale).
• Interview participants about sense of community.
Hypotheses:
1. Sense of community leads to greater game playing and group persistence.
Additionally, this study will generate descriptive data about the nature of and factors
influencing the development of a sense of community within MMOGs.
Follow-up:
Train one group in how to identify and foster a sense of community. Compare this group
with a no training control group.
5.1.4 Research Study IV. Describe the Apprenticeship Process in MMOG.
A multi-user online game might be an ideal environment in which to watch a situated
learning environment unfold over time (Chen et al., 1998). Here, the learner can watch
expert performance and internalize it prior to becoming an active member or legitimate
peripheral participant. In an MMOG, the newcomer has opportunities to observe as well
as actively participate within a community of practice (Wenger, 1998; Wenger,

26
McDermott, & Synder, 2002). The tools and tasks will provide one indicator of the
effectiveness of the apprenticeship process in the online community.
Following the research of Steinkuehler (2003), a cognitive ethnography method will be
employed in this study. A cognitive ethnography thickly describes different cultures
emerging in the MMOG environment. Through this thick description, a better
understanding of the apprenticeship process in MMOGs will emerge. Part of the research
will include a discourse analysis on relevant discussion, e-mail, and chat transcripts to see
how the players are sharing their knowledge and assisting each other as they develop new
skills and status within the game environment. And as detailed by Lave and Wenger
(1991), player movement on a learning trajectory from the periphery to inbound to insider
within the MMOG also will be analyzed. Additionally, players will be asked about how
different tools (e.g., public profiles, group chat, lobby, group Web logs or blogs,
collaborative knowledge bases, calendars, game Web sites) impacted on their
participation in the apprenticeship process.
Qualitative Study: Describe the apprenticeship process in a MMOG.
Methods
1. Observe one or more MMOGs over a series or weeks or months.
2. Collect transcripts of MMOG communications (e.g., chats, bulletin boards, and
blogs) and game logs.
3. Conduct discourse analyses.
4. Track player development and compare to player participation in online
communications.
5. Interview participants about the apprenticeship process and sense of community
felt in MMOG.
6. Interview any instructors, guides, instructional designers, or experts involved in
the apprenticeship process.
5.1.5 Research Study V. Game Authenticity and Constructivism
Constructivism emphasizes the active involvement of learners in real-life concepts or
experiences. Ideally, tools used within MMOGs that are based on constructivist
principles should foster engaged learning and deep processing of information, multiple
representations of reality, an appreciation of multiple perspectives or viewpoints,
collaborative construction of knowledge and social negotiation of meaning, and reflective
practice (Lainema & Makkonen, 2003). Additionally, these tools should provide
authentic experiences for learners as much as possible. In this study, MMOG instructors
and learners will be asked about their sense of game realism (authenticity) and fantasy.
In addition, they will complete a constructivist learning scale to assess the degree of
negotiation, collaboration, and active engagement evident in different MMOGs.
Achievement or performance, attitudes, and persistence will be compared to the sense of
realism and degree of constructivism learners perceive in different MMOGs.

27
Experiment: How Does Game Realism and Active Learning Impact Performance and
Satisfaction.
Independent variable: Sense of game
realism or fantasy; sense of active learning
or constructivism for different MMOGs.
(Note: might conduct regression or path
analyses or simple correlations.)
Dependent variable: Game performance,
attitudes, persistence, motivation.
1. Part I of Study: Complete sense of
realism-fantasy scale and sense of
constructivism or active learning
scale before and after playing
different MMOGs for a month.
2. Part II of Study: Assign new set of
participants to two groups: Group 1:
High sense of realism in MMOGs;
Group 2: Low sense of realism in
MMOGs.
3. Part III of Study: Assign
participants to two groups: Group 1:
High sense of constructivism in
MMOGs; Group 2: Low sense of
constructivism in MMOGs.
• Collect game performance scores.
• Administer game satisfaction
questionnaire.
• Interview players about what makes a
MMOG authenticity and fantasy.
• Record group longevity.
• Interview participants about sense of
community.
• Complete checklist of intrinsic and
extrinsic MMOG motivators.
Hypotheses:
1. Part 1: Sense of realism or authenticity related to higher satisfaction, group
persistence or longevity, and game performance.
2. Part 1: Sense of constructivism related to higher satisfaction, group persistence or
longevity, and game performance.
3. Part II: Higher realism increases game performance, attitudes, and persistence.
4. Part III: The more active the learning environment, the higher the performance,
satisfaction, and persistence.
5.1.6 Research Study VI. Bandwidth Constraints and Differences
In a computer network, latency refers to the length of time a message takes to move from
one designated node to another. Smed, Kaukoranta, and Hakonen (2002) note that the
variance in latency measures is known as “jitter,” and suggest that in MMOGs an
acceptable latency might range from 0.1 to 1.0 seconds. In effect, higher latencies might
be acceptable as long as there is minimal jitter. Participant acceptance of latency might
vary according to whether the activity is performance-based (i.e., first person shooter
games), strategic, or reflective. If participants are dissatisfied with the latency they are
experiencing, they may be less likely to play regularly or truly engage in a multiplayer
online game. This study, therefore, compares the impact of bandwidth variations on
overall game performance as well as player attitudes, persistence, and commitment.

28
Experiment: Impact of bandwidth and latency on online game performance.
Independent variable: Game bandwidth. Dependent variable: Game performance,
attitudes, game commitment or persistence,
motivation.
1. Collect information of game player
hardware platforms and bandwidth
or Internet connections.
• Collect game performance scores.
• Administer game satisfaction
questionnaire.
• Administer game persistence or
commitment instrument.
• Have players record impressions of
system latencies during different points
or levels of game playing.
• Collect computer log data (or use data
mining) to verify system latencies.
Hypotheses:
1. Longer latencies result in lower game satisfaction and game performance.
2. Longer latencies reduce participants’ degree of commitment and motivation.
5.1.7 Research Study VII. Role Assignment: Achievers, Socializers, Explorers,
Killers
There are many different player types in an MMOG environment. In a fairly simple
taxonomy, Bartle (1996) details four key roles in a MUD environment: achievers,
explorers, socializers, and killers. Some may seek high achievement in the system and
the status brought about by winning. According to Bartle, the focus on accumulating
points, wealth, or whatever the currency or goal of the game is a key marker of
individuals who might be labeled “achievers.” Others may be motivated to understand
the system or game well enough to be able to train others in it. They delight in finding
unusual or unknown places and interesting game features (e.g., bugs) and sharing that
knowledge. These players, known as explorers, might help with training manuals and
FAQs and usually will be highly valued as team members for their knowledge. Other
game players are often online simply to search for a social forum rich in personal or
professional relationships. These socializers might excel at mentoring those new to the
game, brokering connections, recruiting new team members, and holding groups together.
Bartle notes that someone in this role is interested in people and human relations and
tends to have a high level of empathy, good listening skills, and a ready ability to joke or
share humor, and generally entertain others. Finally, there are players who are dubbed
killers. They tend to cause other players grief or discomfort. They focus on acquiring
weapons of some type and using them on other players to cause death, havoc, or distress
within the game. Kim (1998) argued that it is fairly common to assume this role online

29
since it tends to be more exciting and interactive for players than other roles, even if
those interactions are negative in nature.
Role assignment has the potential to be an important factor in the educational use of
MMOGs. For example, it is possible that certain roles foster particular types of learning.
Perhaps players should be encouraged to assume roles that are most fitting with their job
requirements. Alternatively, they might be required to rotate through roles to experience
the MMOG from a variety of perspectives.
This study will have two groups. In group one, players will be assigned particular roles
(e.g., achievers, explorers, socializers, and killers). In group two, players will be allowed
to assume any role they want. After playing a MMOG for one to two months,
participants will be encouraged to change their role or perspective. The study will
explore how players assumed their initial roles, how easy it is for players to move among
these roles, and which roles promoted qualities such as leadership, management, and
vision. Additionally, data will be collected about participant performance and persistence
to help determine what effect, if any, roles had on their game play.
Experiment: How do different roles impact game performance, persistence, and decision-
making?
Independent variable: Role assignment. Dependent variable: Game performance,
attitudes, game commitment or persistence,
motivation.
1. Group 1: Assign to particular role
(i.e., socializer, achiever, killer, and
explorer).
2. Group 2: Allow to assume roles.
• Collect game performance scores.
• Administer game satisfaction
questionnaire.
• Administer game persistence or
commitment instrument.
• Collect decision-making style data.
• Interview participants about roles
assumed and roles they changed to.
Hypotheses:
1. Individuals allowed to assume their own roles will be more satisfied with their
performance and online gaming in general.
5.1.8 Research Study VIII. Cognitive Tools and MMOG Performance and
Dialogue
In addition to using post-game reflection, another way to build conceptual knowledge is
to engage in dialogue with peers or experts about the game during game play. Specific
cognitive tools (i.e., discussion forums, bulletin boards, debate tools, concept mapping
tools, surveys and polling tools, etc.) might be used to support MMOG play by mediating
social interaction and fostering depth of discussion. Tools might assist in collaborative
argumentation, questioning, articulation of ideas, idea critique, and elaboration of points
of view. Ravenscroft and Matheson (2002) found that, while there are a plethora of

30
contextual factors to consider, dialogue-based games can significantly improve student
conceptual understanding. In this study, students will be assigned to a control group
(game play only) or a treatment group (game play plus discussion) to determine the
impact of concurrent discussion with cognitive support tools.
Experiment: Can cognitive support tools enhance performance and strategic decision-
making within MMOG?
Independent variable: Supplemental
cognitive tools. Dependent variable: Game performance,
attitudes, depth of understanding,
motivation.
1. Treatment group utilizes
collaboration and conferencing
tools within MMOG.
2. Control group does not access such
tools.
• Collect game performance scores.
• Administer game satisfaction
questionnaire.
• Interview participants about their use of
supplemental cognitive tools.
• Participants complete strategic planning
documents for particular situations
similar to the online game they played.
Hypotheses:
1. Groups with cognitive support tools will outperform those without access to those
tools.
2. Strategic planning scores will be higher for groups which have cognitive tool
support.
5.1.9 Research Study IX. Collaboration and Virtual Teaming in MMOG: Co-
located and Distance Groups
During the last decade there has been intensive growth in collaborative technologies for
online learning as well as work (Bonk & Wisher, 2000). As a result, there is increasing
interest in how to establish and use virtual teams (ThoughtLink, 2002). One relevant area
of research is whether co-located and virtual teams develop and work differently. In this
study, one treatment group or guild will include group members who are co-located,
while the other group or guild will have members who are located at a distance from each
other. The study will explore how strategic plans and ideas are presented and negotiated
within these two types of groups. Relevant points of comparison will include how the
groups are coordinated; the types of communication and teaming structures that are used;
how norms and procedures are negotiated within the MMOG environment (Graham,
2002, 2003); the methods of conflict resolution; the use of communication tools; and
participant expectations and experiences regarding feedback and support.
Additionally, participant satisfaction with their team and perceptions of team success may
vary based on team type. Factors such as group cohesion or timely completion of tasks
may be viewed as indicators of a positive group experience. Interviews with team

31
members can indicate group member satisfaction. Online chats or discussions among
group members also might be used to help determine the effectiveness of the groups.
Experiment: What is the impact of team member location on group or guild performance
within MMOG?
Independent variable: Assignment to group
with co-located or distant team members. Dependent variable: Game performance,
attitudes, group cohesion and attrition,
motivation.
1. Assign half of the participants to
groups with members at different
locations around the globe.
2. Assign half of the participants to
groups with co-located members.
• Collect game performance scores.
• Administer game satisfaction
questionnaire.
• Give participants team effectiveness
survey or questionnaire.
• Analyze team negotiation processes.
Hypotheses:
1. Groups with co-located team members will engage in more social chat or
interaction.
2. Groups with co-located team members will make more assumptions and use
specific lingo and abbreviations.
5.1.10 Research Study X. Decision-making, Leadership, and Interpersonal Conflict
in MMOGs.
Research on group decision support systems (GDSS) has revealed that these systems
result in more time to make a decision, higher decision quality, better overall
performance, and a higher degree of satisfaction with the decision (Dasgupta, 2003).
Dasgupta points out that most GDSS studies have taken place in highly controlled
environments. With greater use of the Internet to support group functioning, there is a
need to replicate some of these studies in more dynamic and decentralized online
environments. Instead of being located in a computer lab, individuals and groups can be
located across different departments, organizations, countries, and continents. MMOGs
offer one such environment to explore the quality, confidence, and speed of online
decision-making.
As indicated earlier, some scholars are increasingly focusing on how simulations and
games can impact group as well as individual decision-making. Aspects of decision-
making that may be explored include the quality of the decision, the level of satisfaction
with the decision among group members, the confidence in the soundness or correctness
of a decision, and the extent of conflict during the decision-making process (Dasgupta,
Chanin, & Ioannidis, 2000). In terms of the latter, styles of handling interpersonal
conflict (e.g., compromising, obliging, dominating, avoiding, integrating, etc.) might be
compared to the quality of decisions or solutions. These styles also are important to help
understand the principles and practices of leadership within MMOGs. For example, how

32
is leadership decided and instantiated within these online games? How do different types
of leaders impact group cohesion and the effectiveness of decision making? How
successful is bottom-up and top-down leadership in MMOGs? What is the overall
quality of decision making and leadership displayed in predetermined versus emergent
groups? Are real-world leaders threatened by gaming scenarios wherein they might not
be the leader?
Qualitative study: How are ways of handling interpersonal conflict in MMOGs related to
the quality of decision-making and group cohesion?
Methods:
1. Collect game performance scores.
2. Administer game satisfaction questionnaire.
3. Observe ways of handling interpersonal conflict online.
4. Record decisions made in MMOGs and categorize decision making types.
5. Collect decision making style information.
6. Record group cohesion and longevity.
Hypotheses:
1. Groups with co-located team members will engage in more social chat or
interaction.
2. Groups with co-located team members will make more assumptions and use
specific lingo and abbreviations.
5.1.11 Research Study XI. Learning from Mistakes and Learning Histories.
Computer log tools and other devices provide a history of learner progress and decision-
making that can be used for review and reflection. Session histories might be replayed or
evaluated immediately after they are recorded or after a significant delay (Rose et al.,
2000). By encouraging learner review, session histories can enhance learner’s
metacognitive skills such as reflection. In this study, half of the participants will review
their histories with a mentor as well as peers, whereas half will reflect individually on
their session histories. Post performance on a series of problem solving tasks will be
compared across groups.
Experiment: Can review of session histories impact on metacognitive skills.
Independent variable: Type of session
histories (mentor/peer and individual). Dependent variable: Game performance,
attitudes, metacognitive skills, strategic
decision making.
1. Group 1: Mentors and peer review
of session history.
2. Group 2: Self-reflection on session
history.
• Collect game performance scores.
• Administer game satisfaction
questionnaire.
• Give participants survey on satisfaction
with game environment.
• Metacognitive awareness of tactical

33
decisions.
• Solve similar scenarios or problems.
Hypotheses:
1. Group with peer and mentor reviews will score better than self-review group.
2. Group with peer and mentor reviews will display greater metacognitive skill and
strategic decision making skill.
5.1.12 Research Study XII. Learning Style and Game Selection.
Learners have different individual learning preferences. For instance, some are more
sensory and others more intuitive; some are more global and others more analytical or
detail-oriented; some prefer hands-on and active learning experiences whereas others
enjoy more passive, receptive learning; and some students might learn best from visual
representations whereas others will strictly want verbal materials. Different types of
games may address preferences for different learning styles or target different ways of
knowing the material (Hill, Ray, Blair, & Carver, 2003). In this study, students will be
administered the Index of Learning Styles questionnaire (Felder & Solomon, 2004).
They will then be allowed to select from 4-8 different online games that had been
previously categorized as high on text, visuals, hands-on experiences, or reflective
exercises. Their choices will be compared to their learning style preferences as well as
their game performance and attitudes.
Experiment: Will participants select MMOGs that match their preferred learning styles?
Independent variable: Learning styles. Dependent variable: Game selection, game
performance, participant satisfaction with
game environment.
1. Administer Kolb’s learning style
inventory; sort participants into
different learning style preferences.
• Allow participants to choose game.
• Collect game performance scores.
• Administer game satisfaction
questionnaire.
• Give participants survey or
questionnaire regarding satisfaction
with game selection.
Hypotheses:
1. Participants will select online games that most closely match their preferred
learning styles.
2. Participants that select a game the matches their learning style will perform better
and be more satisfied than those who do not.

34
5.1.13 Research Study XIII. Game-based Motivation.
Intrinsic motivators (e.g., curiosity, fantasy, and challenge) might be more applicable to
single-user games, whereas multi-player games add extrinsic motivators such as
recognition, competition, and collaboration. A study might be conducted wherein
participants complete a Likert scale rating of statements for the degree of engagement or
the most interesting factors related to their MMOG (Chen et al., 1998). For example,
what are the social and psychological factors (e.g., challenge, fantasy, control, need for
achievement, curiosity, identity, fulfillment, goal setting, sense of persistence,
collaboration and information exchange, social networking, and variety) as well as the
technological factors (e.g., real time chats, custom avatars, animations, and audio) that
engage them in a MMOG experience? They might also rate their agreement with
different issues related to their attitudes and activities in the MMOG in a pre-post
fashion. Motivational factors might be compared for high and low achieving students, per
Chen et al. (1998). Other indicators of motivation include time spent in the system and
completion rates or advancement to higher game levels (Siemer & Angelides, 1995).
Experimental study: What is motivational about MMOGs?
Methods:
1. Administer motivational factor checklist.
2. Engage in MMOG activity for set period of time.
3. Readminister motivational checklist.
4. Compare high and low achieving players on motivational checklist.
5. Compare computer log information to motivational indices.
Hypotheses:
1. Students who play games for challenge and a high need for achievement will
perform better on MMOGs than those who play primarily for social networking
needs.
2. Low achievers will have more social needs met by MMOG environments than
high achievers.
3. Social and psychological motivational factors will be more related to performance
than technological factors.

35
5.1.14 Research Study XIV. Problem Solving Processes and Types of Knowledge
Facilitated by MMOGs.
Researchers have been curious about whether computer game play improves problem
solving skills. Problem solving, in fact, is often viewed as the main reason to use
simulations and games (Leemkuil et al., 2000). According to Mayer and Wittrock
(1996), problem solving is the “cognitive processing directed achieving a goal when no
solution method is obvious to the problem solver” (p. 47). Amory, Naicker, Vincent, and
Adams (1999) had college age students play four different commercial games and then
asked them to assess the importance of different skills (e.g., logic, memory, mathematics,
visualization, reflexes, and problem solving) in playing each game. Not surprisingly,
problem solving was a skill consistently mentioned by the subjects.
A related issue to problem solving is to understand the types of knowledge that are
facilitated by MMOG. In a study of adventure gaming with college students, Ju and
Wagner (1997) found factual knowledge was more easily retrieved than planning skills
and negative knowledge. It remains unknown whether MMOGs foster cause and effect
knowledge, strategic planning, conceptual learning, or just basic facts about a particular
game. Also, do MMOG players retain more procedural or declarative knowledge? And
will different types of MMOGs foster different types of knowledge? In this study,
subjects will be asked different levels or types of questions, “what is the name of…,”
“how would you conduct…,” “what happens if…,” and “how would you get to…”? Of
interest, is the type of knowledge most remembered—strategic or declarative.
Additionally, learner performance in the MMOG environment will be compared to the
types of knowledge remembered.
Experiment: What type of knowledge do MMOG participants focus on?
Methods:
1. Play 3-4 different MMOGs for a set time period (e.g., 1-2 weeks of 1-2 months).
Some games may be more social and others more tactical.
2. Both one week and one month after completion, ask participants procedural,
declarative, technological, and strategic or tactical questions related to each game.
3. Compare knowledge retention to game performance.
Hypotheses:
1. Participants who retain more strategic knowledge will have higher game
performance scores.

36
5.1.15 Research Study XV. Social Skills and Friendship Development.
Online shared spaces are important to group functioning and performance (Schrage,
1990). Socialization during MMOG may take many forms. In terms of social spaces,
players may congregate in launching pads, pit stops, or staging areas (Koster, 2004).
They may also use these spaces need to recover after the adventure. It is vital to
understand how activity is spawned and accelerates in these social spaces as well as what
transpires cognitively, socially, and emotionally in order to better support group
communication and performance.
Orvis, Wisher, Bonk, and Olson (2001) found that social activity was present in about 30
percent of synchronous training. In the context of MMOGs, the average time spent
socializing in the environment is an open research question, along with the participants’
reasons for socializing (e.g., to explain rules, provide strategic advice, or offer support).
Not all game players may value or use the social options. To begin learning about these
issues, in this study, learners will be observed as they engage in game play and negotiate
ideas and their social situation. They will then be interviewed about their preferences for
different types of online social spaces when playing in MMOGs. The results might help
improve the design and availability of social spaces and tools valued by players during
MMOG play as well as enhance their productivity and the overall outcomes of MMOGs.
Experiment: What social activities and support tools are favored when playing MMOGs?
Methods
1. Observe activities and tools favored by online gaming participants.
2. Interview players about their preferred social outlets. What are their favored
social activities and tools when playing MMOGs? When do they use them?
How might they redesign online games for their social preferences?
5.2 MMOG Assessment and Evaluation
In addition to the primary and secondary research studies mapped out above, it is vital to
explore the effectiveness of MMOGs using different assessment and evaluation
techniques. Bonk and Wisher (in press) discuss a range of evaluation methods within
online environments. Of course, MMOG stakeholders will each have different
assessment and evaluation criteria and needs. As O’Neil et al. (2002) argue, the four
levels of Kirkpatrick’s (1998, 2001) model of training evaluation might help analyze the
effectiveness of the MMOG environments. Even though this model was developed
decades prior to the emergence of the Internet, it helps explain why the evaluation of e-
learning is important (Bonk & Wisher, in press).
The first level of the Kirkpatrick model would ask participants questions about their
reactions to the game. Evaluation questions would relate to how to improve the MMOG
in terms of participant satisfaction. Also, participants might be asked about what they
liked most, found most difficult and what obstacles they faced. At the second level, the
evaluation focus shifts to learning – specifically, participant knowledge, attitudes, and

37
skills that were gained or refined as a result of the MMOG. At the third levels, the
evaluation might directly explore what happens when trainees return to their jobs or start
a new one after the training game. The concern here is with the transfer of knowledge,
skills, and attitudes to the real world. Such information might be revealed through direct
observations, supervisor feedback, and peer evaluations. Finally, at the fourth level of
the Kirkpatrick model, the focus might shift to the impact of MMOG on enhanced unit
readiness or productivity as well as reductions in costs or accidents. Given these four
levels of evaluation, it is clear that there are numerous measures and approaches that
might be drawn upon to address particular stakeholder needs and interests (Bonk &
Wisher, in press). Thus, the Kirkpatrick model would provide a useful way of ensuring
that MMOGs are thoroughly evaluated for both immediate and long-term impact on the
organization.
6. Future Directions
While this is not a report on the market potential of MMOG, research is currently needed
on a wide assortment of variables that could, in turn, impact on products and innovations
that vendors offer. Lack of entrepreneurship and innovation in the online gaming
industry will likely never be the problem. Costs will continue to plummet, innovative
features will be added, and processing speed will be significantly increased. The
problems, instead, will arise in how to use them effectively, and, in conjunction,
determining the most relevant and important research questions and avenues to pursue.
The research questions raised in this report relate primarily to higher level skills and
cognitive processes as well as issues of identity and social affiliation. Among the more
salient issues were those related to sense of community, group persistence, intrinsic and
extrinsic motivational factors, player apprenticeship, and the impact of playing MMOGs
on decision making, problem solving, and leadership.
During the coming decade, online games and simulations will become increasingly
interactive and sophisticated. Advances in pedagogical agent technology will provide
additional layers of guidance, challenges, and control within these systems. According to
Dessouky et al. (2001), “the primary role for agents will be to recognize student learning
opportunities and ensure that students recognize them as well” (p. 179). As agents and
artificial intelligence as well as additional customization and options are added to the
MMOG experience, there will be increasing avenues for education, training, and
performance support within these environments. If some of the research recommended in
this report is conducted during the next few years, it should provide some answers and
directions on where and how to take advantage of the next evolution of MMOG
technology.

38
7. References
Alberts, A., Gartska, J., & Stein, F. (1999). Network centric warfare: Developing and
leveraging information superiority (pp. 5-6). Washington, D.C.: C4ISR
Cooperative Research Program, Department of Defense, 1999.
Aldrich, C. (2002). Field guide to educational simulations. Learning Circuits, American
Society for Training and Development. Retrieved April 11, 2004, from
http://www.learningcircuits.org/NR/rdonlyres/3FBFC2B6-4517-4E35-A9BE-
E34EE719D134/1353/Aldrich.pdf
Aldrich, C. (2004). Simulations and the future of learning: An innovative (and perhaps
revolutionary) approach to e-learning. San Francisco, CA: Pfeiffer Publishing.
Anderson, C. A., & Bushman, B. J. (2001, September). Effects of violent video games on
aggressive behavior, aggressive cognition, physiological arousal, and prosocial
behavior: A meta-analytic review of the scientific literature. Psychological
Science, 12(5), 353-359.
Amory, A., Naicker, K., Vincent, J., & Adams, C. (1999). The use of computer games as
an educational tool: Identification of appropriate game types and game elements.
British Journal of Educational Technology, 30 (4), 311-321.
Asakawa, T., & Gilbert, N. (2003). Synthesizing experiences: Lessons to be learned in
Internet-mediated simulation games. Simulation & Games, 34(1), 10-22.
Barab, S. A., & Duffy, T. (2000). From practice fields to communities of practice. In D.
Jonassen, & S. M. Land. (Eds.). Theoretical foundations of learning environments
(pp. 25-56). Mahwah, NJ: Lawrence Erlbaum Associates.
Bangert-Drowns, R. L., Kulik, J. A., & Kulik, C. L. (1985). Effectiveness of computer-
based education in secondary schools. Journal of Computer-Based Instruction,
12(3), 59-68.
Baron, J. (1999, November). Glory and shame: Powerful psychology in multiplayer
games. Proceedings of the Game Developers Conference, San Francisco, CA.
Gamasutra. Retrieved November 10, 1999, from
http://gamasutra.com/features/19991110/Baron_01.htm.
Bartle, R. A. (1996, June). Hearts, clubs, diamonds, spades: Players who suit MUDs.
Journal of MUD Research, 1(1). Retrieved May 10, 2004, from
http://www.mud.co.uk/richard/hcds.htm.
Bartle, R. (2003). Designing virtual worlds. Indianapolis, IN: New Riders Press.

39
Beal, S. A., & Christ, R. E. (2004). Training effectiveness evaluation in the Full Spectrum
Command game (Technical Report No.1140). Alexandria, VA: U.S. Army
Research Institute for the Behavioral and Social Sciences.
Belanich, J., Sibley, D. E., & Orvis, K. L. (2004). Instructional characteristics and
motivational features of a PC-based game (Research Report No. 1822).
Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social
Sciences.
Bonk, C. J., & Wisher, R. A. (2000). Applying collaborative and e-learning tools to
military distance learning: A research framework (Technical Report No. 1107).
Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social
Sciences. Retrieved May 20, from
http://www.publicationshare.com/docs/Dist.Learn(Wisher).pdf.
Bonk, C. J., & Wisher, R. A. (in press). Getting a sense of e-learning evaluation methods.
To appear in S. Reddy (Ed.). Training and Development - Vol. 2: Role of
Technology in Training. Hyderabad, India: ICFAI Press.
Bonk, C. J., Wisher, R. A., & Nigrelli, M. L. (in press). Learning communities,
communities of practice: Principles, technologies, and examples. To appear in K.
Littleton, D. Faulkner, & D. Miell (Eds.), Learning to collaborate, collaborating
to learn. City, State: NOVA Science.
Borland, J., & Kanellos, M. (2004, July 28). South Korea leads the way. CNET
News.com, Retrieved October 1, 2004, from
http://news.com.com/South+Korea+leads+the+way/2009-1034_3-
5261393.html?tag=nl
Brown, R. (2001). The process of community-building in distance learning classes.
Journal of Asynchronous Learning Networks, 5(2). Retrieved December 24, 2002,
from http://www.aln.org/alnweb/journal/Vol5_issue2/Brown/Brown.htm.
Chambers, S. K., Haselhuhn, C., Andre, T., Mayberry, C., Wellington, S., Krafka, S.,
Volmer, J., & Berger, J. (1994). The acquisition of a scientific understanding of
electricity: Hands-on versus computer simulation experience; conceptual change
versus didactic text. Paper presented at the Annual Meeting of the American
Educational Research Association, New Orleans, LA.
Chao, C. (2001). Toward an understanding of sense of community and meaningful
learning experiences in an on-line language education course. Unpublished
doctoral dissertation, University of Indiana, Bloomington.
Chen, G. D., Shen, G. Y., Ou, K. L., & Liu, B. J. (1998). Promoting motivation and
eliminating disorientation for Web based courses by a multi-user game.
Proceedings of the Ed Media/Ed Telecom 99 World Conference on Educational

40
Multimedia and Hypermedia and World Conference on Educational
Telecommunications, Frieberg, Germany.
Chinn, C. A., & Brewer, W. F. (1993). The role of anomalous data in knowledge
acquisition: A theoretical framework and implications for science instruction.
Review of Educational Research, 63, 1-49.
Corbeil, P. (2003). IN THE HIVE: A game on internal communications inspired by
nature. Simulation & Gaming, 34(1), 164-166.
Dasgupta, S. (2003). The role of controlled and dynamic process environments in group
decision making: An exploratory study. Simulation & Gaming, 34(1), 54-68.
Dasgupta, S., Chanin, M., & Ioannidis, A. (2000). Group decision making using
knowledge-based systems—An experimental study. Simulations and Gaming: An
Interdisciplinary Journal of Theory, Practice, and Research, 31(4), 536-544.
Davis, J. (2003, September 8). Soldiers would play against each other. Inside the Army,
Volume, 1.
Defense Advanced Research Projects Agency, Defense Sciences Office (2004). Website:
Training superiority (DARWARS). Retrieved May 2, 2004, from
http://www.darpa.mil/dso/thrust/biosci/training_super.htm
Department of Defense (2004, June 9). Department of Defense training transformation
implementation plan. Washington, DC: The Pentagon, Office of the Under
Secretary for Personnel and Readiness.
de Jong, T., & van Joolingen, W. R. (1998). Scientific discovery learning with computer
simulations of conceptual domains. Review of Educational Research, 68(2), 179-
201.
Delaney, K. (2003, November 3). Are videogames ready to be taken seriously by media
reviewers? The Wall Street Journal, p. B1. Retrieved May 20, 2004, from
http://www.gamegirladvance.com/archives/2003/11/03/the_wsj_gets_it.html
Dessouky, M. M., Verma, S., Bailey, D. E., & Rickel, J. (2001). A methodology for
developing a web-based factory simulator for manufacturing education. IIE
Transactions, 33, 167-180.
DiMascio, J. (2004, April 12). Ft. Benning soldiers among the first to sample commercial
X-box game. Inside the Army, p. 1.
Downes, S. (2004, March 15). Learning by doing: James Paul Gee at RIMA ICEF.
Stephen’s Web. Retrieved March 29, 2004, from http://www.downes.ca/cgi-
bin/website/view.cgi?dbs=Article&key=1079385148

41
Felder, R. M., & Solomon, B. A. (2004). Index of Learning Styles Questionnaire.
Retrieved Mau 31, 2004, from North Carolina State University Web site:
http://www.engr.ncsu.edu/learningstyles/ilsweb.html
Gee, J. P. (2003a, June 20). From video games, learning about learning. The Chronicle of
Higher Education, pp. B13. Retrieved April 10, 2004, from
http://chronicle.com/prm/weekly/v49/i41/41b01301.htm.
Gee, J. P. (2003b). What video games have to teach us about learning and literacy.
Palgrave, NY: MacMillan.
Gokhale, A. A. (1996). Effectiveness of computer simulation for enhancing higher
order thinking. Journal of Industrial Teacher Education, 33(4), 36-46.
Graham, C. R. (2002). Understanding and facilitating computer mediated teamwork: A
study of how norms develop in online learning teams. Unpublished doctoral
dissertation, Indiana University at Bloomington, IN. Retrieved June 3, 2004, from
http://www.byu.edu/ipt/faculty/documents/charles_graham_dissertation.doc
Graham, C. R. (2003). A model of norm development for computer-mediated teamwork.
Small Group Research, 34(3), 322-352.
Gredler, M. E. (1996). Educational games and simulations: A technology in search of a
(research) paradigm. In D.H. Jonassen (Ed.), Handbook of Research for
Educational Communications and Technology (pp. 521-540). New York, NY:
Simon & Schuster Macmillan.
Green, C. S., & Bavelier, D. (2003). Action video game modifies visual selective
attention. Nature, 423, 534-537.
Griffiths, M. D., & Davies, M. N. O. (2002). Excessive online computer gaming:
Implications for education. Journal of Computer Assisted Learning, 18, 379-380.
Harmon, A. (2003, April). More than just a game, but how close to reality? New York
Times. Retrieved May 2, 2004, from
http://www.peostri.army.mil/PAO/pressrelease/moreThan.jsp.
Herz, J. C., & Macedonia, M. R. (2002, April). Computer games and the military: Two
views. Defense Horizons, 11. Retrieved May 3, 2004, from
http://www.ndu.edu/inss/DefHor/DH11/DH11.htm.
Hill, J. D. J., Ray, C. K., Blair, J. R. S., & Carver, C. A. (2003, February). Puzzles and
games: Addressing different learning styles in teaching operating systems
concepts. Proceedings of the 34th SIGCSE technical symposium on computer
science education, Reno, Nevada, 182-186. Retrieved May 11, 2004, from

42
http://delivery.acm.org/10.1145/620000/611964/p182hill.pdf?key1=611964&key
2=6391334801&coll=portal&dl=ACM&CFID=21402756&CFTOKEN=6472580
6
Jayakanthan, R. (2002). Application of computer games in the field of education. The
Electronic Library, 20(2), 98-102.
Johnson, A. (2003, September 21). The learning game--Researchers study video gaming
principles that apply to education. Wisconsin Technology Network. Retrieved May
2, 2004, from http://www.wistechnology.com/article.php?id=243.
Jones, S. (2003). Let the games begin: Gaming technology and edutainment among
college students. Pew Internet and American Life Project, Washington, DC.
Retrieved April 10, 2004, from
http://www.pewinternet.org/reports/toc.asp?Report=93.
de Jong, T., & van Joolingen, W. R. (1998). Scientific discovery learning with computer
simulations of conceptual domains. Review of Educational Research, 68, 179-
202.
Ju, E., & Wagner, C. (1997). Personal computer adventure games: Their structure,
principles, and applicability for training. The DATA BASE for Advances in
Information Systems, 28(2), 78-92.
Kim, A. J. (1998, May). Killers have more fun. Wired News. Retrieved May 14, 2004,
from http://hotwired.wired.com/collections/gaming/6.05_killers_fun1.html
King, B. (2003). Educators turn to games for help. Wired News. Retrieved April 10,
2004, from http://www.wired.com/news/games/0,2101,59855,00.html
Kirkpatrick, D.L (1998). Evaluating training programs: The four levels (2nd ed.). San
Francisco, CA: Berret-Koehler.
Kirkpatrick, D.L (2001). Evaluating training programs: Inside the four levels.
Proceedings of the 17th annual Training Director’s Forum (pp. 545-572), Las Vegas,
NV.
Kirriemuir, J. (2002, February). Video gaming, education and digital learning
technologies. D-Lib, 8(2). Retrieved April 10, 2004, from
http://www.dlib.org/dlib/february02/kirriemuir/02kirriemuir.html
Knight, W. (2003, April). Million-user online game previewed. New.scientist.com.
Retrieved April 7, 2004, from http://www.newscientist.com
Kolbert, E. (2001, May 28). Pimps and dragons. The New Yorker.

43
Koster, R. (2004). On socialization and convenience. Retrieved May 9, 2004, from
http://www.legendmud.org/raph/gaming/socialization.html
Lainema, T., & Makkonen, P. (2003, March). Applying constructivist approach to
educational business games: Case REALGAME. Simulations and Gaming: An
Interdisciplinary Journal of Theory, Practice, and Research, 34(1), 131-149.
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation.
New York, NY: Cambridge University Press.
Learning and Training Innovations (2003, September 18). Bersin offers guide to e-
learning application simulation tools. News. Retrieved May 21, 2004, from
http://www.ltimagazine.com/ltimagazine/article/articleDetail.jsp?id=70007
Leemkuil, H., de Jong, T., & Ootes, S. (2000). Review of educational use of games and
simulations. EC project KITS (Knowledge management Interactive Training
System). EC project KITS (IST-1999-13078), KITS Deliverable D1, Enschede:
KITS consortium. University of Twente, The Netherlands. Retrieved May 14,
2004, from http://kits.edte.utwente.nl/documents/D1.pdf.
Lenhart, A., Rainie, L., & Lewis, O. (2001). Teenage life online. Pew Internet and
American Life Project, Washington, DC. Retrieved April 10, 2004, from
http://www.pewinternet.org/reports/toc.asp?Report=36.
Littman, D., & Soloway, E. (1988). Evaluating ITSs: The cognitive science perspective.
In M. C. Polson & J. J. Richardson (Eds.), Foundations of Intelligent Tutoring
Systems. Mahwah, NJ: Lawrence Erlbaum.
Malone, T. (1980). What makes things fun to learn: Heuristics for designing instructional
computer games. Proceedings of the 3rd ACM SIGSMALL symposium and the
first SIGPC symposium on Small systems, 162-169.
Malone, T. (1981). Toward a theory of intrinsically motivating instruction. Cognitive
Science, 4, 333-369.
Malone, T. W., & Lepper, M. R. (1987). Making learning fun: A taxonomy of intrinsic
motivations for learning. In R. E. Snow and M. J. Farr (Eds.), Aptitude, learning,
and instruction. Hillsdale, NJ: Lawrence Erlbaum Associates.
Mayer, R.E., & Wittrock, M.C. (1996). Problem-solving transfer. In D.C. Berliner &
R.C. Calfee (Eds.), Handbook of educational psychology (pp. 47-62). New York:
Macmillan.
McLoughlin, C., & Oliver, R. (1999). Pedagogical roles and dynamics in telematics
environments. In M. Selinger & J. Pearson (Eds.), Telematics in education:
Trends and issues. Amsterdam, The Netherlands: Pergamon.

44
Morrison, J. L., & Aldrich, C. (2003, September/October). Simulations and the learning
revolution: An interview with Clark Aldrich. The Technology Source. Retrieved
April 11, 2004, from http://ts.mivu.org/default.asp?show=article&id=2032
The National Research Council Computer Science & Telecommunications Board (1997).
Modeling & Simulation: Linking Entertainment & Defense. Washington, D.C.:
National Academy Press.
Nygren, T. E. (2000, November). Development of a measure of decision making styles to
predict performance in a dynamic judgment/decision making task. Paper
presented at the 41st Annual Meeting of the Psychonomics Society, New Orleans,
LA.
Oblinger, D. (2003, July/August). Boomers, gen-xers, and millennials: Understanding the
new students. Educause, 38(4), 37-47. Retrieved April 9, 2004, from
http://www.educause.edu/ir/library/pdf/erm0342.pdf.
O’Neil, H., Baker, E. L., & Fisher, J. (2002). A formative evaluation of ICT games. Los
Angeles, CA: The University of Southern California.
Orvis, K. L., Wisher, R. A., Bonk, C. J., & Olson, T. (2002). Communication patterns
during synchronous Web-based military training in problem solving. Computers
in Human Behavior, 18(6), 783-795.
Palloff, R. M., & Pratt, K. (1999). Building learning communities in cyberspace: Effective
strategies for the online classroom. San Francisco, CA: Jossey-Bass.
Peck, M. (2004, January). DARPA sketches futuristic ‘virtual schoolhouse.’ National
Defense Magazine. Retrieved May 2, 2004 from
http://www.nationaldefensemagazine.org/article.cfm?Id=1312.
Pintrich, P. R., & Schunk, D. H. (1996). Motivation in education: Theory, research, and
applications. Englewood Cliffs, NJ: Merrill.
Prensky, M. (2001). Digital game-based learning. NewYork, NY: McGraw-Hill.
Raduege, H. D. (2004). Net-centric warfare is changing the battlefield environment.
CrossTalk: The Journal of Defense Software Engineering, 17(1), 7-9.
Ravenscroft, A., & Matheson, M. P. (2002). Developing and evaluating dialogue games
for collaborative e-learning. Journal of Computer Assisted Learning, 18, 93-101.
Raybourn, E. M. (1998). Designing DomeCityMOO training collaboratory: A multi-user
simulation in a text-based networked virtual environment that supports non-

45
scripted interactions toward intercultural understanding. SIGGROUP Bulletin,
19(3), 35-37.
Reeve, J. M. (1996). Motivating others: Nurturing inner motivational resources.
Needham Heights, MA; Allyn & Bacon.
Rickard, W., & Oblinger, D. (2004). Higher education leaders symposium: Unlocking the
potential of gaming technology. Redmond, WA: Microsoft Corporation. Retrieved
April 11, 2004, from
http://www7.nationalacademies.org/itru/Gaming%20Technology.pdf.
Riddell, R. (1997, April). Doom goes to war. Wired Magazine. Retrieved September 27,
2004, from
http://hotwired.wired.com/collections/future_of_war/5.04_doom_pr.html
Rose, A., Salter, R., Keswani, S., Kositsyna, N., Plaisant, C., Rubloff, G., &
Shneiderman, B. (2000, April). Simulation based learning environments and the
use of learning histories. Paper presented at the CHI Conference, The Hague, The
Netherlands.
Rovai, A. (2002). Building a sense of community at a distance. International Journal of
Open and Distance Learning, 3(1). Retrieved December 24, 2002, from
http://www.irrodl.org/content/v3.1/rovai.html
Sawyer, B. (2002). Serious games: Improving policy through game-based learning and
simulation. Foresight and governance project. Washington, DC.: Woodrow
Wilson International Center for Scholars, Publication 2002-1. Retrieved May 12,
2004, from http://wwics.si.edu/topics/docs/ACF3F.pdf
Schatz, A. (2001, June 21). The onset of anarchy. The Lycos 50 daily report. Retrieved
April 10, 2004, from http://50.lycos.com/062101.html
Schlager, M. S., Fusco, J., & Schank, P. (2002). Evolution of an on-line education
community of practice. K. A. Renniger & W. Shumar (Eds.), Building virtual
communities: Learning and change in cyberspace. New York, NY: Cambridge
University Press.
Schettler, J. (2003, February). Weapons of mass instruction. Training Magazine, 40(2),
20-22, 24-26.
Schiesel, S. (2003, June 12). Voyager to a strange planet. The New York Times.
Retrieved June 13, 2003, from
http://query.nytimes.com/gst/abstract.html?res=F10F1EFE355D0C718DDDAF08
94DB404482

46
Schrage, M. (1990). Shared minds: The technologies of collaboration. New York, NY:
Random House.
Schwier, R.A. (1999, June). Turning learning environments into learning communities:
Expanding the notion of interaction in multimedia. Proceedings of the World
Conference on Educational Multimedia, Hypermedia and Telecommunications,
Seattle, Washington: Association for the Advancement of Computers in
Education.
Seay, A. F., Jerome, W. J., Lee, K. S., & Kraut, R. E. (2004). Project massive: A study of
online gaming communities. Extended abstracts of the Conference on Human
Factors in Computing Systems, pp. 1421-1424, Vienna, Austria. Retrieved May
14, 2004, from http://delivery.acm.org/10.1145/990000/986080/p1421-
seay.pdf?key1=986080&key2=4316454801&coll=GUIDE&dl=ACM&CFID=21
537523&CFTOKEN=81438270.
Sellers, M. (2002, September 16). Creating effective groups and group roles in MMP
games. Gamasutra. Retrieved June 10, 2003, from
http://www.gamasutra.com/resource_guide/ 20020916/sellers_01.htm.
Sempsey, J. J., & Johnston, D. A. (2000). The psychological dynamics and social climate
of text-based virtual reality. Journal of Virtual Environments, 5(1).
Shwayder, R. (2004, March 26). The MMOG battle: EQ vs FFXI. GameWatchers.net.
Retrieved May 14, 2004, from
http://www.gamewatchers.net/displayarticle.php?article=109.
Siemer, J. (1995). Evaluating intelligent tutoring with gaming-simulations. Paper
presented at the Winter Simulation Conference.
Siemer, J., & Angelides, M. C. (1994). Embedding an intelligent tutoring system in a
business gaming- simulation environment. Proceedings of the Winter Simulation
Conference (pp. 1399-1406), Orlando, Florida. Retrieved May 31, 2004, from
http://delivery.acm.org/10.1145/200000/194914/p1399-
siemer.pdf?key1=194914&key2=4212606801&coll=portal&dl=ACM&CFID=21
954772&CFTOKEN=56354256.
Smed, J., Kaukoranta, T., & Hakonen, H. (2002). Aspects of networking in multiplayer
computer games. The Electronic Library, 20(2), 87-97.
Snider, M. (2002, May 23). Where movies end, games begin. USA Today. Retrieved May
14, 2002, from http://www.usatoday.com/tech/techreviews/2002/5/23/e3.htm.
Snider, M. (2004, May 10). Hollywood at the controls: Video games feed off each other.
USA Today, 1D-2D.

47
Squire, K. D., & Steinkuehler, C. A. (in press). Generating cyberculture/s: The case of
Star Wars Galaxies. To appear in A. Park (Ed.), Cyberlines: Languages and
cultures of the Internet (2nd ed.). Albert Park, Australia: James Nicholas
Publishers.
Steinkuehler, C. A. (2003). Massively multiplayer online videogames as a constellation
of literacy practices. Paper presented at the International Conference on Literacy,
Ghent, Belgium. Retrieved May 13, 2004, from
http://www.sit.wisc.edu/~steinkuehler/papers/SteinkuehlerAILA2003.pdf.
Steinkuehler, C. A. (2004). Learning in massively multiplayer online games. Paper to be
presented at the International Conference of the Learning Sciences (ICLS), Los
Angeles, CA. Retrieved May 13, 2004, from
http://www.sit.wisc.edu/~steinkuehler/papers/SteinkuehlerICLS2004.pdf.
Stevenson, R. (2003, May). U.S. Army creates video game to train soldiers.
Technology—Reuters. Yahoo! News. Retrieved May 15, 2003.
Stipek, D. J. (1998). Motivation to learn: From theory to practice (3rd ed.). Boston, MA:
Allyn & Bacon.
Tharp, R. (1993). Institutional and social context of educational reform: Practice and
reform. In E. A. Forman, N. Minick, & C. A. Stone (Eds.), Contexts for learning:
Sociocultural dynamics in children's development. New York, NY: Oxford
University Press.
Tharp, R., & Gallimore, R. (1988). Rousing minds to life: Teaching, learning, and
schooling in a social context. Cambridge, MA: Cambridge University Press.
ThoughtLink (2001, November). Key drivers for C2 performance: Data mining
SCUDHunt experiment data. Vienna, VA: Thoughtlink, Inc.
ThoughtLink (2002, October). Meta-analysis of team performance accuracy and shared
situational awareness in SCUDHunt experiments. Vienna, VA: Thoughtlink, Inc.
United States Joint Forces Command (2002, November). Millennium challenge 2002
executive report: Thinking differently. Norfolk, VA: United States Joint Forces
Command.
Vaknin, S. (2002, February 2). TrendSiters: Games people play. Electronic Book Web.
Retrieved May 13, 2004, from
http://12.108.175.91/ebookweb/discuss/msgReader$1405
Valenza, J. K. (2002, December 26). Gamers are playing, maybe learning too. The
Philadelphia Inquirer, Philly.com. Retrieved May 15, 2004, from

48
http://www.philly.com/mld/philly/living/columnists/joyce_kasman_valenza/4814
055.htm?1c.
Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. New
York, NY: Cambridge University Press.
Wenger, E., McDermott, R., & Snyder, W. M. (2002). Cultivating communities of
practice: A guide to managing knowledge. Boston, MA: Harvard Business School
Press.
Wired News (2003, October 3). Military training is just a game. Retrieved May 2, 2004,
from
http://www.wired.com/news/games/0,2101,60688,00.html?tw=wn_story_related
Wisher, R.A., & Fletcher, J. D. (2004). The case for advanced distributed learning.
Information and Security: An International Journal, 14, 17-25.
Woodcock, B. S. (2004, March). An analysis of MMOG subscription growth—Version
8.0. Retrieved May 14, 2004, from
http://pw1.netcom.com/~sirbruce/Subscriptions.html
Yee, N. (2003). Learning leadership skills. The Daedalus Project. Retrieved June 2,
2004, from http://www.nickyee.com/daedalus/archives/000338.php
Yee, N. (2004). The Daedalus Project. Retrieved June 2, 2004, from
http://www.nickyee.com/daedalus/
Young, K. (2004). Internet addition: Evaluation and treatment. Retrieved May 3, 2004,
from http://www.studentbmj.com/back_issues/1099/editorials/351.html