Games Research Today: Analyzing the Academic Landscape 2000-2014

Abstract

In this paper we present an analysis of the academic landscape of games research from the last 15 years. We employed a data driven approach utilizing co-word and co-venue analysis on 48 core venues to identify 20 major research themes and 7 distinct communities, with a total of 8,207 articles and 21,552 unique keywords being analyzed. Strategic diagrams and network maps are applied to visualize and further understand interrelationships and underlying trends within the field.

Full text

Games Research Today: Analyzing the Academic
Landscape 2000-2014
Edward Melcer1, Truong-Huy Dinh Nguyen2, Zhengxing Chen2, Alessandro Canossa2,
Magy Seif El-Nasr2, Katherine Isbister1
[1] Game Innovation Lab, NYU Polytechnic School of Engineering, 5 MetroTech Center, Brooklyn, NY, 11201
{eddie.melcer, katherine.isbister}@nyu.edu
[2] PLAIT Lab, College of Arts, Media, and Design, Northeastern University, Boston, MA 02115
{tru.nguyen, m.seifel-nasr, a.canossa}@neu.edu, czxttkl@gmail.com
ABSTRACT
In this paper we present an analysis of the academic landscape of
games research from the last 15 years. We employed a data driven
approach utilizing co-word and co-venue analysis on 48 core
venues to identify 20 major research themes and 7 distinct
communities, with a total of 8,207 articles and 21,552 unique
keywords being analyzed. Strategic diagrams and network maps
are applied to visualize and further understand interrelationships
and underlying trends within the field.
Categories and Subject Descriptors
K.2. [History of Computing]: Theory, People
General Terms
Theory
Keywords
Games research; bibliometric study; co-word analysis; co-venue
analysis
1. INTRODUCTION
Since the turn of the century, games research has both become an
established domain of study and broadly diversified the range of
topics explored under the heading of "games". While this
expansion can be viewed positively for the field as a whole, the
large number of venues and topics can become overwhelming for
researchers trying to understand where their work best fits and
should be published. Some studies have tried to reduce this
confusion by identifying paradigms of games research with
respect to particular domains and venues [9, 15, 24], but little has
been done to investigate the field as a whole.
In the absence of data analysis, there has been an anecdotal
understanding among some game researchers—such as with [3,
4]—that there are two overarching communities within the field,
one with research focused on technical approaches to
understanding and developing games (e.g. artificial intelligence,
computational modeling, visualization, graphics research, etc.)
and another addressing non-technical aspects of games with a
range of research approaches from the humanities, arts, design,
and social sciences (e.g., narrative, user experience, virtual
worlds, role play, design, philosophy, etc.). However, a clear
analysis of the interrelations and synergies among sub-
communities and research themes that comprise the current
academic landscape remains undocumented. In this paper, we
present our efforts at mapping the topology of games research
from the first part of this century.
We collected and analyzed publications from 48 core publication
venues of games research from the years 2000 through 2014. Our
analysis utilizes co-occurrence methods and community detection
techniques to provide a comprehensive overview of the field.
With co-word analysis we identified 20 distinct research themes,
and through co-venue analysis we also determined 7 communities
for these themes. We also note conferences and journals that act
as bridges between communities. Results are visualized using co-
occurrence analysis artifacts such as strategic diagrams and
keyword network maps. Finally, we conclude with a discussion of
the current state of the games research field.
2. RELATED WORK
While our work is among the first of its kind targeting the games
research community as a whole, there has been similar work
tracing the progress of other research fields such as Human
Computer Interaction [17], ubiquitous computing [18], library
and information science [11, 14, 16, 25], consumer behavior [19],
software engineering [10], biology [1, 8], and education [21].
Drawing from this past work, we use popular techniques such as
co-word analysis to examine the current academic landscape of
the games research field.
2.1 Co-word and Co-venue analysis
Co-word analysis is a widely used bibliometric approach that
identifies interactions and hierarchies among problems, concepts,
and ideas in a network [6, 7]. It builds upon the assumption that
an article's keywords provide a summary of its content, and thus
can be utilized to reduce a large space of descriptors (i.e., article
text) to a network graph of smaller related spaces (i.e. keywords)
[8]. Keywords are associated with a paper, and two keywords
associated with the same paper are connected to form a network
graph of keywords. The co-word network can then be analyzed for
clusters to identify a set of closely related themes [18]. A similar
approach can be used for co-venue analysis where venues are
associated with authors, and two venues where the same author
has published are linked to form a network graph of venues. A co-
venue network can be analyzed for clusters to identify a set of
closely related venues, or communities. This approach visualizes
the interrelated concepts and is easier to understand while
maintaining vital information necessary for analysis [10].

Figure 1. Strategic diagram characterizations based upon
density and centrality.
Ultimately, network analysis can be used on network graphs to
describe a field of research. It segments the graph into clusters of
nodes, and each cluster corresponds to a research theme or
community within the field. Depending on how nodes and clusters
are linked (e.g., co-word, co-author, co-venue, etc.), different
network characteristics can be utilized to describe a research field
in vastly different ways [17]. The network characteristics we are
concerned with in our analysis are:
 Centrality: the degree of interaction a cluster has with other
parts of the network [18]. It essentially measures the strength
of the links from one research theme or community to other
research themes or communities, and is an indicator of the
significance of a theme or community in the development of
an entire field [19]. As a cluster obtains more strong links in
a network, the more central it's position becomes [20].
 Density: the measurement of a cluster's development [19]. It
can be understood as the strength of all internal ties (edges)
linking together nodes that make up a theme or community
[13]. Density provides a good representation of a cluster's
ability to maintain itself and grow over time [6]. As a cluster
increases in density, the more coherent it becomes and the
more likely it is to contain inseparable nodes [18].
 Bridges: bridges between two nodes provides communication
and facilitates flow among otherwise isolated regions of the
network [17, 20].
Utilizing centrality and density, a strategic diagram can be created
to better visualize and understand the maturity and cohesion of
network clusters [14]. Strategic diagrams have been used widely
in previous co-word analysis work [1, 14, 16–19, 25] where the
density and centrality of clusters are plotted on a two-dimensional
grid. The x-axis of the grid shows how strongly a cluster is
connected to others and the y-axis shows a cluster's development.
A cluster's location within a strategic diagram characterizes it in
the context of the whole discipline [6] (Figure 1). In quadrant I,
clusters are both coherent and central to the field as a whole.
These mainstream clusters represent the focus of a large portion of
the network. While clusters in quadrant II are also coherent, they
tend to be specialized and separate from the overall focus.
Clusters in quadrant III are in flux, representing emerging or
declining portions of the network. Finally, quadrant IV contains
clusters that represent a common, broad focus or have not yet
matured but have the potential to be a primary network focus.
Table 1. Expert generated list of core games research journals.
Journal
Computers in Entertainment (CIE)
Eludamos. Journal for Computer Game Culture (Eludamos)
Entertainment Computing
Game Studies
GAME The Italian Journal of Game Studies (G|A|M|E)
Games and Culture (G & C)
IEEE Transactions on Computational Intelligence and AI in
Games (TCIAIG)
International Computer Games Association Journal (ICGA)
International Digital Media and Arts Association (iDMAa)
International Journal of Arts and Technology (IJART)
International Journal of Computer Games Technology (IJCGT)
International Journal of Game-Based Learning (IJGBL)
International Journal of Gaming and Computer-Mediated
Simulations (IJGCMS)
International Journal of Role-Playing (IJRP)
International Journal of Serious Games (IJSG)
Journal of Game Design and Development Education (JGDDE)
Journal of Game Development (JOGD)
Journal of Gaming & Virtual Worlds (JGVW)
Journal of Virtual Worlds Research (JVWR)
Simulation & Gaming (S&G)
The Computer Games Journal (TCGJ)
Well Played
3. DATA
To understand different games research communities, we obtained
meta-data of research papers in a set of core publication venues
vetted by field experts. The meta-data contains titles, authors,
published venues, keywords, and citations. This allows us to
establish relationships between papers and within communities.
3.1 Core Venue Identification
The first step in the data collection process was identifying core
games research venues. Having consulted with leading researchers
among major game research groups in the US—such as UCSC,
NCSU, NYU, and NEU—we identified 21 core games research
journals (Table 1) and 27 core games research conferences (Table
2) that researchers often publish in and refer to for new games
research.
Since our data collection is done using electronic means, we do
not account for venues that do not publish their papers in
dedicated online proceedings. We feel this exclusion is reasonable
since modern games research is conducted via electronic medium.
In fact, almost all of games research venues release their articles,
or at least the meta-data, online. We also excluded major
interdisciplinary venues that have published games research
related papers—such as AAAI, SIGCHI and SIGGRAPH—since
their primary focus is not games research itself and the core games
research venues primarily cover the same topics. This exclusion is
acceptable according to Bradford's law of scattering [5] since a
relatively small core of venues will account for as much as 90% of
the literature while attempts to gather 100% will add venues and
articles at an exponential rate [12].

Table 2. Expert generated list of core research conferences.
Conference
AAAI Spring Symposium on AI and Interactive Entertainment
(AAAISAIIE)
ACM SIGRAPH Sandbox Symposium (Sandbox)
Advances in Computer Entertainment Technology (ACE)
Artificial Intelligence and Interactive Digital Entertainment
(AIIDE)
Computational Intelligence and Games (CIG)
Digital Games Research Association Conference (DIGRA)
European Conference on Games Based Learning (ECGBL)
Foundations of Digital Games (FDG)
Future Play
Games Learning Society (GLS)
Gamification
Intelligent Narrative Technologies Workshop (INT)
Intelligent Technologies for Interactive Entertainment
(INTETAIN)
International Computer Games Conference (CGAMES)
International Conference on E-Learning and Games (Edutainment)
International Conference on Entertainment Computing (ICEC)
International Conference on Interactive Digital Storytelling
(ICIDS)
International Conference on Virtual Storytelling (ICVS)
International Games Innovation Conference (IGIC)
International Simulation and Gaming Association Conference
(ISAGA)
Meaningful Play
Serious Games Development and Applications (SGDA)
Technologies for Interactive Digital Storytelling and Entertainment
(TIDSE)
The Philosophy of Computer Games
Under the Mask
Workshop on Network and Systems Support for Games
(NetGames)
3.2 Collection and Completeness Verification
We contacted publishers of each venue for approval to batch
download articles from their repositories, and used a data crawler
to automate the collection process. However, for some venues
publications are only stored in PDF format. In such cases, we
hand-collected articles and used scripts to convert them into text
before running an additional script to extract meta-data.
In order to confirm the completeness of our collection process, we
conducted a verification step that randomly selects two papers
from each venue and checks for their existence in our database.
To make sure that the random selection is fair, for each venue, we
use the following retrieval URL to obtain the list of papers:
https://scholar.google.com/scholar?as_public
ation=<venue_name>&as_ylo=2000&as_yhi=2014
The value "<venue_name>" above is replaced by the names of the
venues, such as DIGRA, FDG, etc. The first two entries in the
retrieved list for each venue are used in our test set, and the
verification process shows that we have collected 86.17% of the
articles from our core games research venues.
3.3 Cleaning the Data
The data set collected originally suffered from entry duplication
and articles that were prematurely reported or not peer-reviewed,
such as extended abstracts, abstracts only, and panels. These
articles were detected and removed. Another issue was the
appearance of editors’ names in the co-author list of papers. In
particular, some online repositories show editors’ names
alongside with authors, which caused inconsistencies when batch
downloaded. Potential editors in the data were identified by
having more than 5 publications in a year. We then verified
whether the potential editors were actually editors or instead
authors and removed them accordingly. After the data was
cleaned, we were left with 8,207 papers from the games research
field in the last 15 years.
3.4 Keyword Generation
Over a third of the papers in the dataset we collected did not
originally contain keyword information. This was due to
incomplete data sources or lack of keywords in the original
publication. Since keywords are essential pieces of information to
connect papers in the same research community (and vital for our
co-word analysis), we implemented an algorithm to generate
keywords from paper titles and abstracts. Below are the steps for
each paper missing keywords:
1. Create a keyword candidate pool of existing keywords from
the papers in the database.
2. Create a 2-gram candidate pool by extracting 2-grams from
titles and abstracts of all papers in the database.
3. Manually screen out inappropriate 1- and 2-grams which are
either meaningless or semantically too general to be good
keywords, such as “recent years” or “considerable amount”.
4. Split titles and abstracts into phrases. Treat the phrases as
keywords if they already exist in the keyword candidate pool.
5. For papers with less than 3 phrases set as keywords, the valid
2-grams from their titles and abstracts are set as keywords.
Our justifications for the above generation steps are as follows:
1. Existing keywords cover N-grams well, for N ≤ 2.
2. 1-grams include too many frequently used words, which are
generally not very useful. (e.g., "Game", "Player", etc.).
3. Keywords are rarely long phrases, thus N-grams where N ≥ 3
are unlikely to be keywords.
This process automatically generates and tags keywords for most
papers without keywords in our database, however it still leaves
about 50 papers without any appropriate keywords. We asked the
researchers from our venue identification task to identify
keywords for these papers based on domain knowledge.
We would also like to note that Rake [22] is one of the widely
used algorithms for keyword generation. However, since the
algorithm adopts a scoring scheme that linearly combines the
scores of word pairs in candidate keywords, it tends to return high
scores for long phrases even though the frequency of such phrases
appearing altogether within one context is low. This led us to use
the modified algorithm above which is based on Rake.

Figure 2. Power-law distribution of keyword frequency with
logarithmic scale. Power-law distributions should appear
similar to a straight line when using a logarithmic scale.
4. ANALYSIS AND RESULTS
4.1 Research Themes from Co-Word Analysis
To improve accuracy of the co-word analysis, we manually
standardized keywords with a frequency greater than 10 through
synonym mergence (e.g., "MMOG" and "Massively Multiplayer
Online Game") and plurality mergence (e.g., "Educational Game"
and "Educational Games"). As a result, from the 8,207 articles
collected, a total of 21,552 unique keywords were identified with
frequencies up to 466.
The frequency of keywords in the last 15 years of games research
follows a power-law distribution with an alpha of 2.28 and R2 of
0.92 (Figure 2). This indicates a scale-free network structure
where a small number of popular nodes (i.e., keywords) act as
hubs connecting other concepts; these hubs in turn shape the
network reflecting the overall intellectual structure of games
research through keywords [17]. The scale-free property of the
network also suggests that major research themes and influences
can be detected using a small subset of the most popular keywords
[25]. We therefore selected keywords that appeared more than 18
times in the dataset, covering 25% of the total keyword frequency.
This resulted in the 264 most frequent keywords for co-word
analysis (Figure 3).
We then constructed a weighted co-word network graph where
each keyword is represented by a node and an edge of weight n is
added between keywords that appeared together on n papers.
Research themes were determined using Blondel et al's
community detection algorithm on the network [2]. A total of 20
clusters (research themes) were detected from the 264 most
frequent keywords (Table 3).
Figure 3. Keyword cloud visualizing the 264 most frequent
keywords used in games research papers
Figure 4. Strategic diagram for games research themes.
For each research theme, Table 3 shows:
 Keywords: the 10 most frequent keywords that constitute the
theme. The three most frequent keywords are shown in bold
and summarize the general topics of the theme.
 Size: the total number of keywords encompassing the theme.
 Frequency: how often a keyword from this theme will occur
on average.
 Centrality: the strength of a theme's interaction with other
themes [13]. A localized version of this metric is calculated
using a K-step reach of 2.
 Density: the strength of the links tying together keywords
within a theme (i.e., internal cohesion) [13].
Based on cluster centrality and density of the 20 themes in Table
3, we constructed a strategic diagram to visualize the maturity and
cohesion of each theme (Figure 4). The origin of the diagram is
set to the average cluster centrality and density (i.e., 0.7229,
2.4245). C19 was excluded from the calculations since its values
were a large enough outlier to notably skew the averages.
Finally, to better understand and visualize the interactions
between games research themes in Table 3, we created a granular
network map of keywords [17, 18] (Figure 5). In the network
map, each keyword is represented as a node and keywords that
appear on the same paper are linked together. The size of a node
in the figure is proportional to the frequency of the keyword and
nodes of the same color belong to the same theme. To reduce
visual clutter, only the two most frequent keywords from each
research theme are used while links between keywords are shown
if their correlation coefficient is above 0.33. A downside of this
visual reduction is that the exclusion of weaker links can cause
clustered nodes to appear disconnected. For instance, "Interactive
Narrative" and "Interactive Storytelling" in Figure 5 appear
separated, however this is not the case. It is simply because
multiple weaker links between the two are not included.

Table 3. Major themes in games research. The cluster ID, top keywords, size (S), frequency (F), centrality (C), and density (D).
ID
10 Most Frequent Keywords
Size
F
C
D
T1
Game Design, Serious Games, Game Based Learning, Educational Games, Game Development,
Motivation, Case Study, Engagement, Gamification, Collaborative Learning
38
67.74
0.947
1.138
T2
Interactive Storytelling, Interactive Narrative, Role Playing, Real World, Multiplayer Online,
Massively Multiplayer, Interactive Drama, Game World, Non Player, Digital Storytelling
24
47.88
0.750
1.641
T3
Real Time, Virtual Reality, Virtual Environments, Virtual Characters, Game Engine, Motion
Capture, Time Strategy, Animation, Virtual Storytelling, Computer Animation
20
51.00
0.840
1.500
T4
Virtual Words, Massively Multiplayer Online Games, Second Life, Online Games, Avatars, Social
Interaction, Gender, Multiplayer, World of Warcraft, Ethnography
20
63.30
0.906
1.942
T5
Gameplay, User Experience, Entertainment, Player Experience, Immersion, Usability, Flow,
Interface, Ludology, Game Environment
18
43.11
0.874
1.059
T6
Narrative, Art, Interactivity, Emotion, Aesthetics, Music, Agency, Interactive Art, Affective
Computing, Interactive Systems
14
37.57
0.744
1.154
T7
Game Theory, Evolutionary Computation, Computational Intelligence, Genetic Algorithms, Search
Problems, Standards, Optimization, Trees Mathematics, Mathematical Model, Statistics
14
45.01
0.616
4.901
T8
Augmented Reality, Mixed Reality, User Interface, Pervasive Games, Mobile Games, Magic Circle,
Mobile, Ubiquitous Computing, Mobile Gaming, Location Based
13
50.23
0.757
1.321
T9
Artificial Intelligence, Decision Making, Planning, Context, Cognition, Game AI, Multi-Agent
Systems, Real-Time Systems, Human Player, Measurement
12
54.75
0.821
3.515
T10
Human Computer Interaction, Digital Media, Interaction Design, New Media, Psychology,
Interactive Media, Human Factors, Interface Design, Level Design, Gesture Recognition
12
42.92
0.817
1.409
T11
Simulation, Role Play, History, Experiential Learning, Cooperation, Representation, Modeling,
Negotiation, Participation, Simulation Gaming
12
48.17
0.718
1.894
T12
Learning Artificial Intelligence, Training, Machine Learning, Reinforcement Learning, Learning
Systems, Game Playing, Data Mining, Sport, Predictive Models, Feature Extraction
11
39.18
0.684
3.073
T13
Servers, Internet, Computer Architecture, Mobile Computing, Delay, Peer to Peer Computing,
Media, Tiles, Cloud Computing, Scalability
10
31.50
0.602
3.200
T14
Learning, Education, Children, Storytelling, Creativity, Teaching, Reflection, Tangible Interfaces,
Educational Technology, Survey
10
59.10
0.736
2.000
T15
Humans, Neural Networks, Software Agents, Testing, Navigation, Computer Simulation, Artificial
Neural Networks, Games of Skill, Robots
9
44.56
0.549
5.861
T16
Interaction, Communication, Role Playing Games, Player Behavior, Content Creation, Personality,
Fun, Multi-Touch
8
29.88
0.633
1.357
T17
Educational Institutions, Computer Aided Instruction, Software Engineering, Software, Computer
Science Education, Technological Innovation
6
31.17
0.570
3.733
T18
Collaboration, Board Games, Social Networks, Multiplayer Games, Social Media
5
46.20
0.656
1.700
T19
Monte Carlo Methods, Tree Searching, Algorithm Design and Analysis, Monte Carlo Tree Search
4
38.00
0.323
16.667
T20
Computational Modeling, Visualization, Engines, Databases
4
50.00
0.515
3.667
Figure 5. Keyword network map (line represents link between two keywords with Pearson correlation coefficient ≥ 0.34).

4.2 Communities from Co-Venue Analysis
Using a similar approach to our co-word analysis, we constructed
a weighted co-venue network graph where each venue is
represented by a node and an edge of weight n is added between
venues that had n authors publish in both venues. Communities
were again determined using Blondel et al's community detection
algorithm on the network, and a total of 7 clusters (research
communities) were detected from the 48 venues (Table 4). In
addition to the most frequent keywords, centrality, and density of
each research community, Table 4 also shows venues that
comprise each community. Venues are ordered from greatest to
least by the total number of unique authors that have published at
that venue.
We also wanted to see which venues were most likely bridges
between communities. As a result, we calculated the betweenness
centrality for all venues in the network since nodes with high
betweenness centrality play a role as bridges between other
portions of the network [16]. The venue with the highest
betweenness centrality for each community is shown in bold. We
then constructed a strategic diagram to visualize the maturity and
cohesion of each research community (Figure 6). The origin of the
diagram is set to the average centrality and density of the
communities from Table 4 (i.e., 0.9393, 17.5656).
Lastly, to better understand and visualize the interaction between
games research communities, we constructed a granular network
map of venues (Figure 7). In the network map, each venue is
represented as a node in the graph and venues that have had the
same author publish are linked together. The size of a node in the
figure is proportional to its degree and nodes of the same color
belong to the same theme. To reduce visual clutter, only four
venues with the highest number of authors from each research
theme are used while links between venues are shown if their
correlation coefficient is above 0.33. Again, this visual reduction
has the same downsides as the visual reduction used for the
keyword network map.
Table 4. Communities in games research. Cluster ID, conferences, keywords, centrality (C), and density (D) are shown.
ID
Conferences
5 Most Frequent Keywords
C
D
C1
DIGRA, S & G, JVWR, G & C, iDMAa, The Philosophy
of Computer Games, JGVW, Game Studies, Eludamos,
Under the Mask, ISAGA, G|A|M|E, IJRP
Virtual Worlds, Simulation, Game Design, Second Life,
Role Playing
0.971
6.744
C2
ACE, ICEC, Edutainment, CIE, Intetain, Entertainment
Computing, IJCGT, NetGames
Augmented Reality, Real Time, Virtual Reality, Game
Design, Virtual Environments
0.975
37.786
C3
IJART, Sandbox, IGIC, FuturePlay, CGames,
Gamification, JGDDE
Game Design, Serious Games, Education, Computer
Aided Instruction, Human Computer Interaction
0.976
4.762
C4
CIG, AIIDE, FDG, TCIAIG, ICGA, INT
Artificial Intelligence, Game Theory, Humans,
Evolutionary Computation, Computational Intelligence
0.976
27.933
C5
ICIDS, ICVS, TIDSE, AAAISAIIE, JOGD
Interactive Storytelling, Interactive Narrative, Real
Time, Interactive Drama, Digital Storytelling
0.884
15.300
C6
GLS, Meaningful Play, IJGCMS, TCGJ, Well Played
Game Design, Gameplay, Educational Games, Real
World, Case Study
0.907
9.100
C7
ECGBL, IJGBL, SGDA, IJSG
Game Based Learning, Serious Games, Game Design,
Learning, Educational Games, Education, Assessment
0.886
21.334
Figure 6. Strategic diagram for games
research communities.
Figure 7. Venue network map (line represents link between two venues with Pearson
correlation coefficient ≥ 0.34).

5. DISCUSSION
We took a data driven approach, using co-occurrence of keywords
and venues to identify major research themes and communities in
the last 15 years, and to understand how these communities and
themes interact. This data driven approach circumvents the pitfalls
of subjectively or intuitively trying to map the field, using
research that has been conducted and published as the basis for
analysis. Ultimately, while previous work has been focused on
specific communities and paradigms within games research, our
work provides a big picture overview of the landscape to offer
insights on common researcher questions such as to which
communities, themes, and venues their work applies and where
they should publish.
5.1 Connecting the Overarching Communities
Our results appear to support anecdotal evidence for the
separation of research with a technical focus (clustered around the
right side of Figure 5, and in the top left quadrant of Figure 4—
e.g., Artificial Intelligence, Decision Making, Neural Networks,
etc.) from other games research topics. Similarly, technically
focused communities and venues in Figure 7 fall more towards the
left side (e.g., venues clustered in C4 and C5—see Table 4) while
less technically focused ones fall more towards the right (e.g., C6
and C1). The two venues often anecdotally cited as the larger
umbrella conferences—FDG and DiGRA—fall near the center of
the venue network map (Figure 7) as we would expect, with FDG
left of center indicating a greater presence of technical papers.
The data also shows that education has established itself as one of
the central research topics within the games research field, with
keywords such as “Serious Games”, “Game Based Learning”,
“Education”, “Educational Games”, "Computer Aided
Instruction", "Collaborative Learning", etc. relating to many
different research themes (T1, T11, T14, T17) and communities
(C3, C6, C7) at different levels of development—as illustrated in
the strategic diagrams for Figures 4 & 6. Education related
keywords such as serious games or game based learning are also
among the most frequently used keywords and make up the largest
cluster (T1) in the network.
5.2 Too Many Venues?
One recent debate that has appeared on the Digital Games
Research Association Gamesnetwork mailing list is the question
of whether there are too many venues for games research [4].
Forty-eight core games research venues is a large number for a
relatively young field and we have noted a comparatively small
number of major communities in Table 4. However, there is a
large number of research themes in Table 3, suggesting that many
small and emerging sub-communities are encompassed within the
larger ones presented here. Additionally, the strategic diagram in
Figure 4 suggests that the games research field is still rapidly
evolving since many themes and corresponding sub-communities
are still isolated from the field (e.g., Monte Carlo methods in T19)
or have the potential to become a primary focus of games research
(e.g., augmented reality in T8). We hope that our presentation of
communities and trends through this paper can provide data-
driven insights that might inform the conversation about venues.
5.3 Where Should I Publish?
For many researchers, the large number of venues raises the
question of appropriate locations to publish their work. This
depends largely on what communities and themes their research
best matches and the interdisciplinary nature of their work. The
venue network map in Figure 7 illustrates the layout and strong
connections between venues and communities. Venues and
communities that are more central to the network tend to cover a
wider range of research and accept more interdisciplinary work,
while more peripheral venues and communities tend to be focused
on specific research themes. Therefore, the more interdisciplinary
a researcher's work is among communities in Table 4, the more
beneficial a venue with high betweenness centrality (more central
to the network in Figure 7) from one of the related communities
is. Conversely, research that is very focused within a specific
community may benefit from more peripheral venues within that
community. For instance, FDG in C4 might be a better venue for
submissions of AI work making use of narrative theory while CIG
in C4 would likely be a better venue for submissions of focused
AI work using neural networks in games.
Notably, the clustering of venues is not as focused with respect to
research themes since there are many venues that publish
interdisciplinary work, spanning a diverse range of topics.
Looking at the strategic diagram in Figure 6 and corresponding
communities in Table 4, a distinction between the breadth of
venues can be understood. For instance, between the two larger
umbrella conferences in Figure 6, FDG (C4) is shown to be more
focused and mainstream than DiGRA (C1). Considering FDG
tends to have a stronger technical focus while DiGRA is more
broad, this distinction seems appropriate.
6. LIMITATIONS AND FUTURE WORK
One of the main limitations of this study is that it does not
account for games research works published in non-core venues—
such as AAAI, SIGCHI, CogSci, and SIGGRAPH—since it is
difficult to auto-detect games research papers published in these
venues and existing methods to do so vary greatly in their success.
Additionally, accounting for papers in non-core but known games
research venues does not solve the general challenge of collecting
all games research papers. There are other interdisciplinary venues
that occasionally publish games papers and tracking every single
one of these venues and papers would be infeasible. However, we
do feel these exclusions are reasonable considering Bradford's law
of scattering, which demonstrates that almost all of a community's
literature can be accounted for using a small set of core of venues.
Another limitation to note is that our co-venue analysis cannot
determine the impact of a particular venue within its community
or the games research field as a whole. This is because (on top of
being subjective) many other factors contribute to the clout of a
venue besides its centrality, connectedness, or size. As a result,
the collection of additional meta-data for analysis—such as
acceptance rates—would be necessary.
Lastly, a limitation and direction for our future work is the lack of
an analysis of the evolution of the field. While our analysis
presented here provides a strong overall look at the current state
of the field over the past 15 years, the subtle nuances of how and
when communities have emerged, grown, merged, and declined is
difficult to grasp using just co-word analysis. We have begun
analyzing the co-evolution of the games research field using the
Evo-NetClus model [23], since it is ideal for better understanding
how games research has changed and what direction it might go in
the future.
7. CONCLUSION
In this paper, we presented an overview of the landscape of games
research over the last 15 years. Our findings identified 20 major

research themes and 7 distinct sub-communities. The results
validated the commonly held assumption that games research has
different clusters of papers and venues for technical versus non-
technical research, and identified interactions and synergies
between these research clusters. We hope the data driven
approach used can provide insight and aid further discussions and
questions that researchers may have about the field as a whole.
8. ACKNOWLEDGEMENTS
The authors would like to thank our colleagues including Tiffany
Barnes, Michael R. Young, T.L. Taylor, faculty members of the
Game Design Program at Northeastern University and at NYU
Polytechnic School of Engineering, for their recommendations
and suggestions in our selection of games research venues. We
would also like to thank the publishers and representatives that
granted us official access to the meta-data of their respective
venues: Jose Zagal (DIGRA), Drew Davidson (Well Played),
Brian Winn (Meaningful Play), and Craig Rodkin and Sandy
Yang (ACM). We would also like to thank Janet Morrow from
Northeastern University Libraries who helped us obtain access
approval from IEEE.
9. REFERENCES
[1] An, X.Y. and Wu, Q.Q. 2011. Co-word analysis of the
trends in stem cells field based on subject heading
weighting. Scientometrics. 88, 3 (2011), 133–144.
[2] Blondel, V.D. et al. 2008. Fast unfolding of communities
in large networks. (2008), 6.
[3] Boellstorff, T. 2009. Method and the Virtual: Anecdote,
Analogy, Culture. Journal of Virtual Worlds Research. 1,
(2009), 3–7.
[4] Bogost, I. 2015. [GAMESNETWORK] Why Are There
so Many Games Research Conferences? Mailing List.
GAMESNETWORK Games Research Network.
[5] Bradford, S.C. 1976. Sources of Information on Specific
Subjects. Collection Management. 1, January 2015
(1976), 95–104.
[6] Callon, M. et al. 1991. Co-word analysis as a tool for
describing the network of interactions between basic and
technological research: The case of polymer chemsitry.
Scientometrics. 22, 1 (1991), 155–205.
[7] Callon, M. et al. 1983. From translations to problematic
networks: An introduction to co-word analysis. Social
Science Information.
[8] Cambrosio, a. et al. 1993. Historical scientometrics?
Mapping over 70 years of biological safety research with
coword analysis. Scientometrics. 27, 2 (1993), 119–143.
[9] Carter, M. et al. 2014. Paradigms of Games Research in
HCI: A Review of 10 Years of Research at CHI.
Proceedings of the First ACM SIGCHI Annual
Symposium on Computer-human Interaction in Play
(New York, NY, USA, 2014), 27–36.
[10] Coulter, N. et al. 1998. Software engineering as seen
through its research literature: A study in co-word
analysis. Journal of the American Society for
Information Science. 49, 1997 (1998), 1206–1223.
[11] Ding, Y. et al. 2001. Bibliometric cartography of
information retrieval research by using co-word analysis.
Information Processing and Management. 37, (2001),
817–842.
[12] Garfield, E. 1971. The Mystery of Transposed Journal
Lists: Wherein Bradford’s Law of Scattering is
Generalized According to Garfield's Law of
Concentration. Current Contents. 17, (1971), 5–6.
[13] He, Q. 1999. Knowledge Discovery Through Co-Word
Analysis. Library Trends. 48, 1 (1999), 133–159.
[14] Hu, C.-P. et al. 2013. A co-word analysis of library and
information science in China. Scientometrics. 97, (2013),
369–382.
[15] Lara-Cabrera, R. et al. 2014. An analysis of the structure
and evolution of the scientific collaboration network of
computer intelligence in games. Physica A: Statistical
Mechanics and its Applications. 395, (2014), 523–536.
[16] Liu, G.Y. et al. 2012. A co-word analysis of digital
library field in China. Scientometrics. 91, December
2011 (2012), 203–217.
[17] Liu, Y. et al. 2014. CHI 1994 - 2013 : Mapping Two
Decades of Intellectual Progress through Co-word
Analysis. Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems (2014), 3553–
3562.
[18] Liu, Y. et al. 2014. Identity Crisis of Ubicomp? Mapping
15 Years of the Field’s Development and Paradigm
Change. Proceedings of the 2014 ACM International
Joint Conference on Pervasive and Ubiquitous
Computing (2014).
[19] Muñoz-Leiva, F. et al. 2012. An application of co-word
analysis and bibliometric maps for detecting the most
highlighting themes in the consumer behaviour research
from a longitudinal perspective. Quality and Quantity.
46, (2012), 1077–1095.
[20] Nielsen, A.E. and Thomsen, C. 2011. Sustainable
development: The role of network communication.
Corporate Social Responsibility and Environmental
Management. 18, February 2010 (2011), 1–10.
[21] Ritzhaupt, A.D. et al. 2010. An investigation of distance
education in North American Research Literature using
co-word analysis. International Review of Research in
Open and Distance Learning. 11, 1 (2010), 37–60.
[22] Rose, S. et al. 2010. Automatic Keyword Extraction from
Individual Documents. Text Mining: Applications and
Theory. Michael J. Berry and J. Kogan, eds. Wiley
Online Library. 1–20.

[23] Sun, Y. and Chen, C. 2013. Co-Evolution of Multi-
Typed Objects in Dynamic Star Networks. … on
Knowledge and …. January 2012 (2013), 1–14.
[24] Yannakakis, G. and Togelius, J. 2014. A Panorama of
Artificial and Computational Intelligence in Games.
IEEE Transactions on Computational Intelligence and
AI in Games. c (2014), 1–1.
[25] Zong, Q.-J. et al. 2012. Doctoral dissertations of Library
and Information Science in China: A co-word analysis.
Scientometrics. (2012), 781–799.