Content uploaded by Carlos Pereira Santos
Author content
All content in this area was uploaded by Carlos Pereira Santos on May 18, 2016
Content may be subject to copyright.
On Utilizing Player Models to Predict Behavior in
Crowdsourcing Tasks
Carlos Pereira Santos1,2, Vassilis-Javed Khan2, and Panos Markopoulos1
1 Eindhoven University of Technology, Eindhoven, The Netherlands
{c.a.pereira.santos, p.markopoulos}@tue.nl
2 NHTV Breda University of Applied Sciences
{santos.c, khan.j}@nhtv.nl
Abstract. Player Modeling is a research field that studies player characteristics
by analyzing in-game behavior. We aim to develop independent models, which
are transferable and useful beyond a game’s context. We shall demonstrate the
feasibility of this approach by applying player models to crowdsourcing to pre-
dict workers’ task completion effectiveness. Specifically, we model a user’s
Need for Cognition based on in-game behavior, and based on that try to assign
appropriate tasks to workers.
1 Introduction
Every single player interaction within a video-game has the potential of generating
data. Monitoring in-game data to gain a better understanding of player behavior has
become common-practice in the industry with the intent of improving game develop-
ment, content and revenue. Each player’s in-game behavior can be tracked, for exam-
ple, the sequence of pressed buttons, the usage of specific words, the response time
for a certain part of the game, or the frequency of game-specific actions such as walk-
ing, picking up items or purchasing items.
By using data analysis techniques like simply counting the occurrences of specific
events [1], or more complex clustering techniques [2] it is possible to obtain general
information about the player, like: personality traits, age groups and skill sets. Player
modeling is a recent research field under the general umbrella of Computational Intel-
ligence directly applied to games, which recent research studies show to attain not
only player skill levels but also predict specific players’ traits.
Those models are currently used to improve video-game experiences, during new
development cycles like associating specific play styles with player age [3], or using
avatars to profile personality traits [4], or even to reduce a game’s difficulty and im-
prove player experience [5].
In a recent chapter [6] defined four different purposes for using player models: i)
game balancing, i.e., adapt the game experience based on the player skill, ii) personal-
ized content, i.e., procedural generation of more content, iii) game authoring, i.e.,
playtest to determine whether it provides the desired player experience and iv) mone-
tization, i.e., increase revenue, popular in free-to-play games.
However, we think that there is a greater potential for player models. The work
we have been developing is focused on creating Context Independent Player Models
(CIPMs), by utilizing existing methodologies to collect and analyze in-game player
data and build player models which can be applied for other than game systems. More
specifically we aim to investigate whether and to what extent player models can be
utilized to identify more suitable workers in crowdsourcing systems. This aim re-
quires a two-step process. First one needs to extract CIPMs and second one would
need to check the effectiveness of it in crowdsourcing task completion.
2 Prior Art
Crowdsourcing tasks are making an important contribution to society by providing
jobs, while still allowing enterprises to maintain quality and cut costs. Apart from the
obvious motivation of financial benefits there are more reasons for people to engage
on crowdsourcing such as: altruism, enjoyment, reputation and socialization [7].
There are already some examples that illustrate the potential of our approach. The
Malaria Training Game [8] is a first example combining gaming aspects and crowd
intelligence. It utilizes learning capabilities of human crowds to conduct reliable mi-
croscopic analysis of biomedical samples. A second example is Quizz [9]. It is a
crowdsourcing game, which assesses a user’s knowledge within a specific field and
enrich the system’s knowledge base. In both systems, the game results (answers) are
used to improve or validate knowledge. The difference in our approach is that these
and other similar systems can be improved if the in-game player behavior is analyzed,
providing a richer knowledge about the player, not only using the game results but
also the way game is played. Hence the need to create CIPMs, which would be valid
not only within the game context, but also, as a cross-domain profiling tool.
Most of crowdsourcing task platforms do not consider individual workers abilities,
personality, and context when assigning tasks. Recent studies have showed that user
profiling tools could improve quality and workers satisfaction [10, 11]. Integration of
CIPM to profile crowdsourcing workers would bring advantages such as: i) keep
workers more motivated, ii) it would be more difficult for workers to cheat in self-
reported profiling tools, iii) it would be an unobtrusive way of creating a player model
and iv) can complement existing services or be integrated within the task.
3 Case Study: Need for Cognition as CIPM
Most of player modeling studies are performed based on data-mining techniques
which try to statistically correlate (large) set of collected variables about player in-
game behavior. In the research line we propose a different approach: to design game
mechanics, which are able to determine specific player models.
For this reason we focused our research on a specific personality trait – Need for
Cognition (NC). NC is a personality trait associated with the extent a certain person is
inclined to perform cognitive activities. NC researches the link between personality
and behavior, and can be reliable measured through a self-report questionnaire [12].
NC is a simple and stable personality model, and it has been applied across con-
texts to make predictions of user behavior, e.g., how to effectively advice based on
whether someone will likely to follow the specific persuasion techniques [13]. High
NC defines a personality which is more inclined to mental problem solving, assess-
ment of situations with higher degree of elaboration; persons with lower NC are prone
to less elaboration and follow more heuristic/empirical strategies. The dualism of this
personality trait creates an area to explore inference using CIPM. Although, the NC
scale has been used within the video-games as a tool to determine players’ profiles
[14, 15], there is no reference of being inferred through in-game behavior.
NC is also important for specific crowdsourcing tasks. Studies on profiling labor
compared crowdsourcing workers to other recruiting methods and concluded that
workers are representative of the general population and substantially less expensive
to recruit. With regard to NC, workers are very similar, but, there is a slight skew to
having more individuals with higher NC within crowdsourcing workers [16].
To our knowledge NC is being used to serve as profiling tool within both contexts
(games and crowdsourcing), but, never being inferred through game behavior or being
used to predict and influence crowd tasks; this makes NC a good and credible per-
sonality trait for us to study as CIPM.
Based on the aforementioned analysis of existing studies, the key research ques-
tions we are interested in addressing are:
• Can Need for Cognition predict people’s behavior in crowdsourcing applications?
• Can certain game mechanics be designed to reliably infer Need for Cognition?
3.1 Method and Current study
We are currently in the exploratory phase, where we are evaluating the possibility of
inferring NC based on users’ gameplay. We have already developed a game which
integrates a mechanic to specifically to measure NC (Fig. 1). This mechanic is based
on the work of Boatman [15] which states that individuals with a high NC are pro-
cess-oriented as opposed to outcome-oriented. More specifically, in our game we ask
players to control the movement of a set of units. We embedded a hint system (using
game AI) which provides one possible (non-optimal) movement to support the deci-
Fig. 1. The mobile game we developed to integrate mechanics to infer a player’s NC
sion making process. In theory, higher NC players are driven by the cognition and the
process, not the outcome, hence, we should observe that high NC players will use and
follow less hints, unlike lower NC which are goal oriented and opt to take mental
shortcuts, consequently, follow hints more frequently. The in-game data will be com-
pared to NC measured using the self-report questionnaire [12]. The game is available
online
1
and we are gathering data.
References
1. Drachen, A., Canossa, A.: Towards gameplay analysis via gameplay metrics. Proc.
MindTrek ’09. pp. 202–209. ACM Press (2009).
2. Mahlmann, T., Drachen, A., Togelius, J., Canossa, A., Yannakakis, G.N.: Predicting
player behavior in Tomb Raider: Underworld. Proc. CIG2010. pp. 178–185 (2010).
3. Tekofsky, S., Spronck, P., Plaat, A., van den Herik, J., Broersen, J.: Play style:
Showing your age. Proc. CIG2013. pp. 1–8. IEEE (2013).
4. Yee, N., Ducheneaut, N.: Introverted elves & conscientious gnomes: the expression of
personality in world of warcraft. Proc. CHI2011 pp. 753–762 (2011).
5. Zook, A.E., Riedl, M.O.: A temporal data-driven player model for dynamic difficulty
adjustment. Proc. AIIDE 2012. pp. 93–98 (2012).
6. Yannakakis, G.N., Spronck, P., Loiacono, D., André, E.: Player Modeling,
http://drops.dagstuhl.de/opus/volltexte/2013/4335/, (2013).
7. Quinn, A.J., Bederson, B.B.: Human Computation: A Survey and Taxonomy of a
Growing Field. Proc. CHI2011 pp. 1403–1412. ACM (2011).
8. Luengo-Oroz, M.A., Arranz, A., Frean, J.: Crowdsourcing malaria parasite
quantification: an online game for analyzing images of infected thick blood smears. J.
Med. Internet Res. 14, e167 (2012).
9. Ipeirotis, P.G., Gabrilovich, E.: Quizz: targeted crowdsourcing with a billion
(potential) users. Proc. 23rd Int. Conf. World wide web. 143–154 (2014).
10. Difallah, D.E., Demartini, G., Cudré-Mauroux, P.: Pick-a-crowd: tell me what you
like, and i’ll tell you what to do. 367–374 (2013).
11. Sarasua, C., Thimm, M.: Microtask Available, Send us your CV! 2013 International
Conference on Cloud and Green Computing. pp. 521–524. IEEE (2013).
12. Cacioppo, J.T., Petty, R.E., Kao, C.F.: The efficient assessment of need for cognition.
J. Pers. Assess. 48, 306–307 (1984).
13. Kaptein, M., Markopoulos, P., Ruyter, B. de, Aarts, E.: Can You Be Persuaded?
Individual Differences in Susceptibility to Persuasion. Proc. INTERACT 2009.
14. Hoffman, B., Nadelson, L.: Motivational engagement and video gaming: a mixed
methods study, (2010).
15. Boatman, P.R., Day, E.A., Kowollik, V., Espejo, J., McEntire, L.E.: Modeling the
links between need for cognition and the acquisition of a complex skill, (2007).
16. Berinsky, A.J., Huber, G.A., Lenz, G.S.: Evaluating Online Labor Markets for
Experimental Research: Amazon's M.T.. Polit. Anal. 20, 351–368 (2012).
1
http://imi.nhtv.nl/nanobots/