Code Hunting Games: A Mixed Reality Multiplayer Treasure Hunt Through a Conversational Interface

Abstract

In this paper, we describe an online multi-player game that challenges players with abstract coding puzzles that are tied to a geo-graphical location. The proposed system transposes the classical scheme of “treasure hunt” games into a mixed-reality game, where players must physically move in order to advance in the game, while at the same time interacting with a chatbot through an online messaging system. The implementation of the online game is described in detail and an overview of different deployments of the system is given, including a large-scale deployment during the European CodeWeek 2017. We discuss details of the proposed system, including lessons learned during the development and operation of the game. We also argue that mobile games like the one proposed can be successfully adopted for many different purposes, from entertainment to education.

Full text

Code Hunting Games: a mixed reality
multiplayer treasure hunt through a
conversational interface
Lorenz Cuno Klopfenstein, Saverio Delpriori,
Brendan Dominic Paolini, and Alessandro Bogliolo
DiSPeA
University of Urbino, Italy
cuno.klopfenstein@uniurb.it
Abstract.
In this paper, we describe an online multi-player game that
challenges players with abstract coding puzzles that are tied to a geo-
graphical location. The proposed system transposes the classical scheme
of “treasure hunt” games into a mixed-reality game, where players must
physically move in order to advance in the game, while at the same time
interacting with a chatbot through an online messaging system. The
implementation of the online game is described in detail and an overview
of different deployments of the system is given, including a large-scale
deployment during the European CodeWeek 2017. We discuss details of
the proposed system, including lessons learned during the development
and operation of the game. We also argue that mobile games like the one
proposed can be successfully adopted for many different purposes, from
entertainment to education.
Keywords:
Bots; instant messaging; conversational UI; mobile gaming;
mixed reality
1 Introduction
Online messaging represents one of the most popular communication channels,
widely adopted since the early days of Internet access and now surged to be the
most important function of modern smartphones for a majority of users, especially
younger ones. As different studies confirm, not only have instant messaging (IM)
systems replaced SMS as the primary text communication vehicle, but many
IM platforms are used more frequently than traditional social networks [4]. Even
if considerable variation exists in the adoption of different platforms, using IM
communication channels is often the prevalent reason for using a smartphone [16].
A large-scale study by Battestini et al. shows that many individuals perceive
IM as a common and preferred communication channel, and they also frequently
engage in many simultaneous conversations throughout the day [1]. Also, IM chats
are more conversational in nature and often are used for group-based communi-
cation, leading to the exchange of very frequent messages if compared to other

communication tools such as SMS or e-mail [7]. In a small-scale study of mo-
bile phone users Dinger et al. found that users are “attentive” (i.e., they react
quickly to incoming messages) for more than 12 hours of a day. Non-attentiveness
typically lasts for at most 5 minutes at daytime [8]. Even if these studies might
not be representative of behaviors in a larger population, they show that instant
messaging is a truly pervasive activity, which frequently requires users to “mul-
titask” in order to maintain ongoing conversations on their smartphones while
performing other activities, such as homework or working on a computer [11].
Starting in 2014, most major online messaging systems have begun introducing
“bots”: conversational agents that can interact with users on the platform through
the exchange of simple text messages, right inside the messaging system itself [15].
Conversations with bots look like any other chat with friends or relatives but
provide an easy way for third-party developers to have a direct communication
channel to users. Bots exhibit many advantages if compared to traditional
mobile applications, including not requiring lengthy installation or updating
processes, the presence of a very familiar interaction model, and independence
from the underlying software platform. Moreover, many IM platform providers
have started enhancing the conversational UI paradigm, expanding it with more
complex messages and interactive elements, which provide much more engaging
experiences than simple text messages [13]. IM platforms increasingly focus on
bots as efficient software distribution services: an alternative to mobile application
stores, which allows IM systems to disrupt the control of mobile OS providers
and to directly reach consumers.
1.1 Related work
Current work on bots and autonomous conversational agents can be traced back
to over 60 years ago, with attempts such as Alan Turing’s Imitation Game [21]
or Weizenbaum’s ELIZA chatbot [22], both centered around computer programs
that would attempt to give the illusion of an actual human-to-human interaction
through the exchange of text messages.
Current applications of conversational agents on IM platforms span a variety
of use-cases [20], including recommender services that extract user preferences
and interests from conversations in natural language [6], which can be applied to
business, e-commerce, or e-citizenship endeavors. Bots are well suited to provide
access to existing data [5] or to provide a meeting point between multiple actors,
for instance, citizens, open data sources, and policy makers for decision-making
purposes [14]. Other, more personal, use-cases include conversational interfaces
for popular fitness or nutrition tracking applications [10]. In the medical field,
bots can be used to provide quick diagnoses based on patient symptoms, by
following an expert-system approach sourced from existing diagnosis flow charts
and procedures in medical literature [9]. Other applications specialize in complex
machine learning in order to generate dialogues on small talk topics [19], with the
aim of making bots and virtual assistants sound more natural when interacting
with humans. Chatbots have also been successfully used to deliver online quizzes
and games, for educational or entertainment purposes [12].

1.2 Contribution
Large parts of chatbot-related research has focused on techniques to engage users
in natural conversations, understanding natural language and context, first using
rule-based systems, pattern matching, and then machine learning techniques [18].
In this work we focus on conversational interfaces and instant messaging systems
as software delivery platforms, providing a sample application that eschews
most natural language in favor of simple and unambiguous interactions through
the UI elements offered by modern IM platforms [13]. We present a system
that provides a multiplayer puzzle-based game through a conversational interface
aimed at simplicity and effectiveness. The system is inspired by previous game-like
applications deployed through IM bot platforms [3] and makes use of the peculiar
features of these systems to provide a reliable and accessible user experience [15].
2 Code Hunting Games
The proposed game is based on the traditional scheme of “treasure hunt” games,
mixing the tasks of physically locating objects in an environment and that of
solving riddles. The base gameplay scheme is transposed into a mixed-reality
experience, that makes use of an IM bot to manage the game.
Each game run is articulated into multi-
Fig. 1.
Part of a chat with the bot
during a game session.
ple steps, each one of which requires players
to first reach a given location, find a hid-
den physical “token”, and then to solve an
abstract coding puzzle, which is provided
and verified by the IM bot. Once a team of
players completes all steps of a game, the
game is over.
Locations to reach are represented by
pinpoints on a map. Physical tokens to find
are realized using deep-link QR Codes, as
described in detail in Section 2.1. Riddles
presented to participants can be created in
any figurative or textual form. The current
deployment of the game focuses exclusively
on coding-oriented puzzles, in particular for
primary and secondary school pupils and
teachers, as further described in Section 2.2.
Currently, the game is implemented as
a bot for the Telegram platform exclusively.
Game administration services and the ac-
tual gameplay are both delivered through
a conversational interface, which can be accessed through any Telegram client
application. Part of a gameplay interaction is shown in Figure 1.

A complete run of a game session is constituted of the steps listed below.
Game states for each team
1.
Registration: the team leader registers to the game, he or
she provides a name and the number of players of the team;
2.
Start: at a set time, players gather in a predetermined loca-
tion to start the game;
3.
First selfie: teams snap the first selfie at the starting location;
4.
First puzzle: teams solve the first puzzle, possibly helped by
organizers at the starting location;
5.
Reach location: the bot assigns a random location to each
team and tells them to reach it;
6.
Selfie: the team leader gives proof of the team having reached
the location and snaps a picture;
7.
Puzzle: the team solves a random coding puzzle and obtains
a hint for the last puzzle. Go back to (5) if not all puzzles
of the game have been solved yet;
8.
Last location: teams are directed to a predetermined last
location;
9. Victory selfie: last picture;
10.
Last puzzle: the team solves one last puzzle putting together
all hints collected during the previous steps;
11.
Victory: the game is over. The team first solving the last
puzzle wins the game.
Selfies at steps (3), (6), and (9) serve a dual purpose: by providing a picture
of the whole team, they serve as proof of the entire team reaching the assigned
location (even if verification is not automated and must be done by the organizers).
Secondarily, selfies are shown to the other teams (this is achieved through a public
Telegram channel, as described in Section 2.3). All players thus automatically
receive a status update from other teams progressing in the game. This expedient
makes the game more engaging, provides a nice “feel” for the game’s progress,
and also pressures players to keep up with other teams. Selfies can be decorated
by the bot with a themed overlay, thus transforming all pictures of team members
in participation badges, that can easily be shared as trophies on social media.
2.1 Physical locations and deep-links
During the game, player teams are required to reach a sequence of different
locations. While target locations are plainly indicated by the bot on a map, they
are physically represented by a “token”, which is hidden around the indicated
spot.
These “token” objects—physical items bound to a geographical position, in
game terms—make use of the deep-linking feature provided by Telegram. Deep-
links are standard URLs that provide a direct pointer to a chat conversation

Fig. 2.
Seamless interface transition from physical QR Code to bot conversation inside
the Telegram client.
with a bot. These direct pointer URLs have one single text-encoded parameter,
which can include any arbitrary data to be sent to the bot. Data can be used to
identify a point in time, an object, contextual information and, in this case, a
geographical location [12]. Deep-link URLs can be used like traditional hypertext
links but—in the context of the game—they are encoded as QR Codes, printed
on paper or other physical supports, and placed for players to find.
Deep-linking makes use of several operating system level features on most
mobile OS platforms in order to provide a seamless experience. As shown in
Figure 2, (a) the deep-link URL is scanned using a QR Code scanner, (b) the
URL is opened in the default browser, which provides a landing page for users
without Telegram client or launches the Telegram client app through a custom
URL scheme, (c) the Telegram client opens up a conversation to the bot and
sends the deep-link’s data payload to the bot.
2.2 Coding puzzles
While the basic game framework can be easily adapted to any kind of puzzles
and riddles, all public “Code Hunting Games” events included puzzles especially
focused on coding and programming. In particular, the puzzles were mainly
inspired by grid movement games and block-based programming languages (as
seen on Code.org, “Hour of Code”, or Scratch). Most puzzles also make use of
the color-coded iconography established by the “CodyRoby” card game [2].
Puzzles are always represented by an image along with a written question.
Answers by players must be provided in simple text form, such as a sequence of
text-coded actions, a color, a number, or similar simple formats, which allow the
bot to evaluate the answer’s correctness using simple text matching.

2.3 System composition
The overall setup of a game session, shown in Figure 3, revolves around a central
server, which powers the game’s bot and interacts with Telegram’s bot back-end
in order to exchange messages with participants. Each team of participants elects
a “team leader”, who makes use of a smartphone with a configured Telegram
client and a QR Code scanner application.
Team leaders engage in separate one-on-one conversations with the bot. Each
conversation keeps a history of the team’s progress in the game. Interactions with
the bot usually start with the team leader scanning a QR Code upon arriving
at a location—steps (2), (5), and (8) in the previous game states list—and then
engaging in short text-based chats in order to get the next puzzle, solve it, and
receive the next location to reach.
Fig. 3. Configuration of a game instance.
The bot server also automatically publishes noteworthy events to a “public
channel” (i.e., a Telegram conversation to which any player can subscribe to, but
only selected users and the bot can write). All team leaders (and possibly other
team members) are invited to subscribe to the channel before the start of the
game.
Updates about teams reaching the next location or getting close to the last
puzzle are published to the channel in order to provide constant updates to other
participants. Also, selfies shot by the teams and sent to the bot—steps (3), (6),
and (9)—are forwarded to the channel, which makes the game more entertaining.
2.4 Hardware and software implementation
The game’s bot is powered by a single server, with a dual-core 2.20 GHz CPU
and 4 GiBs of RAM, running on-premises. In terms of software, the bot has
been developed as a PHP web application, based on a custom Telegram bot
framework
1
. The runtime environment is based on PHP 7, Nginx, and MySQL.
1Source code available at: https://github.com/CodeMOOC/TreasureHuntBot

2.5 Use-cases
The basic game outlined previously can be adapted to a number of different
scenarios. For instance:
1.
In-presence event: the game is prepared for a given event, players register
and play the game under the supervision of the organizers. Player teams
compete against each other to complete the game as quickly as possible.
2.
Offline installation: a persistent game that is installed once and can be played
multiple times. Location codes are installed in fixed locations and persist
from one game to the next. Players can register and start playing on demand,
competing against their previous runs or try to beat the times of previous
teams, which can be kept in an online leaderboard.
3.
City guide: the game is installed permanently and each location code is placed
at a city landmark. The registration and start steps described in Section 2
are skipped and the game is automatically started as soon as a location
code is scanned. The game can be further customized in order to provide an
engaging and interactive visiting experience for tourist groups (for instance,
by providing tourist information or guiding users to specific landmarks).
3 Deployment of the system
The game was deployed in several occasions for small and large-scale events.
Most events were organized in the context of “CodeMOOC”, a coding-oriented
Massive Open Online Course (MOOC) held at the University of Urbino.
1. CodeMOOC Summer School:
two one-week summer schools
2
were or-
ganized at the University of Urbino, in 2016 and 2017 respectively. Both
summer schools had the aim of providing Italian teachers from primary and
secondary schools with the tools to adopt coding in their classes. A “Code
Hunting Game” event was proposed as an engaging evening pastime for
all 150 registrants and their families, split up into 30 competing teams. A
montage of all pictures shot during the game is shown in Figure 4.
2. CodyTrip for schools:
these one-day field trips were organized in order to
allow entire primary school classes to devote a day to immersive coding expe-
riences, among which a session of “Code Hunting Games”. Three CodyTrip
events have been organized in 2017, with a total of over 900 schoolchildren
attending. The third event saw 35 teams competing, with a total of circa
400 participants.
3. EU CodeWeek:
a European grassroots initiative that promotes coding
and computational thinking at every level, through fun and engaging events,
based on the contribution of volunteers, country-wide “ambassadors”, and
the European Commission (http://codeweek.eu).
2
“Coding in your Classroom, now!” 22–27 August 2016 and “Making Apps in your
Classroom, now!” 21–26 August 2017.

On 14 October 2017 an international Europe-wide “Code Hunting Game”
event was conducted, with multiple game sessions running in parallel. All
game sessions, locally organized by national ambassadors and volunteers,
started at 11:00 CET and featured the same puzzles, the same number of
locations, and the same final puzzle. A total of 22 game sessions in Italy,
Hungary, and Slovakia were played, with 167 teams comprising more than
900 individual participants (3–9 players per team on average). Additional
13 game sessions with 97 teams and more than 500 participants were played
after the event3.
Fig. 4.
Montage of team selfies during the CodeMOOC Summer School 2017 treasure
hunt.
3.1 User feedback from CodeWeek 2017
During the last deployment presented above, a final questionnaire was presented
to all teams successfully completing the game. The questionnaire was composed
of 4 questions: 3 statements which the users could rate on a Likert scale and
an open question which users could answer with suggestions and comments as
free-form text.
In total, 66 answers to the questionnaire were collected. Distributions of the
answers to the Likert scale questions are shown in Figure 5. Evaluations of the
game and the technological components were generally positive. Users appeared
to be generally confident in using the bot and the QR Code system.
Free-form suggestions by 5 users mentioned that the puzzles were too easy,
that puzzles should show more variety, or that multiple difficulty levels should
be provided. Two users mentioned problems with cellular data access during the
game, while other two users described issues in using or navigating the maps
inside the Telegram client. Other comments generally described the game as
being entertaining and engaging.
3
Full account of the CodeWeek event: http://blog.codeweek.eu/post/166975540609/the-
codehunting-game-more-than-1000-people

0
10
20
30
40
50
(a) Game (b) Telegram bot (c) QR Codes
Strongly disagree
Disagree
Neutral
Agree
Strongly agree
Fig. 5.
Answer distributions to the following statements: (a) “I liked the treasure hunt
game.” (b) “The Telegram bot was easy to use and well-suited for the game.” (c) “I
understand how QR Codes work and how they interact with the bot.”
4 Conclusions and lessons learned
In this paper, we presented an instant messaging bot-based multiplayer game
that was deployed to handle large-scale treasure hunts. The puzzles provided by
the bot can be adapted to different use-cases, including educational, cultural,
or touristic applications. The system is available both as open-source code and
as an online service that can be used to create custom gaming sessions
4
. It is
currently available in 6 languages.
4.1 Lessons learned in bot development
Three key lessons learned while developing and deploying the system for over
one year are elucidated here.
Conversational UI is effective, but users generally do not read. One of the main
advantages of using the conversational interface of IM systems is that the basic
UI paradigms of messaging are well known, both by novice and expert users [13].
In the many interactions with players, we found that using the bot had a gentle
learning curve and most players were able to proceed quickly in the game after
a short period of hesitation. However—as previous literature on other kinds of
user interfaces confirms as well—we found that in many cases users tend not
to read any text at all [17]. This may be worsened by the general verbosity of
conversational interfaces, by the lack of recognizable UI elements in text messages
(e.g., colors, modal dialogs, etc.), and by the perceived uninterestingness of the
conversation due to the repetitive structure of the game. In many cases, the
4Official website: http://codehunting.games

effectiveness of messages can be improved using terser language and text emojis
to convey tone or status (e.g., a flashing light or a ‘stop’ sign for errors).
Effectiveness of deep-link interactions. While the deep-link mechanism described
in Section 2.1 is reliable and well supported in most mobile operating systems,
it often adds confusion to less experienced users, who may find the process
unfamiliar. The impracticality of switching between QR Code scanner app and
the Telegram client can often be worsened by less than optimal scanner app
interfaces (i.e., apps with invasive ads or clunky UI). This might be mitigated on
IM platforms whose applications feature a built-in QR Code scanner, such as
WeChat.
Low technical requirements but many failure points. The development of a bot-
based multiplayer game is comparatively easy thanks to the availability of
many resources already provided by the IM platforms. For instance, major
infrastructural elements (load balancing, notifications, user management, etc.) are
already built-in for bot developers [13]. On the other hand, chatbot applications
rely on many external components over which developers have little or no control:
Internet access, DNS, the IM platform’s back-end and mobile clients, QR Code
scanners, and more. Each component represents an additional potential failure
point and can cause issues.
4.2 Future work
When the development of Code Hunting Games started, Telegram was picked
as the target chatbot platform for technical reasons. In particular, the ability
of using deep-links and sharing geographical locations (from and to the bot).
Over the course of the past year, other IM platforms have also introduced these
features, thus making a multi-platform version of the bot feasible.
With regards to the perceived monotonous nature of the bot’s conversation de-
scribed previously, natural language processing, scripts, or storytelling approaches
will be considered in making the interaction with the bot more variable.
Acknowledgments
We wish to thank all members of the CodeMOOC community and all participants
that have tested “Code Hunting Games” since the first run in 2016.
We also wish to express our gratitude to the anonymous reviewers of this
paper for their valuable input.
Icons: server by Denis Shumaylov, smartphone by James Fenton, megaphone
by Christopher Reyes in figures 2 and 3 released under Creative Commons 3.0
on The Noun Project.

References
1.
Battestini, A., Setlur, V., Sohn, T.: A large scale study of text-messaging use. In:
Proceedings of the 12th international conference on Human computer interaction
with mobile devices and services. pp. 229–238. ACM (2010)
2.
Bogliolo, A.: Unplugged language-neutral card games as an inclusive instrument to
develop computational thinking skills. In: INTED2015 Proceedings (9th Interna-
tional Technology, Education and Development Conference). IATED (2015)
3.
Boiano, S., Cuomo, P., Gaia, G.: Real-time messaging platforms for storytelling
and gamification in museums: A case history in Milan. In: EVA (2016)
4.
Business Insider Intelligence: Messaging apps are now bigger than social networks.
Tech. rep. (2016), http://www.businessinsider.com/the-messaging-app-report-2015-
11
5.
Chaniago, M.B., Junaidi, A.: Student presence using RFID and Telegram messenger
application. 8th Widyatama International Seminar on Sustainability (WISS 2016),
Widyatama University and IEEE (2016)
6.
Christakopoulou, K., Radlinski, F., Hofmann, K.: Towards conversational recom-
mender systems. In: KDD. pp. 815–824 (2016)
7.
Church, K., de Oliveira, R.: What’s up with WhatsApp?: comparing mobile instant
messaging behaviors with traditional SMS. In: Proceedings of the 15th international
conference on Human-computer interaction with mobile devices and services. pp.
352–361. ACM (2013)
8.
Dingler, T., Pielot, M.: I’ll be there for you: Quantifying attentiveness towards
mobile messaging. In: Proceedings of the 17th International Conference on Human-
Computer Interaction with Mobile Devices and Services. pp. 1–5. ACM (2015)
9.
Fischer, M., Lam, M.: From books to bots: Using medical literature to create a
chat bot. In: Proceedings of the First Workshop on IoT-enabled Healthcare and
Wellness Technologies and Systems. pp. 23–28. ACM (2016)
10.
Graf, B., Kr¨uger, M., M¨uller, F., Ruhland, A., Zech, A.: Nombot: simplify food
tracking. In: Proceedings of the 14th International Conference on Mobile and
Ubiquitous Multimedia. pp. 360–363. ACM (2015)
11.
Junco, R., Cotten, S.R.: Perceived academic effects of instant messaging use.
Computers & Education 56(2), 370–378 (2011)
12.
Klopfenstein, L.C., Bogliolo, A.: The quiz-master bot: a persistent augmented quiz
delivered through online messaging. In: INTED2017 Proceedings (11th International
Technology, Education and Development Conference). pp. 9806–9811. IATED (2017)
13.
Klopfenstein, L.C., Delpriori, S., Malatini, S., Bogliolo, A.: The rise of bots: A
survey of conversational interfaces, patterns, and paradigms. In: Proceedings of the
2017 Conference on Designing Interactive Systems. pp. 555–565. ACM (2017)
14.
Kucherbaev, P., Psyllidis, A., Bozzon, A.: Chatbots as conversational recommender
systems in urban contexts. arXiv preprint arXiv:1706.10076 (2017)
15.
McTear, M.F.: The rise of the conversational interface: A new kid on the block? In:
International Workshop on Future and Emerging Trends in Language Technology.
pp. 38–49. Springer (2016)
16.
Mobile Ecosystem Forum: Mobile messaging report 2016. Tech. rep. (2016),
https://mobileecosystemforum.com/mobile-messaging-report-2016
17.
Nielsen, J.: Designing web usability: The practice of simplicity. New Riders Pub-
lishing (1999)
18.
Ramesh, K., Ravishankaran, S., Joshi, A., Chandrasekaran, K.: A survey of design
techniques for conversational agents. In: International Conference on Information,
Communication and Computing Technology. pp. 336–350. Springer (2017)

19.
Serban, I.V., Sordoni, A., Lowe, R., Charlin, L., Pineau, J., Courville, A.C., Bengio,
Y.: A hierarchical latent variable encoder-decoder model for generating dialogues.
In: AAAI. pp. 3295–3301 (2017)
20.
Shawar, B.A., Atwell, E.: Chatbots: are they really useful? In: LDV Forum. vol. 22,
pp. 29–49 (2007)
21.
Turing, A.M.: Computing machinery and intelligence. Mind 59(236), 433–460 (1950)
22.
Weizenbaum, J.: ELIZA—a computer program for the study of natural language
communication between man and machine. Communications of the ACM 9(1),
36–45 (1966)