Content uploaded by Lorenz Cuno Klopfenstein
Author content
All content in this area was uploaded by Lorenz Cuno Klopfenstein on May 25, 2017
Content may be subject to copyright.
THE QUIZ-MASTER BOT: A PERSISTENT AUGMENTED QUIZ
DELIVERED THROUGH ONLINE MESSAGING
Lorenz Cuno Klopfenstein, Alessandro Bogliolo
University of Urbino, DiSPeA (ITALY)
cuno.klopfenstein@uniurb.it, alessandro.bogliolo@uniurb.it
Abstract
Chatterbots, or bots, have recently enjoyed a dramatic comeback: these automated agents,
communicating with users through the exchange of simple text messages, have overtaken most online
messaging platforms and are increasingly used to provide means to access information or to make use
of services. Because of the pervasive popularity of messaging platforms, as they represent the main
driver in smartphone usage across all demographics, bots represent an attractive development platform
with direct access to a large number of users and a very low access barrier.
Many bot platforms allow the creation of special hyperlinks that invoke a particular dialogue with the bot.
These hyperlinks can be presented to users digitally as URIs (Uniform Resource Identifiers) or visually
as QR Codes that can be embedded in digital media or bound to a physical object or a geographical
location. In both cases, hyperlinks may carry additional information, providing valuable context, while
enabling a seamless transition to the conversation with the bot.
In this work, we describe a system that makes use of an automated data collection bot that delivers quiz
questions to users. Hyperlinks are used to determine the question asked, for instance ensuring their
sequence or tying them to a specific document. The system can work both synchronously during live
events and asynchronously through persistent links.
We also present two large-scale events during which this system has been used to coördinate online
educational coding quizzes and describe the implemented system in detail, illustrating its effectiveness
and its gaming mechanics, and also discussing strengths and weaknesses of the proposed system. We
argue that the bot-aided coding quiz and its game mechanics could be applied profitably to many other
educational events or data collection tasks.
Keywords: Bot, Conversational interfaces, Automated system, Quiz, Instant messaging.
1 INTRODUCTION
The ubiquitous nature of Internet access and mobile devices has rapidly revolutionized many aspects
of everyday life over the course of the last decade. From work to entertainment, including commerce
and healthcare, we expect smartphones to provide immediate access to software or services capable
of serving our needs. While the impact of these developments is still hard to assess, there have been
many studies and initiatives aimed at applying these new and evolving technologies to learning.
1.1 Learning through mobile devices
It has been argued that learning experiences enhanced by mobile technologies can lead to an increase
of motivation and commitment in participants, if compared with more traditional learning activities.
Mobile technologies also are a fertile ground for innovative blends of learning, gaming, and other
engaging activities. For instance, in a study by Facer et al., a mobile game was designed, tying mobile
technologies to direct physical interaction with space and other players, in order to create a powerful
and engaging learning experience [1]. A similar system, connecting online participants to participants
on the streets, equipped with mobile devices, proposed an innovative “mixed reality” experience [2].
The impact of technology in learning has been argued to be particularly noticeable in collaborative
experiences. The educational benefits of learning with others may be difficult to assess in general.
However, in a study by Ryu et al., mobile technology was shown as being conductive to increased
collaboration opportunities in small groups of students, effectively fostering new learning activities. The
authors argued that the communication capabilities of mobile devices are the trigger that enables more
awareness of difficulties and achievements in each group, also leading to the development of unique
learning paths [3].
Digital communication in general, between groups of students and between students and teachers, has
become increasingly popular over the last years and has expanded to encompass many different
channels. Messaging, e-mail, Facebook, and Instant Messaging (IM) are widely used tools across all
demographics, with different characteristics that influence their suitability as learning tools [4].
1.2 Instant messaging as a learning platform
A recent study by MEF, on nearly 6.000 respondents across 9 countries, clearly shows the impact of IM
systems. Even if the study confirms huge variation in what kind of services are used in different
countries, the messaging space is consolidating as one of the—if not the—most important selling point
and use-case of mobile devices [5].
Autonomous learning and schoolwork collaboration, in addition to socializing and event planning, are
activities that have long been strictly associated to the use of SMS or IM tools [6]. In a study by Grinter
and Palen, IM usage appears to be particularly valued by older teenagers wanting to discuss school
matters or to coördinate with friends in order to ultimately improve grades. In some cases, IM is also
used by teachers as a teaching tool [7]. In general, IM has been shown to stimulate collaborative
learning processes, leading to more cognitive interactions [8]. In a study by Marçal et al. the introduction
of messaging was also found to improve student results and effectiveness, even in economically
disadvantaged regions with poor access to broadband Internet [9].
Among their many advantages, including low-cost, simplicity, accessibility, efficiency, and the use of
natural language, IM systems have the clear advantage of already being tools that are used in the daily
lives of both teachers and students alike [10]. IM-based teaching tools can thus be adopted with virtually
no training and no special supervision. IM has arrived in the classroom through its use in private life
and, thanks to the many advantages, lends itself naturally to be adopted as an educational
technology [11].
1.3 Chatbots in online messaging platforms
Chatbots, autonomous agents with a conversational text interface, have been studied since long [12].
They can in fact be traced back to Alan Turing’s “Imitation Game” (better known as the “Turing test”),
which aims to determine whether a machine can give the impression of being human to a human judge,
just by exchanging simple text messages [13]. ELIZA, a chatbot that pretends to be a psychotherapist
giving evasive answers [14], and ALICE, a bot intelligence based on pattern matching expressed
through a declarative language [15] count among the many seminal attempts in this area.
In the last years, IM platforms have started to introduce the concept of “bots” as well: in this case the
term is applied to enhanced conversational agents that can receive requests and chat with users, right
inside the messaging application itself. Many IM services, such as Kik, Telegram, and WeChat, have
opened up bot programming interfaces to third-party developers. In 2016 Facebook’s Messenger and
Skype have also followed suit. Thanks to the popularity of instant messaging, bots now represent an
interesting platform with a large potential audience and a low access barrier for both users and
developers.
1.4 Contribution
In this work we propose an software system for the management of a large-scale data collection tool for
any number of users. Data collection is carried out through an instant messaging conversation with the
user, which follows a “question and answer” format. Questions proposed by the system can be delivered
in real-time or they can be set in advance and persistent. We present a game-like application, in the
form of a coding quiz, that demonstrates the suitability of the system as an augmented form of
participation to a live video streaming event.
2 PROPOSED SYSTEM
The proposed system is an automated bot-based system that is capable of interacting with its users
through instant messaging conversations in textual form. Conversations are carried out following a strict
“question and answer” format: the user initiates conversation by providing a special code, which uniquely
identifies the question the user intends to answer. The bot provides the question and expects the
answer. One the user replies, the answer is evaluated and collected permanently.
Unique codes that identify the questions can be bound to a live event, a physical object, or a
geographical location, in what can be thought of a mixed reality persistent quiz and data collection
system. The mechanism with which questions are linked to codes is detailed in the next section.
The system has been implemented as a bot making use of the Telegram IM platform. It was firstly
deployed in October 2016, using a quiz game mechanic specified in Section 3, during an event
described in Section 4.
2.1 Deep Linking
The mechanism that allows the bot to associate moments in a live event or geographical locations to
specific questions is called “deep linking”. Deep linking makes use of several operating system level
feature in order to seamlessly start a conversation with a specific bot and to pass in an optional
parameter.
In Telegram a “deep link” takes the form an URL, pointing to a specific Telegram domain
1
. URLs include
the public username of the bot they are linking to. The additional parameter (the uniquely identifying
code of the question) to be sent to the bot is also encoded in the query string inside the URL.
The URL can be used directly (embedded in a web page or any other document that allows opening
hyperlinks) or it can be encoded as a QR Code. Deep link URLs thus encoded can be easily shown on
screen, embedded inside a live video, and printed on paper or any other physical support. QR Codes
can be scanned by any user using a common QR Code scanner on a mobile device. Figure 1a shows
a QR Code representing a deep link that is being scanned by a smartphone.
When the link is opened, by default the system browser follows the URL and displays the linked page,
which includes instructions on how to access the bot. If the Telegram client is installed, the web page
itself will invoke the client and open up the conversation with the bot, as shown in Figure 1c.
On some operating systems (Android, for instance) step b is unnecessary and, after scanning the
QR Code, the Telegram app opens directly without opening the web browser first.
Figure 1. Seamless transition from QR Code to bot conversation in Telegram client.
Once the Telegram application is launched, the app sends a hidden message to the bot. By convention,
the Telegram client will send the “/start” command. The transmission of this command gives the bot the
opportunity to react to its invocation, possibly by responding with another text message. Since the start
command is sent, but not shown in the user interface, the system gives the impression of entering a
conversation with the bot through the simple act of scanning a QR code.
If the QR code contains an additional parameter, this optional data is sent along the start command.
Additional parameters are always encoded as simple text and are simply appended to the “/start”
command. Data thus sent to the bot can be used to identify a point in time, an object, a location, or any
other contextual information that can be used in the conversation.
1
All Telegram deep links point to paths on the telegram.me domain.
3 THE QUIZ-MASTER GAME
The “Quiz-Master” bot is an application of the deep linking feature that allows a bot to ask contextual
questions in a persistent quiz session. Participation is possible for any user with a Telegram account.
The bot can work both in online (i.e., synchronized to a live event) and in offline mode (i.e., without any
synchronicity).
3.1 Online mode
In this scenario, the bot is deployed to perform a large-scale quiz tied to a live event, which can be
performed in person or via online streaming.
During the live event, quiz administrators can create new quiz questions. A unique code is generated
for each quiz question and a special QR Code, deep linking to the bot and including the unique code, is
generated by the bot. QR Codes can be shown in person, through a screen, or on an overlay during live
streaming. When quiz participants make use of the deep link, they are redirected to the bot conversation,
where they can input their answer to the question.
Quiz administrators may close a quiz question at any time, by providing the correct answer to the bot.
The bot checks all collected responses by users and verifies if they match the correct answer or not
(matches are detected using a simple and forgiving string comparison method that, for instance,
normalizes accented characters and ignores whitespace). The bot will then compute aggregated
statistics on how many correct answers were given and on the number of participants. Answers are
ranked based on correctness and timeliness, providing a ranking of users. These results can be shared
through the private user-bot conversation or using a public outlet (such as a Telegram channel
2
).
Once quiz questions are closed they can still be answered to, since quiz codes are persistent: users
may keep reaching the question through the appropriate deep link, they will be able to give their
response, and be evaluated by the bot. Their responses will not count towards the online statistics.
3.2 Offline mode
In this scenario, quiz questions are generated beforehand. For each question, the correct answer is
provided and a code is generated. QR Codes for each question can be applied to objects or locations,
according to the intended game.
Once started, users can scan QR Codes and receive quiz questions through the bot conversation.
Responses are evaluated immediately and cumulative statistics can be computed consecutively. The
bot can be setup in such a way to provide custom responses tied to specific questions.
4 CASE STUDY: CODEMOOC CODING QUIZ
In the context of CodeMOOC
3
, a large-scale coding quiz was planned for 20 October 2016, for the
ending days of the Europe Code Week 2016 initiative. Participants of the course and other interested
players were able to take part in the game and compete online with over 900 groups in Italy.
Participants were asked to register to a Telegram channel dedicated to the event and to follow a live
Youtube stream. During the live event, QR Codes to the quiz questions were dealt out through the
Youtube video and on the channel. On closing the questions, the results and rankings were published
on the channel.
Since the generated links are persistent, users can still access the recording of the live event on
Youtube
4
and participate to the quiz in offline mode.
A total number of 974 Telegram conversations were opened with the bot. Since each person could
register to be a team leader for a group of people, a total number of 9689 people were involved with the
game. 14 quiz questions were asked during the live event, collecting a total of 7004 responses over
approximately 80 minutes, exchanging a total of 7.414.458 messages.
2
See https://www.telegram.org/faq_channels.
3
A massive open online course offered by the University of Urbino about computational thinking and coding. http://codemooc.org
4
https://youtu.be/BwD2Q8DySWg
5 CONCLUSIONS
As discussed previously, many studies confirm the importance of bringing technology—in particular
connected and mobile devices—to the classroom, in order to stimulate communication and collaboration
between groups of students or even between students and teachers. The impact that mobile devices—
such as smartphones—can have is noteworthy because they are not a technology that must be
introduced: they are already being used in the classroom.
Instant messaging is a particularly effective technology because of its simplicity and accessibility. In fact,
based on the latest usage demographics, online messaging might be considered one of the most
inclusive technologies, which can easily be adopted even in regions with poor network connectivity.
In this work we proposed an online software system that, through the use of a Telegram bot, can be
used to deliver pre-recorded or live questions to a pool of users and collect the received answers.
In Section 3 we presented a game-like application of the system, used to deliver a large-scale coding
quiz to the audience of a live video streaming event. The feedback to the event has been largely positive,
thanks to the engagement of a large number of teachers and students. The quiz can still be used in
offline-mode, by watching the live stream recording. The application’s source code has been published
on Github under an open-source license: https://github.com/CodeMOOC/QuizzleBot.
We envision that bot-based applications such as the one presented could be used both in small and
large scale initiatives to deliver a form of “augmented” game in a very cost effective and engaging way.
Homework for a classroom of students can be delivered online and collected through the bot, ensuring
that all students turn in their work and fostering discussion through other IM-based channels. QR Codes
for a set of pre-recorded questions can be hidden in specific locations, sending out participants to find
the codes and thus engage in a quiz-based treasure hunt. Laboratory equipment can be tagged with
similar codes, for which the bot would deliver information and guidance. Bots and the seamless transition
from physical QR Codes to an IM conversation provide a wealth of possibilities in developing a “mixed
reality” quiz or game.
ACKNOWLEDGMENTS
The authors wish to thank all participants taking part in the CodeMOOC coding quiz of 20 October 2016.
Icons for Figure 1: Smartphone by Curve, Lens by mikicon, from the Noun Project.
REFERENCES
[1]
K. Facer, R. Joiner, D. Stanton, J. Reid, R. Hull and D. Kirk, "Savannah: mobile gaming and
learning?," Journal of Computer assisted learning, vol. 20, no. 6, pp. 399-409, 2004.
[2]
M. Flintham, S. Benford, R. Anastasi, T. Hemmings, A. Crabtree, C. Greenhalgh, N. Tandavanitj,
M. Adams and J. Row-Farr, "Where on-line meets on the streets: experiences with mobile mixed
reality games," in Proceedings of the SIGCHI conference on Human factors in computing systems,
2003.
[3]
H. Ryu and D. Parsons, "Risky business or sharing the load? - Social flow in collaborative mobile
learning," Computers & Education, vol. 58, no. 2, pp. 707-720, 2012.
[4]
R. Calvo, A. Arbiol and A. Iglesias, "Are all chats suitable for learning purposes? A study of the
required characteristics," Procedia Computer Science, vol. 27, pp. 251-260, 2014.
[5]
MEF Mobile Ecosystem Forum, "Global Insights into Chat Apps and SMS Usage," 2016.
[6]
A. Lenhart, L. Rainie and O. Lewis, "Teenage life online: The rise of the instant-message
generation and the Internet's impact on friendships and family relationships," 2001.
[7]
R. E. Grinter and L. Palen, "Instant messaging in teen life," in Proceedings of the 2002 ACM
conference on Computer supported cooperative work, 2002.
[8]
H. Kim, M. Y. Lee and M. Kim, "Effects of Mobile Instant Messaging on Collaborative Learning
Processes and Outcomes: The Case of South Korea," Educational Technology & Society, vol. 17,
no. 2, pp. 31-42, 2014.
[9]
E. Marçal, R. Andrade, R. Melo, W. Viana and E. Junqueira, "Using mobile message to improve
student participation in blended courses: a Brazilian case study," ACM SIGAPP Applied
Computing Review, vol. 16, no. 3, pp. 15-25, 2016.
[10]
K. Church and R. de Oliveira, "What’s up with WhatsApp? Comparing Mobile Instant Messaging
Behaviors with Traditional SMS," in Proceedings of the 15th international conference on Human-
computer interaction with mobile devices and services, 2013.
[11]
D. Bouhnik and M. Deshen, "WhatsApp goes to school: Mobile instant messaging between
teachers and students," Journal of Information Technology Education: Research, vol. 13, pp. 217-
231, 2014.
[12]
M. McTear, Z. Callejas and D. Griol, The Conversational Interface, Springer, 2016.
[13]
A. M. Turing, "Computing machinery and intelligence," Mind, vol. 59, no. 236, pp. 433-460, 1950.
[14]
J. Weizenbaum, "ELIZA—a computer program for the study of natural language communication
between man and machine," Communications of the ACM, vol. 9, no. 1, pp. 36-45, 1966.
[15]
R. S. Wallace, "The anatomy of ALICE," in Parsing the Turing Test, 2009.
