Bringing An Unplugged Coding Card Game To Augmented Reality

Abstract

The recent spread of coding literacy initiatives, such as Hour of Code, Europe Code Week, or Africa Code Week, have underlined the growing importance and interest for computational thinking, often channeled through the use of innovative teaching tools, which foster creativity, collaboration, and interactivity.

Learning coding notions is generally tied to the use of computers or other electronic devices, and most recent educational tools are based on online visual programming platforms, which may lead to discrimination because of the digital divide, the lack of sufficient infrastructure, or cultural and linguistic barriers.

However, many code learning activities can be performed in an “unplugged” scenario, often with as little as a pencil and some paper. In fact, CodyRoby is an example of a do-it-yourself unplugged programming kit, published in the end of 2014. Through the use of color-coded cards, inspired by the building blocks of visual programming tools, and the use of intuitive symbols instead of words, the kit enables various fully inclusive coding experiences.

In this work we present a smartphone-based augmented reality system that empowers this simple tool and transforms a CodyRoby session into an immersive experience. A printable additional kit of markers allow a smartphone app to detect game components, such as the chessboard on which to play, and to present additional gaming elements on screen or to draw customizable decorative elements to stimulate engagement and creativity, especially in younger players. Several different game modes are presented and discussed. The suitability of the system to intimate, small-scale, or even large-scale coding events is also discussed.

Full text

BRINGING AN UNPLUGGED CODING CARD GAME TO
AUGMENTED REALITY
Lorenz Cuno Klopfenstein, Andriy Fedosyeyev, Alessandro Bogliolo
University of Urbino, DiSPeA (ITALY)
cuno.klopfenstein@uniurb.it, fedosev@gmail.com, alessandro.bogliolo@uniurb.it
Abstract
The recent spread of coding literacy initiatives, such as Hour of Code, Europe Code Week, or Africa
Code Week, have underlined the growing importance and interest for computational thinking, often
channeled through the use of innovative teaching tools, which foster creativity, collaboration, and
interactivity.
Learning coding notions is generally tied to the use of computers or other electronic devices, and most
recent educational tools are based on online visual programming platforms, which may lead to
discrimination because of the digital divide, the lack of sufficient infrastructure, or cultural and linguistic
barriers.
However, many code learning activities can be performed in an “unplugged” scenario, often with as little
as a pencil and some paper. In fact, CodyRoby is an example of a do-it-yourself unplugged programming
kit, published in the end of 2014. Through the use of color-coded cards, inspired by the building blocks
of visual programming tools, and the use of intuitive symbols instead of words, the kit enables various
fully inclusive coding experiences.
In this work we present a smartphone-based augmented reality system that empowers this simple tool
and transforms a CodyRoby session into an immersive experience. A printable additional kit of markers
allows a smartphone app to detect game components, such as the chessboard on which to play, and to
present additional gaming elements on screen or to draw customizable decorative elements to stimulate
engagement and creativity, especially in younger players. Several different game modes are presented
and discussed. The suitability of the system to intimate, small-scale, or even large-scale coding events
is also discussed.
Keywords: Card game, Unplugged activity, Coding, Computational thinking, Augmented reality.
1 INTRODUCTION
Over past years, new curricula and learning initiatives targeting the field of computer science have been
promoted with growing urgency. Computational tools are increasingly ubiquitous and indispensable. The
noticeable lack of core competencies and digital skills has been reported as being a threat to the
economy [1] and hindering adoption of innovative technologies. According to many reports, a large
percentage of population is digitally disadvantaged [2].
Code.org, Hours of Code, Europe Code Week, CoderDojo, and MotherCoders are only few among the
many large-scale coding literacy campaigns started in recent years—both in and outside of the
classroom—that stimulate the crucial skill of reasoning and communicating inside an increasingly digital
world [3, 4]. Many of the proposed tools also have an impact on learning geometry or other spatial
abilities [5], while “computational thinking” and the general understanding of technology has positive
consequences on any student’s working life [6]. The recent trend of education initiatives has increasingly
focused on early learning as well, starting from pre-K and primary education, as a long-term strategy to
foster interest for young children in the fields of science, engineering, technology, and
mathematics (STEM) [7].
The same consideration has been applied in equal measure to the under-representation of women in
ICT jobs or STEM fields in general [8], which is commonly ascribed to the poor awareness of computer
science as a discipline—perceived as being a male-dominated one, as well—and the perception of
related subjects as boring or not interesting [9]. Literature also links this lack of interest in the field to the
underestimation of their own technological skills. As argued by Vitores and Gil-Juárez, gender inequality
should be considered to be an ethical and political issue, which strongly impacts our society and daily
lives [10].

1.1 Unplugged activities and CodyRoby
Many existing coding initiatives are generally focused on innovative teaching tools. These tools are
preeminently accessed online and are based on block-based programming languages, which are
perceived as easier to use than text-based programming [11].
However, in our previous work we have argued that unplugged activities could further contribute to the
diffusion of computational thinking, by lowering the access barriers to the required teaching tools. The
proposed unplugged card game, “CodyRoby”, maintains the immediacy and effectiveness of Hour of
Code in teaching coding concepts, while being completely accessible for any age group, any level of
experience, or in regions affected by digital divide [12]. The game’s DIY starter kit can be easily printed
on paper and has been available since December 2014. The kit can be used freely, under a Creative
Commons copy-left license, with no need for further equipment [13].
CodyRoby is a game where the two aspects of programming meet: writing (Cody) and executing (Roby)
code. Writing code is achieved through the use of simple, color-coded, and clearly marked Cody cards.
The three basic cards shown in Figure 1 represent spatial movement instructions (turn left, move
forward, and turn right). An additional set of cards can be also used, representing conditional instructions
and loops.
Figure 1. The three basic Cody cards, part of the CodyRoby starter kit.
Proposed board games usually make use of a 5 × 5 or 8 × 8 chessboard, also printed on paper. The
layout makes it possible to reuse all Hour of Code puzzles in an “unplugged” fashion, play other
CodyRoby challenges described on the official website
1
, or to conceive new games.
This approach of cost-effective and approachable educational games has been tested successfully in
primary schools without broadband access and in multi-cultural groups of asylum seekers without a
common language [12]. Also, large-scale events have been proposed making use of the same puzzle
schemes and card-based games in an immersive collaborative format [14].
1.2 Augmented reality
The term Augmented Reality (AR) has been defined in many diverse ways in literature. As argued by
Wu et~al., defining AR in a broad sense—such as the bridging of virtual and real worlds, creating an
enhanced reality—can be more beneficial to educators and designers because it implies that AR can
be implemented and designed through different technologies, with far-reaching innovative applications,
instead of pigeonholing the concept [15].
In fact, AR has been picked as one of the key emerging technologies for education over the next
years [16]: the possibility of mixing virtual and physical objects make it possible to visualize complex
spatial relationships, to synthetize and observe abstract concepts, or to experience phenomena that are
not possible or difficult in the real world. These aspects of AR have often been shown to be promising
in the field of science education [17]. Like many innovations, the educational value of AR is not tied to
the technology per se, but closely depends on the methods used in designing, implementing, and
integrating AR in the learning environment [18].
1
Official CodyRoby website: http://codeweek.it/cody-roby/

1.3 Contribution
In this work we propose an Augmented Reality system to transform CodyRoby, a fully unplugged activity,
into an immersive experience. In the spirit of making the game accessible to all ages and all
demographics, the AR system was developed using only widely-available equipment, such as an
Android smartphone.
2 PROPOSED SYSTEM
The proposed system, called “CodyRoby AR” is based on the same color-coded Cody cards of the DYI
kit and makes use of the same chessboard mechanic in order to propose simple coding games. In
addition, a special chessboard marker is provided that enables the AR experience. An Android
application is also provided, that can be installed on any recent Android device with a front camera.
The application requires Android 4.0.3. It is based on the Unity
2
game engine and makes use of Vuforia
3
in order to provide a real-time AR experience. The reference chessboard marker is shown in Figure 2,
with a real-time rendering of the virtual chessboard (a 5 × 5 grid) and a 3D model of Roby on top of it,
as seen through the smartphone screen. The smartphone can be freely moved, allowing the user to
examine the chessboard from all angles.
Figure 2. Rendering of Roby on top of the virtual
chessboard.
Figure 3. Settings screen.
The application provides a set of settings that allow the user to configure the AR experience, as shown
in Figure 3. In particular, the application can switch from front to rear camera and can flip the image in
case the device is mounted upside-down. A persistent mode can be enabled, lowering the computational
impact of image tracking, in case the device and the marker stay stationary.
Figure 4. The augmented chessboard and Roby, playing a game through the use of Cody cards.
2
Cross-platform game engine. https://unity3d.com
3
Augmented Reality Software Development Kit through image and object tracking in real-time. https://www.vuforia.com

Moreover, the virtual chessboard can be freely scaled and moved by an offset relative to the physical
chessboard marker. This allows users of the application to show the virtual grid in other positions,
possibly where the marker itself cannot be positioned. For instance, in Figure 4 the augmented
chessboard has a larger scale than on Figure 2 and has been moved towards the wall. This ensures
that the area occupied by the virtual chessboard is free of other physical objects—including the
chessboard marker—and thus allows players and actors to freely access the area.
The virtual Roby can be programmed using the color-coded Cody cards shown on screen. By touching
the cards, the rendered Roby actor moves across the virtual chessboard. Grid cells are lighted according
to their status: green when Roby is standing on them, gray if Roby has stood on them at least once, and
transparent otherwise. Also, grid cells can be grayed out manually, by tapping on the rendered cell on
the device’s smartphone. Grayed out grid cells can be used to represent unreachable areas or cells
occupied by an obstacle.
Figure 5. A live CodyRoby AR session where 3 children play with a virtual representation of Roby.
The augmented reality application has been successfully applied to a large-scale public event on 20
January 2017 with a participative coding session, based on the event format we proposed
precedingly [14]. Among the possible gameplay variations that can be adopted:
• Collaborative coding: just like in the immersive coding event format, this game is structured in
three steps of participative maze design, coding (using Cody cards), and code execution (with
the aid of the virtual Roby actor and/or additional children from the public);
• Snake: each player gets a hand of 3 random Cody cards. In turn they must play a card to move
the robot, which will perform the selected movement and gray out cells as it steps on them. A
player loses as soon as he or she is forced to move Roby against a wall or a gray cell.
• Duel: a target cell is picked at random. Each player must compose a sequence of instructions
using the cards. The first player laying out a valid sequence to reach the target, wins the game.
In alternative, the player coding a valid sequence using the least number of cards wins.
3 FUTURE WORK
After the initial beta-testing and the public demonstration session, CodyRoby AR is planned to be
released on Google Play for Android. Also, thanks to the multi-platform nature of the systems adopted,
CodyRoby AR will also be ported to iOS and Windows 10 (UWP) devices. Moreover, the application will
allow users to select preset or custom puzzle schemes, with virtual obstacles, along the lines of Hour of
Code puzzles.
As argued by Wu et al., in order to provide more evidence of the educational values provided by tools
and experiences enhanced by AR, it would be advisable to carry out comprehensive studies on larger
population samples. However, AR studies are usually applied on smaller user groups [18].

To this end, the final product will be made available to the CodeMOOC community, a group of more
than 10.000 Italian teachers interested in computational thinking and coding in the classroom, most of
whom are already familiar with CodyRoby and have successfully integrated the method in their teaching
activities. We look forward to conducting studies on the effective benefits of these tools over traditional
methods, when applied in private learning sessions, classroom teaching, or other large-scale events.
4 CONCLUSIONS
Core concepts of “computational thinking”, such as algorithmic thinking, problem decomposition,
evaluation, and abstract thinking, have been shown to be fundamental skills that can be applied to all
disciplines [19]. In our previous work we have presented CodyRoby, a method for conducting low-cost
unplugged activities that overcomes barriers typically tied to the access to online coding resources.
In this work we have introduced a similarly cost-effective, approachable, and accessible augmented
reality system, which makes use of a simple smartphone as an augmented sensor to transform a fully
unplugged coding game into an immersive coding experience.
During preliminary testing and a large-scale live event, where the proposed system was presented,
participants did appreciate the “mixed reality” view on the conventional Hour of Code puzzles. It was
observed that the effects of written code and the steps of its execution can be understood much more
effectively through a direct visual representation. Likewise, in previous studies, the use of AR solutions
was indeed found to be well suited for the study of unobservable scientific phenomena in particular [17].
The CodyRoby AR experiment is a renewed compromise between the use of new and innovative
technologies and our efforts in reducing the requirements—in terms of costly equipment and
infrastructure—to gain access to effective tools to teach and learn coding skills. A card-based game that
can be printed at home is an extreme choice in terms of simplicity and accessibility. CodyRoby AR
however adds very little in terms of requirements, while profoundly enrichening the unplugged learning
experience.
ACKNOWLEDGMENTS
The authors wish to thank all members of the CodeMOOC community and all participants taking part in
the presentation event of CodyRoby AR. Thanks to Donatella Garavella for the picture in Figure 5.
REFERENCES
[1]
B. Southworth, "A UK digital skills gap looms: A failure to address the shortfall in digital skills at an
educational level would be a threat to the economy," The Guardian Media Network Blog, 18 August
2014.
[2]
R. Cellan-Jones, "More than 12 million fall into UK digital skills gap," BBC News, 19 October 2015.
[3]
Y. B. Kafai, Q. Burke and M. Resnick, Connected code: Why children need to learn programming,
MIT Press, 2014.
[4]
H. Partovi, "Transforming US education with computer science," in Proceedings of the 45th ACM
technical symposium on Computer science education, 2014.
[5]
M. Sabatini, "Learning geometry and visuo-spatial abilities through coding," Form@re-Open
Journal per la formazione in rete, vol. 16, no. 1, pp. 201-212, 2016.
[6]
C. Duncan, T. Bell and S. Tanimoto, "Should Your 8-year-old Learn Coding?," in Proceedings of
the 9th Workshop in Primary and Secondary Computing Education, Berlin, Germany, 2014.
[7]
L. Warner, S. L. Gates, C. W and M. Kieman, "The skills gap: Reversing Washington’s lack of
skilled workers through early learning," America's Edge, Washington, DC, 2011.

[8]
C. Wilson, "Hour of code: we can solve the diversity problem in computer science," ACM Inroads,
vol. 5, no. 4, p. 22, 2014.
[9]
S. Banerjee and R. R. Santa Maria, "A Study of Students' Perception of Computer Education: Lack
of Interest in STEM Fields for Female Students," International Journal of Technology, Knowledge
& Society, vol. 8, no. 4, 2012.
[10]
A. Vitores and A. Gil-Juárez, "The trouble with ‘women in computing’: a critical examination of the
deployment of research on the gender gap in computer science," Journal of Gender Studies, vol.
25, no. 6, pp. 666-680, 2016.
[11]
D. Weintrop and U. Wilensky, "To Block or Not to Block, That is the Question: Students'
Perceptions of Blocks-based Programming," in Proceedings of the 14th International Conference
on Interaction Design and Children, Boston, Massachusetts, 2015.
[12]
A. Bogliolo, "Unplugged language-neutral card games as an inclusive instrument to develop
computational thinking skills," in INTED2015 Proceedings, Madrid, Spain, 2015.
[13]
A. Bogliolo, "CodyRoby: Turning the Hour of Code into a DIY Unplugged Game," ICT in Practice,
pp. 8-16, 16 January 2015.
[14]
A. Bogliolo, S. Delpriori, L. C. Klopfenstein and B. D. Paolini, "Immersive Coding: Innovative Tools
and Formats for large-scale Coding Events," in EDULEARN16 Proceedings, Barcelona, Spain,
2016.
[15]
E. Klopfer, Augmented learning: Research and design of mobile educational games, MIT Press,
2008.
[16]
L. F. Johnson, A. Levine, R. S. Smith and K. Haywood, "Key emerging technologies for elementary
and secondary education," The Education Digest, vol. 76, no. 1, p. 36, 2010.
[17]
L. Kerawalla, R. Luckin, S. Seljeflot and A. Woolard, "“Making it real”: exploring the potential of
augmented reality for teaching primary school science," Virtual Reality, vol. 10, no. 3-4, pp. 163-
174, 2006.
[18]
H.-K. Wu, S. W.-Y. Lee, H.-Y. Chang and J.-C. Liang, "Current status, opportunities and challenges
of augmented reality in education," Computers & Education, vol. 62, pp. 41-49, 2013.
[19]
E. Shein, "Should everybody learn to code?," Communications of the ACM, vol. 57, no. 2, pp. 16-
18, 2014.