Designing Narrative Learning in the Digital Era

Abstract

This paper presents a first prototype of Mobeybou, a Digital Manipulative that uses physical blocks to interact with digital content. It intends to create an environment for promoting the development of language and narrative competences as well as digital literacy among pre and primary school children. Mobeybou offers a variety of characters, objects and landscapes from various cultures around the world and can be used for creating multicultural narratives. An interactive app developed for each country provides additional cultural and geographical information. A pilot study carried out with a group of 3rd graders showed that Mobeybou motivated and inspired them to actively and collaboratively create narratives integrating elements from the different countries. This may indicate Mobeybou's potential to promote multiculturalism.

Full text

Designing Narrative Learning in the
Digital Era
Cristina Sylla
Research Centre on Child Studies
University of Minho, Portugal
cristina.sylla@ciec.uminho.pt
Vítor Martins
Research Centre on Child Studies
University of Minho, Portugal
vitor@mobeybou.com
Gabriela Sá
Research Centre on Child Studies
University of Minho, Portugal
gabrielasa@mobeybou.com
Ana Paula Caruso
Polytechnic Institute of Cávado and Ave
Barcelos, Portugal
anapaulacaruso@mobeybou.com
Bruno Amaro
Research Centre on Child Studies
University of Minho, Portugal
bruno.amaro@mobeybou.com
Francisco Sylla
Research Centre on Child Studies
University of Minho, Portugal
sylla@mobeybou.com
Douglas Menegazzi
Department of Graphic Expression and Design
Federal University of Santa Catarina
douglas.menegazzi@ufsc.br
ABSTRACT
1
This paper presents a first prototype of Mobeybou, a Digital Manipulative that uses physical
blocks to interact with digital content. It intends to create an environment for promoting the
development of language and narrative competences as well as digital literacy among pre and
_____________________________
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee
provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the
full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact
the owner/author(s).
CHI’19 Extended Abstracts, May 4-9, 2019, Glasgow, Scotland, UK.
© 2019 Copyright is held by the author/owner(s).
ACM ISBN 978-1-4503-5971-9/19/05.
https://doi.org/10.1145/3290607.3312937
CHI 2019 Late-Breaking Work
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
LBW1412, Page 1

primary school children. Mobeybou offers a variety of characters, objects and landscapes from
various cultures around the world and can be used for creating multicultural narratives. An
interactive app developed for each country provides additional cultural and geographical
information. A pilot study carried out with a group of 3rd graders showed that Mobeybou motivated
and inspired them to actively and collaboratively create narratives integrating elements from the
different cultures. This may indicate Mobeybou’s potential to promote multiculturalism.
1 INTRODUCTION
Digital, discursive and multicultural competences are key dimensions in current international
frameworks for early years’ education [4], and an important part of educational curricula in various
countries. As technology users, children have particular needs that have to be considered when
developing technological products for them [7]. Digital Manipulatives (DMs), an interaction model
that employs objects to manipulate digital content [8], are especially appropriate for young children.
One of the major advantages of DMs is that they promote collaboration [5] and social interaction,
which are driving forces for language development.
Storytelling has been one of the domains targeted in the development of such systems [3]. Besides
playing a central role in children’s lives, stories are a powerful way of transmitting cultural values,
helping children to understand and to become part of the world [1, 2], and as such they provide the
ideal territory to foster multiculturalism, contributing to build a more inclusive and tolerant world.
2 MOBEYBOU – MOVING BEYOND BOUNDARIES
Mobeybou builds and extends previous work [9], but while the former focused on traditional stories
for children, Mobeybou provides elements for creating multicultural narratives, and at the same
time, it empowers children with diverse cultural backgrounds to create and share their own stories.
The manipulation of the digital content is done using physical blocks (see Figure 1 top), which
supports and promotes collaboration and exchange of ideas [9]. Each physical block (4,5 x 4,5 x
1,5cm) has a visual representation on the top and a digital identification (ID) on its base. Placing a
block on an electronic board triggers the corresponding virtual content on a device’s screen. The
visual narratives unfold according to the blocks that the children place on the board while they
verbalize their stories.
An interactive app tailored for each culture presents a story and information about it, e.g. location,
games involving local traditions, food or other elements that are representative for the culture (see
Figure 10). We envision that Mobeybou can be used by young children in formal and informal
educational contexts, individually, with peers or in group to promote the development of language
and narrative competences as well as digital literacy and multiculturalism.
2.1 Design and Development
Mobeybou follows a user-centered design approach. Besides being shaped by research on the use of
props to promote the creation of narratives [9, 10], studies on embodiment and tangible interaction
[5, 10] its design is also informed by sessions carried out with the final users. In these sessions, we
KEYWORDS
Digital Manipulatives; Tangible Interfaces;
Storytelling; Oral expression;
Multiculturalism; Collaboration.
Figure 1: The physical blocks (top); Children
interacting with a paper prototype (bottom).
CHI 2019 Late-Breaking Work
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
LBW1412, Page 2

use low-fidelity paper and functional prototypes (see Figure 1 bottom) to gather information on how
the children use tangible elements to collaboratively create narratives. For the first design session
we created different sets of paper cards representing elements from various cultures around the
world. Each set of cards contained a protagonist, an animal, a mythical creature, a landscape, an
object, and a musical instrument. There were also environment-cards, this is, cards representing rain,
snow, wind and the night. The session took place at school in the classroom with a class of 3rd graders
and their teacher. The children interacted with the prototype in groups of six. During the design
session we talked with the children about the different elements represented on the paper cards and
asked them if they would like to tell stories using the paper cards.
2.2 Observations
The children started using the paper cards like domino pieces, placing them together and creating
rows. It was interesting to observe how the children used the paper cards and to talk with them
about it. They identified the objects as magical, e.g., the Indian shoes could be used for floating; the
Chinese fan could create a strong wind; the guitar would make the characters dance, and could be
used to hit the antagonists on the head; the flute would enchant the characters, and the dragon
would spit fire and burn the other characters.
2.3 Description of the System
The session at school informed the design of the interactions between the elements. There are
various sets of blocks and each set represents a country. The design of each specific cultural set was
based on a careful investigation in order to identify characteristic elements of the culture, which
simultaneously have the potential to trigger the children’s imagination and creativity. Presently we
have completed sets for China, Brazil and India (see Figure 2, 3, 4). Each set comprises: one
landscape, two protagonists (boy and girl), an animal, an antagonist, a musical instrument and a
magical object (see Figure 5). The elements are divided into categories and behave according to a set
of rules that define their actions in relation to the other active elements.
The relations between the elements were developed following the traditional narrative model from
Western cultures [6]. This is, the antagonists attack the protagonists; the animals defend the latter
(see Figure 6, 7). The protagonists and the animals can join forces to defend themselves from the
antagonists, who can also join forces to attack the former. The musical instruments can be used by
the protagonists to make all the other “living” elements dance and become happy (see Figure 8). The
magical objects can be used to help the protagonists escape or to beat the antagonists (see Figure
9). The environment-blocks (rain, snow, wind, rainbow, night) allow to further customize the stories
(see Figure 8). All the elements from the different cultures can be mixed and matched to create
narratives, potentially promoting multiculturalism (Figure 8).
The physicality of the blocks supports and promotes agency, collaboration, and exchange of ideas
[8, 9, 10]. Based on the feedback from the teacher, we have integrated an audio recording function,
which allows the children to record their stories, and can be used, among other functions, to assess
the children’s learning curve along time.
Figure 2 – China’s cultural set.
Figure 3 – India’s cultural set.
Figure 4 – Brazil’s cultural set.
CHI 2019 Late-Breaking Work
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
LBW1412, Page 3

Inspired by the design session with the children, where they used the paper cards like domino pieces,
we are developing a new hardware where the blocks connect to each other through magnets and
communicate via I2C protocol. A master block receives the IDs of the connected blocks (clients), and
sends them to the computer /tablet via Bluetooth, triggering the corresponding digital content on a
device’s screen, always updating the elements present in the game. The master block has a battery
to power all the blocks and a Bluetooth module to communicate with the device. The electronic
circuit uses a peripheral interface controller (PIC) and magnets, which makes it possible to use small
physical blocks.
2.4 The Story App
Ongoing and future work includes the development of a story app for each culture, which informs
about the culture in a playful way, while simultaneously nourishing children’s creativity. The app
presents a story and information about the culture, e.g. countries mostly associated with each
culture and their location, games involving local traditions, food or other elements that belong and
inform about each specific culture (see Figure 10).
3 PILOT STUDY
Following the first development of the software we carried out a pilot study at school with 12
children from the same class of 3rd graders, the intervention took place in a separate room (the
gymnasium). The teacher divided the children in pairs, and one pair at a time came to the room to
play with Mobeybou (for around 20 minutes). The group was gender balanced with the following
composition: G 1 (boy/girl), G 2 (girl/girl), G 3 (boy/girl), G 4 (girl/girl), G 5 (boy/boy), G 6 (boy/boy).
The researchers stood in the background, observing and taking notes, a video camera set up behind
the children recorded the interactions. After each interaction, the researchers talked with each pair
about their experience.
3.1 Observations
The children started by exploring the different blocks and the interactions among them, placing and
removing the blocks from the board, experimenting different combinations. They often commented
on what they were doing, talked about the blocks and the visualizations shown on the screen. The
activity was collaborative, this seems to be also promoted by the blocks, which provided the children
equal access to the input devices (blocks). The children easily identified the protagonists, the
opponents and found out that the animals were the protagonists’ friends.
One boy told us how he came to this conclusion: “I experimented the snake with the panda and the
result was bad, the panda with the lion also resulted badly. From there
, there was only a possible
conclusion
: these two were friends [elephant & panda] and these two were also friends [lion &
snake]. By looking at what they are doing
I concluded
that the panda and the elephant are friends
because they help each other, and with music they calm down.”
Several children spontaneously identified the characters as being Chinese and Indian and explicitly
included China and India in their narratives. Some groups used the musical instruments to calm
down the antagonists and stop the fights. The following narrative quotations exemplify this:
Figure 5– India, China and Brazil’s set.
Figure 6– Fight between
Hati
, the Indian animal and
Nagi
the Indian antagonist.
Figure 7– Outcome of a fight between
Gugu
, the
Chinese animal and
Nian
the Chinese antagonist.
CHI 2019 Late-Breaking Work
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
LBW1412, Page 4

Pair N. 2 - “[…] The girl was asleep and all the animals appeared and they started to fight, but then
the girl played the pipa (Indian flute) to stop the animals from attacking each other, and then the
Chinese siblings appeared” […].
Pair N. 5 - “Tiago went to China and Nuno also went, there they saw a snake and the panda and
they got very scared and started fighting [the animals]”; “and then Tiago played the guitar and the
panda and the lion calmed down […]”
.
Interestingly, Pair 5 gave their own names to the protagonists,
and each one embodied the character of the other, using direct and indirect speech as well as
dialogues in their stories.
3.2 Informal Interviews with the Children
Following each group’s interaction, the researchers talked with the children about their experience.
All the children identified Mobeybou as a tool for creating stories, some interesting descriptions
were: [it is a] “Doll Projector”; “It is for making theatre with the pieces […] we invent sceneries,
stories, and happenings. It is also a game, we have these pieces and when we place scenarios it helps
us in the stories… and we have more ideas with these blocks”. Another pair had the following
conversation with the researchers. Pair: “We did stories with pieces that give us a lot of ideas”.
Researcher: “Is it a game?” Pair: “it’s more than that, it helps us create a story, when we want to
create a story… when the teacher tells us to create a story, we have more things in our head”. Another
pair said that the board was a game to tell stories, saying that it was to: “make more creative stories”.
We also asked each pair about their favorite blocks, surprisingly, a great number of the children
expressed preference for the landscapes and the environment-blocks. The children also told us that
they particularly liked to place the blocks on the platform and see the interaction on the screen.
3.3 Informal Interview with the Teacher
At the end of the intervention, we talked with the teacher, who was very positive about the potential
of Mobeybou for promoting narrative competences, and knowledge of the world, by working across
the different cultures. We talked also about the children participants and were surprised to hear that
one of the children, who had enthusiastically created narratives with Mobeybou, was one of the less
accomplished students in her class. However, during this activity, she used a rich vocabulary in her
storytelling, used adjectives and linked the sentences with connectors e.g., “on a full moon night it
was snowing heavily, suddenly a strong wind started to blew”. The child was curious and engaged,
she asked questions about unknown vocabulary and quickly applied what she had learned, e.g., she
asked the name of the instruments and only called them by their name afterwards.
4 CONCLUSION AND FUTURE WORK
The session with Mobeybou showed that it was easy to understand and use. The elements inspired
the children to create narratives, and they showed interest for the represented cultures. The children
mixed the different elements in their stories, inventing family relations between the protagonists.
The use of the physical blocks and the corresponding visualizations on the screen promoted
verbalizations, exchange of ideas and collaboration, which are driving forces for the development of
Figure 8 –
Meera
(Indian protagonist) and
Gugu
, the
Chinese animal, listen to
Xiao Li
(Chinese
protagonist) playing the pipa, while snow is falling.
Figure 9 – Ju Long, a Chinese protagonist, uses the
fan (magical object) to create a tornado that knocks-
out Nian, the Chinese antagonist.
Figure 10 – Screen shot of the India App.
CHI 2019 Late-Breaking Work
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
LBW1412, Page 5

oral and narrative competencies. Future work includes the optimization of the new hardware, and
the development of more sets for different countries. Following this, Mobeybo will be used in pre
and primary school to investigate its potential for promoting language and narrative competences,
as well as multiculturalism.
ACKNOWLEDGEMENTS
We thank the students from the school EB1 S. Mamede in Braga and their teacher Ana Bizarro. This
work has been financed by national founds through the Portuguese Foundation for Science and
Technology - FCT, grant: SFRH/BPD/111891/2015, and by the European Regional Development Fund
(FEDER) through the Competitiveness and Internationalization Operational Program (POCI) under
the reference POCI-01-0145- FEDER-032580. And by CIEC (Research Centre on Child Studies),
within the Strategic Project UID/CED/00317/2013, by the Portuguese Foundation for Science and
Technology-FCT, and by European Regional Development Funds (FEDER) through the
Competitiveness and Internationalization Operational Program (POCI) reference POCI-01-0145-
FEDER-007562.
REFERENCES
[1] Brockmeier, J., Harré, R. Narrative: Problems and promises of an alternative paradigm, Res. Lang. Soc. Interact. 30 (4)
(1997) 263–283.
[2] Bruner, J.S., Haste, H . (Eds.), Making Sense: The Child's Construction of the World, Methuen, London, 1987.
[3] Catala A., Theune M., Sylla C., Ribeiro P. (2017) Bringing Together Interactive Digital Storytelling with Tangible
Interaction:
[4] Challenges and Opportunities. In: Nunes N., Oakley I., Nisi V. (eds) Interactive Storytelling. ICIDS 2017. Lecture Notes
in Computer Science, vol 10690. Springer, Cham.
[5] Chaudron S., Di Gioia R., Gemo M.; Young children (0-8) and digital technology, a qualitative study across Europe;
EUR 29070; doi:10.2760/294383.
[6] Hornecker, E. A design theme for tangible interaction: Embodied facilitation, in: H. Gellersen, et al. (Eds.), Proceedings
of the Ninth European Conference on Computer-Supported Cooperative Work, Springer International Publishing,
Switzerland, 2005, pp. 18–22.
[7] Labov, W., Waletzky, J. Narrative analyses: Oral versions of personal experience, in: J. Helm (Ed.), Essays on the Verbal
and Visual Arts, University of Washington Press, Seattle, 1967, pp. 12–44.
[8] Plowman, L., McPake, J. and Stephen, C. "Extending opportunities for learning: the role of digital media in early
education". Contemporary Debates in Child Development and Education, S. Suggate and E. Reese, eds., London:
Routledge, 2012.
[9] Resnick, M., Martin, F., Berg, R., Borovoy, R., Colella, V., Kramer, K., Silverman, B. Digital manipulatives: new toys to
think with, in: Proceedings of t he Conference on Human Factors in Computing Systems, ACM Press, New York, NY,
1998, pp. 281–287.
[10] Sylla, C., Coutinho, C., Branco, P., Müller, W. (2015). Investigating the use of digital manipulatives for storytelling in
pre-school. International Journal of Child-Computer Interaction. doi> 10.1016/j.ijcci.2015.10.001.
[11] Sylla, C. Coutinho, C. Branco, P. A digital manipulative for embodied ‘‘stage-narrative’’ creation, Entertain. Comput.
5 (4) (2014) 495–507. http://dx.doi. org/10.1016/j.entcom.2014.08.011.
CHI 2019 Late-Breaking Work
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
LBW1412, Page 6