Conference PaperPDF Available

A Preliminary Study of Interactivity on Visual Narrative in Children's Story Apps

Abstract and Figures

This study discusses how visual narrative in children's electronic books can benefit from research findings on children's literature. In this paper we discuss the concept of interactivity in the context of ebooks as a resource that adds multimodal stimulus to visual narratives. Based on the work from Nikolajeva and Scott, we focus on three aspects to guide the design of interactive digital narratives for children: (1) ambiance, (2) representation of the characters and (3) the perspective of the narrative. The discussion is exemplified through the analysis of highly interactive children's ebooks, such as story apps. The results of this investigation aim at providing professionals with some preliminary guidelines that can helping them to design interactive books for children.
Content may be subject to copyright.
636 CONFIA . International Conference on Ilustration & Animation
Esposende . Portugal . July 2018 . ISBN: 978-989-99861-6-9
A Preliminary Study of Interactivity on Visual Narrative
in Children’s Story Apps
Douglas Menegazzi1, Cristina Sylla2 and Stephania Padovani3
douglas.menegazzi@ufsc.br; cristina.sylla@ie.uminho.pt; stephania.padovani@gmail.com
[Narrativa e Teoria / Narratives & Theory]
Abstract
This study discusses how visual narrative in children’s electronic books
can benet from research ndings on children’s literature. In this paper
we discuss the concept of interactivity in the context of ebooks as a
resource that adds multimodal stimulus to visual narratives. Based on
the work from Nikolajeva and Scott, we focus on three aspects to guide
the design of interactive digital narratives for children: (1) ambiance, (2)
representation of the characters and (3) the perspective of the narrative.
The discussion is exemplied through the analysis of highly interactive
children’s ebooks, such as story apps. The results of this investigation
aim at providing professionals with some preliminary guidelines that can
helping them to design interactive books for children.
1. Introduction
Picturebooks stand out as a successful approach for engaging young
readers that have little experience in reading [8] since the narrative is pre-
sented through images, usually illustrations. In fact, images do not require
any interpretation competence, allowing and encouraging even younger
children to read the images [17]. Therefore, the picturebook is a particular
“format of expression” [8] that promotes the development of children’s
verbal literacy, stimulating the acquisition of aesthetic and cultural no-
tions through dierent understandings of the graphic narrative [15].
In this context, it is important to understand that the picturebook is
the main editorial product for children [8] and it brings together images,
text and dierent forms of narrative. The illustrations play a fundamen-
tal role as they articulate aspects that cannot be easily communicated
through words [6, 8, 15, 17]. According to Salisbury and Styles [17] the pic-
turebook emerged out of the combination of literary texts and the visual
arts resulting into a specic artifact for children. As such, the picturebook
has an important role as an object of narrative and representational
graphic art for children. Moreover, and given the enormous semiotic and
semantic potential of picture books, reading the words and looking at the
pictures in a picture book promotes the development of various skills [6].
Even though the traditional paper picturebook already presents
a complex and rich narrative created by the interrelationship between
1 Federal University of Santa Catarina, Department of Graphic Expression, Florianópolis, Brazil.
2 University of Minho, Education Institute, Braga, Portugal.
3 Federal University of Paraná, Department of Design, Curitiba, Brazil.
Keywords
Children’s book;
interactive storytelling;
ebook; electronic
picturebook.
637
[Narrativa e Teoria / Narratives & Theory] · A PRELIMINARY STUDY OF INTERACTIVITY ON VISUAL NARRATIVE IN CHILDREN’S STORY APPS
Douglas Menegazzi, Cristina Sylla and Stephania Padovani · douglas.menegazzi@ufsc.br; cristina.sylla@ie.uminho.pt; stephania.padovani@gmail.com
text and images, the digital book (e-book) extends the potential of the
printed picture book, by including multimedia resources [21]. Some highly
interactive formats of digital picturebooks incorporate rich multimedia
features, such as sounds or virtual reality (often present in video games) to
intensify the reader’s sensory experience. Some examples of such ebooks
are story applications (Apps), which include multimedia interactivity ele-
ments into traditional narrative models.
The emergence of the touch screen and the mobility created by the
iPad (2010) was a turning point in the history of the book, which greatly
impacted the editorial book setor, resulting in the creation of a new type
of narrative in the electronic format that combines the picturebook with
technology [4, 5, 23]. Unlike the early generations of children’s ebooks
- which consisted essentially in scans of printed books - story apps are
software applications specically designed to take advantage of multime-
dia and interaction features available in modern electronic devices such
as tablets and smartphones [19].
However, despite the growing number of available ebooks “there
is relatively little critique in the reviewing community” [23:585]. While,
story apps provide the opportunity for including meaningful interactions
that promote the user’s engagement with the plot in ways that printed
books do not allow, the role of interactive features has not yet been fully
explored [22]. Moreover, as Sargeant [19] points out, the experts them-
selves recognize the lack of standards to produce children’s electronic
books, since there are no guidelines for the design of interactivity. This
is understood as a problem that may be faced by illustrators, editors and
other professionals working in the eld of children’s interactive ebooks.
2. Interactivity Beyond Visual Narrative
n Electronic Picturebooks
Printed children’s books already aord dierent types of physical interac-
tion and intellectual interaction [22]. Physical interaction can be provided
at dierent levels such as page manipulation, pop-ups or other additional
mechanical structures. Intellectual interaction serves to guide the reader
through a nonlinear plot and to explore the content of the book, fostering
the reader to solve story puzzles or to make decisions in order to under-
stand the story. Children’s book apps provide digital interaction that oers
opportunities to interact with the story content and visual elements of the
narrative in a way that is still unexplored.
The interaction areas in children’s ebooks provide access to multime-
dia content, favouring multimodality [13]. According to Mayer’s Interactive
Multimodal Environments [11], such environments can be dened as envi-
ronments that include two or more modes of representation that combine
verbal and non-verbal representations, which target dierent channels.
In an extensive literature review on digital ebooks’ for children,
Menegazzi [12] found out that the interaction areas are linked to at least
three stimuli modalities: visual, sonorous and tactile. That is, interactivity
in most children’s ebooks extends beyond the visual narrative, includ-
ing animations, music, sound eects and locution. However, in order to
638 CONFIA . International Conference on Ilustration & Animation
Esposende . Portugal . July 2018 . ISBN: 978-989-99861-6-9
trigger the multimedia elements of the application, the reader needs to
perform certain gestures and touch codes on the graphical interfaces of
the display, as illustrated below (Fig. 1).
In electronic picturebooks the visual
narrative can be reinforced through the
use of sound, generating auditory images
that target the visual-auditory chan-
nels [2]. For example, a narrative using
sounds such as knocking on a door,
birds whistling, or an engine running are
processed by the auditory and the visual/
pictorial channel. Such nonverbal coding
enriches the narrative and helps to convey
scenes and meaning like images do [2].
According to Mayer’s Cognitive Theory
of Multimedia Learning [9, 10] by addressing dierent channels (visual,
auditory and sensorial) through images and sounds reduces the readers’
cognitive load, allowing them to better process the information. However,
the design of such interactions needs to be well balanced combining visual,
sensory and auditory stimuli in a coherent way, avoiding redundancy and
an overload of stimuli that may distract the readers instead of engaging
them. According to Mayer [9, 10], the mere inclusion of simple interac-
tion commands such as forward, pause, and back may assist the learner in
processing multimedia information.
Salmon [18] argues that “multimedia and interactive features that
motivate and engage young readers are inuential factors that can potentially
inuence reading frequency.” Although interactive and multimedia content
certainly have this potential, interactivity should be designed in order to
enhance the narrative experience rather than disrupting the story ow or
distracting the reader [2, 14]. When the interactivity is used inconsistently
without a visible connection with the narrative [2, 14] children may fail to
understand the story. On the other hand, ebook that provide well-designed
interactions that are congruent with the literary content and with the reading
task tend to be more attractive and eective promoting children’s literacy [3].
The great part of research on children’s ebooks claims that the
inclusion of any interactive feature needs to be in accordance with the
narrative and promote the understanding of the narrative [12]. However,
the inclusion and role of interactive features has not yet been completely
investigated [2, 20]. In the following we present our research ndings
regarding the integration and design of interactivity in ebooks for children
outgoing from the work of Nikolajeva and Scott’ work [15].
3. Literary Theory as a Guide to Design
Interactive Visual Narrative
According to Nikolajeva and Scott’ [15] the literary structure of the picture-
book is composed of three main constituents: (1) ambiance; (2) characteri-
zation of the characters; and (3) narrative perspective. Outgoing from these
elements we discuss how these properties can be useful to guide the inclu-
Fig. 1. Diagram of mul-
timodality of ebooks
based on Menegazzi
[12].
639
[Narrativa e Teoria / Narratives & Theory] · A PRELIMINARY STUDY OF INTERACTIVITY ON VISUAL NARRATIVE IN CHILDREN’S STORY APPS
Douglas Menegazzi, Cristina Sylla and Stephania Padovani · douglas.menegazzi@ufsc.br; cristina.sylla@ie.uminho.pt; stephania.padovani@gmail.com
sion of interactivity in children’s ebooks. For this, we will exemplify our
approach with two ebooks for children, namely the Goldilocks and Little Bear
[16] and Lil ‘Red - An Interactive Story [1]. These two ebooks were selected
due to their highly interactive visual narratives. Goldilocks and Little Bear
(produced by Nosy Crow) received a special mention in the Fiction Book
Category at the Bologna Ragazzi Award (2016); Lil ‘Red - An Interactive Story
(2012), was selected based on the readers’ evaluation on the App Store.
Conveniently both ebooks are adaptation of well-known fairy tales.
3.1. Literary Ambiance
The ambience in children’s picturebooks denes the situation and the
story universe, where the story events occur, conveying the time and
place where the narrative unfolds. The ambience includes the scenarios,
providing an aective atmosphere that inuences the reader emotionally.
Depending on the literary genre and whether it is a fairy tale or a horror
story the ambience can have dierent elements. The ambience also
represents the setup or environment where the characters are created and
presented and in some cases it becomes personied, assuming the role of
the narrator. Changes in the literary ambience in the dierent pages of the
books are very important as they serve as an indication to show the evolu-
tion and unfolding of the story. Such evolution can be done by changing
the scenery, using chronological variations, changes in the tone of the
narrative or in the stylistic character of the literary works [15].
The Goldilocks and Little Bear app [16] provides a good example of
interaction targeting the visual ambience. The application prompts the
reader to guide the two main characters - Goldilocks and the Little Bear -
through the dierent story scenarios, simultaneously allowing the reader
to discover other elements in the scene and even to interact with the
elements of the scenario. Figure 2 illustrates the moment when Goldilocks
arrives at the home of the three bears. The reader can move the character
within the scene by dragging it for instance to sit on the armchairs of the
bears. By controlling the main character the reader can explore, discover
and interact with the literary scene in his own way while enjoying the
visual narrative. The progression of the story between the dierent nar-
rative spaces, for example, the forest and the house of the bears, prompts
the reader to touch an arrow icon in the lower right corner of the screen,
this action triggers a marked transition between the virtual pages, which
reinforces the change of space and narrative moment.
In the Lil ‘Red app [1] the
multimodal interactivity is also
very well aligned with the narra-
tive ambience. Here, it is possible
to interact with practically all
graphic elements of the scene in
a synesthetic way. For example,
when the reader touches the tree-
tops an animation of the leaves
swaying and a rustling noise is
Fig. 2. A scene of the
interactive ambiance
in Goldilocks and Little
Bear app [16].
640 CONFIA . International Conference on Ilustration & Animation
Esposende . Portugal . July 2018 . ISBN: 978-989-99861-6-9
triggered. It is also possible to interact with the birds and the frogs, to hit
the trunk of the trees and hear the sound of the wood “toc toc”. The in-
teraction with the elements of the scene allows the user to interfere in the
story course. An example of this is the scene in which the user can touch
the pine cones of a tree, which fall down awakening the Wolf (Fig.3). The
clash of drum cymbals increases the eect, creating a funny moment
As Nikolajeva and Scott [15] explain, the literary ambience in children’s
picturebooks, unlike the classic novels that emphasize the text, is also cre-
ated through the visual narrative of images /illustrations. While words can
at most describe space, illustrations can eectively show it. While the verbal
text can help the reader to “see” the ambience and the scenarios, the visual
representation of the scenes gives the reader more freedom for interpreting
them [15]. As previously referred, the interactivity supported by story apps
oers the readers a synesthetic setting, allowing them to play, to listen to
sounds in the dierent scenes and even to actively interfere in the narrative
by controlling the structures of the scene or the story character’s. Thus, it is
important to keep in mind, as it also happens with printed books, that ambi-
ences have the power to highlight and expand situations in the story, but
they can also bring redundancy to the verbal narrative [15]. This needs to be
taken into account when designing interactive ebooks, in order to further
expand the narrative experience, increasing the readers’ engagement while
avoiding unnecessary interactions.
3.2. Characters Representation
The construction of the characters in picture books occurs especially through
images, whereas in other literary genres it usually occurs through verbal
descriptions. The illustrations of the characters allow even to convey their
physical characterization, without the need for a verbal description. Addition-
ally, the illustrations may also serve to emphasize “inner”, emotional qualities
of the characters, such as moods and feelings that reveal their personality and
state of mind. Therefore, the richness of the characterization in picturebooks
lies in the fact that this can be done by combining text and images, so that
both instances can complement or contradict each other to create dierent
eects and stylistic features in the embodiment of the plot [15].
The “Lil ‘Red” app [1] allows the user to interact with the characters
in an interesting way, as for instance, when the reader touches a story
character, the application plays sounds and animations that reveal certain
Fig. 3. Example of the
interaction in the visual
narrative in Lil’ Red -
An Interactive Story [1].
641
[Narrativa e Teoria / Narratives & Theory] · A PRELIMINARY STUDY OF INTERACTIVITY ON VISUAL NARRATIVE IN CHILDREN’S STORY APPS
Douglas Menegazzi, Cristina Sylla and Stephania Padovani · douglas.menegazzi@ufsc.br; cristina.sylla@ie.uminho.pt; stephania.padovani@gmail.com
characteristics of the characters’ personality. For instance, when the user
interacts with the Little Red Riding Hood character, it reacts with a child-
ish laughter, whereas when the reader interacts with the Wolf, it shows his
teeth, triggering a threatening sound.
The Goldilocks and Little Bear application [16] allows interaction
with the characters through virtual reality, similar to a player, the reader
can control the characters in activities linked to the story. During the
interaction the characters address the reader directly and in the automatic
reading version the reader can hear their voices. Both, the voices and the
interactive animations t the characters well, reinforcing the idea that the
Goldilocks character is a curious girl.
3.3. Narrative Perspective
The narrative perspective refers to the “point of view” of the narration,
which can be assumed by the narrator, by the characters or by the implied
reader. It can be a literal perspective, in accordance with the perspective
of the events’ presenter; gurative, when it conveys an ideological view
of the world; or transferred, when the narrator appropriates the story and
tells the events. In picturebooks, the narrative perspective is again related
to the switching between the communication through pictures and words,
between showing and telling [15].
According to Nikolajeva and Scott [15], the main dierential of
picturebooks are the ways in which text and image can jointly benet the
narrative adding its peculiar characteristics. While “only verbal texts can
comment on events and characters or address the reader immediately (“
Now I will tell you ...), [...] the images have their own means of expres-
sion” [15:157]. As for instance, when a character looks directly from the
illustration to the reader, or instead it is presented using dierent modes
and framing angles, being displayed for instance below the reader’s line
of sight, making the character seem fragile.
Highly interactive children’s picturebooks, such as story apps, may allow
interaction throughout the story to show dierent content or narrative per-
spectives. This is the case of the Goldilocks and Little Bear app that allows the
reader to rotate the tablet device, revealing another story version. As in the
“normal version” in the rotated version the Little Bear character also enters
the humans’ house and tastes their food (Fig. 4). The bear sits on the arm-
chair and the reader can choose in which bed it will lie down. By physically
interacting with the tablet, spinning it again, the reader returns to the original
Fig. 4. Dierent types
of narrative perspec-
tives in “Goldilocks and
Little Bear” app. [16]
642 CONFIA . International Conference on Ilustration & Animation
Esposende . Portugal . July 2018 . ISBN: 978-989-99861-6-9
story version where Goldilocks is the one that invades the house of the Baer.
Multimodality associated with the story can contribute for creating
more dynamic and engaging stories. The inclusion of sound and touch
can enrich children’s literary experience. As an example, the Lil ‘Red app
[1] triggers bass sounds when the reader interacts with the Wolf, generat-
ing a sense of danger (Fig.5). However, when the reader interacts with
the Little Red Riding Hood character, it emits sounds of laughter as well
as subtle sound and animation eects that represent the little girl in the
story. These variation of the visual and
the auditory stimuli that are triggered
when the reader interacts with the
story characters create empathy with
the Lil’Red, generating feelings of fear
towards the wolf. Therefore, the interac-
tion can reinforce a particular aspect of
the narrative.
In printed picturebooks children can
benet from the richness of the graphic
forms and in interactive pop-up books
they can additionally explore the books in
a tangible, sensory way, this also allows
them to read the story in a very personal-
ized way. In story apps children are able
to control the characters and the elements of the scenario in a playful
manner, similar to playing a game. Such aordances can promote a deep-
er involvment with the story. However, to achieve this, the application
need to be carefully designed, since the inclusion of the interaction can
also have a disturbing eect [2] impairing the reading activity. Therefore,
when designing the interaction for ebooks, developers and designers need
to focus primarily on the story and carefully consider the types of interac-
tion that they may use. Such approach will allow the readers to focus on
the literary activity, enabling them not only to read, but also to play an
active role in the construction of the narratives.
5. Considerations
Through the illustrated book the child comes into contact with reading in
several ways, since the interaction with the text includes diegetic and mi-
metic characteristics, the images and visuals facilitate the representation
of the ambience and the characters, enriching the storytelling activity.
A well-balanced approach between the story plot and the design of the
interaction in ebooks for children needs to consider these characteristics
and the literary qualities into account.
As children’s books increasingly move into the digital format, the con-
nections between interactivity and literature also grow closer together. “As
books incorporate higher levels of interactivity, the reader, in turn, is likely to
become more active” [19: 2]. If the design of interactivity in children’s ebooks
is geared towards promoting a good reading experience and the understand-
ing of literary contents in a more dynamic and immersive way, this can stimu-
Fig. 5. Perspectives of
the visual narrative in
Lil’ Red - An Interactive
Story [1].
643
[Narrativa e Teoria / Narratives & Theory] · A PRELIMINARY STUDY OF INTERACTIVITY ON VISUAL NARRATIVE IN CHILDREN’S STORY APPS
Douglas Menegazzi, Cristina Sylla and Stephania Padovani · douglas.menegazzi@ufsc.br; cristina.sylla@ie.uminho.pt; stephania.padovani@gmail.com
late higher levels of engagement and reading comprehension [2, 7, 20].
Story books in the form of ebooks have the potential to increase the
visual narrative by including digital and multimedia interactivity. Inter-
activity can have impactful eects on reading and when it is well aligned
with the story plot, it can lead to even greater proximity of the readers
by enabling them to participate more actively in the development of the
story plot. Interactive features that are congruent with the narrative can
contribute to increase the reader’s engagement with the story. This opens
avenues for integrating interaction in ebooks as a resource that allows
children to actively participate in the construction of the story, telling it in
their own way, having access to dierent perspectives of the narrative and
the presence of multimedia eects that enrich the characterization of the
characters making the visual narrative even more synesthetic.
6. Future Work
This paper is part of a research project about the design of interactive
features for children’s ebooks. We presented a theoretical discussion and
proposed possible solutions to help designing better interaction in chil-
dren’s story applications. In future work we intend to further investigate
these issues and to present a set of guidelines for the design of interactive
children’s books.
Acknowledgements
The present work was carried out with the support of CNPq, National Council of Scientic
and Technological Development - Brazil. Process number: 206788/2017-7. The rst
author is a fellow of CNPq - Brazil. The second author acknowledges the support of the
Portuguese Foundation for Science and Technology - FCT- within the grant: SFRH/
BPD/111891/2015.
References
1. Brian Main: Lil’ Red - An Interactive Story, version 1.0.3. (2012) Accessed 12 April 2018
from https://itunes.apple.com/us/app/lil-red-an-interactive-story/id561436050
2. Bus, A. G.; Takacs, Z. K.; Kegel, C. A. T.: Aordances and limitations of electronic
storybooks for young children’s emergent literacy. In: Developmental Review, vol. 35, pp.
79–97. Eslevier (2015)
3. Cahill, M., Mcgill-Frazen, A.: Selecting “app”ealing and “app”ropriate book apps for
beginning readers. In: Reading Teacher, vol. 67, pp. 30–39. Willey Online Library (2013)
4. Frederico, A.: O Futuro do Leitor ou o Leitor do Futuro: o Livro Infantil Interativo e
os Letramentos Múltiplos. In: Cadernos de Letras da UFF Dossiê: A crise da leitura e a
formação do leitor, num. 52, pp. 101-120 (2016)
5 Frederico, A.: The Construction of Meaningin Three Fairy-Tale Enhanced Electronic Picture-
books. In: Children’s Literature, Technology and Imagination. University of Padua, Italy (2013)
6. Hunt, P.: Crítica, teoria e literatura infantil. Cosac Naify, São Paulo (2010)
7. Kao, G. Y.-M., Tsai, C.-C. C., Liu, C.-Y., Yang, C.-H.: The eects of high/low interactive elec-
644 CONFIA . International Conference on Ilustration & Animation
Esposende . Portugal . July 2018 . ISBN: 978-989-99861-6-9
tronic storybooks on elementary school students’ reading motivation, story comprehension
and chromatics concepts. In: Computers and Education, vol. 100, pp. 56–70. Eslevier (2016)
8. Linden, S. V.: Para ler o livro ilustrado. Cosac Naify, São Paulo (2011)
9. Mayer. R. E.: (Ed.) Cognitive Theory of Multimedia Learning. In: The Cambridge Han-
dbook of Multimedia Learning. Cambridge University Press, Cambridge (2005)
10. Mayer, R. E.: Cognitive Theory and the Design of Multimedia Instruction: An Example
of the Two-Way Street Between Cognition and Instruction. In: New Directions for Tea-
ching and Learning, vol 2002, num. 89, pp. 55-71. Wiley Online Library (2002)
11. Mayer, R. E.: Multimedia Learning: Are We Asking the Right Questions? Educational
Psychologist, VOL. 32, num. 1, pp. 1-19. Lawrence Erlbaum Associates, Inc. (1997)
12. Menegazzi, D.: O Design de interfaces de livros infantis apps: uma revisão das carac-
terísticas e recomendações. In: Textura, vol. 20, n. 43 (2018)
13. Moraes, G. L.: Do livro ilustrado ao aplicativo: reexões sobre multimodalidade na
literatura para crianças. In: Estudos de Literatura Brasileira Contemporânea, num. 46, pp.
231-253, July/Dec. Scielo (2015)
14. Morgan, H.: Multimodal Children’s EBooks Help Young Learners in Reading. In:
Early Childhood Education Journal, vol. 41, num. 6, pp. 477–483. Springer (2013)
15. Nikolajeva, M.; Scott, C.: Livro Ilustrado: palavras e Imagens. Cosac Naify, São Paulo (2011)
16. Nosy Crow: Goldilocks and Little Bear, version 1.2. (2015) Accessed 10 April 2018 from
https://itunes.apple.com/us/app/goldilocks-and-little-bear-by-nosy-crow/id991030428
17. Salisbury, M.: Styles, M. Livro infantil ilustrado: a arte da narrativa visual. São Paulo:
Rosari, 2013.
18. Salmon, L. G.: Factors that Aect Emergent Literacy Development When Engaging
with Electronic Books. In: Early Childhood Education Journal, vol. 42, num. 2, pp. 85–92.
Springer (2014)
19 Sargeant, B.: What is an ebook? What is a Book App? And Why Should We Care? An
Analysis of Contemporary Digital Picture Books. In: Children’s Literature in Education:
An International Quarterly, vol. 46, num. 4, pp. 454–466. Springer (2015)
20 Sargeant, B., Sargeant, E.: Interactive Storytelling: How Picture Book Conventions
Inform Multimedia Book App Narratives. In: Australian Journal of Intelligent Information
Systems, vol. 13, num. 3, pp. 29–35. ACM Digital Library (2013)
21. Teixeira, D., Gonçalves, B. S., Vieira, M. L. H.: Organização da multimídia em ebook in-
terativo infantil. In: SIGRADI 2015 Proceedings, vol. 2, num. 3, pp. 292 – 299. Blucher (2015)
22. Timpany C., Vanderschantz N., Hinze A., Cunningham S.J., Wright K.: Shared Reading
of Children’s Interactive Picture Books. In: Tuamsuk K., Jatowt A., Rasmussen E. (eds)
The Emergence of Digital Libraries – Research and Practices. ICADL 2014. Lecture Notes
in Computer Science, vol 8839, pp. 196-207. Springer (2014)
23. Yokota, J., Teale, W. H.: Picture Books and the Digital World. In: Reading Teacher, vol
67, num. 8, pp. 577–58 5. Wiley Online Library (2014)
... The home icon is easy to use (EtU1) [12] Start, Stop, Pause, Previous, Next are large and easy to use (EtU2) [12] Reading mode (Narration and non-narration) button is available (EtU3) [12] Multimedia(M) [13] Features text, graphics, fonts, animation, and audio Large and appropriate font (M1) [14] Exciting animations and appropriate the content (M2) [15] Music effects do not interfere with the content, and it can motivate (M3) [16] Interaction (I) [17] Hyperlink or button icon that triggers an event or action Text interactions support word introduction, vocabulary, and understanding (I1) [18] Educational content interactions (I2) [18] Effective Supplemental games or quizzes (I3) [19] 1 (1,1,1) (1,3/2,2) (3/2,2,5/2) EtU 2 (1/2,2/3,1) ...
... The home icon is easy to use (EtU1) [12] Start, Stop, Pause, Previous, Next are large and easy to use (EtU2) [12] Reading mode (Narration and non-narration) button is available (EtU3) [12] Multimedia(M) [13] Features text, graphics, fonts, animation, and audio Large and appropriate font (M1) [14] Exciting animations and appropriate the content (M2) [15] Music effects do not interfere with the content, and it can motivate (M3) [16] Interaction (I) [17] Hyperlink or button icon that triggers an event or action Text interactions support word introduction, vocabulary, and understanding (I1) [18] Educational content interactions (I2) [18] Effective Supplemental games or quizzes (I3) [19] 1 (1,1,1) (1,3/2,2) (3/2,2,5/2) EtU 2 (1/2,2/3,1) ...
Article
Full-text available
Recently, digital technology is developing rapidly. Digital technology also needs to be developed in the field of education, especially Digital Book. In evaluating digital books, stakeholders need appropriate criteria. Therefore, this study aims to evaluate the criteria of an excellent digital book by using a fuzzy analytical hierarchy process. A digital book evaluated is elementary school mathematics learning books by considering aspects of Azaz Pancasila. The criteria evaluation method uses a fuzzy analytical hierarchy process. Evaluation criteria used are easy to use, multimedia, interaction. The evaluation results of digital book criteria show that easy criteria have the highest importance. Furthermore, multimedia and interaction criteria have the importance of the second and third importance. This research implies that the selection of the right criteria can be a reference in the development of digital books.
Article
Full-text available
Esse artigo explora o livro infantil interativo como forma de expressão literária que exige do leitor infantil orquestrar diversos tipos de letramento para, em última instância, desenvolver sua capacidade de ler criticamente textos multimodais, digitais e interativos diversos. O modelo de análise envolvendo os níveis perceptivo, estrutural e ideológico (SERAFINI, 2010; 2015) é proposto para promover, por meio dos livros digitais, a articulação de múltiplos letramentos na escola.
Article
Full-text available
Este trabalho busca refletir sobre a diversidade de recursos semióticos presente na literatura infantil contemporânea, analisando os efeitos dos avanços tecnológicos sobre o texto literário. Utilizando alguns conceitos originários dos estudos sobre a multimodalidade, analisa os potenciais de sentido resultantes da articulação entre recursos verbais e não verbais na literatura impressa e digital, a partir das obras Flicts, "A girafa" e "Cigarra", da série "Ave, palavra", de Angela Lago, e do aplicativo de literatura para crianças Es así, de Paloma Valdivia. A autora apresenta elementos para afirmar que o avanço dos estudos da multimodalidade contribui para melhorar a experiência literária por parte das crianças na contemporaneidade ao oferecer meios para conhecer e mediar os processos de interpretação e fruição de textos multimodais.
Article
Full-text available
Stories presented on phones, tablets and e-readers now offer an alternative to print books. The fundamental challenge has become to specify when and for whom the manner in which children retain information from stories has been changed by electronic storybooks, for better and for worse. We review the effects of digitized presentations of narratives that include oral text as well as multimedia information sources (e.g., animations and other visual and sound effects, background music, hotspots, games, dictionaries) on children's emergent literacy. Research on preschool and kindergarten children has revealed both positive and negative effects of electronic stories conditional upon whether materials are consistent with the way that the human information processing system works. Adding certain information to electronic storybooks can facilitate multimedia learning, especially in children at-risk for language or reading difficulty. Animated pictures, sometimes enriched with music and sound, that match the simultaneously presented story text, can help integrate nonverbal information and language and thus promote storage of those in memory. On the other hand, stories enhanced with hypermedia interactive features like games and “hotspots” may lead to poor performance on tests of vocabulary and story comprehension. Using those features necessitates task switching, and like multitasking in general, seems to cause cognitive overload. However, in accordance with differential susceptibility theory, well-designed technology-enhanced books may be particularly suited to improve learning conditions for vulnerable children and turn putative risk groups into successful learners. This new line of research may have far-reaching consequences for the use of technology-enhanced materials in education.
Article
Informed by David Lewis’ ecological approach to picturebooks, this study conducted close readings of three enhanced e-book versions of the fairy tale The Three Little Pigs, analyzing how the different modalities that constitute these narratives interanimate each other and participates in the meaning-making process.Fondée sur l'approche écologique du livre d'images de David Lewis, cette étude repose sur une lecture attentive de trois éditions numériques au contenu rehaussé du conte de fées Les trois petits cochons. L'analyse s'intéresse aux différentes modalités qui constituent la trame narrative et qui s'animent entre elles tout en contribuant au processus de création de sens.
Conference Paper
We report on a study of children and parents shared reading of interactive printed books. We investigated the differences between books with interactive features and books with expressive typography in order to evaluate which features within a book encouraged interaction between the reading participants and the book. 11 parent and child groups took part in the study that involved three observed reading sessions. From our observations we offer suggestions for the development of books and eBooks to encourage shared reading practices.
Article
http://link.springer.com/article/10.1007/s10583-015-9243-5?wt_mc=internal.event.1.SEM.ArticleAuthorAssignedToIssue Book apps have developed into a new format for the picture book. Given the crucial role that picture books have played in early childhood education, it seems pertinent to ascertain the ways in which they have been affected by digitisation. In response to concerns regarding a lack of models and design principles within children’s digital publications, this transdisciplinary study attempts to go some way towards addressing the need for more research in this area. The article draws on research into children’s literature and human-computer interaction, analysing a range of digital picture books and arguing that people read ebooks, whereas they use book apps, the latter being far more media-rich and interactive. The article also uncovers ways in which designers can use media-rich interactive features to further children’s engagement with their literature.
Article
The book publishing industry in general and picture book materials for children in particular have undergone rapid and profound changes in recent years with the developments in the digital realm. As a result, teachers, school librarians, and literacy researchers have been largely left to their own devices to figure out basic questions related to e-picture books such as what works, what doesn't, what's good, and so what? This article discusses two fundamental issues related to these materials and young children's interactions with them: (1) Print or Digital—which version of a story should teachers use in particular instances and (2) Quality of Features of Digital Picture Books—evaluation and selection factors that should be considered in assessing digital picture books for use in the classroom.
Article
It is not possible to understand cognition fully without understanding how it works in realistic settings, and it is not possible to reform education appropriately without understanding how people learn and think.