Content uploaded by Allison Druin
Author content
All content in this area was uploaded by Allison Druin
Content may be subject to copyright.
Designing Intergenerational Mobile Storytelling
Allison Druin1, Benjamin B. Bederson2, Alex Quinn2
University of Maryland
Human-Computer Interaction Lab (HCIL)
iSchool1, Computer Science Department2
College Park, Maryland 20742
bederson@cs.umd.edu, allisond@umiacs.umd.edu, aq@cs.umd.edu
ABSTRACT
Informal educational experiences with grandparents and
other older adults can be an important component of
children‟s education, especially in circumstances where
high quality educational services and facilities are not
readily available. Mobile devices offer unique capabilities
to support such interactions. We report on an ongoing
participatory design project with an intergenerational design
group to create mobile applications for reading and editing
books, or even creating all new stories on an Apple iPhone.
AUTHOR KEYWORDS
Intergenerational design, mobile reading, digital libraries,
storytelling, participatory design, children, design, mobile
user interfaces, iPhone.
ACM CLASSIFICATION KEYWORDS
H.5.2. User Interfaces: User-centered design, prototyping
THE NEED FOR RESEARCH
From the Middle East to northern Africa, to right here in the
United States, the world‟s children are growing up
impacted by conflict, poverty, and lack of school resources
[1, 4, 12]. The 20th century model of shipping books and
other educational materials to various parts of the world is
increasingly difficult and expensive. Complicating these
challenges, parents and other family members are lost to
war, famine, and diseases (in particular HIV/AIDS), which
leaves children with little understanding of their own
cultures and personal family histories [2, 20]. The need has
never been greater to educate the world‟s children.
As mobile technologies become ubiquitous (through the
growth of netbooks and common mobile phone
technologies), this “21st century computing platform” has
emerged as one way to address the many challenges of
educating young people even in developing countries [1,
18]. Anytime, anywhere computing, can lead to affordable
and portable paths to information access and learning.
Therefore we have been adapting and enhancing the
technologies of digital books, multi-sensory story creation,
and distributed storytelling for the mobile platform to be
used by disadvantaged learners at the extremes of life (older
adults and children).
From data collection by children for field research [5, 15,
17, 19], to accessing information via text or voice [8], to
use as mobile guides [6, 10], mobile devices in recent years
have begun supporting learning from science to social
studies. However, little research has been done to date with
mobile technologies to support children in developing
multiple forms of literacy through children‟s literature and
storytelling. This is evidenced by the few articles that have
been published in scholarly journals.
It has been found in many struggling economies that access
to educational services and materials has actually declined
in recent years [1, 8]. This coupled with the lack of success
for many children in traditional school settings, seems to
suggest that a different approach to education is needed.
Studies have shown that interactions between older and
younger people, can improve children‟s motivation for
learning, and increase their awareness of personal and
community culture [8, 13]. Yet, little discussion in the
research literature on mobile computing focuses on
Figure 1. Today’s ICDL for iPhone application displays
children’s books that can be read on an Apple iPhone.
(a) Four books to choose from. (b) An overview of the
Arabic/English book, Black Ear… Blond Ear.
intergenerational learning experiences — where
grandparents, “grandfriends,” and/or community elders can
have a role in educating the next generation of children.
Therefore, we have been exploring how informal
educational experiences with older adults can enhance the
literacy development experience for disadvantaged
children. Among the challenges of this work have been the
development of interfaces that are usable by older users and
the young users alike while enabling the two to smoothly
collaborate. Much of the literature has shown that
interfaces for children and older adults may not be the same
or even compatible.
MOBILE INTERFACE TECHNOLOGIES
To make this vision a reality, a team of researchers,
including the founders of the International Children's
Digital Library (ICDL), are leveraging their expertise in
interaction, technology for children, and mobile user
interface design to adapt existing ICDL content and
infrastructure for mobile technologies. Currently, the ICDL
(freely available at www.childrenslibrary.org) is used on
the Internet by over 100,000 unique visitors per month from
around the world. The library‟s use is split between
children, parents, teachers and librarians as determined by
surveys and optional account logon information. At
present, books from 60 countries are available in the ICDL
via a web interface that includes multiple visual querying
tools for selecting books, and three interaction styles for
reading books online. The ICDL is a stable and robust
platform served from Linux machines at the University of
Maryland running custom application code built with Java,
MySQL and Lucene, and served by Apache and Tomcat.
We are currently working toward delivering some of the
ICDL books on Apple‟s iPhone with support for text and
images to be read aloud by children and their trusted adults
(such as a grand parents). To do this, we need access to the
book‟s text, scaled imagery of the book without the text,
and indexed recorded audio of the text. Our recent work
with ICDL to increase readability and accessibility [16]
solves the first two problems by using image processing to
locate and remove the text from the page scans, and then
store many sizes of those images on our servers. We
created an initial iPhone version [3], which can be seen in
Figures 1 and 2.
We are currently developing recording capacity and
acquiring rights to contemporary books to record them
ourselves. If more ICDL content were available in audio
form, it would give us more flexibility for mobile
application design because readability of text on the small
screen could be less of an issue.
INTERGENERATIONAL DESIGN
Seventy-three-year-old Myles and nine-year-old Dana are
sitting on our couch reading the Arabic/English book, Black
Ear… Blond Ear, by Khaled Jumm‟a. It is a book about two
groups of cats, the light-colored cats and the black cats,
who learn to make peace by listening to each other by
actually exchanging their ears. The book has been digitized
and is available on a mobile phone. Myles is reading the
story to Dana, periodically suggesting that she read a page
to him. He helps as Dana stumbles on a word or pronounces
a word incorrectly. As they are reading, they are also
helping to design a new mobile intergenerational reading
application. Dana suggests to Myles, “There should be a
READ ME button, so if you‟re not here, I can hear you
reading the story with me.”
Myles adds, “Yeah, but I think you should read the book to
me with that button. You‟ve got better eyes to look at that
small screen than me.”
One of the lab‟s staff asks, “What if we highlighted the
words here? Would you notice them enough to talk about
them?”
Myles says, “Sure, if they‟re BIG! Let‟s see, what part are
you talking about … ?”
Thanks to this design session with Myles and Dana, along
with three other child/grandparent reading pairs, people can
now read children‟s books from such places as New
Zealand, Egypt, and the United States on their iPhones. We
are now actively developing the story-creation application.
Currently, development on the reading application is
complete, while development is ongoing on a second
application for creating and/or editing books.
4. APPLICATION DEVELOPMENT
For development and experimental purposes, we are
working with the iPhone and iPod Touch platforms. The
latter are less expensive, but lack a camera and readily
Figure 2. Reading a book in the ICDL for iPhone
application. (a) An example of one page of the book.
and (b) The same page with the English text magnified.
usable audio facilities, but are otherwise suitable and
economically feasible for this research. Both the iPhone
and iPod Touch offer relatively large displays (4.5 inches ×
2.4 inches, 115 mm × 61 mm, 320×480 pixels).
Furthermore, both devices have a high quality capacitive
multi-touch touch screen, and built-in graphics processing
unit (GPU), making it possible to create rich, visual
interfaces such as zooming or touch-based manipulation.
Reading
Using the book reading application we are developing
(Figures 1 and 2), families can comfortably read children's
books on the device, allowing opportunities for reading
together in almost any setting. For the first version, the
application is limited to only four books, the content of
which is included in the application. Thus, once the
application has been downloaded, no further Internet access
is necessary; books can be read even in a subway tunnel.
The application is currently available by searching for
“ICDL” in Apple's App Store or through Apple iTunes.
The application uses the unique capabilities of the device to
provide a rich experience. Rotating the device switches
between landscape (2-page) and portrait (1-page) views.
Tapping on the cover of a book zooms in to view the
thumbnails of the book pages. Tapping on a book page
zooms in to view that page. Swiping a finger across the
touch screen advances to the next or previous page. When
text is too small to be read comfortably, tapping on the text
causes it to pop out and be displayed in a larger font size,
but still in the context of the illustrations (Figure 2).
Participants in our intergenerational design group have
helped to refine the design. They have suggested possible
ways to scroll text or transition between pages. In working
with the full team of children with grandparents, we were
surprised that among about eight children and seven
grandparents, there were no significant problems with
access to the device; the pairs were able to easily negotiate
who would hold and manipulate the device at a given time,
with both the child and the grandparent sharing control.
Furthermore, all of the elderly participants said they could
read the text comfortably. We could see them reading the
text with the children so this is believable.
Editing Books
The next step was to build an application capable of editing
books. As a first step, we enable children and their families
to make changes to the content of the books. The current
interface allows them to use touch-based painting to modify
the illustrations and use the device‟s onscreen keyboard to
modify the text. Text boxes can be moved, resized, added,
and deleted. In this way, the interface is similar to a
simplified, touch-based analog of many object-based
drawing programs (i.e., Microsoft PowerPoint). However,
the editing application also allows users to take pictures
using the iPhone‟s camera or insert items stored in the built-
in photo album available on both devices. Those photos
may have been copied from a personal computer or saved
from web pages while browsing on the mobile device. The
editing interface is shown in Figure 3.
We have been working with children in the lab to iterate on
designs, suggest new design directions, and to give
feedback on prototypes. The tone of the sessions has been
marked by excitement and cooperation, largely because the
children have been enthusiastic about using the devices to
create stories and share them with others. The children
were so deeply immersed in creating their own stories with
the application that when implementation bugs in the
prototypes were discovered, they complained passionately.
We quickly discovered the importance of sound for “read-
aloud.” We have also seen the participants being deeply
engaged while annotating photos with voice. In addition,
the need to project or show the story on a larger screen has
come up numerous times in our design experiences.
FUTURE WORK
As development of the story editor application nears
completion, we are planning a formal evaluation of this
work to be conducted during summer 2009. We anticipate
Figure 3. Editing the book, The Three Little Pigs.
(a) Original book page. (b) Altering the illustration.
(c) Changing the text.
using a variety of research methods to understand changes
in children and older adults: interviews, web logs, and
artifact analysis. We will also ask the elders to journal
using voice logs in support of the reflection process.
ACKNOWLEDGEMENTS
This work could not have been accomplished without the
support of the National Science Foundation (#0839222).
We also thank our design partners in the lab: Dana, Tara,
Alexandra and her grandparents, Stephen, Sasha, Brody,
Naja, Caitlin, and Myles. In addition, our colleagues in the
HCIL have helped enormously: Sonny, Greg, Beth, Beth,
Mona Leigh, Jerry and Leshell.
REFERENCES
1. Adesope, O., Susan O., & McCracken, J. (2007).
Implementing mobile learning in developing countries:
Prospects and challenges In Proceedings of world
conference on educational multimedia, hypermedia and
telecommunications 2007, edited by C. Montgomerie
and J. Seale.Chesapeake, VA: AACE.
2. Anderson, J. L. (2006). A structured approach for
bringing mobile telecommunications to the world„s
poor. Electronic Journal of information systems in
developing countries 27 (2):1-9.
3. Bederson, B.B., Quinn, A., Druin, A. (2009) Designing
the Reading Experience for Scanned Multi-lingual
Picture Books on Mobile Phones. In Proceedings of the
Joint Conference on Digital Libraries (JCDL 2009),
Short Paper, (in press).
4. Blachowicz, C.L.Z., Bates, A., Berne, J., Bridgman, T.,
Cheney, J., Perney, J., (2007). Technology and At-Risk
Young Readers and Their Classrooms. Spencer
Foundation Report, Accessed March 19, 2008 at:
http://www.innovationsforlearning.org/Spencer-8.pdf.
5. Chen, Y., Kao, T., Sheu, J., & Chiang, C. (2002). A
Mobile Scaffolding-Aid-Based Bird-Watching Learning
System. In Proceedings IEEE international Workshop
on Wireless and Mobile Technologies in Education
(August 29 - 30, 2002). IEEE Computer Society,
Washington, DC, 15-22.
6. Chittaro, L. & Burigat, S. (2005). Augmenting audio
messages with visual directions in mobile guides: an
evaluation of three approaches. In Proceedings of the
7th international Conference on Human Computer
interaction with Mobile Devices and Services (Salzburg,
Austria, September 19 - 22, 2005). MobileHCI '05, vol.
111. ACM, New York, NY, 107-114.
7. Druin, A. (2009). Mobile Technology for Children.
Boston, MA: Morgan Kaufman.
8. Ellis, J.B. & Bruckman, A.S. (2002). Encouraging
attitudinal change through online oral history. In
International Conference of the Learning Sciences
(ICLS), (Seattle, Washington)
9. Ford, M. & Botha, A. (2007). MobilED - an accessible
mobile learning platform for Africa? In Proceedings of
the 2007 IST-Africa conference, Maputo, Mozambique,
edited by P. Cunningham and M. Cunningham.
10. Hsi, S. (2004). I-Guides in Progress: Two prototype
applications for museum educators and visitors using
wireless technologies to support informal science
learning. In Proceedings of the 2nd IEEE International
Workshop on Wireless and Mobile Technologies in
Education (WMTE'04), (2004), IEEE Computer Society,
187.
11. Jones, M., & Marsden, G. (2006). Mobile interaction
design. New York: John Wiley & Sons.
12. Narayan, G. (2007). Addressing the digital divide: E-
governance and m-governance in a hub and spoke
model. Electronic Journal of Information Systems in
Developing Countries 31 (1):1-14.
13. Ogozalek, V.Z. (1994). The Worcester State College
'Elder Connection': Facilitating intergenerational
education with information technology and multimedia.
In Sociomedia: Multimedia, Hypermedia, and the Social
Construction of Knowledge, Barrett, E. ed. The MIT
Press, Cambridge, Massachusetts, 533-546.
14. Papanikolaou, K. & Mavromoustakos, S. (2006).
Critical success factors for the development of mobile
learning applications. In Proceedings of the 24th
IASTED international Conference on internet and
Multimedia Systems and Applications (Innsbruck,
Austria, February 13 - 15, 2006). ACTA Press,
Anaheim, CA, 19-24.
15. Price, S., Rogers, Y., Stanton, D., & Smith, H. (2003).
A New Conceptual Framework for CSCL: Supporting
diverse forms of reflection through multiple
interactions. In Designing for Change in Networked
Learning Environments. Proceedings of the
International Conference on CSCL 2003, (Bergen,
Norway, 2003), Springer, 513-522.
16. Quinn, A., Hu, C., Arisaka, T., & Bederson, B.B. (2008)
Readability of Scanned Books in Digital Libraries,
Proc. of ACM CHI (CHI 2008), ACM Press, 705-714.
17. Rogers, Y., Price, S., Fitzpatrick, G., Fleck, R., Harris,
E., Smith, H., Randell, C., Muller, H., O'Malley, C.,
Stanton, D., Thompson, M. & Weal, M. (2004).
Ambient Wood: Designing new forms of digital
augmentation for learning outdoors. In Proceeding of
the 2004 Conference on Interaction Design and
Children, (College park, MD, 2004), ACM Press, 3-10.
18. Seong, D. S. (2006). Usability guidelines for designing
mobile learning portals. In Proc. of the 3rd International
Conference on Mobile Technology, Applications and
Systems (Bangkok, Thailand, October 25--27, 2006).
Mobility '06, vol. 270. ACM, New York, NY, 25.
19. Wang, M., Ruimin S., Ren T., Fan Y., & Han, P.
(2005). Mobile learning with cellphones and PocketPCs.
In Advances in web-based learning – ICWL 2005.
20. Woods, G., William G., Stephens, G., Coakley, P.,
Ryan, M., Merry, C. & Grimson, J. (2007). Applying
mobile devices to promote evidence-based practices for
HIV/AIDS in resource deprived environments. Paper
read at IST-Africa Conference 2007, 09-11 May, at
Maputo, Mozambique.
