Learner Centred Design: Applying MobileHCI and Mobile Design Research Methods in Mobile and Informal Learning Contexts

Abstract

This paper is a survey of research and design methodologies used for understanding individual (human) mobile behaviour used by developers within the MobileHCI (Human Computer Interaction) and Mobile Design research communities. This paper summarizes the most commonly used and emerging research methodologies and suggests which methodologies are ideally suited for researchers within informal and mobile learning contexts to help garner the crucial data to help make informed decisions about the design of learner-centred informal and mobile learning environments.

Full text

Research Methods in Informal
and Mobile Learning
Book of Abstracts
WLE Centre
14th December 2007
www.wlecentre.ac.uk

14 December 2007
WLE Centre,
Institute of Education,
London
United Kingdom
Organisers:
Giasemi N.Vavoula
Agnes Kukulska-Hulme
Norbert Pachler
MIL-RM 2007
Proceedings of the Workshop:
Research Methods in Mobile and Informal
Learning How to get the data we really want
Published by WLE Centre, December 2007
ISSN 1753-3385

Mark Kramer
University of Salzbur
ICT&S Center for Advanced Studies
Learner Centred Design:
Applying MobileHCI and Mobile
Design Research Methods in
Mobile and Informal Learning
Contexts
This paper is a survey of research
and design methodologies used
for understanding individual
(human) mobile behaviour used
by developers within the MobileHCI
(Human Computer Interaction)
and Mobile Design research
communities. This paper
summarizes the most commonly
used and emerging research
methodologies and suggests
which methodologies are ideally
suited for researchers within
informal and mobile learning
contexts to help garner the crucial
data to help make informed
decisions about the design of
learner-centred informal and
mobile learning environments.
1. Introduction
The purpose of this paper is to
briefly survey research methods
from Mobile Human Computer
Interaction (MobileHCI) and
Mobile Design research to
ascertain if research methods from
these disciplines could effectively
be transferred to Mobile Learning
and Informal Learning Design
research. Furthermore, this paper
should be viewed as an attempt to
make a small contribution to help
enhance existing research
methods and help inspire the
development of new and novel
research methods for the Mobile
Learning and Informal Learning
Design communities.
The methods highlighted in this
paper have been chosen on the
basis that they could assist in
collecting useful, if not crucial,data
in order to assist in the evaluation
of the effectiveness of mobile
learning and informal learning
scenarios. The crucial data in
question refers to any data that
accurately measures the
effectiveness of mobile and
informal learning scenarios.
Therefore,the crucial data sought
by Mobile Learning and Informal
Learning Design Researcher can
be revealed through the
application of well-suited
methodologies that potentially
help garner the key data that will
help make informed decisions
about current and future designs of
learner-centred informal and
mobile learning environments and
scenarios.
The research conducted for this
paper is exploratory in nature,and
thus, will not be able to cover in
depth many of the methodologies,
concepts and topics surveyed. It is
helpful to view this work as a
medium to encourage thoughtful
discourse and to continue an
ongoing dialogue regarding how
MobileHCI and Mobile Design
research methodologies can be
implemented by Informal and
Mobile Learning Researchers to
gain the crucial data to help
inform how individuals learn within
the context of mobility and
informal learning contexts.
1.1.The Context of Mobile and
Informal Learning Research
One of the greatest challenges
67
www.wlecentre.ac.uk
67

facing Informal and Mobile
Learning Researchers is gathering
large sets of quantitative and
qualitative data from various
observable and non-observable
phenomena within a specific
context or setting. The purpose of
gathering such data can be seen
as crucial in helping to evaluate
the appropriateness and
effectiveness of informal learning
and mobile learning scenarios.
Additionally,due to the ubiquitous
and pervasive nature of mobile
and informal learning it is no easy
task to conduct quantitative
research in natural and context
specific settings with large numbers
of study participants.Therefore,it is
essential to identify and apply the
most effective and appropriate
research methodologies in order to
achieve the desired results of
gathering useful data within a
specific setting and context.
Bearing this challenge in mind let
us reflect and ask what research
methods can help Mobile Learning
and Informal Learning Design
Researchers gather the data
needed?
According to Jensen & Skov (2005)
it is useful to investigate research
methods derived from different
disciplines as these research
methods can help inform on future
directions and influences on a
particular discipline. This paper
argues that the research
methodologies of Mobile HCI and
Mobile Design are ideally suited to
Mobile Learning and Informal
Learning Design and will help face
the challenge of gathering large
sets of quantitative and qualitative
data within a natural setting and
context in order to evaluate the
effectiveness and appropriateness
of informal learning and mobile
learning scenarios.
1.2. Real world learning
It can be observed that recent
advances in mobile information
and communication technologies
have not only increased individual
mobility,but have empowered and
enabled individuals to harness
mobile technologies for the
purpose of using them to augment
and enhance formal and informal
learning contexts. Smaller more
powerful mobile devices with
network connectivity are enabling
individuals to engage in novel
learning situations that are not
easily observable due to the
ubiquitous and pervasive nature of
informal and mobile learning
contexts. Additionally, it can be
argued that the very mobile
information and communication
technologies that enable these
new modalities of learning can
also be used to help Researchers
observe and gather data on
informal and mobile learning
scenarios.
Many MobileHCI and Mobile
Design research methods harness
the mobile technologies and
engaged the users themselves to
assist in the evaluation of the
accessibility, usability, and
appropriateness of mobile devices
and services. Therefore, one of the
unique characteristics of the very
technologies associated with
Mobile Learning and Informal
Learning is that the technologies
used can be harnessed to help
Mobile Learning and Informal
Learning Design Researchers
conduct large-scale quantitative
research to help gather important
68
Learner Centred Design

69
www.wlecentre.ac.uk
69
(crucial) data from real world
mobile and informal learning
scenarios which will could
ultimately help broaden and
advance current methodologies
used to evaluate informal and
mobile learning contexts.
2. Harnessing MobileHCI
Research Methods
The following MobileHCI and
Mobile Design research methods
highlighted are grounded within
the methodological theories of:
Action Research, Ethno-
methodology, Participatory Design
and User Centred Design, which
can be harnessed by Informal and
Mobile Learning Researchers in
order to gain the data which will
inform the effectiveness of mobile
learning and help evaluate
informal learning situations. The
“different research methods have
been adapted in research projects
[ ].This is no different than other
disciplines, but it is important to
understand how research methods
have been adapted in different
disciplines as it potentially informs
us on future directions and
influences on the discipline
(Kjedskov & Graham, 2003).
Wynekoop & Congor (1990) have
conducted a review of research
methods in which they created a
classification scheme to help in
their analysis. A summary of
existing MobileHCI /Mobile Design
research methods (See Table 1.)
has been adapted from the
research of Kjedskov & Graham
(2003), and Jensen & Skov (2005) to
reflect the Wynekoop & Congor
classification of the most common
research methodologies. This
summary highlights the strengths,
weaknesses and uses of various
methods based upon the
Environment:
Method:
Strengths:
Weaknesses:
Use:
Natural Setting
Case Studies
Natural setting, Rich
data
Time consuming,
Cannot be generalized
Descriptions, explanations,
developing hypothesis
Field Studies
Natural Settings,
Replicable
Difficult data collection,
Unknown sample bias
Studying current practice,
Evaluating new practices
Action
Research
First-hand experience,
Applying theory to
practice
Ethics, bias, time
consuming, Cannot be
generalized
Generation & Testing of
Theories / Hypotheses
Artificial Setting
Laboratory
Experiments
Control over variables,
Replicable
Limited realism, Cannot
be generalized
Controlled experiments,
Theory/Scenario testing
Environment
Independent
Survey
research
Easy, low cost, can
reduce sample bias
Applied
Research
Learning scenarios can
be evaluated
May need further design
to make learning
scenario applicable
Scenario development,
testing hypothesis and
concepts
Basic
Research
No restrictions on
solutions, Solve new
problems
Costly, time demanding,
may produce no solution
Theory building
Normative
writings
Insight into first-hand
experience
Opinions may influence
outcome
Descriptions of practice,
building frameworks
Table 1.Summary of existing research methods. (Adapted from Kjedskov & Graham,and Jensen & Skov)

environmental setting the research
is conducted.
The research of Hagen, Robertson,
Kan and Sadler (2005)
demonstrates the emergence of
new research methods used within
the MobileHCI and Mobile Design
communities. These methods are
categorised and presented as an
extension and combination of
existing MobileHCI and Mobile
Design research methods that
evaluate mobile technology
usage. Three main categories
highlighted in their research
“represent various approaches to
accessing and making available
data about different aspects of
mobile technology use, [and]
entail different roles and
responsibilities for both researchers
and participants.”(Hagen 2005)
The following three categories are
as follows and have been
annotated to apply to a learner
centred context and setting:
1.Mediated Data Collection: In
which participants [learners] and
mobile technologies mediate data
collection about use in natural
settings and situated learning
context.
2.Simulations and Enactments:
simulations and enactments are
used to make available
experiential information sensitized
to real contexts of use.
3.Combinations: existing methods,
and/or mediated data collection
and/or simulations and
enactments are combined to allow
access to complementary data.
(Hagen, 2005)
A summary of the above
mentioned approaches are
highlighted below (See Table 2.)
including the description and
derivation of use from established
methods from which these new
approaches are borne.
The above summary of existing
and emerging research methods
used by the MobileHCI and Mobile
Design communities highlights
many new and novel approaches
in acquiring quantitative and
qualitative data in order to
evaluate mobile technology
usage. In conclusion, the question
remains as to why and to what
extent and under what
circumstances are the specific
MobileHCI and Mobile Design
research methods and
approaches are (or, are not)
transferable to the research
conducted by the Mobile Learning
and Informal Learning Design
communities.
70
Learner Centred Design
APPROACH:
DESCRIPTION:
DERIVED FROM:
Mediated Data Collection
Where access to data about actual use practices is mediated by
both Learner & technology combined
•Learner-centred
Learners conduct the data collection using mobile devices.
Self-reporting, Diaries, Probes
•Automated
Learners engage in learning (m-learning scenarios) while data
about use, content and metadata is logged automatically
Use/Data logs
•Mobile recording
Learners go about their normal routines while wearing sensors or
cameras.
Video-observation, Use/Data logs
Simulations & Enactments
Methods for allowing immersive scenarios in which data about
existing or potential use is accessed through some form of
pretending.
•Simulations
Physical, ergonomic or environmental props are used within a
controlled environment in order to simulate m-learning scenarios.
Lab tests, Scenarios, Heuristics,
Prototypes, Emulators, Simulators
•Enactments
Mobile-learning scenarios are played out through visual imagery
or storytelling in order to observe potential outcomes.
Prototyping Scenarios, Role-playing,
Work shopping, Storyboarding
Combinations
Various established and/or new methods are combined to enable
access to complementary data.
Table 2.Emerging Research Methods in MobileHCI.(Adapted from Hagen,et.al.).

3. Conclusion
This paper has briefly surveyed
research methodologies from the
fields of MobileHCI and Mobile
Design in order to suggest and
evaluate the applicability of these
methods to Mobile Learning and
Informal Learning Design research.
In order to determine if research
methods from MobileHCI and
Mobile Design could effectively be
transferred to Mobile Learning and
Informal Learning Design research
it is important to question as to why
and to what extent and under what
circumstances are MobileHCI and
Mobile Design research methods
and approaches transferable to
Mobile Learning and Informal
Learning Design research.
Furthermore,it is important to
question which specific criteria can
be used to judge transferability
and investigate if there are specific
reasons why MobileHCI and
Mobile Design methods would not
be transferable to Mobile Learning
and Informal Learning Design? This
paper will not be able to address
these questions here at this time,
but encourages further evaluation
in subsequent papers in order to
properly evaluate the
transferability of the methods
survey to the repertoire of Mobile
Learning and Informal Learning
Design research methods and
approaches.
What is special about Mobile
Learning and Informal Learning
Design research in relation to
MobileHCI and Mobile Design is
the element of an embedded
pedagogy (or learning design)
inherent in the learning scenarios
evaluated.One of the primary
goals of Mobile Learning and
Informal Learning Design research
is to evaluate the learning and
developmental outcomes of the
individuals. Bearing this in mind it is
possible that MobileHCI and
Mobile Design methodologies are
more suited to informing and
evaluating aspects of usability and
accessibility issues, but cannot truly
evaluate learning and
developmental outcomes of
individuals.
The importance of highlighting
current and emerging MobileHCI
and Mobile Design research
methods is that they are grounded
within the methodological theories
of: Action Research, Ethno-
methodology, Participatory Design
and User Centred Design, which
can be easily adopted, adapted
and augmented into Mobile
Learning and Informal Learning
Design research. The flexibility of
choosing research methods
derived from different disciplines
may open new doors to help
gather the crucial quantitative and
qualitative data needed in order to
properly evaluate the effectiveness
of informal and mobile learning
scenarios; which ultimately will
place the learners at the centre of
research and design and help
them achieve their learning and
developmental goals through the
appropriate informal and mobile
learning scenarios.
71
www.wlecentre.ac.uk
71

72
Learner Centred Design
References
Hagen, Penny., Toni Robertson, Melanie Kan, Kirsten Sadler. (2005)
Emerging Research Methods for Understanding Mobile Technology Use.
Proceedings of OZCHI 2005, Canberra, Australia. November 23 25, 2005.
Jensen, Janne., Mikael B. Sokov. (2005) A Review of Research Methods in
Children’s Technology Design. In ICD 2005: June 8-10,2005, Boulder
Colorado, USA. ACM Press, (pp. 80-87)
Kjeldskov, J.and Graham, C.(2003) A Review of MobileHCI Research
Methods. In Proceedings of the 5th International Conference on Mobile
Human-Computer Interaction (MobileHCI03), LNCS: Springer-Verlag. (pp.
317–335)
Wynekoop, J. L.and Congor,A.A. (1990) A Review of Computer Aided
Software Engineering Research Methods. In Proceedings of the IFIP TC8
WG8.2 Working Conference on the Information Systems Research Arena
of the 90’s, Copenhagen, Denmark.
Bibliography for Further Reading
Axup, J., Bidwell, N. J., & Viller, S. (2004). Representation of self-reported
information usage during mobile field studies: Pilots & Orienteers 2.
Proceedings of OzCHI 2004: Supporting Community Interaction:
Possibilities and Challenges, Wollongong Australia.
Consolvo, S., & Walker, M., (2003). Using the Experience Sampling Method
to Evaluate Ubicomp Applications. In IEEE Pervasive Computing,2 (2), pp.
24-31.
Gabrielli, S, Mirabella, V., Kimani, S., & Catarsi, T., (2005). Supporting
Cognitive Walkthrough with Video Data: A Mobile Learning Evaluation
Study. In Proceedings of MobileHCI 2005,ACM Press, pp. 77–82.
Intille, S. S., Tapia. E. M., Rondoni, J., Beaudin, J. Kukla, C.,Agrwal, S., Bao, L,
& Larson, K.,(2003). Tools for Studying Behavior and Technology in
Natural Settings. In Proceedings of Ubiquitous Computing 2004,pp.
157–174.
Intille, S.S., Bao, L., Munguia Tapia, E., Rondoni, J.: Acquiring in situ training
data for context-aware ubiquitous computing applications. In
Proceedings CHI (2004) 1-8
Intille, S., Larson, K., Beaudin, J., Nawyn, J., Munguia Tapia, E., Kaushik, P.: A
living laboratory for the design and evaluation of ubiquitous computing
technologies. In Proceedings of CHI Extended Abstracts (2005) 1941-1944
Johnson, P. (1998).Usability and mobility: interaction on the move.In
Proceedings of First Workshop on Human-Computer Interaction with
Mobile Devices, Glasgow, UK.
Kjeldskov J. and Stage J. (2004) New Techniques for Usability Evaluation of
Mobile Systems. International Journal of Human Computer Studies
(IJHCS) Elsevier.