Usability Evaluation of a LMS Mobile Web Interface

Abstract

The growth in the use of mobile devices in everyday life determined an increasing demand to access educational information using mobile technology. Nevertheless, most existing computer based Learning Management
Systems still do not have advanced access support for mobile devices. This paper presents the basis of an in-progress research, aiming to enhance our understanding of mobile learning usability considerations and measurement. It also provides the starting point for performing a usability evaluation of the MyMobile web interface of our university’s Moodle LMS. The main contribution of this paper is our proposed approach, based on four perspectives that have to be considered when designing or testing a mobile learning solution: Pedagogical usability, Usability of the device, Usability of the content and Usability of the mobile web interface. Additionally, metrics, methods and guidelines for usability testing are given. The main goal of the paper is to provide an overview on a proposed framework for testing and optimizing a LMS mobile web interface from a usability perspective.

Full text

Usability evaluation of a LMS mobile web interface
Daniel Ivanc, Radu Vasiu, Mihai Onita
Communications Department, “Politehnica” University of Timisoara
Bv, V. Parvan nr. 2, Timisoara, Romania
{dan.ivanc,radu.vasiu,mihai.onita}@cm.upt.ro
Abstract. The growth in the use of mobile devices in everyday life determined
an increasing demand to access educational information using mobile
technology. Nevertheless, most existing computer based Learning Management
Systems still do not have advanced access support for mobile devices. This
paper presents the basis of an in-progress research, aiming to enhance our
understanding of mobile learning usability considerations and measurement. It
also provides the starting point for performing a usability evaluation of the
MyMobile web interface of our university’s Moodle LMS. The main
contribution of this paper is our proposed approach, based on four perspectives
that have to be considered when designing or testing a mobile learning solution:
Pedagogical usability, Usability of the device, Usability of the content and
Usability of the mobile web interface. Additionally, metrics, methods and
guidelines for usability testing are given. The main goal of the paper is to
provide an overview on a proposed framework for testing and optimizing a
LMS mobile web interface from a usability perspective.
Keywords: mobile learning, usability, metrics, mobile devices
1 Introduction
Mobile technology is recognized to be one of the most significant directions of
nowadays knowledge based society concept. The growth in the use of mobile devices
as mobile phones, smartphones, tablets, netbooks and notebooks in everyday life
determined an increasing demand to access educational information, instruction and
tools using mobile technology. Many studies have given encouraging results for using
mobile technologies to support students in the teaching and learning process.
Nevertheless, most existing computer based Learning Management System still do
not have advanced access support for mobile devices, and there are deficiencies in
cross-platform these solutions. Even more, many mobile device’s browsers do not
support scripting or plug-ins, and do not have available memory to display desktop
web-pages and graphics. This influences a lot the usability of mobile learning
systems. Also, web content that is mostly the format of electronic learning content is
not always suited for mobile browsers and the ability to display information in various
multimedia formats is limited. These usability issues of mobile devices and learning

must be considered when different types of interfaces and device oriented applications
are developed and tested [1].
Moodle is one of the most popular open-source e-Learning platforms. It has proven
to be a serious competitor to other paid worldwide-known solutions and is usually the
first choice when a low-cost, robust eLearning solution is needed. It has been used for
several years in the eLearning Centre of our university and several add-ons and tools
have been developed.
Because very few open source mobile extensions for Moodle were available and
none complied with our needs, the first option was to develop a new mobile user
interface for Moodle. This was needed not only for obtaining a functional access to
the platform by using mobile devices, but also in order to test and to analyze usability
issues in mLearning and especially to develop a usability testing methodology for
mLearning because our research revealed that there is a lack of studies on this aspect.
However the release of the 2.0 version of Moodle provided a new way of developing
mobile friendly interfaces and the release of MyMobile Moodle mobile-optimized
interface, based on jquerymobile was considered and later adopted for providing
mobile access to the eLearning platform and for continuing the research on
mLearning usability testing.
Starting from the question of whether there are specific methods and metrics for
mLearning usability testing, the goal of this paper is to provide an overview on our
work in progress on developing a framework for testing and optimizing a LMS
mobile web interface from a usability perspective. It provides an introduction to
mobile learning usability testing, presents several usability testing methods and
metrics, selected or adapted from literature review, the setup needed for the testing,
and the proposed framework for the design and implementation of usability testing of
a LMS mobile web interface.
2 Mobile Learning and Usability
2.1 Defining Usability
ISO/IEC 9126-1 defines usability as “the capability of the software product to be
understood, learned, used and be attractive to the user, when used under specified
conditions”. This definition is primarily concerned with a software product; however,
it can be applied to mobile learning software taking into consideration features
specific to mobile phones and eLearning aspects.
ISO 9241-11 defines usability as: “the extent to which a product can be used by
specified users to achieve specified goals with effectiveness, efficiency and
satisfaction in a specified context of use” and suggests the possibility of using specific
metrics for measuring performances.
A challenge with definitions of usability is that it is very difficult to specify what
its characteristics and its attributes should be, in particular because the nature of the
characteristics and required attributes depend on the context in which the product is
used [2]. Three critical elements are also revealed: usability of the product relates to

specific users, the ones that it was designed for; these specific users have to follow
some specified goals for the product was designed for; a specific context of use.
2.2 Mobile learning usability testing
Most of the mobile devices that are used in mobile learning were not designed for
educational purpose, and usability issues continue to be the most serious restraining.
Frequent approaches to usability incline to be limited to metrics relating to time
taken to complete a task, effort, throughput, flexibility and the user’s satisfaction and
attitude. Usability has to be considered in a different manner when it is being
evaluated in the context of teaching and learning, and the concept of pedagogical
usability can be helpful when considering the close relationship between usability and
pedagogical design.
2.3 A four perspectives approach
The usability is reviewed as a determinant factor of success of an educational
platform and it should be given special attention also when talking about accessing it
on mobile devices that were not designed for educational purposes. Based on the fact
that the device’s and UI usability issues influence in different ways the user
experience [19], and on the fact that mobile learning is usually considered an
extension to eLearning and have similar characteristics, we consider that the usability
of a mLearning platform has to be regarded from four different perspectives that are
presented in Table 1 and described in the following paragraphs.
Table 1. A four perspectives usability testing approach
Device
Web User Interface
Usability
Usability
Pedagogical
Mobile Learning
Usability Testing
Educational
Usability
Content Usability
2.3.1 Pedagogical usability
Pedagogical usability is the analysis of the way an educational application (tools,
content, tasks and interface) supports students in their learning process within various
learning contexts according to learning objectives. It should be especially concerned
with educational aspects such as the learning process, purposes of learning, user’s
needs, the learning experience, learning content and learning outcomes [3].
When developing or improving a mobile learning application is not sufficient to
intend that people can use it, they must want to use it [4].
Some relevant pedagogical usability metrics and a description of what each of the
usability metrics measures are provided in Table 2. Our research on pedagogical
usability testing needs to go further and establish practical methods for evaluating the
mLearning platform from the pedagogical point of view.

Table 2. Pedagogical usability metrics and their measurements [3]
Metric
Measurement
Instruction
Whether the application’s instruction is clear or if it needs a
lecturer’s intervention.
Learning
content
relevance
The extent to which the content of the application supports
students in their learning.
Learning
content
structure
The degree to which the content of the application is organized
in a way [clear, consistent and coherent] that supports learning.
Tasks
The extent to which the tasks performed on the application help
students to achieve their learning goals.
Learner
variables
The degree to which learner variables are considered in the
application.
Collaborative
learning
The extent to which the application allows students to study in
groups.
Ease of use
The provision by the application of clear directions and
descriptions of what students should do at every stage when
questions to the mobile class activities are answered.
Learner
control
The characteristics possessed by the application that allow
students to make instructional choices.
The extent to which the learning material is broken down into
meaningful units.
The extent to which the learning material in the application is so
interesting to students that it compels them to participate.
Motivation
The degree to which the application motivates students.
2.3.2 Usability of the device
Any usability testing methodology has to account for the current limitations of mobile
devices that are supposed to be used as learning mediums and has to provide
information on the ease-of-use, effectiveness and efficiency of the mobile user
interface. There are a series of known usability issues of the mobile devices that can
affect the overall usability of the mLearning system: small screen sizes, low screen
resolution or the form factor, low storage capacity and network bandwidth, limited
processor performance, compatibility issues, lack of data input capability, short
battery life, the use of the devices more often on the move. There are also a multitude
of mobile user interfaces types with their own usability issues: scroll-and-select
interfaces, tilting/sensor based interfaces, speech-based user interfaces, interfaces that
use a stylus, and nowadays most common touch interfaces. Other specific
characteristics are provided by the different operating systems interaction facilities
and navigation scheme of the applications. There are a wide variety of devices,
possessing different characteristics, which the application must be adaptable to all of
them. These usability issues of mobile devices must be considered and carefully

examined during the usability testing of a mobile learning interface in order to select
an appropriate research methodology and reduce the effect of contextual factors in the
usability testing outcomes [5].
2.3.3 Usability of the educational content
There could be usability issues regarding the content delivered by the mLearning
application or web user interface. The format of electronic learning content is not
always compatible to most mobile browsers, scripts and plug-ins are usually not
supported and the ability to display information in various multimedia content is
limited. Integrated graphics and animation should be provided in compatible mobile
formats as well as audio and video files. Placing large images, video, PDF, MS Office
files and usually any types of similar resources directly on the front of course pages
should be avoided, links to images and video via activities, resources and pages is
recommendable to be used. These and similar multimedia content compatibility issues
must be considered when structuring the mLearning content.
As a result of the usability testing of the content/courses provided by the mobile
web interface we intend to elaborate a guideline for designing, developing and
especially organizing content in a Moodle course for best use on mobile devices.
2.3.4 Usability of the mobile web interface
In this case, the mobile web interface strictly refers to the elements and structure of
the mobile-friendly MyMobile Moodle interface. The usability testing can be similar
to any other mobile web-page or application, and unlike the other perspectives, it can
be evaluated using heuristic methods. The goals of the usability testing, in this case
are: discovering navigation issues, improving the positioning and use of the menu
elements, discovering bugs and verifying compatibility and interoperability of the UI
elements on different devices, assuring efficiency, learnability and satisfaction in
using the system and completing the tasks.
3 Research questions and Test preparations
Our first research question was if we could use the classical methods and techniques
used in computer web interfaces usability testing for mLearning web interface
usability testing. Even if there is not much literature on this topic, the main answer
was that these methods are usually used with the observation that some need to be
adapted to be used for mobile devices and some cannot be used at all because of the
lack of technical solutions or specific software. There is a need in developing specific
methods in evaluating mobile learning applications and we are looking forward to
researching and providing such methods.
The second research question was if the metrics used in computer web interfaces
usability testing could be used for our purpose. The review of mobile UI usability

testing guidelines from the research literature reveals some specific metrics that we
proposed to use for mLearning user interface testing.
The third research question is what metrics of the above are best to be used in order
to evaluate the usability of a mobile web interface of LMS?
3.1 Users, materials, devices and location
Usability testing of the mLearning MyMobile web interface for our Moodle
eLearning platform will be performed by four groups of students and one group of
experts:
─ students that are already using our university’s Moodle eLearning platform, that
are used with the eLearning platform’s characteristics and with the course format
─ future users of the mLearning platform which are not used with the existing
eLearning platform
─ students that use their own mobile devices (used with the specific UI)
─ students that use the provided mobile devices
─ experts from the eLearning department with professional background in the
domain
There are three types of devices that are proposed to be used for the evaluation:
─ old-generation smartphones with medium-size display and normal or full keyboard
─ new-generation smartphones / superphones with full touch-screen displays
─ new-generation tablets
We also want to test the ability of these devices to support technologies used by the
MyMobile UI. Their features also dictate whether certain UI features will be
rendered, identified and visible on these devices.
The usability test must set up in a way that is as close to the normal context as
possible. Testing in a usability lab offers some benefits because it can be designed to
create the ideal testing environment by using specific equipment for recording the
session, controlled logging software, special equipment for testing mobile devices. On
field and remote testing is much closer to the normal context of use and requires
special applications that have the ability to automatically collect user interface events
as the user interacts with the application, to capture screenshots and eventually, make
an audio-video recording of the user. Even if closer to the real life use of the final
product, testing out of the lab has specific disadvantages [6][7].
The testing will take place in one of the universities laboratories, a quiet and air-
conditioned room, equipped with audio-video recording system.
One of the difficulties was to find a mobile and tablet usability testing kit [8]
wishing to capture the full context of usage. There are some testing methods which
use a static camera, telling the user to put a mobile device on a table, operating it
within a designated area (where the camera is pointing). Such testing fails to capture

the true experience of real usage context. It does not allow the user to hold the device
in his hand, and operate it as he or she would do in real life scenarios. More than that:
the fact that the user is instructed not to move the device from a designated area, adds
on stress during testing, which in itself is an un-wanted cognitive load, thus being a
negative interference.
The conditions considered when building a homemade mobile and tablet usability
testing kit are:
─ the testing device should allow users to hold the mobile device (smartphone, tablet)
the way they would hold it in real life
─ the testing device should be flexible to fit many types of mobile devices, which
differ in size, and in screen resolution.
─ the device should look professional and communicate trust
The solution that meets the challenges:
─ GPS mounts and mobile device holders
─ a high-end HD camera, which proves to produce high quality outcome in various
light conditions, with autofocus
─ a glass-fiber lead to be used as a camera mount which gives both stability and
flexibility. This allows to adjust the camera to fit many screen sizes, and also keeps
a steady picture when the user moves the device around while using it.
3.2 Methods
User-based evaluation methods are mostly used in mLearning usability testing. These
methods involve collecting quantitative and qualitative data from users while or after
running through well prepared scenarios. Users are invited to do typical tasks with a
product, or simply asked to explore it freely, while their behaviors are observed and
recorded in order to identify design flaws that cause user errors or difficulties. During
these observations, the time required to complete a task, task completion rates, and
number and types of errors, are recorded. [9]. Users are also asked to complete
questionnaires/surveys that provide useful qualitative data from users. Although the
data collected is subjective, it provides valuable information on what the user wants.
Professional observers and audio-video recording are also used to provide useful
information by using the “think aloud protocol” and by measuring performance data.
Traditional laboratory testing can be complemented by using UI event logging
systems that provide useful information for measuring performance data and usability
metrics [10].
Table 3. Usability evaluation methods [11]
User testing
Thinking aloud
protocol
User speaks aloud about what he is thinking
throughout the evaluation.
Wizard of Oz
Log file analysis
Observation
This is done to watch the action in its setting,

and recording as many details as possible,
without obstructing the experience of the
subjects being studied.
Measuring
performance
Inquiry
Cognitive walk-
through
This aims to complement observation, in order
to get more insight into what is observed.
Heuristics evaluation
This involves Usability Experts and requires an
interactive prototype.
Guidelines review
Interviews
The researcher formulates questions about the
product, based on the issues of interest.
Interview representative users are then asked
these questions, in order to gather desired
information
Inspection
Questionnaires for satisfaction
Questionnaires for preferences
Questionnaires for cognitive workload
3.3 Metrics
Upon review of the measures’ relative appearance in the reviewed literature the core
constructs for the measurement of usability appear to be:
─ Efficiency: Degree to which the product is enabling the tasks to be performed in a
quick, effective and economical manner or is hindering performance
─ Effectiveness: Accuracy and completeness with which specified users achieved
specified goals in particular environment
─ Satisfaction: The degree to which a product is giving contentment or making the
user satisfied
These three dimensions also reflect the ISO 9241 standard making a strong case for
its use in related future studies. [12]
Based on the GQM model, Table 4 provides a list of usability characteristics,
goals, guidelines and metrics that can help to collect quantitative or qualitative data
during the usability evaluation process [13].
Table 4. Measures, goals, guidelines and metrics
MEASURE
GOAL
GUIDELINE
METRIC
Effectiveness
Accessibility
Ease of understanding
content
Time taken to understand the
content

Help
Ease of navigation of
help topics
Is/ not easy to learn how to
navigate help topics
Interactivity
Ease of interaction
Amount of interaction required
Ease of use
Ease of customization
Is or not easy to use
Allow/ not allow customization
Navigation
Ease of navigation
Provide/ not provide easy
navigation
Time taken
Effort
required
Loading application
Time to learn
Time taken to respond
Amount of task effort
Time taken to load the application
Time taken to learn the application
Number of mistakes made before
knowing how to use
Time taken before to respond
Amount of task effort required
Amount of time taken before
knowing what to do
Time taken to complete a task
Number of time the user follows
the wrong path when attempting a
task
Features
Provision of task related
clues.
Provision of error
recovery
Provision of help.
Allow personalization
Organize content
appropriately.
Coherent multimedia
usage.
Use of appropriate
controls.
Narratives to structure
the content.
Provide/not task related clues
Provide/not error recovery
assistance
Provide/not help
Allow/ Not cater for
personalization
Rating scale of content
organization
Rating scale of multimedia usage
Use or not appropriate controls
Use or not narratives for content
structure
Rating scale of narratives
appropriateness
Satisfaction
Familiarity
Mental models
Use or not familiar mental models
Rating scale of user familiarity
with user interface
Consistency
Navigation
Rating scale of consistency during
navigation across system
Is or not easy to navigate
Attractiveness
Use appropriate font
style, size, and colors
Rating scale on whether the system
is attractive
Is or not attractive
Help
Sufficient help
information
Provide or not sufficient help
information

Organization of help
topics
Rating scale of usefulness of help
information
Preciseness
Messages precision
Provide or not precise messages
Rating scale of message
preciseness
Feedback
Helpful messages
Suitability for all users
Provide or not helpful feedback
messages
Provide or not suitable feedback
messages for all categories of
users.
Usability can also be defined by other several quality components [14][15]:
─ Learnability: how easy users accomplish their basic tasks the first time they
encounter the design.
─ Efficiency: number of steps it takes for a user to complete a task
─ Memorability: how easy is to memorize how to use the interface of the system and
how easily users can reuse the system after a break
─ Errors: how many errors do users make using the interface of the system and how
serious are these errors?
─ Satisfaction: how do users like using the system’s interface
─ Understandability (display load, clarity of operation possibilities, completeness of
operation menu)
─ Operability (ease of input entering, display self-adjustment possibilities, messages
conciseness, ease of output use, parameters self-adjustment possibilities, tasks
based on user location)
─ Attractiveness (ease of use - displays per output, ease of use - displays per task)
Mathematical formulas, the methods they were obtained and methods of
application for some of the listed usability metrics can provide the source for
developing new personalized metrics [16][17].
Based on testing methods and metrics, a usability testing framework for developing
or testing mobile learning applications or mobile web interfaces can be structured and
useful usability design guidelines can be modeled [18].
This paper presents a work in progress and the reviewed and listed metrics have to
be also analyzed from a pedagogical perspective so that only relevant metrics are
used.
The final step would be the analysis and interpretation of the collected data and the
use of the results in improving the mobile web user interface of the LMS and the
educational content provided.

4 Usability Testing Framework
Fig. 1. A Framework for the Design and Implementation of Usability Testing of a LMS mobile
web interface

In this section, the authors are proposing a generic framework (Figure 1) for the
design and implementation of the usability testing of a LMS mobile web interface.
The design of the usability testing should start with identifying research questions
and objectives, as these determine the selection of the methods, location, tools and
metrics used during the test.
A survey should be done before choosing the mobile devices to be used for the test
as they should be representative for the majority of the users of the system and should
include different types of devices in terms of operating system, keyboard, display size
and multimedia characteristics.
The selected representative mobile devices should be used in the first stage for
testing their own usability issues and the possibility to display the educational content
provided by the learning platform. Experimented users should be used in identifying
the main usability issues of the devices and educational content that could influence
the UI usability testing and the pedagogical usability testing. Furthermore this is the
step where experts are selecting the usability attributes to be measured, methods and
metrics that should be used in the further main usability test, that are best to answer
the research questions and objectives.
After deciding if a laboratory or a field testing is required, and after determining
the representative user profile of the learning platform, the next important step is
selecting the usability attributes to be measured, methods and metrics that are best for
answering the research questions and objectives from the UI and pedagogical
usability testing points of view.
The guidelines for the main usability test, the setup of the environment and
equipment, the preparing of questionnaires and all the necessary preparations can be
done based on the results of the previous steps.
Running the actual usability testing, data collection, analysis and interpretation, are
the last steps, followed by required changes of the mLearning platform.
As this is a research in progress paper, the proposed framework still has to be
improved, detailed and validated by a real test.
5 Discussions
The first contribution of this paper is an usability testing approach based on four
perspectives that have to be considered when designing or testing a mobile learning
web interface: Pedagogical usability, Usability of the device, Usability of the
educational content and Usability of mobile web interface.
There certainly are direct usability influences among the four domains. For
example, usability issues of mobile devices accentuate usability issues of the UI [19].
Therefore, because the school cannot ask the students to change their mobile devices,
usability improvements should be done to improve the user experience on the existing
devices.
The existing educational content cannot be easily changed, but changes can be
easily done in the way courses are structured, and multimedia content can be adapted
be used both on eLearning or mLearning web interfaces.

Further research should be done on understanding the connections and influences
between the usability issues that belong to each of the four usability testing
perspectives and identify the methods and metrics that can reveal best results when
used in a realistic usability test.
Concerning usability metrics that were mentioned in this paper, the next step
should be the selection of the measuring formulas, or developing specific ones based
on quality metric frameworks [16] for learning environments, and also the preparation
of specific questionnaires elements, as our review of the literature indicates a lack of
specific usability metrics for mobile learning environments.
6 Conclusions
Even if it is one of the most popular open-source eLearning platforms, Moodle is still
in its beginnings in providing educational content on mobile devices. One of the
biggest challenges that it has to overcome is the usability issue that mobile devices,
mobile applications and web interfaces reveal.
As the usability is reviewed as a determinant factor of success of an educational
platform, it is very important to achieve a high usability level also on the mobile web
interface.
This research-in-progress presents the basis of a continuing research, which aims to
enhance our understanding of mobile learning usability considerations and
measurement and it provides the starting point for performing a usability evaluation
of the MyMobile web interface of Moodle LMS. The proposed framework still has to
be improved, detailed and validated by a real test. Additionally, metrics, methods and
guidelines for usability testing were given. However, in order to test their
applicability, they have to be applied on the real mLearning web interface with real
users in the form of an actual evaluation.
Research progress in usability design and testing guidelines, focusing on
pedagogical usability also, rapidly improvements in the usability of mobile devices
and their performances will influence in a good way the development of mobile
learning.
Acknowledgments: “This work was partially supported by the strategic grant
POSDRU/88/1.5/S/50783(2009) of the Ministry of Labor, Family and Social
Protection, Romania, co-financed by the European Social Fund–Investing in people.”
References
1. Bekim Fetaji and Majlinda Fetaji. Analyses and review of M-learning feasibility, trends,
advantages and drawbacks in the past decade (2000-2010). In Proceedings of the 5th
European conference on European computing conference (ECC'11), World Scientific and
Engineering Academy and Society (WSEAS), Stevens Point, Wisconsin, USA, 2011,
pp.474-479.

2. Abran, A., Khelifi, A., Suryn, W.,& Seffah, A.(2003). Consolidating the ISO usability
models. Proceedings of 11th International Software Quality Management Conference
(Springer), April 23 – 25, 2003. Glasgow, Scotland, UK
3. Kukulska-Hulme, A. Mobile usability in educational contexts: What have we learnt?
International Review of Research in Open and Distance Learning, 8(2):1-16, 2007
4. Pakiso Joseph Khomokhoana, Using mobile learning applications to encourage active
classroom participation: Technical and pedagogical considerations, Department of
Computer Science and Informatics, University of the Free State, Bloemfontein - South
Africa, May 2011, pp. 58
5. Bekim Fetaji and Majlinda Fetaji. 2011. Analyses and review of M-learning feasibility,
trends, advantages and drawbacks in the past decade (2000-2010). In Proceedings of the
5th European conference on European computing conference (ECC'11), Remi Leandre,
Metin Demiralp, Milan Tuba, Luige Vladareanu, and Olga Martin (Eds.). World Scientific
and Engineering Academy and Society (WSEAS), Stevens Point, Wisconsin, USA, 474-
479
6. Carol M. Barnum, Usability Testing Essentials: Ready, Set...test!, ISBN: 978-0-12-
375092-1, Elsevier, 2011
7. A. Kaikkonen, A. Kekäläinen, M. Cankar, T. Kallio, and A. Kankainen. Usability testing
of mobile applications: A comparison between laboratory and field testing. Journal of
Usability Studies, 1(1): 4--17, 2005.
8. Spotless Interactive - http://www.spotlessinteractive.com/news/mobile-usability-testing-
kit.php , January 2012
9. Mary Ann Chiramattel Kunjachan, Evaluation Of Usability On Mobile User Interface,
University Of Washington, Bothell
10. Xiaoxiao Ma, Bo Yan, Guanling Chen, Chunhui Zhang, Ke Huang, and JillDrury, A
Toolkit for Usability Testing of Mobile Applications, Computer Science Department,
University of Massachusetts, MobiCase 2011
11. Bernhaupt, R., Mihalic, K. & Obrist, M. (2008): Usability Evaluation Methods for Mobile
Applications. In: Lumsden, J. (Ed.). Handbook of Research on User Interface Design and
Evaluation for Mobile Technology, IGI Global. 745-758.
12. Coursaris, C., and Kim, D., (2006). A Qualitative Review of Empirical Mobile Usability
Studies, Proceedings of the Americas Conference on Information Systems (AMCIS),
Acapulco, Mexico, August 4-6, 2006.
13. Kantore, A and van Greunen, D. Metrics for m-learning usability evaluation, WWW2010
September 2010, Durban.
14. Jakob Nielsen. Usability 101: Introduction to Usability, 2003.Available at
http://www.useit.com/alertbox/20030825.html
15. Ruti Gafni, Usability Issues in Mobile-Wireless Information System, Issues in Informing
Science and Information Technology Volume 6, 2009
16. Gafni, R. Framework for quality metrics in mobile-wireless information systems.
Interdisciplinary Journal of Information, Knowledge, and Management, 3, 23-38.2008
17. Gafni, R. Quality Metrics for PDA-based M-Learning Information Systems. A. Koohang,
Ed.) Interdisciplinary Journal of E-Learning and Learning Objects, 5, 2009
18. Fetaji Majlinda, Fetaji Bekim (2011). Devising M-learning Usability Framework. In
Information Technology Interfaces (ITI), Proceedings of the ITI 2011 33rd International
Conference, pp. 275 - 280. IEEE, New York, USA, 6 / 2011. ISBN 978-1-61284
19. Azham Hussain, Maria Kutar, Apps vs Devices: Can the Usability of Mobile Apps be
Decoupled from the Device?, IJCSI International Journal of Computer Science Issues, Vol.
9, Issue 3, No 3, May 2012 pp.11-16