ArticlePDF Available

Leveraging Microsoft’s Mobile Usability Guidelines: Conceptualizing and Developing Scales for Mobile Application Usability

Authors:

Abstract and Figures

This research conceptualizes mobile application usability and develops and validates an instrument to measure the same. Mobile application usability has attracted widespread attention in the field of human-computer interaction because well-designed applications can enhance user experiences. To conceptualize mobile application usability, we analyzed Microsoft’s mobile usability guidelines and defined 10 constructs representing mobile application usability. Next, we conducted a pilot study followed by a quantitative assessment of the content validity of the scales. We then sequentially applied exploratory factor analysis and confirmatory factor analysis to two samples (n=404; n=501) consisting of German consumers using mobile social media applications on their smartphones. To evaluate the confirmatory factor model, we followed a step-by-step process assessing unidimensionality, discriminant validity and reliability. To assess the nomological validity of our instrument, we examined the impact of mobile application usability on two outcomes: continued intention to use and brand loyalty. The results confirmed that mobile application usability was a good predictor of both outcomes. The constructs and scales associated with mobile application usability validated in this paper can be used to guide future research in human-computer interaction and aid in the effective design of mobile applications.
Content may be subject to copyright.
1
Please cite this article as:
Hoehle, H., Aljafari, R., and Venkatesh, V. “Leveraging Microsoft’s Mobile Usability
Guidelines: Conceptualizing and Developing Scales for Mobile Application Usability,”
International Journal of Human-Computer Studies (89:5), 2016, 35-53.
https://doi.org/10.1016/j.ijhcs.2016.02.001
Leveraging Microsoft’s Mobile Usability Guidelines:
Conceptualizing and Developing Scales for Mobile
Application Usability
Hartmut Hoehle
University of Arkansas
Department of Information Systems
228 Business Building
Fayetteville, AR 72701, USA
Ruba Aljafari
University of Arkansas
Department of Information Systems
228 Business Building
Fayetteville, AR 72701, USA
Viswanath Venkatesh
University of Arkansas
Department of Information Systems
228 Business Building
Fayetteville, AR 72701, USA
vvenkatesh@vvenkatesh.us
2
Leveraging Microsoft’s Mobile Usability Guidelines:
Conceptualizing and Developing Scales for Mobile
Application Usability
ABSTRACT
This research conceptualizes mobile application usability and develops and validates an
instrument to measure the same. Mobile application usability has attracted widespread attention
in the field of human-computer interaction because well-designed applications can enhance user
experiences. To conceptualize mobile application usability, we analyzed Microsoft’s mobile
usability guidelines and defined 10 constructs representing mobile application usability. Next,
we conducted a pilot study followed by a quantitative assessment of the content validity of the
scales. We then sequentially applied exploratory factor analysis and confirmatory factor analysis
to two samples (n=404; n=501) consisting of German consumers using mobile social media
applications on their smartphones. To evaluate the confirmatory factor model, we followed a
step-by-step process assessing unidimensionality, discriminant validity and reliability. To assess
the nomological validity of our instrument, we examined the impact of mobile application
usability on two outcomes: continued intention to use and brand loyalty. The results confirmed
that mobile application usability was a good predictor of both outcomes. The constructs and
scales associated with mobile application usability validated in this paper can be used to guide
future research in human-computer interaction and aid in the effective design of mobile
applications.
Keywords: Mobile application usability, mobile human-computer interaction, mobility,
continued use, survey
3
1 Introduction
Over the last decade, the use of mobile devices has grown exponentially, with worldwide
sales of more than 1.9 billion units in 2014 alone (Gartner Research, 2015). In many developed
countries, individuals often own more than one mobile phone (Thong et al., 2002; Fosso Wamba
and Chatfield, 2009; Fosso Wamba, 2012; Anand and Fosso Wamba, 2013). In conjunction with
these developments, mobile devices have become more sophisticated and recent models enable
individuals to interact with mobile applications on the go (Lal and Dwivedi, 2009; Harvey and
Harvey, 2014). Particularly, Internet-enabled smartphones are becoming increasingly popular
and recent reports found that smartphones accounted for more than 60 percent of mobile phone
sales in 2014, resulting in smartphone sales surpassing traditional mobile phone sales (Gartner,
2015).
In spite of high rates of smartphone diffusion, only a third of all firms selling consumer
goods have established mobile strategies and two-thirds of all firms do not provide mobile
applications for their customers (Forrester Research, 2011). Recent market research shows that
managers recognize that they miss out business opportunities in the mobile market and 70% of
firms are currently adjusting their mobile strategies (Forrester Research, 2011). Developing well-
designed mobile applications is a challenge for organizations (e.g., firms, governmental agencies,
libraries) and prior research suggests that the usability of mobile applications is particularly
important for effective user experiences (Venkatesh and Ramesh, 2006; Adipat et al., 2011).
Establishing mobile application usability is difficult because smartphones have relatively small
screens and the input mechanisms are tiny (Chen et al., 2010) relative to traditional computer
keyboards (Hong et al., 2002; Hong et al., 2004a; Hong et al., 2004b; Dwivedi and Kuljis, 2008).
In order to support organizations aiming to develop user-friendly mobile applications, operating
4
system (OS) vendors, including Apple, Microsoft and Google, provide application development
guidelines. These guidelines include general advice on how to design user-friendly and well-
designed mobile applications. For instance, one of Microsoft’s application development
guidelines suggests that it is: “important to take full advantage of design principles to ensure that
your application’s functionality is quickly and clearly conveyed at every step of the user
interaction” (Microsoft, 2014)1. Although this suggests that design principles are an important
aspect for the usability of mobile applications, it does not provide information on how important
design principles are and whether a given application incorporates design principles effectively.
We believe that systematically developed research instruments could help researchers and
practitioners to better address this issue. In particular, we analyze Microsoft’s mobile usability
guidelines to develop and validate a rich conceptualization of mobile application usability and
associated scales2. We extend our previous work (Hoehle and Venkatesh 2015), which was
focused on Apple’s guidelines, by (a) developing a conceptualization and measurement of
mobile application usability based on Microsoft’s mobile usability guidelines and (b) validate
our instrument. We expect our work will help practitioners in achieving better mobile application
design that helps individuals to more effectively interact with the application.
Although a considerable amount of literature has studied mobile application usability, we
found three key shortcomings in the existing literature. First, in much of the literature we found,
the concept of mobile application usability evolved from website usability (e.g., Venkatesh and
Ramesh, 2006), much like website usability evolved from software usability (Thong et al., 2002).
Although a useful starting point, we argue that it is best to develop research instruments that
account for the unique characteristics of mobile applications, such as small screen sizes and
clumsy input mechanisms (Kurniawan, 2008). Second, the majority of studies in the area of
5
human-computer interaction (HCI) have been laboratory experiments to evaluate mobile
application usability. Experimental research design is particularly useful for benchmarking
competing mobile application prototypes using task-based assessment to measure user
performance in terms of speed and accuracy (Lazar et al., 2010). However, such a research
design is less suited for holistically evaluating the usability of mobile applications and for
dissecting design aspects to improve mobile applications. In other words, experimental research
designs help determining if an application prototype allows users to perform a task fast and
accurately, but they may not capture a complete picture of interface design elements. Third, prior
research has used a variety of conceptually dissimilar constructs for evaluating the usability of
mobile applications including readability, ease of learning, design aesthetics and satisfaction
(Zhang and Adipat, 2005; Cyr et al., 2006; Kim et al., 2011). Associating the concept of ease of
learning and design aesthetics with mobile application usability seems problematic because such
a practice could result in interpretational confoundingi.e., if the empirical meaning of a latent
variable varies from the meaning assigned by a researcher (Burt, 1976; Bollen, 2007).
Against this backdrop, we argue that it is important to think from the ground up about
mobile application usability and develop and validate a survey instrument for assessing the
usability of mobile applications. We assess the predictive validity of the survey instrument using
two theoretically relevant outcomes: intention to use and brand loyalty. A systematically
developed survey instrument should help practitioners in designing mobile applications and to
study individuals’ views on to-be-developed or existing mobile applications. Well-designed
mobile applications should help users to effectively interact and become more satisfied with the
application. Likewise, it will be beneficial for research in this area because such a study will
provide theoretical clarity on the underlying factors influencing mobile application usability.
6
Thus, our objectives are: (a) to systematically review and analyze Microsoft’s mobile application
usability guidelines, (b) to develop relevant constructs that represent mobile application
usability, and (c) to develop and validate a survey instrument to measure the constructs by
following the scale development procedure of Lewis et al. (2005). We validate our survey
instrument in the context of social media applications, which are increasingly leveraged for both
hedonic as well as professional purposes (Scheepers et al., 2014).
2 Literature Review
2.1 Mobile application usability
Mobile application usability is defined, drawing from the International Standards
Organization’s (ISO) definition of usability, as the degree to which a mobile application can be
used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction
in a specified context of use (Venkatesh and Ramesh, 2006). Over the last decade, the concept of
mobile application usability has been the focus of much research in HCI and information systems
(IS) and research that falls at the intersection of these two areas, such as mobile commerce and e-
commerce. In order to identify theoretically motivated studies, we searched for mobile
application usability studies in leading HCI journalsnamely, ACM Transactions on Computer-
Human Interaction, AIS Transactions on Human-Computer Interaction, Behaviour and
Information Technology, Human-Computer Interaction, Interacting with Computers,
International Journal of Human-Computer Interaction, International Journal of Human-
Computer Studies, and Journal of Usability Studies. Because IS researchers commonly
investigate HCI-related phenomena (Hong et al. 2004a), we also targeted leading journals in
information systemsnamely, Communications of the ACM, European Journal of Information
Systems, IEEE Transactions on Human-Machine Systems, Information Systems Journal,
7
Information Systems Research, Journal of AIS, Journal of Information Technology, Journal of
MIS, Journal of Strategic Information Systems, and MIS Quarterly. Our search strategy included
various keywords, such as usability theory, mobile application usability, mobile application
usability theory and mobile interface usability. The search yielded 93 peer-reviewed articles. We
studied the identified articles for how mobile application usability was conceptualized, the
proposed usability evaluation methods, and associated scales used to measure mobile application
usability. Based on our assessment, we found three key shortcomings in the literature on mobile
application usability.
First, we found that few, if any, field studies used research instruments that were
specifically designed to evaluate mobile application usability in the field (Hoehle and Venkatesh,
2015; Kjeldskov and Stage, 2004; Nielsen et al., 2006; Avouris et al., 2008). In our recent work
(Hoehle and Venkatesh, 2015), we developed a conceptualization of mobile application usability
and an instrument based on Apple’s user experience guidelines for mobile applications (Apple,
2011). Here, we add to this work because Microsoft’s mobile usability guidelines vary from
Apple’s guidelines in that they emphasize different aspects of mobile application usability. For
example, Microsoft’s mobile application usability guidelines underline color and hierarchy,
whereas Apple’s guidelines include search features as part of mobile application usability. We
also found conceptual overlaps in both guidelines. For example, control obviousness is
highlighted in both guidelines and we therefore conceptualize it here and in our previous work.
The other few field studies we found conceptualized mobile application usability in a more
simplistic manner, which is understandable because research on mobile application usability is
not as mature as research on website usability. For instance, Huang (2012) note that mobile
application usability is a critical success factor for mobile marketing. Mobile application
8
usability in the context of marketing though focused only on whether the application is easy to
use and whether users can effortlessly gather marketing information (Huang, 2012). Hence, most
studies on mobile application usability drew on research instruments that were originally
developed to evaluate the usability of traditional computers and websites (Battleson et al., 2001;
Thong et al., 2002; Wu and Wang, 2005; Hsu et al., 2007; Kim et al., 2010). For example,
Venkatesh and Ramesh (2006) surveyed consumers in Finland and the U.S. to better understand
the differences between website and mobile application usability. In order to measure mobile
application usability, the authors adapted the website usability instrument originally developed
by Agarwal and Venkatesh (2002). In such a study, it is likely that important usability
requirements of the mobile context were omitted. For instance, much experimental research on
mobile application usability found that mobile application buttons should be large and
appropriate to the size of fingertips (Kurniawan, 2008). This would be necessary because mobile
application users navigate through menus using their fingers for smartphone interfaces
(Kurniawan, 2008). Given that traditional computer interfaces including websites are operated
through mouse cursors (see Thong et al., 2002), website usability instruments would be unlikely
to account for the concept of fingertip-sized controls.
Second, we found that the majority of the studies in HCI journals studied mobile
application usability in laboratory environments using experimental research designs. This is in
line with Kjeldskov and Graham (2003) who identified that more than 70% of all mobile
application usability evaluations take place in laboratory settings. In these studies, mobile
application prototypes were typically benchmarked to examine the influence of interface design
in relation to specific outcome variables, such as user performance (Ziefle and Bay, 2005; Adipat
et al., 2011). For example, Adipat et al. (2011) studied the effect of interface structure of mobile
9
applications on user performance. In this experiment, task complexity and mobile interface
structure were manipulated in order to determine the most effective mobile interface structure.
Although experimental research designs yield important findings for better understanding mobile
application usability, they face several limitations including the artificial nature of the setting and
the limited numbers of variables that can be manipulated (Kjeldskov and Graham, 2003).
Third, our literature review found that researchers have defined and conceptualized
mobile application usability inconsistently. For instance, Sonderegger et al. (2012) conducted a
longitudinal experiment in which they conceptualized mobile application usability as a
combination of design aesthetics and readability. Kim et al. (2005) used thirteen mobile
application usability elements, namely predictability, learnability, consistency, memorability,
familiarity, simplicity, feedback, effectiveness, efficiency, flexibility, minimal memory load,
satisfaction and helpfulness. Several studies also integrated concepts commonly seen in the
technology acceptance literature (e.g., ease of use) with concepts from the marketing research
discipline (e.g., satisfaction), as well as HCI principles (e.g., design aesthetics) (see Cyr et al.,
2006). Although some studies suggested that efficiency and effectiveness were part of mobile
application usability (e.g., Kim et al., 2005), others argued that both concepts are a result, or
outcome, of mobile application usability. Table 1 summarizes measurement approaches and
conceptualizations that prior studies have used for evaluating mobile application usability.
Table 1. Prior evaluation methods, research methodologies and conceptualizations used to study
mobile application usability
Usability
Evaluation
Method
Research
Methodology
Mobile Usability
Conceptualization
Study
Expert evaluation
Longitudinal field
experiment
Aesthetics and readability
Sonderegger et al.
(2012)
Cross-sectional
usability expert
survey
Cognition support (predictability,
learnability, structure principle,
consistency, memorability,
familiarity), information support
Ji et al. (2006)
10
Usability
Evaluation
Method
Research
Methodology
Mobile Usability
Conceptualization
Study
(recognition, visibility, simplicity,
subsitutivity), interaction support
(feedback, error indication,
synthesizability, responsiveness),
user support (recoverability,
flexibility, user control,
customizability), performance
support (effectiveness, efficiency,
effort)
Cross-sectional
usability expert
survey
Predictability, learnability,
consistency, memorability,
familiarity, simplicity, feedback,
effectiveness, efficiency, flexibility,
minimal memory load, satisfaction,
helpfulness
Kim et al. (2011)
Laboratory
experiment
Errors
Kim et al. (2005)
Single-user testing
Laboratory
experiment
Ease of learning
Mallat (2007)
Laboratory
experiment
Efficiency, effectiveness
Barnard et al.
(2007)
Laboratory
experiment
Accuracy
Burigat et al. (2008)
Laboratory
experiment
Attractiveness, perspicuity,
dependability, efficiency,
stimulation, novelty
Lin et al. (2007)
Laboratory
experiment and
cross-sectional
survey
Ease of use, playfulness, usefulness
Fang et al. (2003)
Laboratory
experiment and
cross-sectional
survey
Errors, learnability, operability
Kaikkonen et al.
(2005)
Laboratory
experiment and
cross-sectional
survey
Ease of use, effectiveness, and
overall satisfaction
Lai and Zhang
(2015)
Group usability
testing and focus
groups
Laboratory
experiment and
cross-sectional
field survey
Effectiveness, contextual awareness,
task hierarchy, visual attention, hand
manipulation and mobility
Duh et al. (2006)
Observation and
interviews
Control, difficulties with hardware,
software, netware, and bizware
Palen and Salzman
(2002)
11
Usability
Evaluation
Method
Research
Methodology
Mobile Usability
Conceptualization
Study
Interviews
Usability was not explicitly
conceptualized but various relevant
usability dimensions were
discussed, such as effectiveness and
efficiency
Nah et al. (2005)
Case study and
experts’ opinions
Support of task goals, cognitive
interaction, efficient interaction, and
ergonomic
Heo et al. (2009)
Field surveys
Cross-sectional
field survey
Content, ease of use, promotion,
made-for-the-medium, emotion
Venkatesh and
Ramesh (2006)
Multiple field
surveys/instrument
development
Application design, application
utility, user interface graphics, user
interface input, user interface
output, and user interface structure
Hoehle and
Venkatesh (2015)
Cross-sectional
field survey
Design aesthetics, ease of use,
usefulness
Cyr et al. (2006)
Cross-sectional
field survey
Usefulness, ease of use, satisfaction
Hsu et al. (2007)
Cross-sectional
field survey
Usefulness, enjoyment
Kim et al. (2007)
Cross-sectional
field survey
Perceived usefulness, ease of use
Kim et al. (2010)
Cross-sectional
field survey
Ease of use, usefulness and
compatibility
Wu and Wang
(2005)
Cross sectional
field survey
Efficiency, ease of use, and utility
Oliveira et al.
(2013)
Conceptual
Literature analysis
Information presentation, data entry
methods, mobile users and context
Adipat and Zhang
(2005)
Literature analysis
Learnability, efficiency,
memorability, error, satisfaction,
effectiveness, simplicity,
comprehensibility and learning
performance
Zhang and Adipat
(2005)
Literature analysis
Portability, adaptability, availability,
learnability, security, reliability,
attractiveness, interoperability
Terrenghi et al.
(2005)
Literature analysis
Effectiveness, efficiency,
satisfaction, errors, attitude,
learnability, accessibility,
operability, accuracy, acceptability,
flexibility, memorability, ease of
use, usefulness, utility, and
playfulness
Coursaris and Kim
(2011)
Literature analysis
Context, content, community,
customization, communication,
connection and commerce
Lee and Benbasat
(2003)
12
2.2 Microsoft’s usability guidelines
In order to support developers in designing user-friendly mobile applications, Microsoft
provides usability guidelines for its mobile operating system. These guidelines are now available
through Microsoft’s mobile portal (Microsoft, 2014). The guidelines are comprehensive and
cover six distinct aspects for designing mobile applications, including development frameworks,
platform specific advice, web development, mobile design, tools and resources, and natural
languages. Many sections contain technical instructions (e.g., development frameworks) and
were thus not relevant to user perceptions of usability. Most relevant to our work was the mobile
design section because it focuses exclusively on improving the usability of mobile applications.
We found Microsoft’s guidelines particularly suited for developing a usability survey
instrument for several reasons. First, Microsoft, through its acquisition of Nokia, is one of the
leading companies in the smartphone industry and the firm sold approximately 40 million
smartphones in 2013 (Gartner Research, 2013). Therefore, we felt it is reasonable to say that
Microsoft’s guidelines underscore the most critical aspects for designing successful mobile
applications. Second, we believe that using Microsoft’s guidelines for developing a mobile
application usability survey instrument would help us to produce relevant research. Rosemann
and Vessey (2008) propose that relevant research should be based on practitioners’
recommendations and Microsoft’s guidelines provide such an opportunity to develop
practitioner-based research (Rosemann and Vessey, 2008).
3 Instrument Development
To design our mobile application usability instrument, we drew on measurement theory.
Measurement theory emerged from the reference discipline of psychometrics and it aims to
measure human perceptions, behaviors and attitudinal beliefs (Burt, 1976). One particular stream
13
of measurement theory focuses on the development and validation techniques for scientific
research instruments to precisely measure attitudinal beliefs (MacKenzie et al., 2011). Although
there are alternative approaches to develop instruments, we followed the methodology suggested
by Lewis et al. (2005). The proposed methodology consists of three major stages. The first stage
includes the conceptualization of the construct domain and includes a content analysis of the
constructs of interest. The second stage focuses on the scale development process and relates to a
pre-test, pilot test and quantitative assessment of the content validity of the measures. The third
stage aims to validate the survey instrument and includes an exploratory and confirmatory
assessment of the scales (Lewis et al., 2005). Below, we discuss each stage and outline how we
applied them to our research. For each stage, we summarize the recommended activities,
followed by a discussion of our actions undertaken as part of the scale development process.
3.1 Stage 1: Domain
The first stage of the scale development involves establishing the domain of the
conceptual idea (Lewis et al., 2005). Lewis et al. (2005) recommended content analysis, which is
a technique used to draw inferences from text-based material (Lewis et al., 2005). Content
analysis also helps to develop the purpose and/or importance of a conceptual construct and it can
be used to develop a conceptual definition of it (Lewis et al., 2005).
We initially used content analysis to examine Microsoft’s usability guidelines for mobile
applications. In particular, one author systematically reviewed and analyzed the guidelines. To
conduct the content analysis, we applied Strauss and Corbin’s (1990) open and axial coding
procedures. Open coding is the “analytical process through which concepts are identified and
their properties and dimensions are discovered in the data” (Strauss and Corbin, 1990, p. 101).
Axial coding is the process of “relating categories to their subcategories, termed ‘axial’ because
14
coding occurs around the axis of a category, linking categories at the level of properties and
dimensions” (Strauss and Corbin, 1990, p. 123). We applied these coding procedures to identify
conceptually similar themes discussed in Microsoft’s guidelines. Initially, one author reviewed
Microsoft’s guidelines and coded the content using Strauss and Corbin’s (1990, p. 119) line-by-
line analysis. Next, the open codes were grouped and subcategories were formed to identify
conceptually similar codes. Using axial coding, the open codes were examined for similarities or
differences and then organized as conceptual units. For example, we identified two open codes
that focused on the concept of graphics in mobile applications: (1) images and graphics must
enhance and support the user experience and (2) graphics should be designed aesthetically and
should not replace or overlap important textual content.
Both open codes were combined into one subcategory that was labeled as “well-designed
and aesthetic graphics”. Then, using axial coding, the major category was labeled as aesthetic
graphics. Next, the results were organized in a matrix as outlined by Miles and Huberman
(1999). Organizing codes in a data matrix is useful to compress coded information and it
supports drawing conclusions (Miles and Huberman, 1999). Subsequently, a second author
reviewed the usability guidelines and associated coding patterns. In a few cases, there was a
disagreement between the authors. In these instances, we asked two independent judges, who
were unfamiliar with the study, to facilitate a discussion in order to reach a coding consensus.
Table 2 shows the final matrix derived from Microsoft’s guidelines.
Table 2. Coding matrix adapted from Miles and Huberman (1999)
Subcategory
Open codes derived from Microsoft’s guidelines
Well-designed
and aesthetical
graphics
Images and graphics must enhance and support the user experience.
Graphics should be designed aesthetically and should not replace or
overlap important textual content.
Contrast and
Color
The text of applications should have a good contrast with the
background.
Color assists in the organization and grouping of information,
helping to focus attention, convey differentiation, and establish
relationships and visual hierarchies between elements.
15
Subcategory
Open codes derived from Microsoft’s guidelines
Color can help readers scan information and quickly identify
structural or functional elements, such as headers, menu items and
hyperlinks.
When used incorrectly, however, color can easily distract attention
from the task at hand.
If a color is being used to convey a specific meaning (for example,
red to warn of danger or an error), chosen colors should be
universally associated with the intended meaning and potential
conflicts that result from cultural misinterpretation should be
avoided.
Consistent use of
controls
User controls should be obvious and interaction should be familiar,
clear, and trustworthy.
The design should be consistent, logical, and coherent both within
the application and within the target platform.
Controls and application features should be used consistently.
Application
accessibility and
application
entrance points
Users should have several options to choose from if aiming to
access an application.
The application should be designed in a way that it is accessible via
direct controls or application menus, or a combination of both.
Well-designed applications should have several points to access a
menu or an application.
Button size and
control size
Interface elements should not be smaller than the smallest average
finger pad, that is, no smaller than 1 cm (0.4") in diameter or a 1 cm
× 1 cm square.
The width of a finger limits the density of items on screen. If the
items are too close, the user will not be able to choose a single one.
As the user is more likely to touch higher on the button by mistake
than on either side, consider the height of your buttons and icons.
Essential information or features, such as a label, instructions, or
sub-controls should be placed below an interface element that can
be touched, as it may be hidden by the user's own body.
Font style
Font is an important consideration for designing applications
because users appreciate well-chosen font styles.
Devices normally have one standard font style, which should be
used as the application’s default typeface.
Gestalt principles
and proximity of
interface elements
Information and content should be organized in accordance with the
Gestalt principles.
Each part of the application is affected by what surrounds it.
Users should be able to quickly make sense of the elements on-
screen and understand what functionality or data they represent.
Elements that are close together are naturally perceived as being
related.
Because of the small screen size, however, the use of proximity
may be limited.
Hierarchical menu
structure and
application
navigation
Drill-down views offer hierarchical navigation for applications that
need to provide access to hierarchies of information.
The layout of the various views in the navigation chain is not
restricted to lists, and should be optimized for the type of content
and/or functionality.
In all cases, users navigate hierarchies in drill-down views by
tapping items in a view to ‘drill down’ another level in the
information hierarchy.
Users should also be able to move back toward the top or ‘root’ of
16
Subcategory
Open codes derived from Microsoft’s guidelines
the hierarchy and back commands should be also available.
Tapping ‘back’ at any level takes the user up to the previous level
in the hierarchy.
Animation use
and simplicity of
animated content
Animations should be kept simple.
Avoid complex animations and, in particular, multiple simultaneous
timeline-based motion animations.
Avoid unnecessary alpha effects or gradients and do not combine
transitions with changes in transparency or other graphical effects
because they are likely to slow down the animations.
Transition and
flow of user
interface elements
Well-designed transitions help users and make the user interface
more engaging.
Without transitions, the interaction feels less natural.
Transitions can be used to inform the users of what is going on.
Transitions should be used wisely and it is useful to test how users
feel about them.
Transitions can easily create a WOW-factor to applications.
If every user interface element is twitching and turning wildly, it
could as easily exhaust the user.
Next, we used the axial codes, shown on the left hand side of Table 2, as the basis for
conceptualizing each construct. To further inform the construct conceptualization, we compared
the axial codes with the existing literature on mobile application usability. In all instances, we
found literature, whether in the domain of traditional desktop, website usability or mobile
application usability, supporting the identified axial codes shown in Table 1. Below, we discuss
the outcome of this process and define the constructs we derived.
3.1.1 Aesthetic graphics
Based on early research on the aesthetics of desktop and web applications (Lavie and
Tractinsky, 2004) and online shopping environments (Porat and Tractinsky, 2012), more recent
studies started to examine the effect of aesthetic graphics on outcomes (e.g., intention to use) in
the context of mobile applications (Cyr et al., 2006; Nathan-Roberts and Liu, 2015). A review of
the literature shows that researchers conceptualized and measured aesthetic graphics differently.
For instance, Li and Yeh (2010) defined aesthetics as “the balance, emotional appeal, or
aesthetic of a website and it may be expressed through the elements of colors, shapes, language,
17
music or animation” (p. 674). Sonderegger et al. (2012) conceptualized aesthetic qualities less
comprehensively and argued that clearness, symmetry, and color settings are the most critical
factors underlying the concept. To measure aesthetic graphics, Sonderegger et al. (2012) asked
individuals to rate a mobile application along the identified aesthetic graphics dimensions.
In summary, the literature on mobile applications suggests that aesthetic graphics is an
important concept when evaluating the overall mobile application usability. In our work, we
define aesthetic graphics as “the extent to which a user perceives that the mobile application
makes use of aesthetic graphics.” Our conceptualization of aesthetic graphics is consistent with
existing definitions and emphasizes that the mobile application user desires aesthetically pleasing
designs (Hoehle and Venkatesh, 2015).
3.1.2 Color
Prior work found that color is another important factor to consider when studying mobile
application usability (Hartmann, et al. 2008; Sonderegger et al., 2012) because colorful
application interfaces produce initial affective user reactions, which could ultimately impact the
user’s continued intention to use mobile application (Nilsson, 2009; Leung et al., 2011; Dong
and Zhong, 2012). Relevant work on other types of applications suggests that color becomes
important in providing guidance to users (Brandse and Tomimatsu, 2014) or influencing trust in
online shopping contexts (Pelet and Papadopoulou, 2011). Instead of conceptualizing color as an
independent construct, mobile application usability studies typically manipulated color as part of
experimental studies. For example, Sonderegger et al. (2012) manipulated the color of text and
icons as part of a mobile application and asked individuals to rate the color schemes. Color was
manipulated in the experiment and the authors produced low, moderate, and highly colorful
application designs. For instance, a disharmonious combination of magenta, amber, and green
18
was chosen for the design with moderate aesthetic appeal (Sonderegger et al., 2012). The results
showed that highly colorful interface designs yielded the most favorable user reactions
(Sonderegger et al., 2012).
In sum, much prior work suggests that color is important for a user’s overall evaluation of
mobile application usability (Nilsson, 2009; Leung et al., 2011; Dong and Zhong, 2012). Instead
of examining specific color combinations and attributes, we treat color as an independent
construct. Specifically, we focus on whether the use of color is appropriate from the user’s
perspective. We closely followed Microsoft’s guidelines and did not identify specific color
combinations or attributes because the main purpose of our conceptualization is to be able to
examine usability across different types or contexts of mobile applications. In our work, we
define color as “the degree to which a user perceives that the mobile application uses colors
effectively.” Thus, we extend existing work in this area because the color scope is not limited to
aesthetical perceptions.
3.1.3 Control obviousness
The existing literature on mobile application usability suggests that the application’s
controls should be immediately obvious to application users (Ji et al., 2006). The importance of
control obviousness for mobile applications is consistent with research on traditional web
applications, which also emphasizes ease of searching and executing shopping tasks within
online stores with minimal effort (Nah and Davis, 2002; Porat and Tractinsky, 2012). However,
work on HCI emphasized that mobile applications’ functionalities should be obvious to users
because they are displayed on small screens (Huang et al., 2006). In these studies, researchers
typically exposed research participants to mobile applications, asked them to execute predefined
tasks using the mobile application, and surveyed participants afterwards. For instance, Ji et al.
19
(2006) surveyed mobile application developers and usability experts and asked them if executing
controls would be consistent and clear. Examples of predefined tasks included the examination
of confirmation, input, termination, cancel, and search tasks (Ji et al., 2006). Kim et al. (2011)
added that the location of soft keys is most critical for the controls obviousness as part of mobile
applications. In summary, mobile application’s controls and buttons should make it easy for
users to pick the desired functions (Hoehle and Venkatesh, 2015).
We define control obviousness as “the degree to which a user perceives that the mobile
application deploys controls that are immediately obvious. Our measurement captures the
extent to which the main function is apparent and whether the application makes use of
commands and controls that are intuitive and obvious. Our conceptualization is consistent with
existing studies on mobile application usability in that it emphasizes the easiness of finding and
executing controls (see Ji et al., 2006).
3.1.4 Entry point
The concept of entry point focuses on a user’s ability to access a given mobile application
via several alternative entry points. Our literature review suggests that there is a lack of
discussion on usability issues pertaining to entry points to mobile applications. Our
conceptualization of entry point follows Microsoft’s guidelines closely and is different from the
W3C’s view of accessibility, which emphasizes making the applications more acceptable to
people with cognitive or physical disabilities (W3C, 2015). Although we were unable to find
studies that conceptualized and measured entry point in the context of mobile applications,
Benbunan-Fich and Benbunan (2007) found that smartphone users become frustrated if they are
unable to find an application on the mobile phone after downloading it. These findings overlap
20
with the concept of entry point because they emphasize that it is critical that users can enter
mobile applications easily (Benbunan-Fich and Benbunan, 2007).
We focus on accessing the mobile application from an interface design perspective and
define entry point as “the degree to which a user perceives that the mobile application can be
accessed through alternative entry points.” This view is consistent with Benbunan-Fich and
Benbunan’s (2007) findings in that it emphasizes making mobile applications as accessible as
possible for users. Due to the fact that Benbunan-Fich and Benbunan (2007) exclusively focused
on the accessibility of newly downloaded applications, we extend this notion and study whether
or not an application can be accessed using different icons and menu access points.
3.1.5 Fingertip-size controls
Due to the hardware limitations of smartphones (Romano et al., 2014), such as limited
screen size and relatively small keyboards, mobile applications developers should consider the
size of buttons (Brewster, 2002). For instance, Kurniawan (2008) studied the effect of control
size on application usability and surveyed elderly mobile application users. The study found that
relatively large, i.e., fingertip-size controls, helped users to select functions and menus in mobile
applications (Kurniawan, 2008).
In line with Kurniawan’s (2008) work, we use the term fingertip-size controls as
suggested in Microsoft’s guidelines. Consistent with other studies, we define fingertip-size
controls as “the degree to which a user perceives that the mobile application deploys fingertip-
size controls.” This definition captures the extent to which the users can tap on controls easily,
which requires an appropriate size of controls.
3.1.6 Font
21
Font is another relevant design element that has been studied from different perspectives
in the context of mobile application usability and traditional desktop applications (Bernard et al.,
2003; Ling and Schaik, 2006). For example, several variations of font, such as style (e.g., Arial
versus Times) and size, have been studied in relation to readability of application content (see
Ling and Schaik, 2006; Moshagen and Thielsch, 2010). Kim et al. (2005) suggested that font size
is a critical part of mobile application usability because it influences how efficiently the
information is shown, how easy it is to read the presented information, and how effectively the
information is presented to users (Kim et al., 2005).
We define font as “the degree to which a user perceives that the mobile application uses
font effectively.” Our definition covers not only font size but also the extent to which font is
perceived as good and appealing.
3.1.7 Gestalt
Gestalt theory has been utilized to study aspects of mobile application usability (see for
example Paay and Kjeldskov, 2007) as they have proven to be highly effective in the context of
traditional desktop and website applications (Moller et al., 2012). Gestalt theory includes several
gestalt laws (e.g., proximity of objects) and explains how humans perceive objects in their
environment and how they form such perceptions (Wertheimer and Riezler, 1944). For instance,
Paay and Kjeldskov (2007) applied gestalt theory to study location-based mobile applications
and they identified five relevant gestalt laws in this context, namely proximity, closure,
symmetry, continuity, and similarity. For each law, Paay and Kjeldskov (2007) developed 2-3
survey questions that developers could use to examine adherence to the identified gestalt rules.
The results of a qualitative study showed that the identified gestalt rules helped in explaining
22
how users perceive and make sense of mobile location-based services (Paay and Kjeldskov,
2007).
Microsoft’s guidelines also emphasize gestalt principles and apply the laws of similarity
and proximity to mobile application usability. Hence, our conceptualization of gestalt is
consistent with these two laws based on gestalt theory. We define gestalt as “the degree to which
a user perceives that the mobile application uses gestalt principles effectively.”
3.1.8 Hierarchy
Application hierarchy is another relevant concept for the organization of application
content and elements. In traditional website applications, hierarchy has been emphasized as an
important design aspect that embeds structure (Agarwal and Venkatesh, 2002), which makes it
easier for users to perceive the overall organization of the website. For example, Adipat et al.
(2011) suggest that mobile sites should have a hierarchal structure because it informs mobile
users about the inherent logic of the site. Integration of titles and sub-titles would indicate several
hierarchal levels and help users in navigating the mobile application easily (Adipat et al., 2011).
Further, Kim et al. (2005) emphasized the concept of hierarchy as part of mobile applications
and suggested categorizing, labeling and sequencing application menus in order to help users
during navigation. Most recently, Hoehle and Venkatesh (2015) noted that users should be able
to perceive an effective structure in the mobile application interface.
We define hierarchy as “the degree to which a user perceives that the mobile application
has a hierarchical structure.” The term is consistent with existing views of mobile application
literature in that it identifies application hierarchy as a critical part of mobile application usability
(Kim et al., 2005).
3.1.9 Subtle animation
23
Research on traditional website applications emphasized the need for using media
appropriately and to effectively communicate the content (Agarwal and Venkatesh, 2002).
Animation and media use are also important design aspects of mobile application interfaces. For
example, Venkatesh and Ramesh (2006) examined media use as a subcategory of application
content and the concept captured the extent to which media is used appropriately and effectively
to communicate content (Venkatesh and Ramesh, 2006). Although media use in wireless
contexts was deemed less important than media use in web contexts (Venkatesh and Ramesh,
2006), mobile users still need to perceive media use as appropriate. From a design and user
perspective, the use of richer and complex graphics and animations, for instance, does not
guarantee successful communication of the content, because such use might be perceived as
distracting (Mayer, 2001). Based on this assumption, we use the term subtle animation to capture
the preciseness of media use and define it as “the degree to which a user perceives that the
mobile application uses subtle animations effectively.” This term captures the user’s perception
of appropriateness of animation use and content communication effectiveness.
3.1.10 Transition
It is important that mobile applications are designed in a way that they help users
transitioning from one page to another (Adipat et al., 2011). The importance of simple
transitioning within mobile applications is consistent with the need for easy navigation in the
context of traditional website applications (Nielsen, 2000; Porat and Tractinsky, 2012).
Transitioning from a page to another without problems also captures efficiency, as in traditional
usability studies (Nielsen, 2000), because it reduces the time to navigate within an application. In
the context of mobile applications, Lee et al. (2009) emphasized that users should be able to
effectively navigate between screens because this would improve their overall perception of
24
system quality. Likewise, Benbunan-Fich and Benbunan (2007) proposed that navigation
problems could be identified by measuring how users move between pages and how users access
specific information within each application page.
We define transition as “the degree to which a user perceives that the mobile application
transitions from one page to another.” Application transition needs to be smooth so that the user
can easily determine his/her position while navigating through the application.
Next, the identified constructs were reviewed for conceptual similarities. This is an
important step for identifying higher-order constructs (Lewis et al., 2005). Through iterations
following discussions between the authors and our literature review described above, we
converged on ten independent constructs forming mobile application usability. Table 3 provides
a summary of construct definitions based on the content analysis and literature review.
Table 3. Construct definitions based on the content analysis and literature review
Construct name
Construct definition
The degree to which a user perceives that the mobile application…
Aesthetic graphics
......makes use of aesthetic graphics.
Color
......uses colors effectively.
Control obviousness
......deploys controls that are immediately obvious.
Entry point
......can be accessed through alternative entry points.
Fingertip-size controls
…..deploys fingertip-size controls.
Font
…..uses font effectively.
Gestalt
…..uses gestalt principles effectively.
Hierarchy
…..has a hierarchical structure.
Subtle animation
…..uses subtle animations effectively.
Transition
…..transitions from one page to another.
3.2 Stage 2: Instrument construction
The second stage of the construct development methodology focuses on the survey
instrument development and involves three distinct phases (Lewis et al., 2005). In the first phase,
researchers develop items for the identified constructs and pre-test the scales (Lewis et al., 2005).
In the second phase, a pilot study should be conducted in order to purify the wording of the items
and to obtain initial feedback on the survey instrument (Lewis et al., 2005). The third phase
25
involves the screening items and assessing the scales for content validity (Lewis et al., 2005).
Content validity is the extent to which a scale represents all facets of a given construct (Anderson
and Gerbing, 1991; Hinkin and Tracey, 1999; Lewis et al., 2005; Lawshe, 1975).
We drew on the codes derived from Microsoft’s usability guidelines in order to develop a
pool of items. Particularly, the open codes listed in Table 2 were helpful during this stage and we
also leveraged existing literature that previously measured usability. We created 4-6 items for
each construct to assure a reliable measurement of the conceptual domain. This led to an initial
pool of 58 items. Next, we conducted a pre-test of the survey instrument and asked six Australian
University staff members to complete a paper-based survey containing the newly developed
items. Three administrative staff members, two PhD students and one Masters student completed
the survey. Before asking the participants to pre-test our survey, we asked them if they owned a
smartphone and had experience with mobile applications. We felt this was necessary in order to
avoid confusion about the questions asked in the survey. All items were randomized and we
included feedback fields within the survey. We asked all respondents to flag unclear items or
sections of the survey instrument that they viewed as confusing or vague. Out of the 58 items, 42
were identified as clear and none of the participants suggested altering these questions. For 4 of
the 58 items, the participants proposed minor changes. We modified these items in accordance
with the obtained feedback and kept 46 questions in the item pool. All items that the respondents
flagged as unclear were excluded from the item pool.
The next step of the survey instrument development included a pilot test of the survey
instrument. Lewis et al. (2005) recommended that participants for the pilot test should come
from the main population of interest. Thus, we collected 30 responses from German consumers
recruited by a market research firm. The firm invited potential respondents to complete the
26
survey online and participation was encouraged via small monetary incentives. The respondents
were provided with instructions and the survey was available to them in German, e.g., the items
were translated and back-translated by bilingual professionals to ensure cross-language
equivalence in meaning. This procedure is common in cross-cultural research (see Zhang et al.,
2007). The respondents’ demographics are shown in Appendix 1. As can be noted from the
descriptive statistics in Appendix 1, the respondents interacted with different types of social
media applications. We did not find any significant differences in the constructs based on the
type of social media application. Regarding the pre-test, respondents were provided with
opportunities to give feedback on the survey structure and items. The results suggested that the
survey instructions were clear and we obtained positive feedback from most respondents. Out of
the 46 newly developed items, 12 questions were flagged as worded vaguely by these
respondents. These items were excluded from the item pool. This led to 3-4 items for each
construct identified in stage 1 of the instrument development process. In order to have at least 4
items per construct, we modified the wording for some of the flagged items and opted to have 4
items for each construct. This led to 40 items based on the pilot study. Table 4 lists the items.
We next evaluated the content validity of the new scales, which can be done using
multiple approaches. Lewis et al. (2005) recommended using Lawshe’s (1975) content validity
ratio that requires subject matter experts to judge how essential each item is in relation to a given
construct. We sought to pursue this approach but after many experts declined our request due to
lack of time and others repeatedly rescheduling and failing to complete the requested assignment,
we turned to the literature for an alternative. Anderson and Gerbing (1991) proposed an
alternative approach to assess the content validity of newly developed scales. This approach
works on the assumption that each item represents only one construct (Anderson and Gerbing,
27
1991; Yao et al., 2007). This procedure includes the use of a matrix in which construct
definitions are listed on top of the columns and items are placed in the rows. Individuals can be
asked to select the most appropriate item-to-construct combination and raters are not required to
be experts in the field of study (Anderson and Gerbing, 1991). We followed the procedure
outlined by Anderson and Gerbing (1991) and developed four matrices in which we organized
our construct definitions in rows and listed them on top of the columns. The items were listed in
the columns and we hired the same market research firm that was employed to conduct the pilot
study. The firm invited potential respondents via email. The invited individuals were asked to
complete the survey online and participation was encouraged via small monetary incentives
provided by the market research firm. In total, 318 U.S. consumers who were familiar with
mobile applications evaluated how well our items fit with our construct definitions. Appendix 1
includes the demographic information on the research participants. Next, we computed the
proportion of substantive agreement (PSA) and substantive validity coefficients (CSV) as
explained by Anderson and Gerbing (1991)3. These values can range between 0 and 1 where
higher values indicate a high degree of content validity and low values indicate that the item does
not overlap with the intended construct definition. Yao et al. (2007) suggested 0.25 as a cut-off
point for PSA and CSV values. Table 5 shows that the content validity ratios obtained were high,
thus indicating that most respondents sorted the majority of items into the posited construct
definitions. Out of 40 items, only HIER1 was lower than the recommended threshold of 0.25.
Hence, we re-worded the item in order to align it better with the construct domain.
Table 4. Item pool, proportion of substantive agreement and substantive validity coefficients based on the
content validity survey
Code
Items
PSA
CSV
Aesthetic
graphics
AEST1
The mobile application uses beautiful artwork.
0.92
0.87
AEST2
The mobile application uses rich, beautiful, and engaging graphics
that draw you into the application.
0.89
0.83
AEST3
The mobile application uses stunning graphics.
0.93
0.88
28
Code
Items
PSA
CSV
AEST4
The mobile application benefits from beautiful and engaging
graphics.
0.91
0.85
Color
COL1
The mobile application uses appropriate colors.
0.93
0.90
COL2
The mobile application makes use of appropriate colors.
0.91
0.88
COL3
The mobile application has great colors.
0.91
0.88
COL4
The mobile application doesn’t misuse colors.
0.93
0.90
Control
obviousness
COOB1
The mobile application makes the main function of the application
immediately apparent.
0.89
0.84
COOB2
The mobile application uses intuitive commands.
0.76
0.58
COOB3
The mobile application uses controls that are immediately obvious.
0.89
0.85
COOB4
The mobile application employs controls that are intuitive.
0.81
0.66
Entry point
ENPO1
The mobile application can be accessed using two different ways.
0.88
0.83
ENPO2
The mobile application can be accessed via two different menus.
0.90
0.86
ENPO3
The mobile application can be started either through an icon or
menu.
0.90
0.86
ENPO4
The mobile application is accessible using different icons or menu
access points.
0.88
0.83
Fingertip-
size controls
FTSC1
The mobile application uses fingertip-size controls.
0.89
0.84
FTSC2
The mobile application makes use of fingertip-size buttons.
0.89
0.84
FTSC3
The mobile application uses large-size controls.
0.91
0.87
FTSC4
The mobile application uses small controls that require you to aim
carefully before you tap it.
0.64
0.46
Font
FON1
The mobile application makes use of a good font.
0.92
0.89
FON2
The mobile application has a good font.
0.92
0.90
FON3
The mobile application uses a good font size.
0.92
0.89
FON4
The mobile application uses a font that I find appealing.
0.89
0.85
Gestalt
GEPR1
The mobile application uses similar shapes for elements that are
identical.
0.80
0.67
GEPR2
The mobile application groups elements together that are similar.
0.86
0.80
GEPR3
The mobile application groups things that belong together.
0.86
0.81
GEPR4
The mobile application makes use of similar shapes for elements
that are identical.
0.75
0.58
Hierarchy
HIER1
The mobile application guides users from top to bottom (original).
0.36
-0.01
The mobile application has a well-defined hierarchical structure
(modified).*
NA
NA
HIER2
The mobile application uses a clear hierarchy.
0.84
0.78
HIER3
The mobile application makes use of headings to develop a
hierarchy on the screen.
0.80
0.71
HIER4
The mobile application employs headings to establish a hierarchy.
0.78
0.69
Subtle
animation
SANM1
The mobile application uses animations effectively to communicate
content.
0.79
0.75
SANM2
The mobile application uses animations appropriately.
0.91
0.88
SANM3
The mobile application doesn’t overuse animations.
0.92
0.89
SANM4
The mobile application uses subtle animation to communicate
content.
0.92
0.89
Transition
TRAN1
The mobile application informs you when it transits from one screen
to another.
0.79
0.68
TRAN2
The mobile application tells the user when switching from one
screen to another.
0.72
0.55
TRAN3
The mobile application moves from one screen to another without
any problems.
0.81
0.72
29
Code
Items
PSA
CSV
TRAN4
The mobile application switches from one screen to the next
smoothly.
0.81
0.76
*Note: We listed the original item for HIER1 used for the content validity check as well as the modified item. The
modified item was used during stage 3 of this study.
30
3.3 Stage 3: Evaluation of measurement properties
The third stage of the instrument development process focuses on evaluating the
measurement properties of the new scales. Lewis et al. (2005) recommended using two
independent samples that are relevant to the population of interest. Exploratory factor analysis
(EFA) should be used to discover the factor structure in the first sample. Then, using the second
sample, confirmatory factor analysis (CFA) should be used to validate the scale properties
(Lewis et al., 2005). In the confirmatory phase, researchers should also assess the nomological
network of the scales by testing if the constructs of interest predict theoretically relevant
dependent variables.
3.3.1 Exploratory study
Following Lewis et al. (2005), we initially collected a sample of German consumers
using mobile applications. Germany is a large European economy where mobile smartphones are
widely used. Germany also has a very high population density and the 3G mobile networks cover
most areas of the country. Forrester Research suggests that around 40% of all German consumers
use mobile data services and access Internet-based mobile applications on their smartphones
(Savvas, 2010).
Due to this, we felt surveying German consumers regarding their perceptions toward the
usability of mobile applications would be particularly interesting for companies developing and
distributing mobile applications in Europe. As with the pilot study, we executed the data
collection for the exploratory phase through a market research firm that recruited German
consumers. We used the instructions developed for the pre-test and pilot study. All items listed in
Table 4 were measured using a 7-point Likert-type scale (1=strongly disagree…7=strongly
agree) and we tailored the questions toward mobile social media applications, such as Facebook.
31
Tailoring questions to the context of a particular study is a well-accepted practice in IS research
(Venkatesh et al., 2003; Venkatesh and Ramesh, 2006). At the start of the survey, we provided a
list of the most common social media applications, including Facebook, LinkedIn, Twitter, My
Space and Google+. Depending on a respondent’s choice, we programmed the survey to carry
over the response individuals provided at the beginning of the survey. This way, the items were
displayed as “Facebook (mobile) uses beautiful artwork” instead of “The mobile social media
application uses beautiful artwork.”
For the exploratory study, we collected data from 464 actual consumers. We hired a
different market firm from the one used for the pilot study to ensure that the respondents came
from a different respondent pool. As with the pilot study, the instructions and survey was
available to the respondents in German. We hired professionals to translate and back-translate
the items to ensure cross-language equivalence in meaning. Initially, all responses were
scrutinized for the time respondents took to complete the survey. Respondents who took too little
time and/or did not correctly answer reverse-coded filler items were excluded from our sample.
This led to 404 usable responses. We also tested for non-response bias and the data showed no
significant differences in terms of demographic characteristics between the respondents and non-
respondents. Further, we examined how well the profile of our respondents corresponded to the
profile of the sampling frame provided by the market research firm. The results confirmed that
the respondents’ characteristics of our sample matched the sampling frame provided by the
market research firm. We did not see a need to compare early versus late responses because all
responses were collected during a single weekend and no reminders were employed (Churchill,
1979; Hair et al., 1998). Appendix 1 summarizes the respondent demographics.
32
Next, we conducted an exploratory factor analysis (EFA) with direct oblimin rotation to
allow for correlated factors. It is recommended that the item-to-response ratio be in the range
from 1:3 to 1:8 (Hair et al., 1998). Given that our scales included 40 newly developed items, our
ratio of items to responses seemed adequate for exploratory analysis. The results of the EFA
confirmed a solution with ten factors, each with eigenvalues greater than 1.0. As shown in Table
5, the items explained a reasonable amount of covariance in the associated constructs ranging
from 14.6% to 32.3%. All item loadings were greater than .70. Given these results, we felt that
dropping items was unnecessary. We inspected the reliability of the items by computing
Cronbach’s α coefficients for all scales—all of which were above .77 or greater and thus higher
than the recommended threshold of .70 (Fornell and Larcker, 1981).
Table 5. Exploratory study covariance explained by each factor, item loadings and Cronbach’s alpha
reliability
Construct
name
Covariance
explained
Loadings
Cron.α
Construct
name
Covariance
explained
Loadings
Cron.α
Aesthetic
graphics
(AEST1-4)
32.3%
.75
.80
Font
(FON1-4)
21.8%
.74
.80
.74
.77
.82
.79
.80
.80
Color
(COL1-4)
24.6%
.75
.83
Gestalt
(GEPR1-4)
14.6%
.74
.77
.77
.73
.78
.74
.81
.74
Control
obviousness
(COOB1-4)
17.4%
.84
.85
Hierarchy
(HIER1-4)
17.5%
.75
.85
.88
.77
.80
.73
.81
.77
Entry point
(ENPO1-4)
19.8%
.85
.87
Subtle
animation
(SANM1-4)
16.5%
.79
.84
.80
.84
.80
.83
.83
.80
Fingertip-size
controls
(FTSC1-4)
16.6%
.75
.84
Transition
(TRAN1-4)
16.9%
.82
.83
.77
.80
.75
.75
.84
.77
33
3.3.2 Confirmatory study
Following the procedure used for the exploratory study, we collected a new sample for
the confirmatory phase of this research. As part of the confirmatory assessment of the survey
instrument development process, Lewis et al. (2005) recommended evaluating the nomological
network of the scales. To do this, theoretically related variables should be included in the survey
to test the predictive validity of a given construct of interest. Therefore, based on existing
information sciences and mobile application research, we included items for two dependent
variables, namely continued intention to use and brand loyalty. Intention in particular is a critical
indicator of success of newly implemented information technologies and associated services (Hu
et al., 2010; Hu et al., 2005; Hu et al., 2009). Prior research suggests aesthetic and colorful
graphics positively influence consumers’ continued intention to use and brand loyalty toward
mobile applications (Scornavacca et al., 2006). Similarly, obvious controls, multiple entry points
and fingertip-size controls of mobile applications will have a positive effect on consumers’
continued intention to use as well as their brand loyalty toward mobile applications (Barnes,
2002; Barnes, 2003; Barnes and Huff, 2003; Kurniawan, 2008). Research also proposes that the
type of font, hierarchical structure and gestalt principles (e.g., similar application components are
grouped together) positively influence users’ continued intention to use and brand loyalty toward
mobile applications (Barnes, 2003). Likewise, research found that a user’s continued intention to
use and brand loyalty toward a mobile application is positively influenced if menus follow a
clear hierarchy, pages flow smoothly from one page to another, incorporate subtle animations
(Scornavacca et al., 2006). The scales used to measure continued intention to use and brand
loyalty constructs were adapted from prior research (Johnson et al., 2006; Venkatesh and Goyal,
2010). Appendix 2 lists the items used to measure the outcome variables. As shown in Figure 1,
34
mobile application usability is conceptualized using 10 unique constructs identified based on
Microsoft’s guidelines.
Fig. 1. Structural model
Similar to the exploratory study, we collected data from a new sample consisting of 550
German consumers using mobile social media applications. We hired the same market research
firm we used for the exploratory study. Care was taken not to invite respondents who
participated in the exploratory study. Following the steps undertaken for the exploratory study,
we excluded problematic responses (e.g., those who spent too little time on the survey and
responded incorrectly to reverse-coded items). In total, we received 501 usable responses.
Appendix 1 shows the demographic information of the respondents. Similar to the exploratory
study, the instructions and survey were professionally translated and were made available to the
35
respondents in German. As with the exploratory study, we did not find any significant
differences in terms of demographic characteristics between the respondents and non-
respondents. Our sampling frame also matched the sampling frame provided by the market
research firm. Like in the exploratory study, we felt that it was unnecessary to compare early
versus late responses because all responses were collected during a single weekend and no
reminders were employed (Churchill, 1979; Hair et al., 1998).
As recommended by Lewis et al. (2005), we also carefully assessed our sample for the
shape of the distribution and checked for skewness and kurtosis before starting the data analysis,
and found no significant issues. Next, following Lewis et al. (2005), we used confirmatory factor
analysis (CFA) to evaluate the psychometric properties of the scales. AMOS was used to assess
all factors separately, then in pairs and then as a collective network as outlined by Lewis et al.
(2005). We then examined the construct validity of the scales (see Lewis and Byrd, 2003). The
results are shown in Table 6. All items loaded highly on the intended construct, with item-to-
construct loadings between 0.71 and 0.87, thus supporting convergent validity.
36
Table 6. Confirmatory studymeasurement properties of the usability model from the confirmatory factor analysis
Construct
name
Mean
Std.
deviation
Loading
T-Value
Construct
name
Mean
Std.
deviation
Loading
T-Value
Aesthetic
graphics
(AEST1-4)
4.44
1.13
.73
14.28***
Font
(FON1-4)
4.78
1.75
.75
13.22***
4.17
1.17
.77
15.18***
4.75
1.71
.78
15.87***
4.38
1.17
.75
16.25***
4.72
1.73
.74
16.44***
4.12
1.19
.79
15.45***
4.77
1.70
.77
17.37***
Color
(COL1-4)
4.75
1.20
.84
14.78***
Gestalt
(GEPR1-4)
4.17
1.60
.73
16.44***
4.82
1.22
.82
13.89***
4.44
1.64
.73
13.28***
4.35
1.28
.81
14.66***
3.98
1.62
.71
19.66***
4.78
1.33
.84
17.74***
3.89
1.60
.72
17.40***
Control
obviousness
(COOB1-4)
4.66
1.28
.80
16.28***
Hierarchy
(HIER1-4)
4.44
1.57
.71
15.68***
4.35
1.30
.77
17.98***
4.75
1.55
.79
14.21***
4.68
1.35
.75
14.38***
4.78
1.50
.82
12.38***
4.37
1.32
.73
15.38***
4.71
1.44
.84
14.98***
Entry point
(ENPO1-4)
4.91
1.31
.77
16.74***
Subtle
animation
(SANM1-4)
4.28
1.42
.83
17.55***
4.87
1.37
.79
15.48***
4.27
1.42
.84
16.23***
4.77
1.39
.80
16.70***
4.25
1.44
.85
17.14***
4.75
1.41
.84
21.22***
4.46
1.49
.87
16.84***
Fingertip-size
controls
(FTSC1-4)
4.28
1.43
.84
17.84***
Transition
(TRAN1-4)
4.44
0.89
.86
12.44***
4.27
1.42
.81
14.80***
4.42
0.91
.85
13.84***
4.55
1.44
.77
13.71***
4.41
1.01
.84
14.77***
4.75
1.48
.75
14.77***
4.18
1.03
.82
13.87***
*** p <.001.
37
Following Lewis et al. (2005), we next examined if the correlations between pairs of
factors were significantly different from unity. Such results would suggest discriminant validity
between the pair of factors (Lewis and Byrd, 2003; Lewis et al., 2005). Table 7 shows the results
for the pairwise tests among the mobile application usability factors. The significant χ2 tests
confirmed discriminant validity.
Table 7. Confirmatory study discriminant validity tests for the usability factor
Test
Loading
T-Value
χ2 Diff.
Aesthetic graphics
Color
.221
4.28
29.1*
Control obviousness
.241
4.44
33.0*
Entry point
.261
4.28
17.5*
Fingertip-size controls
.173
5.45
44.5*
Font
.144
4.66
38.5*
Gestalt
.128
4.56
36.7*
Hierarchy
.190
5.21
28.7*
Subtle animation
.141
5.28
29.5*
Transition
.130
5.16
31.4*
Color
Control obviousness
.187
6.12
32.3*
Entry point
.142
4.14
37.4*
Fingertip-size controls
.177
9.39
26.8*
Font
.241
4.87
27.9*
Gestalt
.130
6.65
34.5*
Hierarchy
.132
6.22
42.8*
Subtle animation
.108
6.10
51.6*
Transition
.098
4.55
52.1*
Control obviousness
Entry point
.244
3.98
35.5*
Fingertip-size controls
.172
3.44
32.8*
Font
.120
5.12
37.4*
Gestalt
.080
5.17
36.8*
Hierarchy
.041
5.07
33.1*
Subtle animation
.028
5.18
44.4*
Transition
.170
5.62
46.8*
Entry point
Fingertip-size controls
.134
4.42
29.8*
Font
.040
4.24
23.3*
Gestalt
.101
4.29
26.7*
Hierarchy
.074
3.99
30.4*
Subtle animation
.048
3.95
32.8*
Transition
.049
4.47
30.1*
Fingertip-size controls
Font
.019
4.71
77.4*
Gestalt
.028
4.75
76.8*
Hierarchy
.034
4.38
76.5*
38
Test
Loading
T-Value
χ2 Diff.
Subtle animation
.018
4.55
70.2*
Transition
.020
5.01
71.4*
Font
Gestalt
.121
4.42
68.6*
Hierarchy
.123
4.44
66.4*
Subtle animation
.007
4.38
70.1*
Transition
.044
4.37
73.4*
Gestalt
Hierarchy
.057
3.87
66.5*
Subtle animation
.068
3.42
62.8*
Transition
.074
3.75
62.8*
Hierarchy
Subtle animation
.058
4.58
55.4*
Transition
.055
4.51
52.1*
Subtle animation
Transition
.060
5.01
56.8*
* p <.05.
We next used our sample to assess the fit of the measurement model. Following Lewis et
al. (2005), we initially examined the factor-centric fit indexes. This step is useful for determining
the extent to which the set of items assessing a given factor defines the latent trait of the factor
under investigation (Lewis et al., 2005). Overall, the goodness of fit indexes were well in line
with the cutoff values recommended by Hair et al. (1998). Table 8 shows the factor-centric fit
indexes.
Table 8. Confirmatory study measures of model fit factor-centric
Adjusted χ2
Goodness of Fit
RMSR
Factor reliability
Aesthetic graphics
<5***
.955
.04
.82
Color
<5***
.971
.04
.84
Control obviousness
<4***
.947
.04
.85
Entry point
<5***
.958
.03
.80
Fingertip-size controls
<5***
.967
.02
.75
Font
<5***
.954
.04
.71
Gestalt
<5***
.912
.03
.78
Hierarchy
<5***
.948
.02
.74
Subtle animation
<5***
.974
.01
.77
Transition
<4***
.933
.02
.74
*** p <.001.
39
We continued our analysis by determining the model fit indexes of the overall model.
The results are shown in Table 9. Overall, the goodness of fit indexes were well in line with the
cutoff values recommended by Hair et al. (1998), thus supporting the validity of our model.
Table 9. Confirmatory study model fit
Model
Fit
Adjusted χ2
6.28
RMSR
.05
GFI (≥.90)
.94
RMSEA (≤.06)
.04
SRMR (≤.08)
.06
CFI (≥.95)
.96
NFI (≥.90)
.93
TLI (≥.80)
.89
To further evaluate the psychometric properties of the scales, we examined the
Cronbach’s αs, AVEs and inter-construct correlations. Table 9 shows that the AVEs were all
above .70, which is the recommended threshold (Straub et al., 2004). The results also confirmed
that the AVEs for each construct exceeded the squared correlation of the construct with other
constructs (Fornell and Larcker, 1981), thus providing further evidence of discriminant validity.
Table 10 also shows that the reliabilities, assessed using Cronbach’s αs, for all scales were above
the threshold of .70.
40
Table 10. Confirmatory study reliabilities, AVEs and correlations
Cron.α
Mean
SD
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
1. Gender
NA
NA
NA
NA
2. Age
NA
NA
NA
.03
NA
3. Income
NA
NA
NA
.04
.23***
NA
4. Aesthetic graphics
.87
4.42
1.20
.04
.03
.03
.74
5. Color
.83
4.31
1.21
.05
.05
.02
.13*
.75
6. Control obviousness
.77
3.71
1.29
.07
.02
.06
.02
.03
.74
7. Entry point
.75
3.98
1.31
.02
.12*
.04
.04
.10
.14*
.81
8. Fingertip-size controls
.79
4.01
1.55
.03
.13*
.06
.05
.08
.05
.02
.75
9. Font
.84
4.30
1.78
.05
.07
.04
.05
.15**
.02
.04
.03
.73
10. Gestalt
.73
4.40
1.70
.05
.02
.07
.02
.03
.01
.14*
.16**
.08
.75
11. Hierarchy
.75
4.03
1.36
.02
.05
-.13*
.04
.02
.04
.13*
.07
.10
.19**
.77
12. Subtle animation
.73
4.08
1.41
.07
.05
-.10
.13*
.01
.05
.03
.02
.07
.07
.04
.71
13. Transition
.72
3.77
1.51
.06
.08
.04
.07
.04
.04
.15**
.01
.04
.10
.04
.07
.74
14. Continued intention to use
.82
4.02
1.33
.14*
-.19**
.07
.15**
.07
.13*
.14*
.21***
.13*
.24***
.19**
.21***
.17**
.83
15. Brand loyalty
.83
4.17
1.28
-.13*
.15**
.04
.04
.03
.19**
.10
.24***
.07
.28***
.21***
.17**
.15**
.25***
.80
* p < 0.05, ** p < 0.01 and *** p < 0.001.
41
Next, we examined the structural model results, which are shown in Table 11. The 10
usability constructs explained 21% of variance in continued intention to use. The R2 was slightly
higher for mobile application loyalty (25%). Seven paths between the mobile application
usability constructs and continued intention to use were significant, with gestalt (.21), followed
by fingertip-size controls (.17), followed by subtle animation (.16) being the strongest
determinants. Six mobile application usability constructs were significant predictors of brand
loyalty, with gestalt, fingertip-size controls and control obviousness being the strongest.
4 Discussion
Due to the widespread diffusion of mobile technologies, more and more organizations are
seeking to incorporate mobile presences into their existing channel strategies. Although mobile
vendors, such as Microsoft, Apple and Google, provide general guidance for developing mobile
applications, we could not identify any scientific instruments that help practitioners to accurately
measure the overall usability of mobile applications. Against this backdrop, we developed a rich
conceptualization and psychometrically sound instrument for mobile application usability.
Following the methodology for construct development suggested by Lewis et al. (2005), we
Table 11. Confirmatory study structural model results
Continued intention
to use
Brand loyalty
R2
.21
.25
Gender
.03
.02
Age
.04
.02
Income
.05
.04
Aesthetic graphics
.13*
.02
Color
.02
.01
Control obviousness
.10
.17**
Entry point
.12*
.05
Fingertip-size controls
.17**
.21***
Font
.05
.04
Gestalt
.21***
.24***
Hierarchy
.13*
.15**
Subtle animation
.16**
.13*
Transition
.13*
.12*
* p < 0.05, ** p < 0.01 and *** p < 0.001.
42
initially analyzed Microsoft’s development guidelines to conceptualize mobile application
usability. Based on a content analysis of the guidelines, we conceptualized mobile application
usability via ten unique constructs. Next, we developed, pre-tested and pilot tested the new
scales. Then, we quantitatively assessed the content validity of the newly created items. To
validate our instrument, we collected two independent samples consisting of German consumers
who use mobile social media applications on their smartphones. We initially used exploratory
factor analysis to identify the factor structure in the first sample. Subsequently, using
confirmatory factor analysis, we validated the findings of the exploratory study and tested the
generalizability of our mobile application usability scales. The findings confirmed that our
conceptualization and associated instrument were strong predictors of consumers’ continued
intention to use and brand loyalty toward mobile social media applications. Our research has
several implications for research and practitioners.
2.1. Research implications
First, our comprehensive conceptualization of mobile application usability contributes to
HCI literature studying the design and use of information technology at the individual level.
Much HCI-related work drew on well-established IS acceptance theories, such as TAM (Davis et
al., 1989), IS success model (Delone and McLean, 1992) and UTAUT (Venkatesh et al., 2003) to
predict why and how individuals interact with IS artifacts. Due to their parsimony, these theories
do not aim to specifically explain why individuals find a specific application interface as easy to
use. From an HCI perspective, our research provides a more detailed view on why individuals
continue to use mobile applications because we conceptualized 10 unique mobile application
usability constructs. We found that 7 constructs contributed significantly to explaining continued
intention to use mobile applications. We believe that our fine-grained and context-specific
43
instrument helps HCI researchers to accurately predict mobile application usability and provide a
better understanding of why individuals continue to use mobile application interfaces for
obtaining information that is of value to them.
Second, much prior HCI literature has studied mobile application usability in laboratory
environments using experimental research designs, which have the limitations of artificial
settings and limited numbers of variables that can be manipulated (Kjeldskov and Graham,
2003). Our study complements such work by providing a validated research instrument for
comprehensively assessing the usability of mobile applications using a survey method. This
should be helpful for future HCI studies researching real world mobile applications in the field.
For instance, if researchers are interested in studying particular design aspects of mobile
application usability, such as user interface output and input, they can draw on our instrument
and leverage relevant constructs separately. If the goal is to study mobile application usability
holistically (e.g., if studying the usability of existing mobile applications), they could use our
entire instrument. Our instrument should help future studies aiming to evaluate user experiences
with mobile applications even in laboratory environments to understand participants’ reactions to
mobile application prototypes. By leveraging our scale, studies evaluating prototypes will be able
to effectively identify the most effective prototypes and discover avenues for improving them.
Also, design science researchers focusing on developing effective mobile application interfaces
could use our instrument to evaluate usability as part of the artifact development.
Third, our study is among the first studies that developed usability scales tailored to the
interactions with mobile applications. When evaluating mobile application usability, we found
that it is important to identify factors that are unique to mobile applications. For example, our
findings suggest that gestalt principles are critical in a mobile context because design elements
44
that are logically ordered and organized help users to navigate mobile applications on small
screens. Although it may be that gestalt principles are also important for websites, our study
supports that it is one of the most important aspects of mobile application usability. Likewise,
fingertip-size controls was found to be a significant part of the overall usability of mobile
applications. This aspect of mobile application usability is less likely to be relevant in the
website context due to the fact that most user interfaces on stationary computers (e.g., in
libraries) are operated via mouse movements. Similarly, animations should be designed
particularly subtle in context of mobile applications. One reasonable explanation for this is the
limited screen size on which mobile applications are displayed. When using animations
extensively as part of a mobile application, users might feel overwhelmed and the animations
may distract the user. We also found that some usability elements were less important than
expected. For instance, color neither significantly influenced individuals continued intention to
use nor their brand loyalty.
Fourth, this study contributes to measurement theory (Straub et al., 2004; MacKenzie et
al., 2011). Lewis et al. (2005) suggested a comprehensive methodology for developing survey
instruments. To the best of our knowledge, the current study is among the first studies that
closely followed Lewis et al. (2005) to develop a survey instrument. We applied the
recommendations and did not encounter major issues by following Lewis et al. (2005). In a few
instances, we deviated from their recommendations for practical reasons. For example, for the
content validity check, we initially followed Lawshe’s (1975) recommendations and invited
industry experts to judge the content validity of our scales. Few subject-matter experts indicated
that they were available to participate in our study due to time constraints. Thus, we decided to
employ Anderson and Gerbing‘s (1991) approach because the method proposes that judges are
45
not required to be subject-matter experts. It is also important to note that during all stages of the
instrument development procedure, we asked research participants to provide feedback and this
was generally positive.
Fifth, we felt that Microsoft’s guidelines helped us to provide a relevant contribution to
practitioners aiming to mobilize the workforce and/or business operations. Rosemann and
Vessey (2008) suggest that developing relevant research is “not necessarily based in theory, [but]
involves examining a practical intervention using a well-established, rigorous research approach”
(p. 7). Our study followed this recommendation and employed Microsoft’s user experiences
guidelines. These guidelines were developed by practitioners, and we rigorously developed the
constructs and an associated survey instrument to represent mobile application usability.
Finally, the present work is expected to serve as a critical starting point for future
scientific investigations of mobile application usability as the wireless technology revolution
continues to grow. Specifically, our study could guide future research through two avenues.
First, the developed constructs could be leveraged to assess usability of applications based on
their purpose. For example, the relative importance of the constructs might be assessed for
hedonic versus utilitarian applications, such as games versus office or productivity apps. Second,
our instrument could be used to understand organizational phenomena, such as job stress and job
satisfaction (Morris and Venkatesh, 2010; Sykes 2015). With an emphasis on organizational
phenomenon in the context of utilitarian mobile applications (Nah et al., 2005), future research
could also examine the influence of important individual differences, such as age (Morris and
Venkatesh, 2000) and physical or cognitive disabilities on mobile application usability.
Further, our work examined the applicability of the mobile application instrument in one
country (i.e., Germany) but future studies should test the generalizability of our scales in
46
different countries (see Venkatesh and Ramesh, 2006). With the extensive diffusion of
smartphones in European, Asian and North American markets, theoretically motivated studies
would serve an important scientific purpose and provide insights for practitioners distributing
mobile applications on a global scale. Studies that investigate other existing mobile applications,
such as mobile news, mobile libraries and mobile entertainment, could help practitioners in other
industries. It will be also useful to compare guidelines, both from academic and practitioner
outletse.g., Microsoft usability guidelines (see Venkatesh and Ramesh, 2006) and Apple
experience guidelines (Apple, 2011)with what we have developed and validated here. As
discussed earlier, in briefly examining the various guidelines, we find that each of them provides
some unique recommendations. For instance, only Microsoft recommends Gestalt principles.
Interestingly, our findings confirmed that the Gestalt construct was most influential on continued
intention to use and brand loyalty. In contrast, aesthetic graphics are emphasized in Apple’s and
Microsoft’s guidelines. Aesthetic graphics were also found to be important for continued
intention to use but not for brand loyalty. Although we believe that it is beyond the scope of the
current paper, future studies should analyze the guidelines offered by Microsoft’s competitors in
order to explore additional factors that are relevant to mobile application usability.
2.2. Practical implications
Enterprise mobility fundamentally changes the IT landscape of enterprises. Our study
should help organizations to integrate mobile applications into their day-to-day business
activities including supply chain management, sales force automation and field force automation
as mobile technologies have become a central component of organizational IT infrastructures.
Further, the instrument should be useful for understanding the usability of enterprise software
from the perspective of platform providers (e.g., Kude et al., 2012; Tiwana et al., 2010).
47
Specifically, our conceptualization of mobile application usability is helpful for designing mobile
information systems that are no longer bounded by fixed organizational systems. Despite the fact
that our conceptualization and associated scales of mobile application usability are
contextualized to mobile social media applications, we believe that companies could leverage
our scales in an organizational context to explore the meaning and implications of organizational
mobility. Specifically, our implications are of value for different application areas. Those
include, but are not limited to requirement engineering for mobile applications as well as the
development and maintenance of mobile applications in consumer and organizational contexts.
Requirement engineering encompasses tasks that developers perform to determine the
needs or conditions for an application to be developed (Sonderegger et al., 2012). As part of this
process, application developers often leverage interviews or focus groups in order to determine
the desired features of a given mobile application. During this phase, our survey instrument
could be used to develop interview protocols featuring open-ended questions, especially with
user-centered approaches to mobile applications design (Kangas and Kinnunen, 2005). An
example of a non-structured question presented to the participants could be: “How important do
you consider aesthetic graphics as part of mobile applications?” This question was adapted from
the items we developed for aesthetic graphics and application designers could derive similar
questions for the remaining usability constructs.
During the development of mobile applications, firms typically use a variety of software
application methods, such as agile methods (see Strode, 2006). One particular form of agile
methods is the scrum methodology. Scrum is a methodology that emphasizes iterative software
development and it provides just enough rules for teams to be able to focus on innovation
(Strode, 2006). For example, at the initial phases of a scrum project, application owners
48
determine the scope of what needs to be built in a given timeframe. Once the development team
has built the software, the outcome is demonstrated to the application owner and subsequent
steps can be determined (Strode, 2006). Our coding matrix could be particularly useful for such
situations because it could help the involved parties in discussing the mobile application progress
including the usability of the completed software components. For example, the codes derived
for the subtle animation construct emphasize that the mobile applications use subtle animations
effectively and avoid complex animations. The mobile application owner could test the
completed software components and decide whether the animations are subtle and not overly
complex.
Extreme programing (XP) is another form of agile methods and it is typically used in
high change environments using small teams (see Strode, 2006; Kude et al., 2012; Kude et al.,
2014; Stuckenberg et al., 2014). Extreme programming empowers developers to respond to
changing customer requirements, even in late application development stages. Another unique
characteristic of XP is that developers constantly communicate with their customers and fellow
programmers in order to get feedback regarding the developed software. To facilitate the process
of obtaining customer feedback, XP teams could employ our scales to survey customers
regarding their perceptions toward the usability of the developed application. Based on the
feedback obtained, applications could be modified. For instance, if customers dislike the color
scheme of the mobile application, the XP programmers could modify the color scheme and ask
customers to re-evaluate the new design.
With the demonstrated predictive validity of the instrument, organizations will also have
a useful tool to maintain and monitor the performance of their newly developed or existing
mobile applications. The ubiquity of mobile technology offers opportunities for organizations to
49
reach and maintain relationships with customers. To accrue such opportunities, organizations
need to carefully capture customers’ perceptions of mobile application usability, as mobile
applications are increasingly utilized in creating and delivering products. For instance, in the
healthcare industry, more mobile applications are introduced to offer home healthcare, hospice,
and personal care services. For example, there are mobile applications that match blood donors
with those who need it (Ramya, 2013), applications that help users manage their nutrition intake
(Philippine Daily Inquirer, 2013), applications that help users manage their dental claims (Delta
Dental, 2013), and applications that enable users to evaluate their symptoms and manage
diseases (PRWeb Newswire, 2013). For such critical applications, understanding users’
perceptions of mobile application usability becomes important so that healthcare providers can
offer better services and operate more efficiently. For example, when designing these mobile
applications, developers should particularly pay attention to gestalt principles, fingertip-size
controls, subtle animation, aesthetic graphics and transition because these constructs were most
influential in predicting continued intention to use of mobile applications. This illustrates that
practitioners should emphasize different aspects of mobile application usability depending on the
specific outcome of interest.
Organizations will also have a useful tool to disseminate information internally or
monitor knowledge management processes that are conducted via mobile applications. For
employees, social media communities offer group interactions through which knowledge is
created and exchanged, which could ultimately be integrated in product development. For
example, knowledge workers could access their corporate Wikis through their mobile phones
and collaborate on creating knowledge, which could ultimately be integrated in product
development. There is also an increasing trend toward using mobile applications to execute
50
business processes. For example, the airline industry is using in-flight mobile point-of-sale
applications (Delta, 2013) and the trucking industry will be using mobile applications to manage
vehicle and operator data (XRS, 2013). With such diverse and critical mobile applications,
organizations need a validated usability instrument that captures the most relevant mobile
usability dimensions because such applications are becoming important in creating and
delivering products.
3. Conclusions
Internet-enabled smartphones have become increasingly accepted in recent years. Due to
this, consumers today expect user-friendly mobile applications from organizations in many
industries. Yet, little systematic guidance is available that supports mobile application designers
in capturing consumers beliefs regarding the usability of mobile applications in the field.
Therefore, the current study analyzed Microsoft’s usability guidelines and conceptualized mobile
application usability and validated an associated survey instrument. We found strong support for
the psychometric properties for our scales. We also found support for the constructs in predicting
two critical outcomesi.e., continued intention to use and brand loyalty. The findings are
relevant to both academics and practitioners as they shed light on a topic of significance to
researchers and practitioners alike. In particular, researchers can leverage our conceptualization
and scales to study mobile usability and practitioners can use them to evaluate existing and to-be
developed mobile applications.
Endnotes
1 Microsoft acquired Nokia's core cell phone business in September 2013. In our work, we refer to
Microsoft’s cell phone division, which is still branded as Nokia (Microsoft 2014).
2 We also considered examining guidelines from Microsoft’s competitors, including Apple, Google, and
Blackberry. Due to the extensive detail in each set of guidelines and the length of the paper even as it
stands now, we were unable to integrate all guidelines in this paper. Instead, we focused only on
Microsoft’s mobile application usability guidelines. Further information on the usability guideline
selection criteria is provided in the literature review section.
51
3 We used the equation proposed by Anderson and Gerbing (1991) to calculate the CSV, which is equal to
the difference between the number of panelists judging an item to be essential and the highest number of
assignments of the item to any other construct in the set divided by the total number of panelists. We also
used the equation proposed by Anderson and Gerbing (1991) to calculate the PSA, which is equal to the
number of respondents assigning a measure to its posited construct divided by the total number of
respondents.
5 References
Adipat, B., Zhang, D. 2005. Interface design for mobile applications. In Proceedings of the 11th
Americas Conference on Information Systems, Omaha, USA, pp. 22812290.
Adipat, B., Zhang, D., Zhou, L. 2011. The effect of tree-view based presentation adaptation on
mobile web-browsing. MIS Quart. 35(1), 99121.
Agarwal, R., Venkatesh, V. 2002. Assessing a firm's web presence: A heuristic evaluation
procedure for the measurement of usability. Info. Sys. Res. 13(2), 168186.
Anand, A., Fosso Wamba, S. 2013. Business value of RFID-enabled healthcare transformation
projects. Bus. Proc. Manage. J. 19(1), 111145.
Anderson, J. C., Gerbing, D. W. 1991. Predicting the performance of measures in a confirmatory
factor analysis with a pretest assessment of their substantive validities. J. App. Psych.
76(5), 732740.
Apple. 2011. iOS developer library-user experience guidelines.
Avouris, N., Fiotakis, G., Raptis, D. 2008. On measuring usability of mobile applications. In
Proceedings of the International Workshop on Meaningful Measures: Valid Useful User
Experience Measurement, Reykjavik, Iceland, pp. 3844.
Barnard, L., Yi, J., Jacko, J., Sears, A. 2007. Capturing the effects of context on human
performance in mobile computing systems. Person. Ubiq. Comp. 11(2), 8196.
Barnes, S. J. (2002). Provision of services via the wireless application protocol: A strategic
perspective. Elect. Mark. 2(1), 1421.
Barnes, S. J. 2003. Mbusiness: The strategic implications of wireless communications.
Elsevier/Butterworth-Heinemann, Oxford.
Barnes, S. J., Huff, S. L. 2003. Rising sun: iMode and the wireless internet. Comm. ACM.
46(11), 7884.
Battleson, B., Booth, A., Weintrop, J. 2001. Usability testing of an academic library web site: A
case study. J. Acad. Lib. 27(3), 188198.
Bernard, M., Chaparro, B., Mills, M., Halcomb, C. 2003. Comparing the effects of text size and
format on the readability of computer-displayed Times New Roman and Arial text. Int. J.
Human-Computer Stud. 59(6), 823835.
Benbunan-Fich, R., Benbunan, A. 2007. Understanding user behavior with new mobile
applications. J. Strat. Info. Sys. 16(4), 393412.
Bhattacherjee, A. 2001. Understanding information systems continuance: An expectation-
confirmation model. MIS Quart. 25(3), 351370.
Bollen, K. 2007. Interpretational confounding is due to misspecification, not to type of indicator:
Comment on Howell, Breivik, and Wilcox. Psych. Meth. 12(2), 205218.
Brandse, M., Tomimatsu, K. 2014. Using color guidance to improve on usability in interactive
environments, HCI International 2014: Posters’ Extended Abstracts, 3–8.
Brewster, S. 2002. Overcoming the lack of screen space on mobile computers. Person. Ubiq.
Comp. 4(3), 209212.
52
Burt, R. S. 1976. Interpretational confounding of unobserved variables in structural equation
models. Socio. Meth. Res. 5(1), 352.
Burigat, S., Chittaro, L., Gabrielli, S. 2008. Navigation techniques for small-screen devices: An
evaluation on maps and web pages. Int. J. Human-Computer Stud. 66(2), 7897.
Chen, T., Yesilada, Y., Harper, S. 2010. What input errors do you experience? Typing and pointing
errors of mobile web users. Int. J. Human-Computer Stud. 68(3), 138157.
Churchill, G. A. (1979). A paradigm for developing better measures of marketing constructs. J.
Market. Res. 16(1), 6473.
Coursaris, C. K., Kim, D. J. 2011. A meta-analytical review of empirical mobile usability
studies. J. Usability Stud. 6(3), 117171.
Cyr, D., Head, M., Ivanov, A. 2006. Design aesthetics leading to m-loyalty in mobile commerce.
Info. Manage. 43(8), 950963.
Davis, F. D., Bagozzi, R. P., Warshaw, P. R. 1989. User acceptance of computer technology: A
comparison of two theoretical models. Manage. Sci. 35(8), 9821002.
Delone, W. H., McLean, E. R. 1992. Information systems success: The quest for the dependent
variable. Info. Sys. Res. 3(1), 6095.
Delta. 2013. Delta streamlines in-flight customer service with new windows phone handheld
devices for flight attendants. Retrieved December 17, 2014, from
http://news.delta.com/2013-08-22-Delta-Streamlines-In-Flight-Customer-Service-with-
New-Windows-Phone-Handheld-Devices-for-Flight-Attendants.
Delta Dental. 2013. Delta dental’s digital revolution. Retrieved August 26, 2013, from
https://www.deltadental.com/Public/NewsMedia/NewsReleaseDDMobileApplication201
308.jsp.
Dong, M., Zhong, L. 2012. Chameleon: A color-adaptive web browser for mobile OLED
displays. IEEE Tran. Mobile Comp. 11(5), 724738.
Duh, H., Tan, G., Chen, V. 2006. Mobile usabilityusability evaluation for mobile device: A
comparison of laboratory and field tests. In Proceedings of the 8th Conference on Human-
Computer Interaction with Mobile Devices and Services, Helsinki, Finland, pp. 416.
Dwivedi, Y. K., Kuljis, J. 2008. Profiling IS research published in the European journal of
information systems. Euro. J. Info. Sys. 17(6), 678693.
Fang, X., Chan, S., Brzezinski, J., Xu, S. 2003. A study of task characteristics and user intention
to use handheld devices for mobile commerce. In Proceedings of the 2nd Annual Workshop
on HCI Research in MIS, Seattle, USA, pp. 9094.
Fornell, C., Larcker, D. 1981. Evaluating structural equation models with unobservable variables
and measurement error. J. Market. Res. 18(1), 3950.
Forrester Research. 2011. How mature is your mobile strategy?
Fosso Wamba, S. 2012. RFID-enabled healthcare applications, issues and benefits: An archival
analysis (19972011). J. Medical Sys. 36(6), 33933398.
Fosso Wamba, S., Boeck, H. 2008. Enhancing information flow in a retail supply chain using
RFID and the EPC network: A proof-of-concept approach. J. Theo. App. Elect. Comm.
Res. 3(1), 92105.
Fosso Wamba, S., Chatfield, A. T. 2009. A contingency model for creating value from RFID
supply chain network projects in logistics and manufacturing environments. Euro. J. Info.
Sys. 18(6), 615636.
53
Fosso Wamba, S., Lefebvre, L. A., Bendavid, Y., Lefebvre, E. 2008. Exploring the impact of
RFID technology and the EPC network on mobile B2B eCommerce: A case study in the
retail industry. Int. J. Product. Econ. 112(2), 614629.
Gartner Research. 2014. Worldwide mobile device sales in 2013.
Gartner Research. 2015. Worldwide mobile device sales in 2014.
Harvey, M. J., Harvey, M. G. 2014. Privacy and security issues for mobile health platforms. J.
Assoc. Info. Sci. Tech. 65(7), 13051318.
Hair, J. F., Anderson, R. E., Tatham, R. L., Black, W. C. 1998. Multivariate data analysis with
readings, 5th edition. Prentice Hall, Englewood Cliffs, NJ.
Hartmann, J., Sutcliffe, A., Angeli, A. 2008. Towards a theory of user judgment of aesthetics and
user interface quality. ACM Trans. Computer-Human Interact. 16(4), 130.
Heo, J., Ham, D. H., Park, S., Song, C., Yoon, W. C. 2009. A framework for evaluating the
usability of mobile phones based on multi-level, hierarchical model of usability factors.
Interact. Comp. 21(4), 263275.
Hinkin, T. R., Tracey, J. B. 1999. An analysis of variance approach to content validation. Org.
Res. Method. 2(2), 175186.
Hoehle, H., Venkatesh, V. 2015. Mobile application usability: Conceptualization and instrument
development. MIS Quart. 39(2), 435472.
Hong, W., Thong, J. Y. L., Tam, K. Y. 2004a. Does animation attract online users attention? The
effects of flash on information search performance and perceptions. Info. Sys. Res. 15(1),
6086.
Hong, W., Thong, J. Y. L., Tam, K. Y. 2004b. Designing product listing pages on e-commerce
websites: An examination of presentation mode and information format. Int. J. Human-
Computer Stud. 61(4), 481503.
Hong, W., Thong, J. Y. L., Wong, W. M., Tam, K. Y. 2002. Determinants of user acceptance of
digital libraries: An empirical examination of individual differences and system
characteristic. J. Manage. Info. Sys. 18(3), 97124.
Hsu, C. L., Lu, H. P., Hsu, H. H. 2007. Adoption of the mobile Internet: An empirical study of
multimedia message service (MMS). Omega, 35(3), 715726.
Hu, P. J. H., Brown, S. A., Thong, J. Y. L., Chan, F. K. Y., Tam, K. 2009. Determinants of
service quality and continuance intention of online services: The case of eTax. J.
American Soc. Info. Sci. Tech. 60(2), 292309.
Hu, P. J. H., Lin, C., Chen, H. 2005. User acceptance of intelligence and security informatics
technology: A study of COPLINK. J. American Soc. Info. Sci. Tech. 56(3), 235244.
Hu, P. J. H., Hsu, F. M., Hu, H. F., Chen, H. 2010. Agency satisfaction with electronic record
management systems: A large-scale survey. J. American Soc. Info. Sci. Tech. 61(12),
25592574.
Huang, R. Y. 2012. The identification, ranking and categorization of mobile marketing success
factors. Int. J. Mobile Market. 7(2), 8697.
Huang, S. C., Chou, I. F., Bias, R. G. 2006. Empirical evaluation of a popular cellular phone’s
menu system: Theory meets practice. J. Usability Stud. 1(2), 91108.
Johnson, M. D., Herrmann, A., Huber, F. 2006. The evolution of loyalty intentions. J. Market.
70(2), 122132.
Ji, Y., Park, J., Lee, C., Yun, M. 2006. A usability checklist for the usability evaluation of mobile
phone user. Int. J. Human Computer Interact. 20(3), 207231.
54
Kaikkonen, A., Kallio, T., Kekäläinen, A, Kankainen, A., Cankar, A. 2005. Usability testing of
mobile applications: A comparison between laboratory and field testing. J. Usability Stud.
1(1), 416.
Kangas, E., Kinnunen, T. 2005. Applying user-centered design to mobile application
development. Comm. ACM. 48(7), 5559.
Kim, H., Chan, H., Gupta, S. 2007. Value-based adoption of mobile internet: An empirical
investigation. Decision Supp. Sys. 43(1), 111126.
Kim, K., Jacko, J., Salvendy, G. 2011. Menu design for computers and cell phones: Review and
reappraisal. Int. J. Human-Computer Interact. 27(4), 383404.
Kim, H., Kim, J., Lee, Y. 2005. An empirical study of use of contexts in the mobile Internet:
Focusing on the usability of information architecture. Info. Sys. Front. 7(2), 175186.
Kim, K., Proctor, R. W., Salvendy, G. 2011. The relation between usability and product success
in cell phones. Behav. Info. Tech. 31(10), 969982.
Kim, S., Lee, I., Lee, K., Jung, S., Park, J., Kim, Y. B., Kim, S. R., Kim, J. 2010. Mobile web 2.0
with multi-display buttons. Comm. ACM. 53(1), 136141.
Kjeldskov, J, Graham, C. 2003. A review of mobile HCI research methods. In Proceedings of the
5th International Mobile HCI Conference (pp. 317335). Udine, Italy.
Kjeldskov, J., Stage, J. 2004. New techniques for usability evaluation of mobile systems. Int. J.
Human-Computer Stud. 66(5), 599620.
Kude, T., Dibbern, J., Heinzl, A. 2012. Why do complementors participate? An analysis of
partnership networks in the enterprise software industry. IEEE Trans Eng. Manage. 59(2),
250265.
Kurniawan, S. 2008. Older people and mobile phones: A multi-method investigation. Int. J.
Human-Computer Stud. 66(12), 889901.
Lai, J. S., Zhang, D. 2015. ExtendedThumb: A target acquisition approach for one-handed
interaction with touch-screen mobile Phones. IEEE Trans. Human-Machine Sys. 45(3),
362370.
Lal, B., Dwivedi, Y. K. 2009. Homeworkers' usage of mobile phones; social isolation in the
home-workplace. J. Enterp. Info. Manage. 22(3), 257274.
Lavie, T., Tractinsky, N. 2004. Assessing dimensions of perceived visual aesthetics of web sites.
Int. J. Human-Computer Stud. 60(3), 269298.
Lawshe, C. H. 1975. A quantitative approach to content validity. Personnel Psyc. 28(4), 563
575.
Lazar, J., Feng, J. H., Hochheiser, H. 2010. Research methods in human-computer interaction.
Wiley, New York City, NY.
Lee, Y., Benbasat, I. 2003. A framework for the study of customer interface design for mobile
commerce. Int. J. Elec. Comm. 46(12), 4852.
Lee, S., Shin, B., Lee, H. 2009. Understanding post-adoption usage of mobile data services: The
role of supplier-side variables. J. Assoc. Info. Sys. 10(12), 860888.
Leung, R., McGrenere, J., Graf, P. 2011. Age related difference in the initial usability of mobile
device icons. Behav. Info. Tech. 30(5), 629642.
Lewis, B. R., Byrd, T. A. 2003. Development of a measure for the information technology
infrastructure construct. Euro. J. Info. Sys. 12(2), 93109.
Lewis, B. R., Templeton, G. F., Byrd, T. A. 2005. A methodology for construct development in
MIS research. Euro. J. Info. Sys. 14(4), 388400.
55
Li, Y., Yeh, Y. 2010. Increasing trust in mobile commerce through design aesthetics. Comp.
Human Behav. 26(4), 673684.
Lin, M., Goldman, R., Price, K. J., Sears, A., Jacko, J. 2007. How do people tap when walking?
An empirical investigation of nomadic data entry. Int. J. Human-Computer Stud. 65(9),
759769.
Ling, J., Schaik, P. 2006. The Influence of font type and line length on visual search and
information retrieval in web pages. Int. J. Human-Computer Stud. 64(5), 395
404.MacKenzie, S. B., Podsakoff, P. M., Podsakoff, N. P. 2011. Construct measurement
and validation procedures in MIS and behavioral research: Integrating new and existing
techniques. MIS Quart. 35(2), 293334.
Mayer, R. 2001. Multimedia learning. Cambridge University Press.
Microsoft. 2014. Resources library. Retrieved December 6, 2013, from http://developer.nokia
.com/resources/library.
Miles, M. B., Huberman, A. M. 1994. Qualitative data analysis: A sourcebook of new methods,
2nd edition. Thousand Oaks, CA: Sage Publications.
Moller, B., Brezing, C., Unz, D. 2012. What should a corporate website look like: The influence
of gestalt principles on the degree of acceptance and recommendation. Behav. Info. Tech.
31(7), 739751.
Morris, M. G., Venkatesh, V. 2000. Age differences in technology adoption decisions:
Implications for a changing work force. Personnel Psych. 53(2), 375403.
Morris, M. G., Venkatesh, V. 2010. Job characteristics and job satisfaction: Understanding the
role of enterprise resource. MIS Quart. 34(1), 143161.
Moshagen, M., Thielsch, M. 2010. Facets of visual aesthetics. Int. J. Human-Computer Stud.
68(10), 689709.
Nah, F. F. H., Davis, S. 2002. HCI research issues in e-commerce. J. Elec. Comm. Res. 3(3), 98
113.
Nah, F. F. H., Siau, K., Sheng, H. 2005. The value of mobile applications: A utility company
study. Comm. ACM. 48(2), 8590.
Nathan-Roberts, D., Liu, Y. 2015. Testing aesthetic and function design preference for touch
screen and non 13touch screen mobile phones using interactive genetic algorithms. Int. J.
Human-computer Interact. 31(4), 353262.
Nielsen, J. 2000. Designing web usability: The practice of simplicity. New Riders Publishing,
Indianapolis, IN.
Nielsen, C., Overgaard, M., Pedersen, M., Stage, J., Stenild, S. 2006. It's worth the hassle!: The
added value of evaluating the usability of mobile systems in the field. In Proceedings of
the 4th Nordic Conference on Human-computer Interaction, Oslo, Norway, p 272280.
Nilsson, E. G. 2009. Design patterns for user interface for mobile application. Adv. Eng. Soft.
40(12), 13181328.
Oliveira, R. D., Cherubini, M., Oliver, N. 2013. Influence of personality on satisfaction with
mobile phone services. ACM Trans. Computer-Human Interact. 20(2), 1023.
Paay, J., Kjeldskoy, J. 2007. A gestalt theoretic perspective on the user experience of location-
based services. In Proceedings of the 19th Australasian Conference on Computer-Human
Interaction: Entertaining User Interfaces (pp. 16). Adelaide, Australia.
Palen, L., Salzman, M. 2002. Beyond the handset: Designing for wireless communications
usability. ACM Trans. Computer-Human Interact. 9(2), 125151.
56
Pelet, J., Papadopoulou, P. 2012. The effect of e-commerce websites’ colors on customer trust.
Int. J. E-Business Res. 7(3), 118.
Philippine Daily Inquirer. 2013. First hospital-backed preventive healthcare mobile app. in PH.
Retrieved August 26, 2013, from
http://bi.galegroup.com/essentials/article/GALE|A340631269/d72c2bad3602d48a421320
d2954c1f52?u=faye28748.
Porat, T., Tractinsky, N. 2012. It’s a pleasure buying here: The effects of web-store design on
consumers’ emotions and attitudes. Human-Computer Interact. 27(3), 235276.
PRWeb Newswire. 2013. Interim healthcare releases new mobile health app. Retrieved August
26, 2013, from
http://bi.galegroup.com/essentials/article/GALE|A339375023/9cf6b271a9a6fc39ce39fe5b
1f1680a0?u=faye28748.
Ramya, M. 2013. A mobile application to find blood donor at the right time. The Times of India.
Retrieved August 26, 2013, from
http://bi.galegroup.com/essentials/article/GALE|A340193936/25f584349c2200fbb7001a6
3642d746f?u=faye28748.
Romano, M., Paolino, L., Tortora, G., Vitiello, G. 2014. The tap and slide keyboard: A new
interaction method for mobile device text entry. Int. J. Human-Computer Interact. 30(12),
935945.
Rosemann, M., Vessey, I. 2008. Toward improving the relevance of information systems
research to practice: The role of applicability checks. MIS Quart. 32(1), 122.
Savvas, A. 2010. Forrester: Marketing firms underestimate prevalence of mobile web and app
usage. Computer World UK. Retrieved December 17, 2014, from
http://dellkv.computerworlduk.com/news/mobile-wireless/3248014/forrester-marketing-
firms-underestimate-prevalence-of-mobile-web-and-app-usage/.
Scheepers, H., Scheepers, R., Stockdale, R, Nurdin, N. 2014. The dependent variable in social
media use. J. Comp. Info. Sys. 54(2), 2534.
Scornavacca, E., Barnes, S. J., Huff, S. 2006. Mobile business research published in 2000-2004:
Emergence, current status, and future opportunities. Comm. Assoc. Info. Sys. 17, 635
646.
Shackel, B. 2009. Usabilitycontext, framework, definition, design and evaluation. Interact.
Comp. 21(5), 339346.
Sonderegger, A., Zbinden, G., Uebelbacher, A., Sauer, J. 2012. The influence of product
aesthetics and usability over the course of time: A longitudinal field experiment. Ergo.
55(7), 713730.
Straub, D., Boudreau, M. C., Gefen, D. 2004. Validation guidelines for IS positivist research.
Comm. Assoc. Info. Sys. 13, 380427.
Strauss, A., Corbin, J. 1990. Basics of qualitative research: Grounded theory procedures and
techniques. Sage Publications, Newbury Park, CA.
Strode, D. 2006. Agile methods: A comparative analysis. In Proceedings of the 19th Annual
Conference of the National Advisory Committee on Computing Qualifications (pp. 257
264). Wellington, New Zealand.
Stuckenberg, S., Kude, T., Heinzl, A. 2014. Understanding the role of organizational integration
in developing and operating Software-as-a-Service. J. Bus. Econ. 84(8), 10191050.
Sykes, T.A., 2015. Support structures and their impacts: A longitudinal study of an enterprise
system implementation. MIS Quart. 39(2), 473495.
57
Tarasewich, P., Gong, J., Nah, F. F. H., DeWester, D. 2008. Mobile interaction design:
Integrating individual and organizational perspectives. Info. Know. Sys. Manage. 7(2),
121141.
Terrenghi, L., Kronen, M., Valle, C. 2005. Usability requirements for mobile service scenarios. In
Proceedings of the HCI International Conference, Las Vegas, USA, p 110.
Thomas, P., Macredie, R. D. 2002. Introduction to the new usability. ACM Trans. Computer-
Human Interact. 9(2), 6973.
Thong, J. Y. L., Hong, W., Tam, K. Y. 2002. Understanding user acceptance of digital libraries:
What are the roles of interface characteristics, organizational context, and individual
differences?. Int. J. Human-Computer Stud. 57(3), 215242.
Tiwana, A., Konsynski, B., Bush, A. A. 2010. Platform evolution: Coevolution of platform
architecture, governance, and environmental dynamics. Info. Sys. Res. 21(4), 675687.
Venkatesh, V., Morris, M., Davis, G., Davis, F. D. 2003. User acceptance of information
technology: Toward a unified view. MIS Quart. 27(3), 425478.
Venkatesh, V., Ramesh, V. 2006. Web and wireless site usability: Understanding differences and
modeling use. MIS Quart. 30(1), 181206.
Venkatesh, V., Goyal, S. 2010. Expectation disconfirmation and technology adoption:
Polynomial modeling and response surface analysis. MIS Quart. 34(2), 281303.
W3C. 2015. Web design and applications: Accessibility. Retrieved November 20, 2015, from
http://www.w3.org/standards/webdesign/accessibility.
Wu, J., Wang, S. 2005. What drives mobile commerce? An empirical evaluation of the revised
technology acceptance model. Info. Manage. 42(5), 719729.
Wertheimer, M., Riezler, K. 1944. Gestalt theory. Social Res. 11(1), 7899.
XRS. 2013. XRS develops industry’s first integrated fleet platform for compliance and
performance. Retrieved August 26, 2013, from http://xrscorp.com/blog/news/xrs-
develops-industrys-first-integrated-fleet-platform-for-compliance-and-performance/.
Yao, G., Wu, C. H., Yang, C. T. 2007. Examining the content validity of the whoqol-bref from
respondents’ perspective by quantitative methods. Social Indic. Res. 85(3), 483498.
Zhang, D., Adipat, B. 2005. Challenges, methodologies, and issues in the usability testing of
mobile applications. Int. J. HumanComputer Interact. 18(3), 293308.
Zhang, D., Lowry, P. B., Zhou, L., Fu, X. 2007. The impact of individualism-collectivism, social
presence, and group diversity on group decision making under majority influence. J.
Manage. Info. Sys. 23(4), 5380.
Zhang, Y. 2014. Beyond quality and accessibility: Source selection in consumer health
information searching. J. Assoc. Info. Sci. Tech. 65(5), 911927.
Ziefle, M., Bay, S. 2005. How older adults meet complexity: Aging effects on the usability of
different mobile phones. Behav. Info. Tech. 24(5), 375389.
58
Appendix 1. Respondent demographics.
Pilot study
Content validity study
Exploratory study
Confirmatory study
Demographic
Category
N = 30
%
N =318
%
N = 404
%
N = 501
%
Gender
Men
11
37
179
56
230
56.9
312
62.3
Women
19
63
139
44
174
43.1
189
37.7
Age groups
Under 20
1
3
48
15
14
3.5
22
4.4
20-29
23
77
210
66
142
35.1
159
31.7
30-39
5
17
54
17
110
27.2
141
28.1
40-49
1
3
4
1
60
14.9
77
15.4
50-59
0
0
0
0
49
12.1
69
13.8
60 or older
0
0
2
1
29
7.2
33
6.6
Income
(Annual, in
USD)
0-10,000
5
17
42
13
12
3.0
17
3.4
10,000-19,000
5
17
39
12
30
7.4
35
7.0
20,000-29,000
5
17
40
13
32
7.9
41
8.2
30,000-39,000
5
17
41
13
40
9.9
43
8.6
40,000-49,000
1
3
31
10
55
13.6
57
11.4
50,000-74,000
5
17
42
13
90
22.3
123
24.6
75,000-99,000
4
13
42
13
105
26.0
131
26.1
100,000-150,000
0
0
30
9
25
6.2
33
6.6
Over 150,000
0
0
11
3
15
3.7
21
4.2
Job
ICT
11
37
37
12
121
30.0
139
27.7
Banking and Finance
2
7
12
4
40
9.9
47
9.4
Insurance, Real Estate and Legal
1
3
3
1
58
14.4
40
8.0
Government and Military
2
7
6
2
38
9.4
29
5.8
Medical Healthcare
0
0
0
0
51
12.6
53
10.6
Construction and Engineering
0
0
10
3
0
0.0
0
0.0
Retail and Wholesale
1
3
1
0
19
4.7
110
22.0
Education
0
0
17
5
12
3.0
10
2.0
Marketing and Advertising
3
10
18
6
29
7.2
31
6.2
Student
7
23
167
53
31
7.7
35
7.0
Other
3
10
47
15
5
1.2
7
1.4
Social media
preference
Facebook
21
70
NA
NA
202
50.0
209
41.7
LinkedIn
0
0
NA
NA
50
12.4
141
28.1
Twitter
5
17
NA
NA
84
20.8
89
17.8
My Space
0
0
NA
NA
31
7.7
40
8.0
Google+
4
13
NA
NA
37
9.2
22
4.4
Access to
mobile sites
Application on phone
26
87
NA
NA
355
87.9
451
90.0
Web browser
4
13
NA
NA
49
12.1
50
10.0
iPhone
11
37
NA
NA
77
19.1
58
11.6
BlackBerry
2
7
NA
NA
30
7.4
21
4.2
59
Pilot study
Content validity study
Exploratory study
Confirmatory study
Demographic
Category
N = 30
%
N =318
%
N = 404
%
N = 501
%
Primary
phone use
Android
17
57
NA
NA
21
5.2
28
5.6
Windows Mobile
0
0
NA
NA
68
16.8
82
16.4
Symbian
0
0
NA
NA
208
51.5
262
52.3
Other
0
0
NA
NA
0
0
50
10.0
60
Appendix 2. Scales used to measure the outcome variables.
Outcome
variable
Items used
Scales adapted
from
Continued
intention to
use
I intend to continue using the mobile application.
(Bhattacherjee,
2001;
Venkatesh and
Goyal, 2010)
I want to continue using the mobile application rather than discontinue.
I predict I will continue using the mobile application.
I plan to continue using the mobile application.
I don’t intend to continue using the mobile application in future.
Chances are high that I will continue using the mobile application in future.
Brand
loyalty
I encourage friends and relatives to be the customers of the mobile
application.
(Johnson et al.,
2006)
I say positive things about the mobile application to other people.
I will use more services offered by the mobile application in the next few
years.
I would recommend the mobile application to someone who seeks my advice.
I consider the mobile application to be my first choice.
... Different measurement criteria are being employed to analyze other usability elements, with the system's effectiveness being assessed by assessing the task completion rate. Efficiencies are examined by looking at how long it takes users to accomplish tasks, and satisfaction can be measured by looking at how confident users are that the application will meet their expectations [64]- [66], [68]. ...
... Various studies show that SUS is an easy-to-use usability scale was created to measure System Usability [56], [60], [62], [64]- [66], [68]. SUS is a ten-item measure that provides subjective system usability assessments. ...
... According to Tullis and Stetson, 68 is a cutoff SUS score. If the score is over 68, it's above average; below 68, below average [64], [66]. ...
Article
Full-text available
Teachers’ professional development plays a significant role in the sustainability of quality education. However, providing sustainable professional development programs is difficult because of various challenges, such as a scarcity of qualified personnel and Lack of training opportunities, lack of access and resources, etc. For teachers’ professional development, educational institutions currently favor adopting and using modern educational technologies. As a result, various mobile apps have been developed and implemented in educational institutions in recent years. This paper presents the findings of a study that looked at the Usability of our designed mobile App for teachers training in Sindh by employing one of the most widely used usability criteria, SUS. It also investigates the relationship between these variables and the impact of teachers’ demographic characteristics on the SUS Score. The study’s findings achieved a 100% completion rate for all tasks, indicating an extremely effective, efficient, and sustainable system. Moreover, the satisfaction percentage was also found to be 84.1 percent, considered highly satisfactory. © 2022,International Journal of Interactive Mobile Technologies.All Rights Reserved
... Developers of such mobile apps have been focusing on improving user experience; however, due to the unique characteristics of mobile phones (e.g., screen size, connectivity, power consumption, processing capabilities, input modalities and context of use), many mobile apps remain difficult to use [4]. Prior research shows that mobile app usability is especially important for a positive user experience [5]. A high level of usability provides users with rich utilitarian and hedonic experiences. ...
... To the best of our knowledge, such an approach has not been taken. Third, mobile app usability is usually inspired by website usability concepts [17] or software usability [5]. It can be argued that there is a lack of research that addresses the unique characteristics of mobile apps, such as small screens and complex context of use. ...
... According to previous studies, a conceptualization of mobile app usability can be developed from traditional software and website usability [5]. For example, Joyce et al. [17] evaluated the usability of mobile applications by proposing a set of usability heuristics based mainly on Nielsen's website usability guidelines [32]. ...
Article
Full-text available
Advances in mobile technologies and wireless Internet services have accelerated the growth of the mobile app market. To nurture such growth, the usability of mobile apps must be addressed as a priority. Indeed, the unique characteristics of mobile phones, such as the screen size, connectivity, processing capabilities, and context of use, require a high level of usability for mobile apps. Mobile app usability requirements vary with the category of the app. However, mobile app usability is not well understood, and this may lead to the ineffective design of mobile apps and may thus influence users’ acceptance of mobile apps. This study systematically reviews and discusses mobile app usability. It identifies and interprets the relationships among the usability principles, attributes, and design features with the objective of informing mobile app usability design. It also identifies and discusses a set of common usability design features in mobile app design and a set of usability design features that are required in specific mobile app categories.
... Through a case study addressing an international perspective of healthcare practitioners, the research purposes of this work are: (1) to investigate the effects of intelligent agents on technology adoption and especially security, risk, and trust; (2) to increase our understanding of differences in the determinants of trust and acceptance Short in technology use; and (3) to improve the explanatory power and predictive accuracy of a parsimonious questionnaire based on the well known Unified Theory of Acceptance and Use of Technology (UTAUT) constructs for broader application in Human-Computer Interaction (HCI) and AI research communities. To this end, we used the UTAUT model as reported in several works [12,27,39,41,59,70,73] to analyze clinicians' intention to use an AI system. However, the novelty of this paper is to reach a sustainable model built from UTAUT in a specific healthcare domain, such as the medical imaging workflow. ...
... Models of technology adoption are among the most used for studying individual intentions to adopt technology [41,73]. The Technology Acceptance Model (TAM) [26] and UTAUT [101] is designed to measure people's attitude towards technology. ...
... Thus, considering the mobile application is an effective approach, a well-designed mobile application could help users achieve their goals efficiently and increase their satisfaction with the application (Hoehle et al., 2016). Venkatesh and Ramesh (2006) suggested that the usability (an extension of ease of use) of a mobile application is specifically essential to attain an effective and remarkable user experience. ...
Article
Full-text available
The expeditious development of mobile applications has transformed how travel e-commerce firms enhance customer relationships to implement their business strategies successfully. However, the literature has identified that a major determinant affecting customers' considerations to reject a mobile application is a deficiency of usability. Therefore, unlike the typical spotlight on general usability variables found in the prior literature, this study concentrated on investigating more specific usability factors that drive the continued intention to use and mobile application loyalty of a mobile travel e-commerce application. To achieve this, this study utilized the mobile application usability, which we adapted to form an Indonesian version. The dataset was obtained through a survey of 248 users of an Indonesian mobile travel e-commerce application, Traveloka. The collected data were analyzed using partial least square structural equation modeling (PLS-SEM) to test the hypotheses. The results indicated that application utility and the user interface structure emerge as significant drivers for both outcomes. This study could help travel e-commerce firms to develop their mobile strategies effectively, especially in terms of their usability, which may ensure that their customers continue to use the mobile application.
... Security features and application interfaces are of concern to users as well as application localization (Hoehle, Aljafari, & Venkatesh, 2016). The localization of the application makes it easy for users to set the location automatically so they don't have to reset it at checkout. ...
Article
Full-text available
The advancement of information technology, combined with increased bank competition, sparked innovation in non-cash payment options. In Indonesia, mobile payment services had grown in tandem with the rising usage of smartphones in the previous five years, resulting in a shift in financial behavior toward digitization. This study aimed to examine the financial herding behavior of using mobile payments using the Unified Theory of Acceptance and Use of Technology (UTAUT2) approach. The data used for the respondent is 355 Indonesian students who used the mobile payment application on a mobile device. The results found that the variable performance expectance and price value had a significant effect on the behavioral intention of users of mobile payment applications in Indonesia. Then behavioral intention was proven to significantly influence user behavior and herd behavior as a form of financial behavior deviation which was proven to affect performance expectations.
... Research on human-computer interaction and mobile app design has suggested that users easily recognize and like realistic images and icons representing the real world (Flavián et al., 2006). Furthermore, aesthetic graphics and realism have been recognized as key aspects of mobile app usability (Hoehle & Venkatesh, 2015) across multiple mobile app platforms (Hoehle et al., 2016). ...
... For example, the decline of spatial ability, which plays a crucial role in users' understanding of the relationships between pages and locating the page in a whole website (Ziefle & Bay, 2006), is reported to negatively influence older adults' web navigation performance by increasing difficulties in forming an accurate mental model of the website (Castilla et al., 2016;Puerta Melguizo et al., 2012;Wagner et al., 2014). However, with the development of touchscreen-based mobile technologies, information hierarchies are becoming flat and broad, which have considerately lessened the problems of disorientation caused by declined spatial ability (Hoehle et al., 2016;Zhou et al., 2012Zhou et al., , 2013. At the same, it may propose a higher demand for users' visual attention such as perceptual speed (Li & Luximon, 2019;Punchoojit & Hongwarittorrn, 2017;Yang et al., 2012). ...
Article
Full-text available
The interface metaphor was suggested to facilitate a user’s mental model development while navigating, whereas its effectiveness is still unknown among the group of older adults. We developed an experiment to investigate how older adults navigate the mobile interfaces with and without metaphors, by evaluating the possible effects of users’ perceptual speed, task complexity, content similarity, and other related user characteristics in an integrated fashion. Results indicated that the use of interface metaphor could assist in the older adults’ navigation performance, but only for those with a higher level of perceptual speed. Additionally, we found that the older adult’s navigation behavior was significantly influenced by the task complexity, content similarity and the users’ level of technology experience. These findings can help researchers and practitioners to evaluate the effectiveness of interface metaphor on older adults’ mobile navigation behavior by identifying the possible influential factors and interpreting the behind reasons.
Article
Artificial intelligence has the potential to transform many application domains fundamentally. One notable example is clinical radiology. A growing number of decision-making support systems are available for lesion detection and segmentation, two fundamental steps to accomplish diagnosis and treatment planning. This paper proposes a model based on the unified theory of acceptance and use of technology to study the determinants for the adoption of intelligent agents across the medical imaging workflow. We tested the model via confirmatory factor analysis and structural equation modeling using clinicians’ data from an international evaluation of healthcare practitioners. Results show an increased understanding of the vital role of security, risk, and trust in the usage intention of intelligent agents. These empirical findings provide valuable theoretical contributions to researchers by explaining the reasons behind the adoption and usage of intelligent agents in the medical imaging workflow.
Conference Paper
Full-text available
The evaluation of the mobile user interfaces remains one of the most challenging issues in order to improve the interaction between the user and the mobile applications. In this context, many methods and tools have been proposed to evaluate the quality of Graphical Mobile User Interface (GMUI), like heuristic evaluation and field-testing. However, these available techniques tend to be time-consuming and error-prone. An evident solution is to define a set of rules as a combination of metrics quality, context criteria and a list of possible defect types. In fact, the preferences of users are different and heterogeneous. Indeed, the users don't have the same profiles and to satisfy all users' preferences, we can highly decrease the accuracy of the detection rules. In this paper, we consider the generation of the MUI quality evaluation rules as a multi-objective problem where the goal is to find the best rules maximizing the number of detected aesthetic quality issues and minimizing the complexity of the rules. In addition, we studied the impact of the context criteria in the detection of mobile user defects by applying same feature selection techniques in order to select the most pertinent feature of context criteria and to increase the detection rules process.
Article
Mobile app usability has been attracting the attention of academics and practitioners for sometime because well-designed mobile apps can build a close link between businesses and users in addition to enhancing user experiences. To better understand mobile app usability, this study develops a unified hierarchical approach that characterizes mobile app usability from the high level of “usability principles” to the intermediary level of “usability attributes” to the detailed level of “usability features” to assess the usability of current mobile apps from different categories. Using an online survey, we identified a set of usability design features that are common to all categories of apps. Furthermore, the study identifies a set of important and less important usability features within specific mobile app categories. In addition to the practical implications consisting of insights for mobile app design, the study found important relationships among usability principles, attributes, and features.
Article
Full-text available
Usability testing a mobile application in the laboratory seems to be sufficient when studying user interface and navigation issues. Aki Kek?l?inen
Book
Research Methods in Human-Computer Interaction is a comprehensive guide to performing research and is essential reading for both quantitative and qualitative methods. Since the first edition was published in 2009, the book has been adopted for use at leading universities around the world, including Harvard University, Carnegie-Mellon University, the University of Washington, the University of Toronto, HiOA (Norway), KTH (Sweden), Tel Aviv University (Israel), and many others. Chapters cover a broad range of topics relevant to the collection and analysis of HCI data, going beyond experimental design and surveys, to cover ethnography, diaries, physiological measurements, case studies, crowdsourcing, and other essential elements in the well-informed HCI researcher's toolkit. Continual technological evolution has led to an explosion of new techniques and a need for this updated 2nd edition, to reflect the most recent research in the field and newer trends in research methodology. This Research Methods in HCI revision contains updates throughout, including more detail on statistical tests, coding qualitative data, and data collection via mobile devices and sensors. Other new material covers performing research with children, older adults, and people with cognitive impairments.
Article
The statistical tests used in the analysis of structural equation models with unobservable variables and measurement error are examined. A drawback of the commonly applied chi square test, in addition to the known problems related to sample size and power, is that it may indicate an increasing correspondence between the hypothesized model and the observed data as both the measurement properties and the relationship between constructs decline. Further, and contrary to common assertion, the risk of making a Type II error can be substantial even when the sample size is large. Moreover, the present testing methods are unable to assess a model's explanatory power. To overcome these problems, the authors develop and apply a testing system based on measures of shared variance within the structural model, measurement model, and overall model.
Article
A critical element in the evolution of a fundamental body of knowledge in marketing, as well as for improved marketing practice, is the development of better measures of the variables with which marketers work. In this article an approach is outlined by which this goal can be achieved and portions of the approach are illustrated in terms of a job satisfaction measure.
Article
MBusiness: The Strategic Implications of Mobile Communications presents recent theory and evidence on wireless information technologies and their application in business. Such information is useful in both the classroom and for interested practitioners contemplating new technological developments. MBusiness is a very fast moving area and consequently there is a need to provide relevant material for those who wish to understand the subject. This text provides recent teachings, moving beyond existing wireless technology publications, and differentiates itself strongly via its emphasis on the strategic business application of mobile communications. It stands apart by virtue of the degree of academic rigour with which it tackles the material and is an excellent resource for teaching and learning in the classroom.