Accounting for user familiarity in user interfaces

Abstract

Previous work discussed a model of cognitive distance with the novel concepts of "tech bias", "velocity" and "inertia". This paper examines the human aspects of the model by seeking to verify the expected user familiarity behaviour. It describes a pilot study that suggests the model presented allows for a very high degree of confidence in predicting the effect a user's familiarity with a problem domain and specific implementation will have on their perception of the directness of the user interface, allowing for greater insight into the construction of optimally effective novel Augmented Reality interfaces.

Full text

Accounting for User Familiarity in User Interfaces
C. A. D’H Gough, R. Green, M. Billinghurst
HITLabNZ, University of C anterbury, New Zealand
Email: Christiaan.gough@hitlabnz.org
Abstract
Previous work discussed a model of cognitive distance with the novel concepts of “tech bias”, “velocity” and
“inertia”. This paper examines the human aspects of the model by seeking to verify the expected user familiarity
behaviour.
It describes a pilot study that suggests the model presented allows for a very high degree of confidence in
predicting the effect a user’s familiarity with a problem domain and specific implementation will have on their
perception of the directness of the user interface, allowing for greater insight into the construction of optimally
effective novel Augmented Reality interfaces.
Keywords
: LaTeX style, conference paper, New Zealand conference
1 Introduction
Direct Manipulation is an approach to designing user
interfaces, which forms the basis of Graphical User
Interfaces.
A good understanding of how Direct Manipulation
works is essential in engineering optimal user
interfaces; especially in cases such as Augmented
Reality, where the interfaces are often novel and
highly unusual. A model allowing the prediction of
the effectiveness of such novel interfaces prior to
construction may save much time in their construction
and subjective evaluation.
A previous paper
1
presented a model of the
relationship between the user and computer in a
Direct Manipulation interface. This model related
cognitive distance with user familiarity and the novel
concepts of “tech bias”, “velocity” and “inertia”. This
model may be used to compare user interfaces and
explain or predict differences in the degree of
“directness” or “distance” perceived by the user. The
model defines the difference between perceived
distance and directness as being “User Factors” –
primarily that of “User Familiarity”.
In this paper these “Human Factors” are examined
more closely, and studies intended to verify the exact
effect of these factors are discussed.
2 The model
The model describes 2 key indices – the index of
directness and the index of distance. The difference
between the index of distance and directness is that of
the human factors that contribute to a user’s
perception of how direct a user interface is.
2.1 Index of Distance
The first of these indexes is the Index of Distance (S),
which may be used to predict the distance a proposed
user interface may present.
Cognitive Distance
3
is a measure of the gulfs of
execution and evaluation – the conceptual gap
between the user’s ideas and intentions, and the way
in which they are expressed to, or represented by, the
system.
Tech bias (T:(0 < T < 1)) is defined as “a measure of
how well a given device succeeds in the role for
which it is intended”
1
.
The index of distance scales the cognitive distance of
the input and output channels by the Tech Bias of
those channels respectively, as shown in equation (1):





+
+





+
=
o
oo
i
ii
T
SS
T
SS
S
asas
Mature technologies - such as CRT and LCD
displays, mice and keyboards are effective at
providing their intended experience and, as such, tend
to have a high tech bias. Conversely, less
commonplace technologies usually have a relatively
low tech bias.
In most cases the primary aim of developing an
interface is to minimise distance irrespective of user
experience, due to the variability of a large user base.
(1)

The index of distance is therefore useful for
comparing or considering user interfaces in terms that
ignore the user factors, such as in the case of
engineering a UI for mass-market acceptance.
2.2 Index of Directness
The sensation, as perceived by a user, of increased
usability and interactivity provided by a good DM
user interface is known as “directness”.
The components of directness are those of the
cognitive “distance” between the user and the
computer (S), and certain user-related factors (U).
The Index of Directness (D) describes how direct a
given user perceives a given implementation of a
given user interface to be. It is computed by scaling
the index of distance by user factors (U: (0 < U < 1)):
SUD
×
=
These user factors (U) were previously defined as
familiarity with the user interface (F:(0 < F < 1)),
yielding the following complete model:










+
+





+
×=
o
oo
i
ii
T
SS
T
SS
F
D
asas
1
The Index of Directness is an important measure
when dealing with a specific, specialised user
scenario; where the overall perceived directness might
be more relevant than the cognitive distance alone.
2.3 Application
The minimum attainable distance of a given UI is
determined by the semantic and articulatory
components of cognitive distance of the input and
output channels, and the degree to which it is possible
to achieve this theoretical minimum is governed by
the user factors and tech bias of the hardware used.
This paper uses two layers of interaction – semantic
and articulatory, but other common configurations
could be used
5, 6, 7
.
Due to the inherent difficulties of deriving meaningful
values for any of the coefficients used in the model,
any evaluation of indices using this model should be
used relatively rather than absolutely.
For example, it should not be assumed that an index
of directness computed for one case may necessarily
be compared directly with another, unless care were
taken to use the same scales, assumptions and
methodology in both cases.
2.4 Velocity of Mixed Interfaces
An interesting observation may be made in the case of
applications where the user is exposed to “mixed
distance interfaces”, where various elements of the
interface have differing distances.
A good example is that of a recording studio
application, where a part is implemented tangibly as a
“mixing desk” and a part is implemented via a
traditional GUI, mouse and keyboard.
Such mixed-distance interfaces are a sensible
approach to improving directness, as they allow a
commonly used subset of tasks or operations to have a
lessened cognitive distance without sacrificing the
flexibility of a more traditional user interface for the
less common tasks.
In such cases, it is useful to consider the change of
distance that the user must overcome when switching
focus between the interface elements. Such variations
in distance within an interface can be described as
“velocity”.
By taking a weighted average of the Index of Distance
for each of the interface types, we can derive a single
overall Index of Distance and Index of Directness for
the whole interface. This in turn means the
theoretically optimal “blend” of interface types can be
determined using linear programming.
Figure 1: A typical recording studio application
represents a good mixed-distance user interface.
2.5 Inertia
If a user interface is significantly altered in order to
improve distance, it must be determined if the gains in
directness due to decreased distance are greater than
the loss of directness caused by the decreased user
familiarity. A small improvement in the distance of a
system used by very expert users may not be enough
to counter the expertise lost in changing the interface,
resulting in a net loss of perceived directness to the
user.
Thus, any reductions of distance in an existing user
interface must be large enough to overcome the
“inertia” of the users’ experience if it is to be a
worthwhile improvement without requiring re-
learning by the users.
For example, air traffic controllers spend a long time
attaining expertise in using their systems. Because
these systems are complex and because the safety of
(2)
(3)

hundreds of lives relies on their effective use, there is
much research on improving the user interfaces in
order to reduce distance. It would be possible to
engineer a new interface that greatly reduced distance
using the Index of Distance; but in doing so, much of
the acquired directness of the system by the controller
may be lost.
In this case the index of directness should be used
instead, in order to assess the improvements in light
of the inertia of the controller using the system.
It is possible to argue that the primary focus should
always be that of directness, as new systems may be
re-learned and thus, with time, a new expertise may
be joined with the decreased distance to achieve the
most optimal possible usability. But consider that in
some cases the user may have so much inertia that it
is almost impossible to overcome.
For example, surgeons are provided important
information via auditory cues during an operation,
such as heart rate. Surgeons become so expert at using
this system that their use of the interface is almost
completely subconscious.
If the interface were re-engineered in such a way that
this information was no longer provided, it could
result in life-threatening performance decreases for
the surgeon that are unable to be re-learned. Any
replacement would in essence be a substitute, rather
than a replacement, for the auditory approach.
2.6 User Factors
Previous work suggested that the user’s sense of
directness will be inversely proportional to their level
of experience
2, 3
with the system because, as users
become familiar with the interface, less cognitive
effort is required to express their desires
3
.
The user factors that differentiate the index of
distance from the index of directness were therefore
previously expressed
1
as the reciprocal of user
familiarity:
F
U1
=
3 New Model
This paper proposes an expanded definition of the
user factors. It was reasoned that the user’s familiarity
with the problem domain of the application would be
equal in effect to that of familiarity of the
implementation of the application used – all other
factors held constant - when determining the user’s
perceived directness with a given application.
The value of U was therefore updated to take the
following form:
id
FF
U+
=2
where F
d
represents the familiarity of the user with
the problem domain, and F
i
represents the familiarity
of the user with the specific application in question.
4 Pilot study
A pilot study was performed to gain insight into the
validity of this model. Participants were provided
with a URL of a website containing a questionnaire.
Participants were able to log in to this website and
answer a series of questions regarding their degree of
experience with, and perception of, various
implementations of operating systems and file
operation environments. Several of these questions
gave insight into the participant’s experience with 4
specific operating systems – Windows XP, Mac OSX,
Linux and Command Line Interfaces such as DOS.
Table 1: Questions asked in the pilot study. These
questions were presented to the participant 4 times
with a different OS replacing "X".
1 How familiar are you with X? [1 = very
unfamiliar, 5 = very familiar]
2 How would you rate your mastery of X? [1 =
not good, 5 = very good]
3 How competant do you feel in performing tasks
with X? [1 = very incompetant, 5 = very
competant]
4 How much do you enjoy performing tasks with
X? [1 = not very much, 5 = very much]
5 How confident are you when using X to perform
tasks? [1 = very unconfident, 5 = very
confident]
6 If you had to give an overall rating of X, what
would it be? [1 = very bad, 5 = very good]
7 How easy do you feel it was to learn to use X?
[1 = very difficult, 5 = very easy]
8 How easy do you feel it is to learn new features
of X? [1 = very difficult, 5 = very easy]
9 How confident are you in your ability to retain
your current mastery of X? [1 = very
unconfident, 5 = very confident]
10 How eager would you be to demonstrate the use
of X or train novices in using X? [1 = not very
eager, 5 = very eager]
11 How much do you want to explore the more
powerful aspects of X? [1 = not at all, 5 = very
much]
12 How easily do you feel you can achieve a given
task using X? [1 = not very easily, 5 = very
easily]
13 How much do you feel that X is a tool or
extension of yourself, rather than part of the task
to be achieved? [1 = not at all, 5 = very much]
(4)
(5)

The remaining questions were intended to gather an
appreciation of the participant’s perceived directness
of the operating systems, based on the list of proposed
benefits of a good DM interface described by
Shneiderman. All questions were to be answered
using a Likert Scale of 1-5.
The questions were duplicated exactly for each
operating system so that, in effect, each participant
was completing the same questionnaire 4 times for
different Operating Systems.
5 Results
The results from 22 participants were processed in
such a way that 88 samples were obtained, where each
sample represented a set of results of one participant’s
rating of their experience and perceived directness of
an individual Operating System. Each of these results
are represented on figures 2, 3 and 4 as a single
diamond.
The results of the questions pertaining to the
participant’s familiarity with a given OS were
averaged for each sample to obtain a value for their F
i
for that OS, and the remaining questions of that
sample were averaged to represent the participant’s
perceived Directness (D) for that sample.
The resulting correlation between F
i
and measured D
gave a good correlation (R=0.863, R
2
=0.745) (fig 2)
1.00
2.00
3.00
4.00
5.00
1.00 2.00 3.00 4.00 5.00
Fi
D (Measured)
Figure 2: Fi versus Measured D
The F
d
of each participant was then computed by
averaging the F
i
of each of the 4 OS samples for that
participant. Plotting the correlation between each
participant’s F
d
and the D for each of their samples
gave a low correlation (R=0.448, R
2
=0.201)(fig. 3).
1.00
2.00
3.00
4.00
5.00
1.00 2.00 3.00 4.00 5.00
Fd
D (Measured)
Figure 3: Fd versus Measured D
Finally, the U for each sample of each participant was
computed using the method proposed by this paper –
by averaging the F
d
and F
i
for each sample.
1.00
2.00
3.00
4.00
5.00
1.00 2.00 3.00 4.00 5.00
U
D (Measured)
Figure 4: U versus Measured D
The resulting U for each sample of each participant
was very highly correlated (R=0.983, R
2
=0.967)(fig.
4).

6 Conclusion
The results of this pilot study suggest that a given
user’s perception of directness of a given user
interface may be accurately predicted using the model
described in this paper.
This model is significantly more accurate than the
traditional approach of simply assuming perceived
directness will be proportional to their familiarity with
the interface alone.
The pilot study suggests that the model described may
allow confidence of over 90%, although this needs to
be verified with more rigorous experimentation.
A good understanding of how this effect works is
essential in engineering optimal user interfaces;
especially in cases such as Augmented Reality, where
the interfaces are often novel and highly unusual. A
model allowing the prediction of the effectiveness of
such novel interfaces prior to construction may save
much time in their construction and subjective
evaluation.
7 Future Work
This study was an initial stage in verifying and
examining the theories described by Gough et al.
More exhaustive studies are currently being carried
out to provide greater insight into the results indicated
by the pilot study described in this paper. These
experiments are as follows:
7.1 Experiment 1: File operations
This experiment is to be a more exhaustive and
rigorous version of the pilot study described in this
paper.
The problem domain will be restricted to that of file
operations alone, rather than more general usage of
Operating Systems.
Participants will be asked to perform a variety of file
operation tasks using a variety of approaches,
including some approaches that will be custom-
implemented so as to allow greater focus on the
relationship between Fd and Fi.
The purpose of this experiment is to gain insight into
the role of user familiarity with a problem domain and
implementation, and to replicate the results of this
paper with more accuracy and rigour.
7.2 Experiment 2: Creative content
Participants will be asked to “mix” a song based on
content provided to them.
Participants will have varying experience in the use of
computers, audio editing and mixing, and in
performing and creating music.
Mixing will take place on a mixing desk alone, a
computer alone, and a mixed-distance interface
consisting of an automated mixing desk coupled with
an interoperable software environment on a connected
computer.
Once again, the purpose of the experiment is to gain
insight into the role of user familiarity with a problem
domain and implementation, and to replicate the
results of this paper with more accuracy and rigour.
There will, however, be additional scope to gain
insight into the effect of the effect of mixed-distance
user interfaces on perceived Directness, and
specifically the interrelation between the indices of
distance and directness under a mixed-distance
situation.
7.3 Experiment 3: DB Query
Participants will be asked to perform a series database
search queries. The queries will be via traditional user
interfaces such as a web-based search engine, an SQL
command string and a form-based Access GUI.
Participants will also be provided with several new
graphical and tangible approaches based on both new
existing metaphors.
The benefit of this research will once again be
primarily that of verification of the existing model,
but will also allow unique insight into other potential
factors unaccounted for at present, as well as into the
interplay of the indices of directness and distance.
7.4 Experiment 4: IDE Usage
The final experiment will require users to perform a
series of common tasks using development
environments. The questions listed in table (1) will be
asked of the participants, and correlated in the same
way as the pilot study and previous experiments.
8 References
[1] Gough, C. Green, R. Billinghurst, M.,
“Better Realising Direct Manipulation”,
Image and Vision Computing New Zealand
2005, pp. 455-460.
[2] Shneiderman, B., “The Future of Interactive
Systems and the Emergence of Direct
Manipulation”, Behaviour and Information
Technology 1982, v.1 n.3, pp. 237-256
[3] Hutchins, E. Hollan, J. Norman, D., "Direct
Manipulation Interfaces", Human-Computer
Interaction, Volume 1, 1985. pp. 311-338
[4] Frohlich, D, “The history and future of direct
manipulation" Behaviour & Information
Technology 12, 6, 1993. pp. 315-329.
[5] Neilsen, J, “A Layered Interaction Analysis
of Direct Manipulation”, 1992

[6] Hix, D. and Hartson, H., “Developing User
Interfaces”. John Wiley & Sons, Inc, 1993
[7] Taylor, M., “Layered protocol for computer-
human dialogue”, I: Principles. International
Journal of Man-Machine Studies, 28, 1988.
pp. 175-218
[8] Antifakos, S. “Improving Interaction with
Context-Aware Systems”, Selected Readings
in Vision and Graphics, volume 35, 2005
[9] Ishii, H., Kobayashi, M., Grudin, J.
"Integration of Interpersonal Space and
Shared Workspace: ClearBoard Design and
Experiments," ACM Transactions on
Information Systems (TOIS), ACM, Vol. 11,
1993
[10] Nelles, C. “Graphical vs Tangible User
Interface”, unpublished PhD thesis,
eingereicht am Fachochschul-
Diplomstudiengang, Medientechnik und
Design, Hagenburg, 2005