Conference PaperPDF Available

Trust and mistrust of online health sites

Authors:

Abstract and Figures

Do different design and information content factors influence trust and mistrust of online health sites? Fifteen women faced with a risky health decision were observed while searching the Internet for information and advice over four consecutive weeks. In some sessions their searches were unstructured, whilst in other sessions they were directed to review specific sites, chosen for their trust design elements. Content analysis of concurrent verbalisations and group discussion protocols provided support for a staged model wherein design appeal predicted rejection (mistrust) and credibility of information and personalisation of content predicted selection (trust) of advice sites.
Content may be subject to copyright.
Trust and Mistrust of Online Health Sites
Elizabeth Sillence, Pam Briggs, Lesley Fishwick
PACT Lab
Northumbria University
Newcastle, NE1 8ST, UK
p.briggs@unn.ac.uk
+44 191 2437250
Peter Harris
Psychology Department
Sheffield University
Sheffield, U.K.
p.harris@sheffield.ac.uk
+44 114 222 6627
ABSTRACT
Do different design and information content factors
influence trust and mistrust of online health sites? Fifteen
women faced with a risky health decision were observed
while searching the Internet for information and advice over
four consecutive weeks. In some sessions their searches
were unstructured, whilst in other sessions they were
directed to review specific sites, chosen for their trust
design elements. Content analysis of concurrent
verbalisations and group discussion protocols provided
support for a staged model wherein design appeal predicted
rejection (mistrust) and credibility of information and
personalisation of content predicted selection (trust) of
advice sites.
Author Keywords
Trust, credibility, health, social identity, Internet, computer-
mediated communication.
ACM Classification Keywords
H3.3 [Information storage and retrieval]: search processes;
K4.1 [Public Policy Issues]: computer related health issues.
INTRODUCTION
There are between ten and twenty thousand health-related
sites available on the Internet and it has been estimated that
over 21 million people have been influenced by the health
information provided therein [25]. Young people in
particular are turning to the Internet rather than to a family
doctor or a parent to get health information and advice, and
the appeal of the Internet is particularly strong for those
people who wish advice on important but sensitive matters
[18]. However less than half of the medical information
available online has been reviewed by doctors [25] and few
sites provide sufficient information to support patient
decision-making with many also heavily jargon-laden and
difficult to read [30].
There have been numerous detailed assessments of the
quality of health information on the web embracing diverse
topics such as Viagra, rheumatoid arthritis and diabetes.
Eysenbach et al [12] carried out a systematic review of
health website evaluations and noted that the most
frequently used quality criteria included accuracy,
completeness and readability and design. Accuracy referred
to the degree of concordance of the information provided
with the best evidence or with generally accepted medical
practice. Completeness was generally calculated as the
proportion of a priori-defined elements covered by the
website and design covered subjective design features such
as the visual appeal of the website and its layout.
Readability formulas were also used to establish the reading
level of a document. In their review, the authors noted that
70% of the studies concluded that quality is a problem on
the Internet.
The large body of research on online health advice belies
the fact that very little is known about how health
consumers seek advice. Almost all of the existing studies
have evaluated the quality of information and advice
available on the Internet from a medical perspective [30].
This is a problem, because we know that ordinary
consumers search for and appraise information in a
different way to experts. They are more likely to be
influenced by the attractiveness of the design [31] and they
will begin their search for advice from a general
information portal [3] – which means that they gain access
to information indiscriminately. Eysenbach & Köhler [11]
noted that consumers (as opposed to experts) failed to check
the authorship or owners of the website or read disclosure
statements, despite suggesting these as important quality
markers beforehand. However their study made use of an
experimental search task and the authors suggested that
people in a ‘real setting’ with a greater stake in the outcome
may well pay more attention to the content of the websites,
in terms of markers of quality.
There is thus a real need for systematic explorations of the
ways in which people evaluate the trustworthiness of health
information and advice online. However, there is a useful
literature about the way in which consumers evaluate the
trustworthiness of non-health information available to them
Permission to make digital or hard copies of all or part of this work for
p
ersonal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
b
ear this notice and the full citation on the first page. To copy otherwise,
or republish, to post on servers or to redistribute to lists, requires prio
r
specific permission and/or a fee.
CHI 2004, April 24–29, 2004, Vienna, Austria.
Copyright 2004 ACM 1-58113-702-8/04/0004…$5.00.
CHI 2004 ׀ Paper 24-29 April ׀ Vienna, Austria
Volume 6, Number 1
663
in an e-commerce context - see Grabner-Krauter, S. and
Kaluscha for a recent review [15].
Based on this literature we can assume that various factors
are likely to govern the extent to which individuals feel they
can trust health advice online. Firstly, they may be
influenced by the look and feel of the site – trusting, for
example, those sites rated high in visual appeal and
mistrusting those sites with poor visual design or with
unprofessional errors. Secondly, they may be influenced by
the branding of the site or by presence of familiar or images
or trusted logos. Thirdly, they may be influenced by the
quality of information available on the site, trusting those
sites with greater perceived expertise, and fourthly, they
may be influenced by the extent to which the advice is
personalised to the individual – i.e. the extent to which the
advice appears to come from and be directed to similar
individuals (i.e. those with a shared social identity).
Although we know that these various factors are influential,
there is disagreement about their relative importance in
fostering trust. For example, some researchers argue that
consumer trust (or a related construct, credibility) is
primarily driven by an attractive and professional design
[13, 31] or is influenced by the presence or absence of
visual anchors or prominent features such as a photograph
or kitemark [15, 28, 32]. Others argue that trust reflects the
perceived competence, integrity predictability and/or
benevolence of the site [2, 22, 19] and a few authors also
highlight the importance of personalisation in the formation
of trust judgments [3,4].
The picture becomes clearer when a staged model of trust is
adopted in which users engage in some fast preliminary
assessment of a site, before moving on to a process of in-
depth evaluation of the information available in a selected
few sites, and then finally developing a long-term trusting
relationship with a particular site. The first two stages make
sense when one considers that most users have to engage in
a rapid screening process of the large numbers of sites
accessed via a general search engine but can then spend
longer getting to know a handful of sites in greater depth.
They also reflect different cognitive processes or activities
identified in the literature and roughly divided into a
heuristic or affect-based stage where an initial trust
impression is formed, and (ii) an analytic stage where a
decision to engage properly with the site is made [3, 21]. It
remains to be seen whether or both heuristic and analytic
processes are employed effectively in a healthcare context
to underpin a preliminary stage of (i) intention to trust and a
later stage of (ii) trusting activity [22].
The final long-term relationship stage has been rather
overlooked in the trust literature, although it was originally
proposed in the Cheskin/Sapient report [6] and also appears
in MoTEC (a Model of Trust for E-Commerce) [9,10],
where the authors described a stage of trust maintenance,
wherein the consumer develops an informal habit-like
relationship with the vendor.
Sadly there is little empirical evidence for a staged model of
trust in e-commerce (although it can be found in the
managment literature) quite simply because most published
studies of trust do not investigate the act of trusting, but
rather investigate the intention to trust. In other words the
online trust literature suffers from some of the same
methodological problems as the health information
literature – few studies involve real consumers engaged in
real tasks. In a recent review of over thirty papers on trust in
e-commerce, only two of the cited papers involved users
who actually went on to buy a product [15]. Many of the
experimental studies on trust recruited students to the task
and most of studies of genuine consumers involved hands-
off, no-purchase (and in some cases no interaction)
evaluations.
In a recent critique of the trust literature, it has been argued
that these methodological biases have led to a skewed
understanding of the way in which trust develops in an
online context and further, that genuine consumers viewing
information and advice over longer periods of time are less
likely to be influenced by the visual appeal of a site, and
more likely to be influenced by relationship issues such as
the degree of personalisation of a site, and the extent to
which the site reflects their own social identity [4]. The
online community literature stresses the importance of
social and personal identities in the formation of long-term
relationships [26, 20].
The current study is part of a longer-term project which
aims to fill the gap in our knowledge of the ways in which
real-world consumers evaluate information and advice
online. Fifteen women at various stages of the menopause
were observed searching for information and advice online
over a four week period. This paper reports the outcome of
observations and discussions during that period, and acts as
an empirical test of the first two stages in the three-phase
model outlined above. The third, relationship phase will be
investigated in a subsequent diary study and six-month
follow-up interview which will also explore the extent to
which the women were influenced by the advice they
encountered online, and the ways in which they integrated
this advice with other sources.
The Menopause
Mid-age is often a time when women reassess their lives
and the menopause can provide a focus for the
consideration of health and lifestyle issues. During this
time, women can be faced with a number of risky health
decisions. Deciding whether to take prescribed medication
or alternative remedies is one such decision.
Hormone Replacement Therapy (HRT) has been available
in the UK since 1956. The treatment involves implants,
pills, patches or creams containing oestrogen and (unless
the woman has had a hysterectomy) progestogen to replace
the natural hormones which cease to be produced after the
CHI 2004 ׀ Paper 24-29 April ׀ Vienna, Austria
Volume 6, Number 1
664
menopause. HRT is usually prescribed to relieve
menopausal symptoms such as hot flushes, night sweats and
sleep problems but it is also recommended by doctors to
those patients at risk of osteoporosis.
However, HRT also increases the risk of two serious
conditions: deep vein thrombosis (DVT) and breast cancer.
Indeed part of the Women’s Health Initiative study (WHI)
was stopped early in 2002 because the risks were perceived
to outweigh the benefits when they reported a large increase
in breast cancer cases (http://www.whi.org)
The decision to take HRT is therefore complex and the
uncertainty about its risks and benefits makes the decision
more difficult [17]. The media and social contacts are often
a woman’s major source of information about HRT and the
menopause [16]. More recently a number of specialized
websites have appeared dedicated to women’s health issues
and to the menopause in particular [23]. Reed & Anderson
[27] examined the ownership of menopause related
websites in terms of their quality. They recommended
pharmaceutical websites highly in terms of accuracy.
METHOD
Fifteen women at various stages of the menopause
participated in the study (41-60 years, mean 49). All the
women were interested in finding out more about the
menopause and all used the Internet at least once a week
although they had different degrees of confidence with
respect to being online.
Each participant attended a total of four 2-hour sessions
held in an Internet café in Newcastle-upon-Tyne, UK.
During all four sessions, participants used the Internet to
search for information and advice on the menopause,
followed by a group discussion with a facilitator.
Participants were told to freely surf the web during sessions
1 and 4, and were directed to specific web sites during
sessions 2 and 3. These specific sites were chosen for their
trust design elements.
Earlier focus group work had identified a number of issues
that people anticipated would be important in terms of
trusting online health advice. The issues were primarily
content and provider based. They included the site being
provided by a well- known organization, contact details on
the website, simple easy to understand language and up-to-
date information. These requirements are in line with those
found by Eysenbach & Köhler [11]. They found that
consumers reported wanting a reputable source of
information, a professional layout and some sort of
endorsement or quality seal. In this current study, the sites
that the participants were directed to in weeks 2 and 3
contained various provider, content and design features
varied for trust.
The participants were asked to record their perceptions of
each site visited in a logbook and use this information
during the discussion sessions. In addition, participants
engaged in concurrent verbal protocols as they searched
through some of the sites. The group discussion themes
were developed in line with the research questions and were
piloted during earlier focus groups. The discussion guide
covered the following main areas: 1) current information
sources, 2) search strategies, 3) liked and disliked websites,
4) first impressions and 5) revisiting websites.
All discussions and verbal protocols were transcribed and
subject to content analysis. At the end of the fourth week
the participants were given diaries in order to record their
ongoing information and advice searches and were invited
to take part in a follow up interview. An overview of the
study procedure is shown in Table 1.
Week 1 hour
Concurrent
10
minutes
50
minutes
1 Free web
search
Verbal
protocol
Break Group
discussion
2
Directed
search to
specific
sites
Verbal
protocol
Break Group
discussion
3
Directed
search to
specific
sites
Verbal
protocol
Break Group
discussion
4 Free web
search
Verbal
protocol
Break Group
discussion
Table 1. Overview of study procedure
RESULTS
Search strategies and effectiveness
During the unstructured sessions participants used a variety
of search engines and portals to search for menopause
information and advice. None of the participants used
medical portals as a starting point for their searches,
although a number of participants tried to guess the web
address of the National Health Service (NHS) (without
success). During week 1 the participants looked at between
2 and 6 sites. In total, more than 40 sites were visited during
the first session.
The search strategies used by the participants varied
according to specific areas of interest. To begin with
participants carried out simple searches using single
keywords. They soon modified their searches making use of
explicit Boolean operators (in most cases “AND”) and
regularly altered search terms to alter the search engine
results. The participants tended to work through the search
results choosing to click on ones that sounded promising.
They looked for certain keywords and source identifiers.
Several participants limited their search engines to the co.uk
versions to ensure UK relevant information.
The participants reported that they had been successful in
finding the kind of information and advice that they were
searching for. In general the participants preferred websites
CHI 2004 ׀ Paper 24-29 April ׀ Vienna, Austria
Volume 6, Number 1
665
that were specifically based around the menopause as
opposed to women’s health sites or general health and
medical sites.
Content analysis of transcripts
The transcripts of the verbal protocols and the discussion
groups were analysed for emerging themes concerning
markers used by the participants to assess the
trustworthiness of the online health advice. A coding
scheme was developed in accordance with the aims of the
research, the discussion guide and the emerging and
recurring themes. In this paper we summarise those aspects
of discussion relevant to the selection of sites. Responses
were firstly coded according to the selection or rejection of
sites and then in terms of design and content issues. A
number of themes relating to the first impressions of the
website and characteristics of trustworthy sites emerged.
Rejection of Websites
Participants discussed their first impressions of a website.
There were two factors that led them to reject or mistrust a
website quickly. These are summarized in table 2, with
numbers included to give an indication of relative
importance. The overwhelming majority of comments
related to the design of the website.
Type of
factor Specific aspects of the site Weighting
(see legend)
Design
Inappropriate name for the
website
Complex, busy layout
Lack of navigation aids
Boring web design especially use
of colour
Pop up adverts
Slow introductions to site
Small print
Too much text
Corporate look and feel
Poor search facilities/indexes
94%
Content Irrelevant or inappropriate content 6%
Table 2. Factors relating to the rejection and mistrust of
websites. The final column shows the number of times a factor
was mentioned as a percentage of the total number of
comments about rejection.
The look and feel of the website was clearly important to
the participants. Visual appeal, plus design issues relevant
to site navigation appeared to exert a strong influence on
people’s first impressions of the site. Poor interface design
was particularly associated with rapid rejection and mistrust
of a website. In cases where the participants did not like
some aspect of the design the site was often not explored
further than the homepage and was not considered suitable
for revisiting at a later date. One participant mentioned that
an instant rejection could also occur if the website was
completely irrelevant and not related to the topic of interest
at all.
you have thrown a word in and you’ve come up with a
website and its nothing in terms of what you are looking for
and you come straight out ….you get a lot of stuff which has
absolutely got no relevance and you think well how did I get
to this screen (female, 53 years old)
However, as table 2 shows the main reason that websites
were rapidly rejected was due to the design of the interface.
Design issues affected first impressions and could lead to
the mistrust of a website.
It’s so clinical, so pasty, lots of white lots of pale blue
obviously trying to be gentle on the eye (female, 48 years old,
verbal protocol)
the banners, when they are trying to sell you something or
click down here for your free whatever, you just get turned
off (female, 49 years old)
Negative comments in the group discussion also related to
poor interface design:
And one of them I didn’t like the colour of. I couldn’t wait to
get out it was an insipid green backdrop it just put me off
reading it (female, 53 years old)
There was just nothing I liked about it at all. I didn’t like the
colours, the text, the layout (female, 52 years old)
I found the screen too busy I couldn’t quite latch onto
anything straight away (female, 66 years old)
An example of a mistrusted site is shown in Figure 1.
Figure 1. An example of a mistrusted and rejected website.
This site illustrates some of the features associated with
mistrust including choice of colour, unusual layout (right
hand side menus), and a corporate look and feel.
Participants were also influenced by the website’ name. A
good name was specific and to the point but was not
patronizing or too gimmicky. A poor name was not
trustworthy and could lead to a rapid rejection of the site.
CHI 2004 ׀ Paper 24-29 April ׀ Vienna, Austria
Volume 6, Number 1
666
Well I didn’t like it, I think it was possibly the name but I
didn’t hold out any confidence in something called Netdoctor
at all, it sounds more like an IT company to me (female, 41
years old)
Participants thought that the visual appeal of the site was
important, poor visual appeal did not encourage further
exploration.
It’s a very visual thing, anything that’s covered in ads and
pop ups and stuff like that I’m just not interested (female, 48
years old)
Selection of websites
The participants mentioned a number of factors in terms of
the sites that they had chosen to explore in more depth. The
themes are summarized in table 3. Trust was an important
feature of the selected websites. The participants liked sites
that contained a great deal of information but that was
presented in such a manner that an individual could quickly
pinpoint their own specific areas of interest. The
participants’ selection of websites to explore and revisit was
also dependent on the assessment of the content in relation
to their own personal angle.
Table 3 indicates that content factors were more important
than design features in describing trusted or well-liked sites.
Type of
factor Specific aspects of the site Weighting
Design
factors
Clear layout
Good navigation aids
Interactive features e.g.
assessment tools
17%
Content
Factors
Informative content
Relevant illustrations
Wide variety of topics covered
Unbiased information
Age specific information
Clear, simple language used
Discussion groups
Frequently asked questions
83%
Table 3. Positive features about selected, trusted sites
The favourite sites were usually described in terms of their
content.
I found an absolutely marvellous site I was really, really
taken with it, it went into so such clear explanations and with
a breakdown of the different, oestrogen, progesterone,
testosterone and what they actually do and how they link
together all along the way it kind of encouraged you at the
beginning to work through the site progressively if you
wanted to get like a whole raft of background knowledge and
then it would help you make decisions, it was great (female,
48 years old)
Participants trusted the selected sites because they
demonstrated an in-depth knowledge of a wide variety of
relevant topics and put forward unbiased clear information.
Participants were more likely to trust the information if they
could verify it and cross check it with other websites.
If I’d read something about that information before that sort
of backed it up I would be more likely to trust it (female, 55
years old)
Most individuals preferred sites that were run by reputable
organizations or had a medical or expert feel about them.
They trusted the information on such websites especially
when the credentials of the site and its authors were made
explicit. Sites that indicated that the advice originated from
a similar individual was also well received. Most
participants showed some distrust of the advice and
information on websites sponsored by pharmaceutical
companies or those explicitly selling products.
Participants were looking for sites that were written by
people similar to themselves, who shared similar interests.
In this way advice feels personalized for them. Figure 2
illustrates some of the features of a trusted site. Project
Aware is a “website by women for women.” The site is split
into menopause stage specific areas. The site covers a wide
variety of relevant topics and provides links to original
research materials. The language is clear and simple and the
layout out is easy on the eye.
Figure 2. A trusted website
Although the participants were all interested in the
menopause, they all had their own personal agendas and
interests. One participant, for example, had a pre-existing
condition, which prevents her from taking HRT. She
searched for and selected sites, which gave her information
and advice relevant to her condition and was particularly
interested in alternative remedies.
CHI 2004 ׀ Paper 24-29 April ׀ Vienna, Austria
Volume 6, Number 1
667
I can’t take HRT that’s my thing, that’s the angle like you,
everybody comes at it from their own angle (female, 48 years
old)
Another participant had been through the menopause and
felt that a lot of unanswered questions remained. She was
searching through websites trying to find information and
advice that matched her own experiences. The participants
were keen to read about other women’s experiences on
website discussion boards although they did not feel
immediately comfortable posting their own messages to the
site.
There was a case where someone was given it [HRT] and it
did sort of relate to how it happened to me, it was good to
know that I wasn’t alone (female, 60 years old)
“I’m not always looking for a medical opinion, I think its
nice to read about other people’s experiences, see how
everyone else is coping (female, 49 years old)
One other participant was currently taking HRT but was
keen to know more about the risks and benefits in terms of
deciding whether or not to stay on the drugs.
I know that I look at it and I have a slight HRT drugs bias I
am on HRT and I am interested in finding out more about it
I’m interested in seeing what the risks are (female, 52 years
old)
Revisiting websites
Participants expressed their desire to revisit a number of the
sites that had found or had been directed to over the course
of the study. Reasons for revisiting a site included a change
of symptoms or new information reported in the media.
Participants talked about information rich sites, sites that
you could become immersed in and the importance of
bookmarking good sites.
[This site] has a special interest to me personally and I’m
absolutely fascinated…I shall definitely, definitely visit it
again and take quite a lot of notes from it very, very useful
(female, 41 years old)
SUMMARY AND DISCUSSION
This work is concerned with the issue of how trust develops
in an online health context. As reported in the introduction,
almost all studies of online health advice have evaluated the
quality of information and advice available on the Internet
from a medical perspective. In this study it has been shown
that consumers, especially those with an interest in the
health topic, search for and appraise information in a
different way to experts. Reed & Anderson [27]
recommended that women search for health information
regarding the menopause on pharmaceutical websites
because of the high levels of accuracy on such sites. The
participants in our study, however, mistrusted sites
sponsored by pharmaceutical companies and disliked sites
with a corporate look and feel.
In their recent review of trust literature Briggs et al [4] have
suggested a three-stage model of the process of trust
development. In this paper we have reported on the first two
stages of this model. During the first stage participants
carried out a rapid screening process rejecting sites they did
not trust. Mistrust was found to relate to poor design appeal.
The participants were less likely to trust sites that contained
adverts, pop up surveys or that were poorly laid out. Poor
design gave a negative first impression and the name of the
website could lead participants to mistrust the site and its
authors intentions.
The selection of sites was based on trust. Participants
trusted sites that provided informative content on a wide
range of relevant topics. The information was trusted if it
was unbiased and if the information on such sites was
supported by research articles or original sources. Sites that
were selected contained a variety of content features
including Frequently Asked Questions (FAQs) and a
section on hints and tips. Trusted sites were selected by the
participants for further examination. This second stage of
trust development relied more heavily upon a careful
evaluation of website content. The participants were quick
to notice website sponsorship, (even if buried in the small
print), currency (how up-to-date the information was), any
information biases, cultural differences or inconsistencies.
The participants trusted sites that reflected their own social
identity. They liked sites that were written by women and
those that they felt were specifically for women like
themselves. Although there was not time during the four-
week study for the participants to develop proper
relationships with any of the websites they were already
showing preferences for sites that reflected their own social
identity. Social identity appears to be an important part of
the trust process even during the early stages of trust
formation a point which has not been fully recognized
within the e-commerce literature on trust.
As predicted the participants in our study, with a greater
stake in the outcome of their web searches, paid close
attention to the content of selected sites and were careful
and critical evaluators of the information. We can presume
that for these users - faced with genuine health risks
associated with taking the online advice - involvement with
the site was high. This is important when we consider trust
and reflects the work of Chaiken [5] who described two
experiments that show that the degree of involvement in an
issue affects processing strategy - those participants with
low involvement adopted a heuristic approach to evaluating
a message and were primarily influenced by its
attractiveness, whereas those with high involvement
adopted a systematic approach - presenting more arguments
to support their judgment. Our results are also consistent
with a number of other studies in the persuasion literature
that show that people use cognitively intense analytical
processing when the risks involved are great, or the task is
particularly engaging, whereas they use affect or other
simple heuristics to guide their decisions in low risk
situations when they lack the motivation or capacity to
think properly about the issues involved [7,21,24].
CHI 2004 ׀ Paper 24-29 April ׀ Vienna, Austria
Volume 6, Number 1
668
Future Work
We are concerned with exploring a three-stage model of
trust as shown in figure 3. In this paper we have reported
upon the first two stages of trust development The third
stage, that of maintaining trust is a longer-term process is
one which is unlikely to develop over a four week period,
but it is also one where social identity influences may
become more important. Many health decisions are subject
to review based on changes in personal circumstances or the
publication of new research findings. As such consumers
are likely to revisit websites over a period of months to re-
examine information and advice.
Figure 3. Staged model of trust
To investigate the longer-term process of maintaining trust
the participants in this study have been given diaries to keep
over a number of months and have been invited to take part
in follow up interviews. It is anticipated that this data will
allow us to examine the process of maintaining trust. We
would predict that long term engagement with a health
website is more likely to be influenced by issues such as the
degree of personalisation of a site and the extent to which
the site reflects the participants own social identity. As
noted above, the participants were already starting to select
and trust sites on the basis of the extent to which they were
written by like-minded people and were targeted at people
like themselves. Relationship building through involvement
in discussion boards, however, was limited in our study,
and is something that is more likely to develop over time in
a more private setting. The diary and interview data will
also allow us to examine the extent to which information
and advice sought online is integrated with other sources of
advice from friends and family, doctors, and government
health campaigns. Finally, it should be possible to examine
the extent to which the participants genuinely acted upon
the advice sought during this four-week study.
CONCLUSION
Our work provides evidence for a staged model of trust in
which visual appeal influences early decisions to reject or
mistrust sites, whilst credibility and personalization of
information content influences the decision to select or trust
them. These findings overcome methodological criticism
that many studies of online trust do not engage real users
with genuine concerns. They also reconcile troublesome
contradictions in the trust literature with respect to the
importance of visual design and information content. The
methodology employed here would seem a useful one for
further investigations of trust although it is worth asking
whether participants in a focus group setting might be
tempted to make socially desirable responses when
discussing content factors in the selection of sites. We have
no reason to believe this to be true - the discussions
genuinely reflect the logged behavioural data and
participants were happy to stress the importance of design
features in website trust (rather than try to impress others
with careful considerations of site content). Nonetheless
this is an issue that will be addressed explicitly in the
individual follow-up interviews.
ACKNOWLEDGMENTS
The authors would like to acknowledge the support of the
ESRC E-Society Initiative.
REFERENCES
1. Albarracin, D. & Kumkale, G., T. Affect as
Information in Persuasion: A Model of Affect
Identification and Discounting, Journal of Personality
and Social Psychology, 84, 3, (2003) 453-469.
2. Bhattacherjee, A. Individual Trust in Online Firms:
Scale Development and Initial Test. Journal of
management Information Systems, 19,1 (2002) 211-
241.
3. Briggs, P., Burford, B., De Angeli, A. & Lynch, P.
Trust in Online Advice, Social Science Computer
Review, 20, 3, (2002) 321-332.
4. Briggs, P., de Angeli, A. and Simpson, B.
Personalisation and Trust: A Reciprocal Relationship?
Forthcoming in M.C. Kalat, J. Blom and J. Kalat (Eds).
Designing Personalized User Experiences for E-
Commerce. Kluwer, 2004.
5. Chaiken, S. Heuristic versus systematic information
processing and the use of source versus message cues
in persuasion. Journal of Personality And Social
Psychology, 39 (1980) 752-766
6. Cheskin Research and Studio Archetype/Sapient
(1999). Ecommerce trust study. Retrieved (July 16,
2003) from:
http://www.cheskin.com/p/ar.asp?mlid=7&arid=40&art
=0&isu=1
7. Clore, G., L., Schwarz, N.,& Conway, M. Affective
causes and consequences of social information
processing. In R. S. Wyer & T. K. Srull (Eds.)
Handbook of Social Cognition (1994) 323-417.
Hillsdale, NJ: Erlbaum.
8. Corritore, C.L., Kracher, B. and Wiedenbeck, S. On-
line trust: concepts, evolving themes, a model.
International Journal of Human-Computer Studies, 58,
(2003) 737-758.
9. Egger, F.N. "Trust Me, I'm an Online Vendor":
Towards a Model of Trust for E-Commerce System
Design. Proc. CHI 2000. ACM Press.
Rapid screening of
sites based upon
he ur is tic a nal ys is
Systematic
evaluation of
site content
Longer term consultation
and self d isclos ure
processes
Integration of
infor matio n acro ss s ites
and sources
CHI 2004 ׀ Paper 24-29 April ׀ Vienna, Austria
Volume 6, Number 1
669
10. Egger, F.N. Affective Design of E-Commerce User
Interfaces: How to Maximise Perceived
Trustworthiness. Proc CAHD: Conference on Affective
Human Factors Design (2001) 317-324.
11. Eysenbach, G. and Köhler, C. How do consumers
search for and appraise health information on the world
wide web? Qualitative study using focus groups,
usability tests, and in-depth interviews. British Medical
Journal 324 (2002), 573-577.
12. Eysenbach, G. Powell, J., Kuss, O., and Sa, E-R.
Empirical studies assessing the quality of health
information for consumers on the world wide web, a
systematic review. Journal of the American Medical
Association 287, 20 (2002), 2691-2700.
13. Fogg, B. J., Kameda, T., Boyd, J., Marchall, J., Sethi,
R., Sockol, M. and Trowbridge, T.. Stanford-Makovsky
Web Credibiltiy Study 2002: Investigating what makes
Web sites credible today, A Research Report by the
Stanford Persuasive Technology Lab & Makovsky &
Company, Stanford University. Retrieved from :
http://www.webcredibility.org .
14. Fogg, B.J. Prominence-Interpretation Theory:
Explaining How People Assess Credibility Online.
Proc CHI 2003, ACM Press.
15. Grabner-Krauter, S. and Kaluscha, E.A. Empirical
research in online-trust: a review and critical
assessment. International Journal of Human-Computer
Studies, 58 (2003) 783-812.
16. Griffiths, F. Women’s control and choice regarding
HRT. Social Science & Medicine 49 (1999), 469-481.
17. Hunter, M.S. and O’Dea, I. Perceptions of future health
risks in mid-age women: estimates with and without
behavioural changes and hormone replacement therapy.
Maturitas 33 (1999) 37-43.
18. Klein, J.D and Wilson, K.M. Delivering quality care:
adolescents' discussion of health risks with their
providers. Journal of adolescent health, 30(3) 2003,
190-195.
19. Lee, M.K.O., and Turban, E. . A trust model for
consumer Internet shopping. International Journal of
Electronic Commerce, 6, 1 (2001), 75-91.
20. Longmate, E. & Baber, C. How useful are online
community design guidelines: A case study of two fan
communities. Paper Presented at Association of
Internet Researchers Conference,AIRC 3.0, 2002.
21. McAllister, D.J. Affect-Based And Cognition-Based
Trust As Foundations For Interpersonal Cooperation In
Organizations Academy Of Management Journal 38,1
(1995) 24-59.
22. McKnight, D. H. and Chervany, N.L. Trust and
Distrust Definitions: One Bite at a Time. In R. Falcone,
M. Singh and Y.-H. Tan (Eds.) Trust in Cyber-
societies. Berlin: Springer-Verlag, 2001.
23. Pandey, S.K., Hart, J.J., and Tiwary, S. Women’s
health and the internet: understanding emerging trends
and implications. Social Science & Medicine 56
(2003), 179-191.
24. Petty, R.E. and Wegener, D. T. The elaboration
likelihood model: Current status and controversies. In
S. Chaiken & Y. Trope (Eds.),Dual-process theories in
social psychology New York: Guilford Press (1999)
41-72.
25. Pew Research Center. The online health care
revolution: How the Web helps Americans take better
care of themselves (2000). Available at:
http://www.pewinternet.org
26. Preece, J. Online communities: Designing usability and
supporting sociability. John Wiley & Sons, Chichester,
UK, 2000.
27. Reed, M. and Anderson, C. Evaluation of patient
information Internet web sites about menopause and
hormone replacement therapy. Maturitas 43 (2002),
135-154.
28. Riegelsberger, J., Sasse, M. A., & McCarthy, J. Shiny
happy people building trust? Photos on e-commerce
websites and consumer trust. Proc CHI 2003. ACM
Press.
29. Shelat, B. and Egger, F.N. What makes people trust
online gambling sites? An empirical study to identify
on- and offline factors that influence gamblers’
perception of an online casino’s trustworthiness. Proc
CHI 2002. ACM Press.
30. Smart, J M and Burling, D. Radiology and the internet:
a systematic review of patient information resources,
Clinical Radiology, 56(11), (2001) 867-870
31. Stanford, J., Tauber, E., Fogg, B.J. and Marable, L.
Experts vs. Online Consumers: A Comparative
Credibility Study of Health and Finance Web Sites.
Consumer Web Watch Research Report (2002).
Retrieved (August 2003) from:
http://www.consumerwebwatch.org/news/report3_credi
bilityresearch/slicedbread_abstract.htm
32. Steinbruck, U., Schaumburg, H., Duda, S. and Kruger,
T. A picture says more than a thousand
words: photographs as trust builders in e-commerce
websites. Proc. CHI 2002. ACM Press.
CHI 2004 ׀ Paper 24-29 April ׀ Vienna, Austria
Volume 6, Number 1
670
... The authors' literature analysis of eHealth publications found 38 different trust attributes in 5 categories: personal elements and individual antecedents (5); website-related antecedents (9); service provider-related elements (20); informational elements, i.e., design and content factors (9); and information sources (5) (Appendix B). A meaningful finding was that informational elements were the most meaningful attributes in eHealth [112]. According to Liu et al., direct experience is the most reliable information factor for trust measurement [18]. ...
... The authors declare no conflict of interest. Informational elements (design and content factors) from [49,112,122,124] • Quality of links; • Information quality and content (accuracy of content, completeness, relevance, understandable, professional, unbiased, reliable, adequacy and up-to-date), source expertise, scientific references; • Information source credibility, relevant and good information, usefulness, accuracy, professional appearance of a health website; • Information credibility; • Information impartiality. ...
... Information sources from [49,67,112,122,127] • Personal interactions; • Personal experiences; • Past (prior) experiences; • Presence of third-party seals (e.g., HONcode, Doctor Trusted™, TrustE). ...
Article
Full-text available
The use of eHealth and healthcare services are becoming increasingly common across networks and ecosystems. Identifying the quality and health impact of these services is a big problem that in many cases it is difficult determine. Health ecosystems are seldom designed with privacy and trust in mind, and the service user has almost no way of knowing how much trust to place in the service provider and other stakeholders using his or her personal health information (PHI). In addition, the service user cannot rely on privacy laws, and the ecosystem is not a trustworthy system. This demonstrates that, in real life, the user does not have significant privacy. Therefore, before starting to use eHealth services and subsequently disclosing personal health information (PHI), the user would benefit from tools to measure the level of privacy and trust the ecosystem can offer. For this purpose, the authors developed a solution that enables the service user to calculate a Merit of Service (Fuzzy attractiveness rating (FAR)) for the service provider and for the network where PHI is processed. A conceptual model for an eHealth ecosystem was developed. With the help of heuristic methods and system and literature analysis, a novel proposal to identify trust and privacy attributes focused on eHealth was developed. The FAR value is a combination of the service network’s privacy and trust features, and the expected health impact of the service. The computational Fuzzy linguistic method was used to calculate the FAR. For user friendliness, the Fuzzy value of Merit was transformed into a linguistic Fuzzy label. Finally, an illustrative example of FAR calculation is presented.
... This cue-heuristic perspective raises an important question: what cues are made available and what heuristics can be triggered by a given technology? Web researchers have answered the question empirically [15,56]. By surveying 2500 participants, Fogg [14] summarizes 18 types of cues people frequently notice on a website to base their trust judgments on, such as information structure, name recognition, and advertising, with the most frequently mentioned cue being the "design look. ...
... A conceptual analysis as we did in this paper is not enough. Future work should empirically study what people actually pay attention to and how they process them when making trust judgments, similar to what has been done in the web trust literature [14,15,40,56]. To understand the effect of a trustworthiness cue, we echo the point made by Jacovi et al. [25] that it should be studied in relation to different levels of model trustworthiness. ...
... This cue-heuristic perspective raises an important question: what cues are made available and what heuristics can be triggered by a given technology? Web researchers have answered the question empirically [15,56]. By surveying 2500 participants, Fogg [14] summarizes 18 types of cues people frequently notice on a website to base their trust judgments on, such as information structure, name recognition, and advertising, with the most frequently mentioned cue being the "design look. ...
... A conceptual analysis as we did in this paper is not enough. Future work should empirically study what people actually pay attention to and how they process them when making trust judgments, similar to what has been done in the web trust literature [14,15,40,56]. To understand the effect of a trustworthiness cue, we echo the point made by Jacovi et al. [25] that it should be studied in relation to different levels of model trustworthiness. ...
Preprint
Current literature and public discourse on "trust in AI" are often focused on the principles underlying trustworthy AI, with insufficient attention paid to how people develop trust. Given that AI systems differ in their level of trustworthiness, two open questions come to the fore: how should AI trustworthiness be responsibly communicated to ensure appropriate and equitable trust judgments by different users, and how can we protect users from deceptive attempts to earn their trust? We draw from communication theories and literature on trust in technologies to develop a conceptual model called MATCH, which describes how trustworthiness is communicated in AI systems through trustworthiness cues and how those cues are processed by people to make trust judgments. Besides AI-generated content, we highlight transparency and interaction as AI systems' affordances that present a wide range of trustworthiness cues to users. By bringing to light the variety of users' cognitive processes to make trust judgments and their potential limitations, we urge technology creators to make conscious decisions in choosing reliable trustworthiness cues for target users and, as an industry, to regulate this space and prevent malicious use. Towards these goals, we define the concepts of warranted trustworthiness cues and expensive trustworthiness cues, and propose a checklist of requirements to help technology creators identify appropriate cues to use. We present a hypothetical use case to illustrate how practitioners can use MATCH to design AI systems responsibly, and discuss future directions for research and industry efforts aimed at promoting responsible trust in AI.
... Yet, from works addressing usability as an antecedent of trust specifically, we can derive some concrete suggestions. First, ease of navigation and user guidance are beneficial [23,73,76]. Similarly, consistency in design and color schemes improves usability and trustworthiness [19,23,81]. ...
... Similarly, consistency in design and color schemes improves usability and trustworthiness [19,23,81]. For non-intuitive interfaces, learnability was found to be effective [6,73,76]. This can mean giving users the opportunity to learn about the functions of the system and encouraging them to explore it [6]. ...
Preprint
Interface design can directly influence trustworthiness of a software. Thereby, it affects users' intention to use a tool. Previous research on user trust has not comprehensively addressed user interface design, though. We lack an understanding of what makes interfaces trustworthy (1), as well as actionable measures to improve trustworthiness (2). We contribute to this by addressing both gaps. Based on a systematic literature review, we give a thorough overview over the theory on user trust and provide a taxonomy of factors influencing user interface trustworthiness. Then, we derive concrete measures to address these factors in interface design. We use the results to create a proof of concept interface. In a preliminary evaluation, we compare a variant designed to elicit trust with one designed to reduce it. Our results show that the measures we apply can be effective in fostering trust in users.
... A website design is the art and process of combining elements of design such as (line, shape, texture, logo, navigation bar) into a pleasant arrangement that are displayed on the websites to the online users [1]. People can be influenced by the look and feel of the website [14]. In this context, deception in Islamic websites is defined as the websites text-based content and design that were deviated and contradict the Islamic teaching and learning (by Ahlus-Sunnah wa'l-Jama'ah). ...
... The credibility of web information has become a serious social issue. For example, Sillence et al. reported that more than half of the health information available on the web has not been verified by experts (Sillence et al., 2004). Therefore, if web search users may believe misinformation, they cannot distinguish correct and incorrect web information. ...
Article
Full-text available
In this study, we analyzed the relationship between confirmation bias, which causes people to preferentially view information that supports their opinions and beliefs, and web search behavior. In an online user study, we controlled confirmation bias by presenting prior information to participants that manipulated their impressions of health search topics and analyzed their behavioral logs during web search tasks. We found that web search users with poor health literacy and negative prior beliefs about the health search topic did not spend time examining the list of web search results, and these users demonstrated bias in webpage selection. In contrast, web search users with high health literacy and negative prior beliefs about the search topic spent more time examining the list of web search results. In addition, these users attempted to browse webpages that present different opinions. No significant difference in web search behavior was observed between users with positive prior beliefs about the search topic and those with neutral belief.
... Canalizing trust in AI-enabled healthcare has great potential for clinicians and patients alike. In terms of operational efficiency, it can decrease wait time for patients (Kennedy 2018) and decrease health-related information asymmetries (Sillence et al. 2004;Weaver III et al. 2009;Factors and Society 2014;Murray et al. 2003). The latter is a step in the right direction towards increasing patient adherence to prescribed treatment, in combination with a healthy relationship with the prescribing physician (Weaver III et al. 2009;Emanuel and Wachter 2019;Murray et al. 2003). ...
Preprint
Full-text available
In healthcare, the role of AI is continually evolving and understanding the challenges its introduction poses on relationships between healthcare providers and patients will require a regulatory and behavioural approach that can provide a guiding base for all users involved. In this paper, we present ACIPS (Acceptability, Comfortability, Informed Consent, Privacy, and Security), a framework for evaluating patient response to the introduction of AI-enabled digital technologies in healthcare settings. We justify the need for ACIPS with a general introduction of the challenges with and perceived relevance of AI in human-welfare centered fields, with an emphasis on the provision of healthcare. The framework is composed of five principles that measure the perceptions of acceptability, comfortability, informed consent, privacy, and security patients hold when learning how AI is used in their healthcare. We propose that the tenets composing this framework can be translated into guidelines outlining the proper use of AI in healthcare while broadening the limited understanding of this topic.
... The objectivity of the online health and nutrition information i.e. how unbiased is the online health information source and as a content factor, convenience of use i.e. how much the online health and nutrition information content appeals to or connects with a consumer's own perceptions and biases [145] are also potentially imperative factors which can influence perceived online health information source and content quality in OHIS. Studies have also shown that the irrelevance and inappropriateness of specific online health and nutrition information can possibly be counter-productive in fostering consumers' trust is OHIU [146]- [148], more particularly information complexity [149] and information bias [150]. The consumers' perceived information source and content quality of online health and nutrition information can possibly be based on multidimensional consumer information evaluation influences covering reliability, accuracy, clarity, credibility and trustworthiness [151]; and also the relevance of "referral links" i.e. other online health information sources that are linked to that particular information source, which is an indication of that information source's importance as an information provider in the online environment on a particular health or nutrition search topic [4]. ...
Article
Full-text available
This study examines the influence of contemporary literature-supported cognitive influences on consumer Online Health Information Seeking (OHIS) and Online Health Information Usage (OHIU) Intentions; taking a case of supplemental nutrition-related information seeking and Kuala Lumpur’s resident young urban professionals (23 to 38 years of age, educated professionals or business owners of Malaysian nationality) as the target survey population. This study found that Internet Self-Efficacy and Perceived Health Risk both exhibit a strong positive influence on consumers’ Intention of OHIS, while Perceived Health Value and Perceived Value of Information Seeking exhibit a moderate positive influence on Intention of OHIS. On the other hand, Health Self-Efficacy and Perceived Value of Privacy were found to exhibit a moderate negative influence on Intention of OHIS. Similarly, Perceived Information Content Quality exhibit a strong positive influence on consumers’ Intention of OHIU, while Perceived Information Source Quality and Perceived Information Value exhibit a moderate positive influence on Intention of OHIU. It was also found that Personal Bias exhibit a moderate negative influence on Intention of OHIU. Also Intention of OHIS was found to exhibit a moderate to strong positive influence on Intention of OHIU. The study utilised a Self-Completion Questionnaire Survey and analyzed the collected survey data using a PLS Algorithm Path Analysis Test on Smart-PLS 3 statistical software..
Article
This current paper describes the results of our study on users' health information seeking in Wikipedia and on their subsequent satisfaction as an indicator of future Wikipedia usage for health information needs. We examine six factors associated with user’ satisfaction. Based on multiple regression analysis of 322 responses, the paper identifies information quality factors associated with continued use of Wikipedia as a health information source. The results show that trustworthiness, reliability, topic coverage, and information presentation significantly affect user’ satisfaction with Wikipedia health information. Among these factors, information presentation and trustworthiness were most influential on user’ satisfaction.
Chapter
During a web search, the search engines provide relevant pages while the users decide the usefulness of them. The usefulness assessment has been found to be related to various factors. In this study, the effects of task domain and task type on webpage usefulness assessment were investigated. The two task domains involved were health and travel and the two task types were fact finding and decision making. In an experimental environment, 24 participants were asked to do internet search on 4 pre-defined tasks. In the interview after each search task, the users were asked to evaluate the usefulness of each web page they had clicked and talked about why the page was evaluated as such. According to the results of the experiment, we found that the task domain, task type, and various other factors had impacts on the usefulness assessment during web search.
Article
Full-text available
This study addressed the nature and functioning of relationships of interpersonal trust among managers and professionals in organizations, the factors influencing trust's development, and the implications of trust for behavior and performance. Theoretical foundations were drawn from the sociological literature on trust and the social-psychological literature on trust in close relationships. An initial test of the proposed theoretical framework was conducted in a field setting with 194 managers and professionals.
Article
In Exp I, 183 undergraduates read a persuasive message from a likable or unlikable communicator who presented 6 or 2 arguments on 1 of 2 topics. High involvement (HI) Ss anticipated discussing the message topic at a future experimental session, whereas low-involvement (LI) Ss anticipated discussing a different topic. For HI Ss, opinion change was significantly greater given 6 arguments but was unaffected by communicator likability. For LI Ss, opinion change was significantly greater given a likable communicator but was unaffected by the argument's manipulation. In Exp II with 80 similar Ss, HI Ss showed slightly greater opinion change when exposed to 5 arguments from an unlikable (vs 1 argument from a likable) communicator, whereas LI Ss exhibited significantly greater persuasion in response to 1 argument from a likable (vs 5 arguments from an unlikable) communicator. Findings support the idea that HI leads message recipients to employ a systematic information processing strategy in which message-based cognitions mediate persuasion, whereas LI leads recipients to use a heuristic processing strategy in which simple decision rules mediate persuasion. Support was also obtained for the hypothesis that content- vs source-mediated opinion change would result in greater persistence. (37 ref) (PsycINFO Database Record (c) 2004 APA, all rights reserved)