Content uploaded by Jan Maarten Schraagen
Author content
All content in this area was uploaded by Jan Maarten Schraagen on Apr 06, 2021
Content may be subject to copyright.
Article
Journal of Information Science
38(6) 566–577
ÓThe Author(s) 2012
Reprints and permission: sagepub.
co.uk/journalsPermissions.nav
DOI: 10.1177/0165551512459921
jis.sagepub.com
Propensity to trust and the influence of
source and medium cues in credibility
evaluation
Teun Lucassen
University of Twente, The Netherlands
Jan Maarten Schraagen
University of Twente, The Netherlands
Abstract
Credibility evaluation has become a daily task in the current world of online information that varies in quality. The way this task is per-
formed has been a topic of research for some time now. In this study, we aim to extend this research by proposing an integrated layer
model of trust. According to this model, trust in information is influenced by trust in its source. Moreover, source trust is influenced by
trust in the medium, which in turn is influenced by a more general propensity to trust. We provide an initial validation of the proposed
model by means of an online quasi-experiment (n= 152) in which participants rated the credibility of Wikipedia articles. Additionally,
the results suggest that the participants were more likely to have too little trust in Wikipedia than too much trust.
Keywords
credibility; information; medium; propensity; source; trust
1. Introduction
Credibility evaluation in online environments has been shown to be a largely heuristic process [1, 2]. Internet users are
not willing to spend a lot of time and effort on verifying the credibility of online information, which means that various
rules-of-thumb are applied to speed up the process. One important strategy is to consider the source of the information
[3]. In the pre-Internet era, this was a solid predictor of credibility, but nowadays it is hard to point out one single author
as being responsible for the credibility of information. Sources are often ‘layered’ [4, 5], multiple authors collaborate on
one piece of information, and with the advent of Web 2.0, it is often unclear who actually wrote the information.
The diminished predictive power of the credibility of a source could mean that people no longer use it. However,
research on online credibility evaluation has shown otherwise. Consider, for instance, the case of Wikipedia. It was
shown that numerous Internet users made their decision to trust (or not trust) articles from this source solely based on
the fact that they came from Wikipedia [6]. For trusting users, considering the source means that they are also likely to
trust the occasional poor-quality information from this source (i.e. potential overtrust). In contrast, distrusting users miss
out on a lot of high-quality information (i.e. potential undertrust). Hence, the diminished predictive power of the source
does not mean that it is no longer used.
Trust in multiple, comparable sources may generalize to trust in a medium [7]. An example of such a medium is the
Internet as a generalization of several websites. It has been shown that people often refer to ‘the Internet’ or even ‘the com-
puter’ as the source of information they found online, rather than a specific website [8]. This generalization may be the rea-
son why users have already established a baseline of trust when encountering new sources of the same type (i.e. websites).
In this study, we examine the influence of trust in the source and trust in the medium on credibility evaluation. A
more general propensity to trust is also considered, as this may serve as a disposition for more case-specific trust (i.e.
trust in a medium, source or piece of information). We hypothesize a layer model in which each type of trust influences
Corresponding author:
Teun Lucassen, Department of Cognitive Psychology and Ergonomics, University of Twente, PO Box 215, 7500 AE, Enschede, The Netherlands.
Email: t.lucassen@utwente.nl
the next (see Figure 1). The core of this model is trust in a particular piece of information, which is influenced by trust
in the source of this information [3, 6]. Trust in the source is seen as a specification of trust in a medium (as a collection
of sources). Therefore, trust in the medium may serve as a baseline for trust in a source. Furthermore, we hypothesize
that trust in a medium is influenced by a user’s propensity to trust. Overall, we theorize that trust becomes more speci-
fied with each layer; each preceding layer serves as a baseline for the subsequent layer. The proposed model can help us
better understand how trust in information is formed.
We study these influences through an online quasi-experiment in the context of the Internet (as a medium) and
Wikipedia (as a source), starting with a discussion on each proposed layer of trust individually, after which we present
our research model in which we combine them. We introduce three hypotheses, aimed at validating the research model.
Next, we describe our methodology to test the hypotheses, followed by the results. Finally, the results are discussed, lim-
itations are identified, and conclusions are drawn.
1.1. Trust in online environments
A common definition of trust is ‘the willingness of a party to be vulnerable to the actions of another party based on the
expectation that the other will perform a particular action important to the trustor, irrespective of the ability to monitor or
control that other party’ [9]. This definition implies that a certain risk is taken when someone trusts someone else [10];
this trust may prove to be unjustified. In the online domain, trust is an especially relevant concept, as Internet users often
interact with parties they do not have prior experiences with. Consider, for instance, online financial transactions (e.g.
buying a product through a web shop): the consumer is at risk of losing their money when the vendor fails to meet their
expectations.
Four levels have been proposed at which trust may be studied [10], namely individual (as a personality trait), interper-
sonal (one actor trusting another), relational (mutual trust) and societal (trust in a community). When considering trust in
information, the appropriate level is interpersonal trust, as the reader puts their trust in the author of the information.
In order to reduce the risk associated with trusting someone or something, a credibility evaluation may be performed.
In such an evaluation, the ‘trustor’ searches for cues on the credibility of the ‘trustee’. Such evaluations are largely heur-
istic processes, as the user often lacks the motivation and/or ability for a systematic (thorough) evaluation [11].
According to the MAIN model [8], today’s information technology has resulted in numerous affordances in which credi-
bility cues can be found. Such cues may trigger cognitive heuristics; simple judgment rules to estimate the various
dimensions of the quality of information. These dimensions also play an important role in the judgment of credibility.
1.2. Trust in information
Another model that clarifies the use of various cues of credibility is the 3S-model of information trust [6]. This model
asserts that the most direct strategy for evaluating credibility is to search for semantic cues in the information. By doing
so, Internet users try to answer the question: ‘Is this information correct?’ Cues such as factual accuracy, neutrality or com-
pleteness of the information are considered by users who follow this strategy. This implies that some domain knowledge
of the topic at hand is required. However, a typical information search concerns information that is new to the user, as it
normally does not make any sense to search for information one already has. This means that users may often lack the
required domain expertise to evaluate the semantics of the information, which makes it impossible to apply this strategy.
To work around this deficit, users may also consider surface cues of the information to evaluate credibility. This strat-
egy concerns the manner of presentation of the information. Examples of cues evaluated when following this strategy
are the writing style, text length or number of references in the information. While it is a less direct way to evaluate
credibility, no domain knowledge is needed. Instead, by considering surface cues, users bring to bear their information
skills. Such skills involve knowledge of how certain cues (e.g. a lengthy text, numerous images) relate to the concept of
credibility.
Following dual-processing theory [12], it is tempting to see the strategy of evaluating semantic cues as a systematic
evaluation and the strategy of evaluating surface cues as heuristic. However, both strategies can be performed at various
levels of processing. For instance, recognizing something one already knows (semantic) is considered largely heuristic
behaviour [13]. On the other hand, checking the validity of each of the references of an article on Wikipedia (surface)
can be seen as largely systematic. The choice between systematic or heuristic processing in credibility evaluation pri-
marily depends on the motivation and ability of the user [11].
Thus, the cues used in credibility evaluation depend heavily on user characteristics. This has also been proposed in
the unifying framework of credibility assessment [14], which suggests that there are three levels of credibility evaluation
between an information seeker and an information object (namely the construct, heuristics and interaction layers). The
Lucassen and Schraagen 567
Journal of Information Science, 38 (6) 2012, pp. 566–577 ÓThe Author(s), DOI: 10.1177/0165551512459921
first layer is the construct layer, in which it is posited that each user has their own definition of credibility, which means
that different elements of the information object are salient to different users when evaluating credibility (see also Fogg
[15]).
1.3. Trust in the source
A third, more passive way of evaluating credibility is also posited in the 3S-model [6], namely the strategy of consider-
ing the source of information. Following this strategy, earlier interactions with a particular source may serve as a cue for
the credibility of the current information. For instance, if someone has numerous positive experiences with information
from a particular website, this user may choose to trust new information from that source without actively evaluating its
credibility. The opposite is also possible: when one has negative experiences with a source, one may choose to avoid
new information from this source without even looking at it (at the semantic or surface level).
The approach of transferring the credibility of the source of information to the credibility of the information itself only
works well when the credibility of information from a source is stable over time. However, in online environments, infor-
mation from one source may vary greatly in credibility. Consider again Wikipedia: information quality is generally very
high [16], but numerous examples of incorrect information from Wikipedia are readily available [17–19]. This means that
trusting this source involves taking the risk of encountering false information. On the other hand, distrusting this source
means that the user may miss out on much high-quality, valuable information. Nevertheless, it has been shown that, also
in the case of Wikipedia, the source-strategy is applied very often. It was found that around 25% of experts on the topic
and 33% of novices trusted or distrusted information solely because it came from Wikipedia [6]. This is an indication that
users weighed the benefits of Wikipedia (much information) against its risks (poor information). For some users, the ben-
efits clearly outweighed the risks. For others, they did not.
A second drawback of considering the source of information is that nowadays it is often difficult to determine a sin-
gle author who is responsible for the information. Online news, for instance, is often carried through multiple sources
(e.g. a blogger writing a piece on something she read on Facebook, which was a reaction on an article on the CNN news
page). This concept is known as ‘source layering’ [4], and makes it increasingly difficult to determine which source is
responsible for the credibility of the information. However, a recent study on this phenomenon [5] has shown that only
highly involved users considered more distal sources when evaluating credibility. Users with low involvement were only
influenced by the credibility of the most proximate source (i.e. the website on which they read the news). For this rea-
son, we consider the most proximate source (i.e. Wikipedia) as ‘the source’ of information in this study.
Two key factors for the credibility of a source have been identified [20]. First, sources should have the appropriate
knowledge (expertise) to provide correct information. For instance, a doctor is able provide credible health information,
whereas a patient may not be. Second, sources should be trustworthy, that is, have the intention to supply correct infor-
mation. To clarify this concept, consider the difference between a manufacturer of a product and an independent party
testing this product. Both may provide similar information about the product, but have very different intentions. The
manufacturer wants to sell the product, whereas the tester wants to provide consumer advice. This may have large conse-
quences for the credibility of the information supplied.
1.4. Trust in the medium
Traditional linear communication models generally encompass a source (sender) of information, who transmits a mes-
sage through a medium to a receiver. However, it has been shown that a medium may also be treated as a more general
type of source by information seekers [4]. People tend to say that they got information ‘off the Internet’, or even ‘off the
computer’ rather than naming one specific website [8]. As such, credibility may also be attributed to a medium rather
than a single source.
Examples of different media channels are the Internet (or a subset, such as Internet vendors [21]), television, newspa-
pers or school books. It has been shown that trust in the Internet is primarily influenced by experience [7]. It is hardly
possible to assign a value to the credibility of online information without having used the Web. Such experience with
the Web means that users have interacted with various online sources (websites). The experiences in these interactions
are accumulated into trust in the Internet as a whole.
Trust in a medium can be brought to bear when encountering a new source on this medium (i.e. an unfamiliar web-
site). Users may evaluate the credibility of this website, as well as the information on it, but trust in the Internet in gen-
eral may serve as a baseline.
In the context of research on the Internet, this medium is often compared with traditional sources such as books or
newspapers [22]. Differences are found at various levels, such as organization, usability, presentation and vividness. In
Lucassen and Schraagen 568
Journal of Information Science, 38 (6) 2012, pp. 566–577 ÓThe Author(s), DOI: 10.1177/0165551512459921
various instances, the Internet has been shown to be more credible (e.g. political information [23]) and less credible (e.g.
health information [24]) than traditional media.
1.5. Propensity to trust
As stated earlier, when studying trust in information, the appropriate level is interpersonal trust [10]. However, this does
not mean that trust on the other levels has no influence. Consider, for instance, trust at the individual level, or ‘propensity
to trust’. Propensity to trust is a personality trait, a stable factor within a person, that affects someone’s likelihood to trust
[6].
One’s propensity to trust, or dispositional trust, serves as a starting point, upon which more case-specific trust builds
[25]. In an experiment with an X-ray screening task with automation available, trust in the automation moved from dis-
positional to history-based [25]. In other words, trust became more calibrated to the automation. Owing to the heuristic
character of online credibility evaluation, we expect that the propensity to trust will also be visible in more case-specific
trust, such as trust in the medium, source or actual information.
The relationship between propensity to trust and trust in online information has been studied before. Propensity to
trust has been shown to be among the most influential factors predicting consumers’ trust in Internet shopping [21, 26].
In these studies, propensity to trust is seen as a mediating factor between trustworthiness of the vendors and the external
environment on the one hand, and trust on the other. Some researchers distinguish a propensity to trust from a propensity
to distrust [27]. Again, in the context of e-commerce, it was shown that the former has an influence on trust in low-risk
situations, whereas the latter influences trust in high-risk situations.
It is not surprising that much research on trust in online environments has focused on e-commerce. In this domain,
users take a direct, measurable risk (of losing money), which makes trust a very important construct. This risk may be
less salient (or at least measurable) in other domains, such as online information search, as it heavily depends on the pur-
pose of the information. However, wrong decisions may be taken based on this information, which makes online infor-
mation search an important area of study.
1.6. Proposed research model
In the literature discussed here, the concepts of trust in information, sources and media, and a general propensity to trust
are mostly studied in isolation from each other. Some exceptions can be found (e.g. [6, 28]), but an integrated approach
featuring all these concepts in one study is yet to be seen. In this study, we present a novel model of trust in information,
explaining how these concepts are related to each other.
As presented in Figure 1, we hypothesize that not all types of trust discussed here have a direct influence on trust in
each particular piece of information. Instead, we suggest a layer model, in which trust is built from a general propensity
Figure 1. Proposed ‘layer’ model of trust. Each layer is a further specification of trust, and influences the next layer.
Lucassen and Schraagen 569
Journal of Information Science, 38 (6) 2012, pp. 566–577 ÓThe Author(s), DOI: 10.1177/0165551512459921
to trust to case-specific trust in a particular piece of information. In this model, we consider general propensity to trust
as the general baseline of trust of a person [25] in all situations, hence not only for trust in (online) information, but also
trust in, for example, others, society or technology. With each layer, trust becomes more specific for a single situation
(i.e. evaluating the credibility of a single piece of information).
The second layer is labelled ‘trust in the medium’ and concerns trust of a user in a particular type of medium (e.g.
newspapers, the Internet). While this is clearly a more case-specific form of trust than a general propensity (at least one
feature of the information at hand is considered), it is still a generalization of trust in the source of the information [4].
Trust in a medium can also be seen as trust in a collection of sources.
Trust in the medium is followed by the layer ‘trust in the source’. Again, trust is further specified, as the specific
source of the information is considered rather than the medium through which the information is communicated.
Considering the source of information is perhaps the most traditional form of credibility evaluation [6].
The most specific form of trust is trust in the information itself. Especially when the credibility of the source is doubt-
ful, users may search for cues in the information itself to estimate credibility [28]. In sources where the credibility varies
between different pieces of information (e.g. Wikipedia), trust is best calibrated with the actual information quality when
cues from the information itself are considered, rather than cues from the source or medium [6].
In this study, we seek initial validation for this model of trust, by evaluating the influence of each of these layers on
trust in Wikipedia articles. Thereby, we assume that all participants actively evaluate the credibility of the information
to a certain extent (as this is the task imposed on them). However, the layer at which credibility is evaluated in practice
may largely vary between users and contexts. Motivation and ability to evaluate have been identified as important fac-
tors for the extent to which credibility is evaluated [11]. Only relying on one’s propensity to trust does not require any
effort when encountering a piece of information. Each next layer requires more effort from the user to evaluate credibil-
ity. Hence, in situations with a low risk of poor information, or with users with a low motivation or ability, the outer
layers may have a larger influence on trust in the information than when the risk, motivation or ability are higher.
We thus hypothesize a direct influence of each layer on the next. Moreover, we expect that the influence of each layer
can also be observed in more distant layers (e.g. the influence of trust in a medium on trust in information). However, we
hypothesize that this influence is mediated by the intermediate layer, which can better explain how the two non-adjacent
layers are related (e.g. trust in a source explains the relationship between trust in a medium and trust in information).
Hence, the following hypotheses are tested through mediation analysis in order to examine the validity of the proposed
research model. The following three hypotheses can be derived from the hypothesized model:
Hypothesis 1. Propensity to trust has a direct positive influence on trust in the medium. Its influence on trust in the
source is mediated by trust in the medium; its influence on trust in information is mediated by both trust in the
medium and trust in the source.
Hypothesis 2. Trust in the medium has a direct positive influence on trust in the source. Its influence on trust in
information is mediated by trust in the source.
Hypothesis 3. Trust in the source has a positive influence on trust in a particular piece of information from this
source.
2. Method
2.1. Participants
Invitations for participation in an online experiment were posted on several online forums and social media, and via direct
email contact. This resulted in a total of 152 participants who completed the whole experiment. Three participants were
excluded for bogus participation; they gave the same answer to every question. Of the remaining 149 participants, the major-
ity (81.9%, n= 122) was male. The average age was 25.7 years (standard deviation, SD = 10.1). The participants came from
Europe (67.8%), North America (25.5%), Australia (2.7%), South America (2.0%) and Asia (2.0%). It was ensured that each
participant could only partake once by registering their IP-addresses and placing a cookie on their computer.
2.2. Task and procedure
The experiment was conducted using an online questionnaire. When following the link to the questionnaire in the invita-
tions, an explanation of the task was presented first. Participants were informed that they had to evaluate the credibility
of two Wikipedia articles, without specifying how to perform this task (Wikipedia Screening Task, [29]). The
Lucassen and Schraagen 570
Journal of Information Science, 38 (6) 2012, pp. 566–577 ÓThe Author(s), DOI: 10.1177/0165551512459921
explanation also stated that, after the evaluation, a few questions about trust and Internet use would be asked. Moreover,
the participants were warned that they could not use the back and forward buttons of their browser.
After reading the instructions, the participants could decide to participate by clicking on the ‘next’ button. After doing
this, they were asked to enter their gender, age, and nationality on the subsequent page.
The actual experiment started after the participants clicked on the ‘next’ button again. A full-page screenshot of a
Wikipedia article was shown in the questionnaire. Underneath the article, the participants had to answer three questions.
First, they had to rate how much trust they had in the article on a seven-point Likert scale. Second, they could provide an
explanation for their answer through an open-ended question. This explanation was mainly used as an indicator for bogus
participation, but the explanations were subsequently also categorized according to the 3S-model. Third, the participants
had to rate how much they already knew about the topic at hand, again on a seven-point Likert scale. After answering
these questions and clicking ‘next’, the procedure was repeated for the second article.
After evaluating both articles, three separate webpages asked for (1) propensity to trust in general, (2) trust in
Wikipedia and the Internet, and (3) general remarks on the experiment.
2.3. Stimuli
Each participant viewed one article of high quality and one article of low quality. The ratings of the Wikipedia Editorial
Team [30] were used to make this distinction. For the high-quality articles, the highest quality class (Featured Articles)
was used whereas for the low-quality articles, the second lowest quality class (Start-class Articles) was used. The lowest
quality class (Stub articles) was deliberately avoided, as these are often single-sentence articles.
Next to quality, length of the articles was also taken into account. Some featured articles tend to be extremely lengthy.
This could cause problems in the experiment, as it could take too much time for the participants to evaluate such articles.
Therefore, only featured articles with fewer than 2000 words were selected. Moreover, we enssured that the articles in
the poor-quality condition were sufficiently long to perform a meaningful credibility evaluation (i.e. they contained
enough cues to evaluate). Therefore, only start-class articles with more than 300 words were selected.
Following these considerations, three topics with a typically encyclopedic character were used, namely:
•food (‘Andouilette’ and ‘Thomcord’);
•historical persons (‘Princess Amelia of Great Britain’ and ‘Wihtred of Kent’);
•animals (‘Bobbit worm’ and ‘Australian Tree Frog’).
The first of each pair served as a low-quality article, and the latter as a high-quality article. Each participant was ran-
domly assigned to one of the topics and evaluated two articles on this topic. The order of articles was counterbalanced
between subjects.
2.4. Measures
2.4.1. Propensity to trust. Propensity to trust was measured using a subsection of the NEO-PI-R personality test [31]
regarding trust. This consisted of eight questions, to be answered on five-point Likert scales (see Appendix). Although
the NEO-PI-R is not intended for partial usage, we decided not to use the full questionnaire, as this would extend the
duration of the experiment substantially, which is not desirable in online experiments. Moreover, the other personality
characteristics of the full test did not bear relevance to the scope of this study. A reliability analysis (see Results) ensured
the reliability of the remaining questions.
2.4.2. Trust in the Internet. Trust in the Internet was measured on seven-point Likert scales using six questions about (1)
usage, (2) perceived credibility, (3) trust in the institutes behind the Internet, (4) confidence in other Internet users, (5)
usefulness and (6) privacy protection. Question 2–4, and 6 are based on the Net-confidence and Net-risks scales [7],
extended with questions about usage (1) and usefulness (5), which have been shown to be other salient indicators of trust
[10, 32]. See Appendix for the full questionnaire.
2.4.3. Trust in Wikipedia. Trust in Wikipedia was measured on seven-point Likert scales using basically the same six ques-
tions as used for trust in the Internet, replacing ‘the Internet’ with ‘Wikipedia’. However, some issues could not be easily
converted, such as the issue of privacy. Therefore, the nearest related concept applicable to Wikipedia was used (in this
case: accuracy). See Appendix for the full questionnaire.
Lucassen and Schraagen 571
Journal of Information Science, 38 (6) 2012, pp. 566–577 ÓThe Author(s), DOI: 10.1177/0165551512459921
2.4.4. Trust in information. Trust in the information was measured on a seven-point Likert scale after each article, asking
the question ‘How much trust do you have in this article?’ As each participant viewed one article of high quality and one
article of low quality, the average rating was taken for the construct ‘trust in information’ in the analyses.
2.5. Data analyses
For each of the constructs measured through questionnaires, its reliability was calculated using Cronbach’s α. We took
an αof at least 0.70 as an acceptable value for all three constructs (propensity to trust, trust in the Internet and trust in
Wikipedia).
In order to find validation for our research model, bootstrapping mediation analysis was performed [33, 34] to esti-
mate direct and indirect effects with multiple mediators using the PROCESS toolkit for SPSS [35]. The advantages of
this technique are that all mediators can be tested simultaneously, normal distribution does not need to be assumed, and
the number of inferential tests is minimized (reducing the risk of a type 1 error). Following the proposed research model
(see Figure 1), a model with trust in the medium and source as sequential mediators was tested.
The motivations for trust in the articles that could be provided by the participants were classified in accordance with
the 3S-model [6]. This means that each comment was categorized as referring to a semantic, surface or source feature.
Comments that could not be categorized as referring to any of these features were classified as ‘other’. Half of the com-
ments were categorized by two raters. Based on this overlap, Cohen’s kwas calculated to ensure inter-rater reliability. A
kof 0.91 indicated a near-perfect agreement.
3. Results
3.1. Validity of the questionnaires
Cronbach’s αfor the participants’ propensity to trust derived from the NEO-PI-R questionnaire [31] was 0.82, indicating
good reliability. For the trust in the Internet scale, Cronbach’s αwas 0.70, indicating acceptable reliability, and for the
trust in Wikipedia scale, Cronbach’s αwas 0.88, again indicating good reliability.
3.2. Trust
Propensity to trust, as measured through the eight questions on this construct on the NEO-PI-R [31], is divided into five
categories, displayed in Table 1.
Trust in the Internet ranged from 1.17 to 5.33 (on a Likert-scale from 0 to 6), with an average of 3.63 (SD = 0.79).
Trust in Wikipedia ranged from 0.00 to 5.83 (on a Likert-scale from 0 to 6), with an average of 3.51 (SD = 1.11).
Trust in high-quality information (mean, M= 4.91, SD = 1.64) was higher than trust in low-quality information
(M= 4.42, SD = 1.63), t(148) = 2.82, p≤0.01. Average trust in the information was 4.67 (SD = 1.24).
3.3. Validity of the research model
Figure 2 shows a cross section of the layer model presented in Figure 1, with unstandardized regression coefficients
between all constructs. Trust in the information was entered as the dependent variable, propensity to trust as the predic-
tor variable and trust in the medium and trust in the source as (sequential) mediators.
Basic regression analysis showed that the effect of propensity to trust on trust in the information was 0.20 (p≤0.05).
However, when (either or both of) the two mediating variables were entered into the model, this direct effect became
insignificant (0.05, p= 0.56).
Table 1. Participants in each of the five categories of the NEO-PI-R trust scale
Category Percentage
Very high trust 10.7% (n= 16)
High trust 17.4% (n= 26)
Average trust 45.6% (n= 68)
Low trust 16.8% (n= 25)
Very low trust 9.4% (n= 14)
Lucassen and Schraagen 572
Journal of Information Science, 38 (6) 2012, pp. 566–577 ÓThe Author(s), DOI: 10.1177/0165551512459921
The total indirect effect was estimated at 0.08, with a 95% bias-corrected bootstrap (1000 samples) confidence inter-
val of 0.03–0.16. Hence, trust in the medium and trust in the source mediated the effect of propensity to trust on trust in
the information. Moreover, a model with only trust in the medium or trust in the source as mediating variable proved to
be less valid, with a total indirect effect of respectively 0.05 (95% CI −0.02–0.16) and 0.01 (95% CI −0.05–0.08).
More light can be shed on the relationship between trust in Wikipedia and trust in information when considering the
difference between high-quality and low-quality information. A median split on trust in Wikipedia was performed to dis-
tinguish participants with high and low trust in this source. Based on this split, we performed a repeated-measures
ANOVA with article quality as a within-subject variable and trust in Wikipedia as a between-subject variable. A main
effect of article quality and trust in Wikipedia on trust in the information was found, respectively F(1, 147) = 7.45,
p≤0.01 and F(1, 147) = 31.71, p≤0.001. An interaction effect between article quality and trust in Wikipedia on trust
in the information was only significant at the 10% significance level, F(1, 147) = 3.02, p= 0.08. Visual inspection of
the data suggested that users with low trust in Wikipedia were less influenced by article quality than users with high trust
in Wikipedia. As expected, the same analysis applying a median split on propensity to trust and trust in the medium did
not yield a significant interaction effect, F(1, 147) = 0.37, p= 0.55.
3.4. Motivations for trust
A total of 131 of the 149 participants entered a motivation for their trust in the article on at least one occasion. This
resulted in a total of 252 comments, which were categorized in accordance with the 3S-model [6]. Table 2 gives an over-
view of these comments.
As can be expected from a user group with limited domain knowledge on the topic at hand, most comments could be
classified as referring to surface features (e.g. ‘This article is well-cited’). However, semantic features (e.g. ‘It appears to
be historically correct, as far as my knowledge of the subject goes’) and source features (e.g. ‘Wikipedia has yet to fail
me’) were also mentioned as a motivation to trust the article. A remainder of 14.7% of the comments could not be classi-
fied in the 3S-model (e.g. ‘I have no reason not to trust this particular article’).
4. Discussion
In this paper, we propose a novel layer model of trust, with an inner core of trust in a particular piece of information,
surrounded by trust in the source of the information, trust in the medium and propensity to trust in general. A mediation
analysis on the results of the online experiment provided initial validation for this model. Moreover, a marginally signif-
icant interaction between trust in Wikipedia and trust in high-quality and low-quality information was found. The main
contribution of this study is that the concepts of trust in a source, trust in a medium and propensity to trust in credibility
Figure 2. Cross section of the proposed layer model, showing unstandardized regression coefficients between all proposed
constructs. Coefficients marked with three asterisks are significant at the 0.001 level; other coefficients were not significant.
Table 2. Motivations given for trust in the articles, coded in accordance with the 3S-model [6]
Motivation Percentage
Semantic features 11.5% (n= 29)
Surface features 59.9% (n= 151)
Source features 13.9% (n= 35)
Other 14.7% (n= 37)
Lucassen and Schraagen 573
Journal of Information Science, 38 (6) 2012, pp. 566–577 ÓThe Author(s), DOI: 10.1177/0165551512459921
evaluation are investigated in one, integrated study. This means that the presented model can be useful in explaining
how these concepts are related to trust in information, and to each other.
Of course, the predictive power of each layer on the next is limited as numerous other aspects are likely to influence
trust at the various levels as well (e.g. familiarity, information skills [6]). However, significant coefficients were found
for each pair of layers. This means that we can draw the following conclusions:
•Trust in information is influenced by trust in its source.
•Trust in a source is influenced by trust in a medium.
•Trust in a medium is influenced by a propensity to trust.
Next to the correlation between trust in the source and trust in the information, a marginal interaction between trust
in the source and trust in high-quality and low-quality information was found. In particular, this interaction can explain
quite clearly how these two constructs are related. While it was only significant at the 10% level, it suggests an impor-
tant difference between Internet users with a sceptical or trusting attitude towards Wikipedia. Earlier, we suggested that
users with low trust in a source may skip it altogether, regardless of the information itself. We confirmed this behaviour
in the experiment: participants with low trust in Wikipedia did not perceive any difference between high-quality and
low-quality articles. This indicates a negative ‘Halo effect’ of the source on the information [36]; it is perceived differ-
ently (worse) because of characteristics of its source. Users demonstrating this behaviour are prone to undertrust, as they
are likely not to use this source at all, even when the information quality is high.
On the other hand, participants with high trust in Wikipedia did perceive a difference between high-quality and low-
quality information. This means that, even though they had a positive attitude towards the source, they still considered
the quality of the information itself. Hence, no evidence for potential overtrust based on trust in the source was found in
this experiment.
These findings are not in line with an earlier study [28], in which it was shown that low source credibility led to the
accumulation of cues from the information itself. Of course, the context of that study (various online news sources) was
quite different from this one (Wikipedia). Thus, the effect found in our study may be specific for the case of Wikipedia.
An alternative explanation is the extent to which the source was found not to be credible. It is possible that, when the
source credibility is perceived to be limited, more cues in the information are sought, but when perceived source credi-
bility is below a critical value, it is discarded entirely.
Moreover, it should be noted that, although a statistical difference was found between trust in high-quality and low-
quality articles, this difference was quite small. Several explanations can be given for this finding. First, the motivation
of the participants was likely to be limited, leading to a quick, heuristic evaluation of credibility at most [11]. Also, the
categorization of the Wikipedia Editorial Team was taken as a measure for quality. However, articles of lesser quality
are not necessarily less credible, as the editorial team predominantly judges how far each article is from a distribution-
quality article. In other words, completeness is a dominant factor for the editorial team, but this does not necessarily play
a central role in the credibility evaluations of our participants.
Surprisingly, the link between trust in the source and trust in the information itself proved to be rather weak in this
study. This finding seems to contradict much of the literature on the topic of source credibility, which mostly suggests
that this is in fact a very strong relationship [3, 10, 12, 28]. Two explanations can be given for the lack of a strong corre-
lation between trust in the source and trust in the information in this study.
First, the participants in this study were asked to evaluate multiple articles, which had one common characteristic,
namely its source. This means that the participants were able to compare the articles with each other. In such compari-
sons, it is of no use to consider the source of the information, as this is a constant. The notion that the participants in this
experiment indeed only made limited use of source cues is also supported by the percentage of comments (Table 2)
regarding source features, which was rather low in this study (~14%). An earlier study [6] featuring only one stimulus
article yielded a much larger percentage of comments on the source (~30%). In contrast, in a think-aloud study with 10
stimulus articles [29], no utterances on the source of information were recorded at all. Hence, the ability to compare var-
ious articles could have diminished the influence of source cues.
Second, the particular source used in this study may have led to a limited influence on trust in the information. As
already noted in the Introduction, it is problematic to consider Wikipedia as one single source in the traditional sense [6].
The information quality heavily varies between articles and over time, which makes the source credibility of Wikipedia a
poor predictor for information credibility. Critical participants may have been aware of this, and thus attributed less value
to source credibility. Some evidence of particularly critical participants can be found in the open-ended motivations, for
instance in comments such as ‘I don’t really trust Wikipedia because someone from the public can make changes to the
topic or article’ and ‘Wiki is an open source; anyone can comment on it and a slight change in the wording can cause
Lucassen and Schraagen 574
Journal of Information Science, 38 (6) 2012, pp. 566–577 ÓThe Author(s), DOI: 10.1177/0165551512459921
misguidance.’ This reasoning, as illustrated by these examples, may thus mean that at least some participants were aware
of the limited transfer of source credibility to information credibility in this particular context, which also may have led
to a limited use of source cues.
No influence of trust in the medium or propensity to trust on trust in high-quality and low-quality information was
found. This is in line with the hypothesized research model, as these constructs are more distant from trust in
information.
A strong tie was found between trust in the Internet and trust in Wikipedia. This can partly be explained by the fact
that, as opposed to the other layers, very similar questions were used to measure the two constructs (mostly only repla-
cing ‘the Internet’ by ‘Wikipedia’). However, we reckon that the relationship between these two is in fact among the
most powerful, as Wikipedia is one of the most visited sites on the Internet [37]. As stated [7], trust in the Internet is
largely built on experience with this medium. The prominent place Wikipedia has online means that its influence on
trust in this medium is large. An interesting follow-up question is whether this relationship is equally strong in other con-
texts, such as television, newspapers or other printed materials.
Propensity to trust had a large influence on trust in the medium. This supports the notion that trust is specified from
dispositional trust to more case-specific trust when needed [25]. However, since credibility evaluation in this context is
largely heuristic, the disposition still has an influence on trust in information, albeit limited. It is expected that, in situa-
tions where the perceived need for credible information is higher (e.g. health information, financial transactions), propen-
sity to trust is less influential, as trust is better calibrated to the actual credibility of the information as a result of a more
profound evaluation of credibility [11].
4.1. Limitations
Only one medium (the Internet) and one source (Wikipedia) were taken as a case study to demonstrate the validity of the
research model in this experiment. Future research in this direction could utilize the same model, but with different media
and/or sources to verify its validity in other contexts.
The interaction effect found between trust in the source and quality of the information on trust in the information was
not significant at the customary 5% level. However, the trend found in this experiment suggests a larger risk of undertrust
in Wikipedia than overtrust. More research on the effect of the source of information on trust should confirm whether this
is actually the case.
In this experiment, trust on various levels was measured through (partially validated) questionnaires. In future
research, attempts should be made to manipulate trust more systematically, in order to rule out the effect of potentially
confounding variables (e.g. age, Internet experience).
We cannot rule out the possibility that the order of the experiment (specifically the general questions on trust after
the administration of the stimuli) had an influence on the answering of the questions. However, we are convinced that a
reverse order in which the general questions would have been presented before the stimuli would have influenced the
answering of the questions regarding the stimuli to an even larger extent, as this would have primed the participants on
the issues of source, medium and propensity to trust. Future research could completely preclude this potential issue by
counter-balancing the order of questions.
Finally, the results found here may not generalize to the entire population of Wikipedia (or Internet) users, as the sam-
ple is demographically biased towards European and North-American males. Gender differences in trust in Wikipedia
[38] and in general [39] have been shown before. Moreover, six different articles were used as stimuli in this experiment.
Although we attempted to rule out specific effects of this selection (e.g. very short or long articles), we cannot be sure
that exactly the same effects are found when other articles are selected.
5. Conclusions
In this study, we proposed a novel model of trust. In this model, trust in information is influenced by trust in its source,
which is in turn influenced by trust in the medium of this source. Moreover, trust in the medium is influenced by the
user’s propensity to trust. An online quasi-experiment has provided a first validation in the context of Wikipedia (as a
source) and the Internet (as a medium). Moreover, some evidence for potential undertrust in Wikipedia was found, as
participants with low trust in this source disregarded the presented information, without considering its actual quality in
their credibility evaluation. No evidence for potential overtrust was found, as participants with high trust in Wikipedia
were still influenced by the quality of the presented information, rather than having blind faith in this source.
The proposed layer model can serve as a framework for future studies on the role of propensity to trust, trust in a
medium and trust in a source in credibility evaluation, for example in other contexts than the Internet or Wikipedia.
Lucassen and Schraagen 575
Journal of Information Science, 38 (6) 2012, pp. 566–577 ÓThe Author(s), DOI: 10.1177/0165551512459921
Acknowledgements
The authors would like to thank Chris Kramer and Tabea Hensel for their valuable contributions to this study.
References
[1] Taraborelli D. How the Web is changing the way we trust. In: Waelbers K, Briggle A and Breg P (eds) Current issues in com-
puting and philosophy. Amsterdam: IOS Press; 2008, pp. 194–204.
[2] Metzger MJ, Flanagin AJ and Medders RB. Social and heuristic approaches to credibility evaluation online. J Commun 2010;
60: 413–439
[3] Chaiken S and Maheswaran D. Heuristic processing can bias systematic processing: Effects of source credibility, argument
ambiguity, and task importance on attitude judgment. J Personal Social Psychol 1994; 66: 460–473.
[4] Sundar SS and Nass C. Conceptualizing sources in online news. J Commun 2001; 51: 52–72.
[5] Kang H, Bae K, Zhang S and Sundar SS. Source cues in online news: Is the proximate source more powerful than distal
sources? Journalism Mass Commun Q 2011; 88: 719–736.
[6] Lucassen T and Schraagen JM. Factual accuracy and trust in information: The role of expertise. J Am Soc Inform Sci Technol
2011; 62: 1232–1242.
[7] Dutton WH and Shepherd A. Trust in the Internet as an experience technology. Inform Commun Soc 2006; 9: 433–451.
[8] Sundar SS. The MAIN model: A heuristic approach to understanding technology effects on credibility. In: Metzger MJ and
Flanagin AJ (eds) digital media,youth, and credibility. Cambridge, MA: The MIT Press, 2008, pp. 73–100.
[9] Mayer RC, Davis JH and Schoorman FD. An integrative model of organizational trust. Acad Mgmt Rev 1995; 20: 709–734.
[10] Kelton K, Fleischmann KR and Wallace WA. Trust in digital information. J Am Soc Inform Sci Technol 2008; 59: 363–374.
[11] Metzger MJ. Making sense of credibility on the Web: Models for evaluating online information and recommendations for future
research. J Am Soc Inform Sci Technol 2007; 58: 2078–2091.
[12] Chaiken S. Heuristic versus systematic information processing and the use of source versus message cues in persuasion. J
Personal Social Psychol 1980; 39: 752–766.
[13] Klein GA, Calderwood R and Clinton-Cirocco A. Rapid decision making on the fire ground. In: Proceedings of the Human
Factors and Ergonomics Society annual meeting, vol. 30. Santa Monica, CA: Human Factors and Ergonomics Society, 1986,
pp. 576–580.
[14] Hilligoss B and Rieh SY. Developing a unifying framework of credibility assessment: Construct, heuristics, and interaction in
context. Inform Process Mgmt 2008; 44: 1467–1484.
[15] Fogg BJ. Prominence-interpretation theory: explaining how people assess credibility online. In: CHI ’03 extended abstracts on
human factors in computing systems. New York: ACM, 2003, pp. 722–723.
[16] Giles J. Internet encyclopaedias go head to head. Nature 2005; 438: 900–901.
[17] Cross T. Puppy smoothies: Improving the reliability of open, collaborative wikis. First Monday 2006; 11.
[18] Waters NL. Why you can’t cite Wikipedia in my class. Commun ACM 2007; 50: 15–17.
[19] Dooley PL. Wikipedia and the two-faced professoriate. In: Proceedings of the 6th international symposium on wikis and open
collaboration, WikiSym ’10. New York: ACM, 2010, pp. 1–2.
[20] Fogg BJ and Tseng H. The elements of computer credibility. In: Proceedings of the SIGCHI conference on human factors in
computing systems: the CHI is the limit. CHI ’99. New York: ACM, 1999, pp. 80–87.
[21] Lee MKO and Turban E. A trust model for consumer Internet shopping. Int J Electron Commerce 2001; 6: 75–91.
[22] Wathen CN and Burkell J. Believe it or not: Factors influencing credibility on the Web. J Am Soc Inform Sci Technol 2002; 53:
134–144.
[23] Johnson TJ and Kaye BK. Cruising is believing? Comparing Internet and traditional sources on media credibility measures.
Journalism Mass Commun Q 1998; 75: 325–340.
[24] McClung HJ, Murray RD and Heitlinger LA. The Internet as a source for current patient information. Pediatrics 1998; 101: e2.
[25] Merritt SM and Ilgen DR. Not all trust is created equal: Dispositional and history-based trust in human–automation interactions.
Human Factors 2008; 50: 194–210.
[26] Cheung CM and Lee MK. Trust in Internet shopping: Instrument development and validation through classical and modern
approaches. J Global Inform Mgmt 2001; 9: 23–35.
[27] McKnight DH, Kacmar CJ and Choudhury V. Dispositional trust and distrust distinctions in predicting high- and low-risk inter-
net expert advice site perceptions. e-Service J 2004; 3: 35–58.
[28] Sundar SS, Knobloch-Westerwick S and Hastall M R. News cues: Information scent and cognitive heuristics. J Am Soc Inform
Sci Technol 2007; 58: 366–378.
[29] Lucassen T and Schraagen JM. Trust in Wikipedia: How users trust information from an unknown source. In: Proceedings of
the 4th workshop on Information credibility, WICOW ’10. New York: ACM, 2010, pp. 19–26.
[30] Wikipedia. Wikipedia: Version 1.0 editorial team, http://en.wikipedia.org/wiki/Wikipedia:Version_1.0_Editorial_Team
(accessed June 2012).
[31] Costa PT and McCrea RR. Revised NEO personality inventory (NEO PI-R) and NEO five-factor inventory (NEO-FFI). Odessa,
FL: Psychological Assessment Resources, 1992.
Lucassen and Schraagen 576
Journal of Information Science, 38 (6) 2012, pp. 566–577 ÓThe Author(s), DOI: 10.1177/0165551512459921
[32] Davis FD, Bagozzi RP and Warshaw PR. User acceptance of computer technology: a comparison of two theoretical models.
Mgmt Sci 1989; 35: 982–1003.
[33] Preacher KJ and Hayes AF. Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple med-
iator models. Behav Res Meth 2008; 40: 879–891.
[34] Hayes AF. Beyond Baron and Kenny: Statistical mediation analysis in the new millennium. Commun Monogr 2009; 76: 408–
420.
[35] Hayes, AF. PROCESS: A versatile computational tool for observed variable mediation, moderation, and conditional process
modeling [White paper], http://www.afhayes.com/public/process2012.pdf (2012, accessed June 2012).
[36] Nisbett, RE and Wilson TD. The halo effect: Evidence for unconscious alteration of judgments. J Personal Social Psychol
1977; 35: 250–256.
[37] Alexa Internet. Alexa Top 500 Global Sites, http://www.alexa.com/topsites (accessed June 2012).
[38] Lim S and Kwon N. Gender differences in information behavior concerning Wikipedia, an unorthodox information source?
Library Inform Sci Res 2010; 32: 212–220.
[39] Feingold A. Gender differences in personality: a meta-analysis. Psychol Bull 1994; 116: 429–456.
Appendix
Propensity to trust
•Regarding the intentions of others I am rather cynical and sceptical.
•I believe that you will be used by most people if you allow them to.
•I believe that most people inherently have good intentions.
•I believe that most people, with whom I have dealings, are honest and trustworthy.
•I become distrustful when someone does me a favour.
•My first reaction is to trust people.
•I tend to assume the best of others.
•I have a good deal of trust in human nature.
Trust in the Internet
•When you are looking for information, how often would you use the Internet as opposed to offline sources?
•What do you think is the credibility of the Internet?
•How much do you trust the institutes and people ‘running the Internet’?
•How much confidence do you have in the people with whom you interact through the Internet?
•If you are in need of information, how confident are you that you can find it on the Internet?
•How well do you think your privacy is protected on the Internet?
Trust in Wikipedia
•When you are looking for information, how often would you use Wikipedia as opposed to other sources?
•What do you think is the credibility of Wikipedia?
•How much do you trust the institutes and people ‘running Wikipedia’?
•How much confidence do you have in the people who add information to Wikipedia?
•If you are in need of information, how confident are you that you can find it on Wikipedia?
•How large do you think the risk of getting inaccurate information on Wikipedia is?
Lucassen and Schraagen 577
Journal of Information Science, 38 (6) 2012, pp. 566–577 ÓThe Author(s), DOI: 10.1177/0165551512459921
