ArticlePDF Available

(Why) does comment presentation order matter for the effects of user comments? Assessing the role of the availability heuristic and the bandwagon heuristic

Authors:

Abstract

Research has shown that user comments influence peoples’ perceptions, with recent evidence suggesting that comment presentation order (e.g., whether comments are presented prior to/after the commented item) may alter the strength of comments’ effects. Considering the implications of this finding for content producers, the informed design of experiments, and the interpretation of prior studies, this study aimed (1) to replicate a recent study on comment presentation order, and (2) to identify why (negative) comments presented after an article seem to have stronger effects on users’ perceptions. A pre-registered experiment with 325 participants provided inconclusive evidence of the predicted presentation order effect and did not allow for further investigations of the underlying reasons. Overall, the findings highlight the ongoing need for studying the conditions under which effects of user comments occur.
(WHY) DOES COMMENT PRESENTATION ORDER MATTER? 1
(Why) Does Comment Presentation Order Matter for the Effects of User Comments?
Assessing the Role of the Availability Heuristic and the Bandwagon Heuristic
Anna Sophie Kümpel1 & Julian Unkel2
2Institute of Media and Communication, TU Dresden
2Department of Media and Communication, LMU Munich
This pre-registered report has been accepted for publication in Communication Research
Reports (published by Taylor & Francis)
Author Note
Anna Sophie Kümpel https://orcid.org/0000-0001-7184-4057
Julian Unkel https://orcid.org/0000-0001-9568-7041
Correspondence concerning this article should be addressed to Anna Sophie Kümpel,
Institute of Media and Communication, TU Dresden, Zellescher Weg 17, 01069 Dresden,
Germany. E-mail: anna.kuempel@tu-dresden.de
(WHY) DOES COMMENT PRESENTATION ORDER MATTER? 2
Abstract
Research has shown that user comments influence peoples’ perceptions, with recent
evidence suggesting that comment presentation order (e.g., whether comments are presented
prior to/after the commented item) may alter the strength of comments’ effects. Considering the
implications of this finding for content producers, the informed design of experiments, and the
interpretation of prior studies, this study aimed (1) to replicate a recent study on comment
presentation order (Kümpel & Unkel, 2020), and (2) to identify why (negative) comments
presented after an article seem to have stronger effects on users’ perceptions. A pre-registered
experiment with 325 participants provided inconclusive evidence of the predicted presentation
order effect and did not allow for further investigations of the underlying reasons. Overall, the
findings highlight the ongoing need for studying the conditions under which effects of user
comments occur.
Keywords: online news, user comments, social media, journalistic quality, presentation
order, elaboration, bandwagon perceptions
(WHY) DOES COMMENT PRESENTATION ORDER MATTER? 3
(Why) Does Comment Presentation Order Matter for the Effects of User Comments?
Assessing the Role of the Availability Heuristic and the Bandwagon Heuristic
User comments—defined here as public statements about (news) content in online
media—have repeatedly been shown to affect peoples’ judgments and perceptions (for an
overview see Ksiazek & Springer, 2018). Among other outcomes, experimental research has
identified effects of user comments on third-person perceptions (e.g., Chen & Ng, 2016),
perceptions of public opinion (e.g., Zerback & Fawzi, 2017), or journalistic quality (e.g.,
Prochazka et al., 2018). However, considering recent changes in information environments—
provoked by news providers moving their comment sections to social network sites (SNS)—
researchers have started to investigate whether the associated shift in comment presentation
order might influence the effects of user comments (Kümpel & Unkel, 2020). While news
websites have traditionally placed comments below the article, suggesting that the article is read
beforehand, SNS invite users to first turn to the comments—already available in the SNS feed
together with a short teaser of the article—and then read the linked full story afterwards.
Investigating whether this changes the effects of user comments, a recent study found that
presentation order seems to matter; but only for negative user comments (Kümpel & Unkel,
2020). Specifically, it was found that comments criticizing the quality of a journalistic article that
were presented after the article had a stronger effect on users’ perceptions of journalistic quality
than those presented beforehand.
Considering the implications of this finding for news providers, the informed design of
user comment experiments, and the interpretation of prior studies on the effects of user
(WHY) DOES COMMENT PRESENTATION ORDER MATTER? 4
comments1, we aim to (1) replicate this effect of comment presentation order, and (2) identify
why (negative) comments presented after an article seem to have stronger effects on users’
perceptions of journalistic quality. Building on theoretical assumptions about online users’
reliance on different heuristics, we focus on two related, but distinct information processing
strategies that might be responsible for the observed effects: the role of the availability heuristic
(i.e., comments have an effect because they are readily available and easy to be recalled) and the
role of the bandwagon heuristic (i.e., comments have an effect because users believe they reflect
the majority’s opinion and thus jump on the proverbial bandwagon). Finding stronger evidence
for the bandwagon heuristic would suggest that user comments are actually acting as exemplars
(i.e., perceived to represent the opinion of larger groups, see Peter et al., 2014; Zerback & Fawzi,
2017) and that the effects of comments are social in nature. However, finding stronger evidence
for the availability heuristic would suggest that (opinions in) comments are simply easier to
recall and used for judgments because they are ‘fresh’ and accessible.
The Influence of Comment Presentation Order
Why should (negative) user comments matter more when they are read after an article?
User comments in general have been conceptualized as cues that hold the potential to trigger
different cognitive heuristics (e.g., Bellur & Sundar, 2014; Weber et al., 2019). Prior research in
the domain of journalistic quality perceptions has shown that users have a pronounced tendency
to rely on cues in the form of user comments when asked to judge the quality of news articles.
1 Looking at the 13 user comment experiments discussed in Kümpel & Unkel’s (2020) paperChen & Ng,
2016; Dohle, 2018; Kümpel & Springer, 2016; Lee & Jang, 2010; Lee, Kim, & Cho, 2017; Peter et al., 2014;
Prochazka et al., 2018; von Sikorski, 2016; Waddell, 2018; Waddell & Sundar, 2017; Weber et al., 2019; Winter et
al., 2015; Zerback & Fawzi, 2017, nine opted for a simultaneous presentation of main stimulus and comments,
with comments usually being placed below the main stimulus (e.g., a news article). Two studies presented the user
comments first and the main stimulus second, while another two studies presented the main stimulus first and the
user comments second. If comment presentation order indeed has a consistent influence, this has implications for the
interpretation of these studies.
(WHY) DOES COMMENT PRESENTATION ORDER MATTER? 5
This is due to the fact that (1) comments are usually “more salient than the core qualities of the
journalistic content” (Weber et al., 2019, p. 26) and that (2) most readers are unlikely to be
particularly concerned with making quality judgments (ibid., see also Kümpel & Unkel, 2020).
Although these general assumptions should be valid regardless of presentation order, we have to
assume that they are stronger when comments are read after the article. Rather than taking into
account all relevant information, people might rely only on the latest piece of information,
because it is the easiest to bring to mindparticularly when exhibiting low levels of message-
relevant elaboration (Haugtvedt & Wegener, 1994; Petty et al., 2001).
Considering our first goal of conceptually replicating the effect of comment presentation
order observed by Kümpel & Unkel (2020), we start by proposing that negative user comments
that are read after a journalistic article influence usersquality perceptions of the respective
article more strongly than comments that are read prior to the article. We purposefully exclude
positive user comments from this study as previous investigations (ibid., see also Waddell, 2018;
Waddell & Sundar, 2017; Winter et al., 2015) consistently found that “positive comments fail to
have a comparable effect” (Waddell, 2018, p. 3070) on attitudes or perceptions. Thus, we
assume:
H1: Negative user comments presented after a news article lead to lower journalistic
quality perceptions than negative user comments presented before a news article.
The Availability Heuristic
The first mechanism we aim to test relates to the assumption that comments are simply
used for (quality) evaluations, because they happen to be available. The availability heuristic
(Tversky & Kahneman, 1973) builds on the notion that if an information can readily be recalled,
it must be (more) important and should thus be used for judgments. Although the availability
(WHY) DOES COMMENT PRESENTATION ORDER MATTER? 6
heuristic originally refers to judgments about frequency, it is also regularly referenced when it
comes to the effects of user comments on perceptions or evaluations in general (e.g., Neubaum &
Krämer, 2017; Zerback & Fawzi, 2017). Like all heuristics, the availability heuristic should
reduce cognitive load and thus lead people to come to their judgments more quickly. However,
the main aspect we are interested in is not whether the availability or the bandwagon heuristic is
‘true,’ but rather in the relative importance of both mental shortcuts. Should we find more
evidence for the availability heuristic, previously observed effects of user comments might at
least partly be a methodological artifact instead of a genuine social effect that stems from “the
thoughts, feelings, communication, or behavior of one or more other people” (Kim &
Hollingshead, 2015, p. 165). To test the ‘pure’ availability heuristic, we propose:
H2: As compared to negative user comments placed before an article, negative user
comments presented after a news article will lead to a decrease in users’ elaboration, which will
then lead to a decrease in quality perceptions.
The Bandwagon Heuristic
The second mechanism we aim to test relates to the assumption that comments set off the
so-called bandwagon heuristic (Sundar, 2008)—the rule of thumb that if others think about a
piece of information in a certain way, then I should think the same (ibid., see also von Sikorski,
2016; Waddell, 2018). But why should a handful of user comments be perceived as representing
the opinion of a larger group? Exemplification theory (Zillmann & Brosius, 2000) suggests
that—due to a “deep-rooted inclination to generalize observed phenomena(p. 11)user
comments can act as exemplars (i.e., illustrative individual cases) and thus as cues to public
opinion (see also Lee & Jang, 2010; Peter et al., 2014; Zerback & Fawzi, 2017). If comments
indeed act as cues to public opinion, they should trigger the bandwagon heuristic, consequently
(WHY) DOES COMMENT PRESENTATION ORDER MATTER? 7
reduce cognitive load and lead people to come to judgments more quickly. Considering the
underlying assumption of a genuine social influence, peoplesjudgments of the commented news
article (in our case: quality perceptions) should then mirror the negative opinion expressed in the
user comments. More formally, we propose:
H3: As compared to negative user comments placed before an article, negative user
comments presented after a news article will lead to an increase in bandwagon perceptions,
which will then lead to a decrease in users’ elaboration, ultimately leading to a decrease in
quality perceptions.
Method
To test the hypothesized effects (see Figure 1), an experiment with German online users
was conducted between February 17 and March 3, 2021. Participants were randomly distributed
to one of two groups (negative comments presented before the article vs. negative comments
presented after the article). Participants were first asked to read a news article, with a Facebook
teaser of said article containing two negative user comments displayed either before or after the
article. We then assessed participantsjournalistic quality perceptions, their bandwagon
perceptions, and their degree of elaboration while providing the quality perceptions.
All data, analysis scripts, and materials are available in an OSF repository:
http://doi.org/10.17605/OSF.IO/VFRMB. The hypotheses and the data analysis plan were
preregistered following a template for experiments (van ’t Veer & Giner-Sorolla, 2016) on
February 15, 2021: https://doi.org/10.17605/OSF.IO/HKZPV.
Participants
Participants were recruited from a pool of about 70,000 German residents who signed up
for the non-commercial online access panel SoSci Panel (Leiner, 2016). An accuracy in
(WHY) DOES COMMENT PRESENTATION ORDER MATTER? 8
parameter estimation (AIPE; Maxwell et al., 2008) simulation approach was used to determine a
minimum sample size of 300 participants to reliably estimate effects within an expected
credibility interval width of 0.5 (see the preregistration form for details). The final sample
consists of 325 participants (self-identified gender: 56% female, 43% male, 1% non-binary; age:
M = 44.5, SD = 15.9) after applying all exclusion criteria (insufficient stimulus engagement,
failed treatment check, missing data, and predefined outliers; see preregistration form for
details). As an incentive for participation, three 25€ vouchers for an online bookstore were
raffled among all participants who completed the study.
Stimuli
Just like in the to-be-replicated original study (Kümpel & Unkel, 2020), participants were
asked to read and evaluate an article that was allegedly published by the German regional news
website Stuttgarter-Zeitung.de. The article was about 420 words long and focused on the Supply
Chain Act (‘Lieferkettengesetz’), a legislation obliging German businesses to implement due
diligence in their supply chains in regard to ethical and ecological aspects.
In the two experimental groups, a Facebook teaser for the article containing two negative
comments was displayed either before or after the article on a separate questionnaire page. These
commentsidentical in wording to those of the original studycriticize the article’s writing
style, quality of research, and impartiality, thus referring to key aspects of journalistic quality
(see Urban & Schweiger, 2014). The Facebook teaser also features the article’s headline, lead,
and teaser image. Apart from the comments, no further social cues (i.e., reactions) were present.
(WHY) DOES COMMENT PRESENTATION ORDER MATTER? 9
Measures
Journalistic Quality Perceptions
Participants were asked to rate the journalistic quality of the news article on a 7-item
scale ranging from 1 (does not apply at all) to 7 (does fully apply). The items focused on various
aspects of journalistic quality (e.g., impartiality, comprehensiveness, comprehensibility) and
were adopted from Jungnickel (2011). Participants’ overall journalistic quality perceptions were
then calculated with a mean index of all seven items, with higher values indicating a higher
perceived journalistic quality (M = 4.90, SD = 1.05, ω = .82).
Degree of Elaboration
Participants’ degree of elaboration on the journalistic quality perception scale was
measured as their response latency for said scale (Knobloch-Westerwick & Meng, 2009; Weber
et al., 2019). Response latencies on two other 7-item scalesneed for cognition (Müller et al.,
2016; surveyed before the stimulus) and need for consistency (von Collani & Blank, 2013;
surveyed after the stimulus; negative items inversed)—served as the baseline. Response latencies
(in ms) were measured for each of the three scales’ remaining six items after participants had
responded to one item of the respective scale. All latencies were then subjected to a negative
reciprocal transformation (Fazio, 1990). The calculated mean baseline response latency was
subtracted from each journalistic quality perception item latency, and the results were then
averaged. Larger values thus indicate higher response latency and, consequently, a higher degree
of elaboration (M = -0.07, SD = 0.33).
Bandwagon Perceptions
Building on previous research (Waddell & Bailey, 2017; Waddell & Sundar, 2017; Xu,
2013), we developed a 5-item scale ranging from 1 (not at all likely) to 7 (very likely) to measure
(WHY) DOES COMMENT PRESENTATION ORDER MATTER? 10
bandwagon perceptions. Participants were asked how likely it is that other people “will not like
the article,” “will have a negative opinion about the article,” “mistrust the article,” “consider the
article to be of inferior quality,” and “would advise their friends to not read the article.”
Participants’ overall bandwagon perceptions were then calculated with a mean index, with higher
values indicating a higher bandwagon perception (M = 4.10, SD = 1.35, ω = .91).
Results
All hypotheses were tested with a Bayesian serial mediation model as specified in Figure
1, with comment presentation order as the independent variable, journalistic quality perceptions
as the dependent variable, bandwagon perceptions (M1) and degree of elaboration (M2) as serial
mediators, and uninformative priors using the R package brms (Bürkner, 2017). Journalistic
quality perceptions, bandwagon perceptions, and degree of elaboration were standardized for the
model to allow for the interpretation of standardized coefficients. All coefficient estimates are
displayed in Table 1.
Both the direct effect of comment presentation order on journalistic quality perceptions
(c': Mdn = -0.15; 89% CI [-0.31; 0.01]; pd = .93) and the total effect (c: Mdn = -0.14; 89%
CI [-0.32; 0.03]; pd = .91) provided uncertain evidence for the existence of the presentation order
effect. Likewise, both hypothesized indirect effects of comment presentation order on
journalistic quality perceptions through degree of elaboration (a2b2: Mdn = 0.00; 89% CI [-0.01;
0.02]; pd = .58) and through bandwagon perceptions and degree of elaboration (a1d21b2:
Mdn = 0.00; 89% CI [0.00; 0.00]; pd = .50) were not supported by the model. Thus, H1, H2, and
H3 had to be rejected. Notably, we observed a very likely existing effect of bandwagon
perceptions on journalistic quality perceptions (b1: Mdn = -0.33; 89% CI [-0.41; -0.24];
pd > .99), with quality perceptions being lower the higher the bandwagon perceptions.
(WHY) DOES COMMENT PRESENTATION ORDER MATTER? 11
Discussion
This study set out to replicate a recently observed effect of negative user comments
presented after a news article having a stronger effect on users’ perceptions of said article than
user comments presented beforehand (original study: Kümpel & Unkel, 2020). This replication
effort seemed necessary as the original pre-registered study predicted the opposite (i.e., negative
comments would have stronger effects when presented beforehand). Moreover, our aim with this
study was to identify possible reasons for the existence of the effect. Specifically, we focused on
the role of two heuristics: the availability heuristic (i.e., comments have an effect because they
are readily available and easy to be recalled) and the bandwagon heuristic (i.e., comments have
an effect because users believe they reflect the majority’s opinion and thus jump on the
proverbial bandwagon).
Our pre-registered online experiment with 325 participants provided inconclusive
evidence of the predicted presentation order effect. While the probability of the direction of the
effect being negative (as hypothesized in H1) is 93%, this is still below the threshold set for a
successful replication. Even if the effect exists, it is likely to be rather small and negligible with
the 89% credibility interval spanning from -0.32 (which is approximately the strength of the
effect found in the original study) to 0.03 (which would suggest that the effect is practically
equivalent to zero). Assuming that the presentation order effect is indeed very small or even
absent, we might conclude that (1) prior user comment experiments can be interpreted regardless
of the order in which the stimulus was presented, and (2) that news providers do not have to feel
overly apprehensive about the information environment in which comments on their work
appear. Accordingly, we found no support for H2 and H3, that is, comment presentation order
did neither affect participants’ degree of elaboration nor their bandwagon perceptions (and,
(WHY) DOES COMMENT PRESENTATION ORDER MATTER? 12
consequently, had no indirect effects on journalistic quality perceptions). However, our model
shows a substantial, non-hypothesized association between users’ bandwagon perceptions and
their journalistic quality perceptions, which occurred regardless of comment presentation order.
Notably, higher bandwagon perceptions did not lead to a decrease in people’s elaboration (and
then to reduced quality perceptions)—a route that we would have expected to occur if a heuristic
was at play (see our reasoning for H3). From our point of view, two explanations could account
for this finding: First, the direction of effects might be different from what one might expect.
Instead of others’ opinions influencing one’s own perceptions (a bandwagon effect), people’s
own perception might rub off on how they think others evaluate the article (i.e., the more
negative people’s own perceptions, the more they assume that others evaluate the article
negatively as well). This would suggest the presence of a “false consensus effect” (Ross et al.,
1977) according to which people tend to attribute their own sentiments to others. However, due
to our cross-sectional design and the non-randomization of bandwagon perceptions—similar to
previous studies (Waddell, 2018; Waddell & Sundar, 2017)—we can only speculate about the
actual direction of effects. Second, the finding could also mean that response latencies, albeit
being extensively used in prior research on heuristic information processing (for an overview see
Bellur & Sundar, 2014), might not be the best proxy to determine users’ degree of elaboration
and/or need to be triangulated with other measures focusing on people’s ease of recalling
judgment-relevant information.
Overall, our results highlight the continuing need for studying the conditions under which
effects of user comments occur, the necessity of replicating prior findings, and the commitment
to monitor how changing information environments might (not) alter previously observed effects
of user comments.
(WHY) DOES COMMENT PRESENTATION ORDER MATTER? 13
References
Bellur, S., & Sundar, S. S. (2014). How can we tell when a heuristic has been used? Design and
analysis strategies for capturing the operation of heuristics. Communication Methods and
Measures, 8(2), 116–137. https://doi.org/10.1080/19312458.2014.903390
Bürkner, P.-C. (2017). brms: An R package for Bayesian multilevel models using Stan. Journal
of Statistical Software, 80(1), 1–28. https://doi.org/10.18637/jss.v080.i01
Chen, G. M., & Ng, Y. M. M. (2016). Third-person perception of online comments: Civil ones
persuade you more than me. Computers in Human Behavior, 55, 736–742.
https://doi.org/10.1016/j.chb.2015.10.014
Dohle, M. (2018). Recipients’ assessment of journalistic quality. Do online user comments or the
actual journalistic quality matter? Digital Journalism, 6(5), 563–582.
https://doi.org/10.1080/21670811.2017.1388748
Fazio, R. H. (1990). A practical guide to the use of response latency in social psychological
research. In C. Hendrick & M. S. Clark (Eds.), Research methods in personality and
social psychology: Review of personality and social psychology (pp. 74–97). Sage.
Haugtvedt, C. P., & Wegener, D. T. (1994). Message order effects in persuasion: An attitude
strength perspective. Journal of Consumer Research, 21(1), 205–218.
https://doi.org/10.1086/209393
Jungnickel, K. (2011). Nachrichtenqualität aus Nutzersicht. Ein Vergleich zwischen
Leserurteilen und wissenschaftlich-normativen Qualitätsansprüchen. Medien &
Kommunikationswissenschaft, 59(3), 360–378. https://doi.org/10.5771/1615-634x-2011-
3-360
(WHY) DOES COMMENT PRESENTATION ORDER MATTER? 14
Kim, Y. J., & Hollingshead, A. B. (2015). Online social influence: Past, present, and future.
Annals of the International Communication Association, 39(1), 163–192.
https://doi.org/10.1080/23808985.2015.11679175
Knobloch-Westerwick, S., & Meng, J. (2009). Looking the other way: Selective exposure to
attitude-consistent and counterattitudinal political information. Communication Research,
36(3), 426–448. https://doi.org/10.1177/0093650209333030
Ksiazek, T. B., & Springer, N. (2018). User comments in digital journalism: Current research
and future directions. In S. A. Eldridge & B. Franklin (Eds.), The Routledge Handbook of
Developments in Digital Journalism Studies. Routledge.
Kümpel, A. S., & Springer, N. (2016). Commenting quality. Effects of user comments on
perceptions of journalistic quality. Studies in Communication | Media, 5(3), 353–366.
https://doi.org/10.5771/2192-4007-2016-3-353
Kümpel, A. S., & Unkel, J. (2020). Negativity wins at last: How presentation order and valence
of user comments affect perceptions of journalistic quality. Journal of Media Psychology,
32(2), 89–99. https://doi.org/10.1027/1864-1105/a000261
Lee, E.-J., & Jang, Y. J. (2010). What do others’ reactions to news on internet portal sites tell us?
Effects of presentation format and readers’ need for cognition on reality perception.
Communication Research, 37(6), 825–846. https://doi.org/10.1177/0093650210376189
Lee, E.-J., Kim, H. S., & Cho, J. (2017). How user comments affect news processing and reality
perception: Activation and refutation of regional prejudice. Communication Monographs,
84(1), 75–93. https://doi.org/10.1080/03637751.2016.1231334
(WHY) DOES COMMENT PRESENTATION ORDER MATTER? 15
Leiner, D. J. (2016). Our research’s breadth lives on convenience samples: A case study of the
online respondent pool “SoSci Panel.” Studies in Communication and Media (SCM),
5(4), 367–396. https://doi.org/10.5771/2192-4007-2016-4-367
Maxwell, S. E., Kelley, K., & Rausch, J. R. (2008). Sample size planning for statistical power
and accuracy in parameter estimation. Annual Review of Psychology, 59(1), 537–563.
https://doi.org/10.1146/annurev.psych.59.103006.093735
Müller, P., Schneiders, P., & Schäfer, S. (2016). Appetizer or main dish? Explaining the use of
Facebook news posts as a substitute for other news sources. Computers in Human
Behavior, 65, 431–441. https://doi.org/10.1016/j.chb.2016.09.003
Neubaum, G., & Krämer, N. C. (2017). Monitoring the opinion of the crowd: Psychological
mechanisms underlying public opinion perceptions on social media. Media Psychology,
20(3), 502–531. https://doi.org/10.1080/15213269.2016.1211539
Peter, C., Rossmann, C., & Keyling, T. (2014). Exemplification 2.0: Roles of direct and indirect
social information in conveying health messages through social network sites. Journal of
Media Psychology: Theories, Methods, and Applications, 26(1), 19–28.
https://doi.org/10.1027/1864-1105/a000103
Petty, R. E., Tormala, Z. L., Hawkins, C., & Wegener, D. T. (2001). Motivation to think and
order effects in persuasion: The moderating role of chunking. Personality and Social
Psychology Bulletin, 27(3), 332–344. https://doi.org/10.1177/0146167201273007
Prochazka, F., Weber, P., & Schweiger, W. (2018). Effects of civility and reasoning in user
comments on perceived journalistic quality. Journalism Studies, 19(1), 62–78.
https://doi.org/10.1080/1461670X.2016.1161497
(WHY) DOES COMMENT PRESENTATION ORDER MATTER? 16
Ross, L., Greene, D., & House, P. (1977). The “false consensus effect”: An egocentric bias in
social perception and attribution processes. Journal of Experimental Social Psychology,
13(3), 279–301. https://doi.org/10.1016/0022-1031(77)90049-X
Sundar, S. S. (2008). The MAIN model: A heuristic approach to understanding technology
effects on credibility. In M. J. Metzger & A. J. Flanagin (Eds.), Digital media, youth, and
credibility (pp. 73–100). The MIT Press.
Tversky, A., & Kahneman, D. (1973). Availability: A heuristic for judging frequency and
probability. Cognitive Psychology, 5(2), 207–232. https://doi.org/10.1016/0010-
0285(73)90033-9
Urban, J., & Schweiger, W. (2014). News quality from the recipients’ perspective: Investigating
recipients’ ability to judge the normative quality of news. Journalism Studies, 15(6),
821–840. https://doi.org/10.1080/1461670X.2013.856670
van ’t Veer, A. E., & Giner-Sorolla, R. (2016). Pre-registration in social psychology—A
discussion and suggested template. Journal of Experimental Social Psychology, 67, 2–12.
https://doi.org/10.1016/j.jesp.2016.03.004
von Collani, G., & Blank, H. (2013). Konsistenzstreben. Zusammenstellung
sozialwissenschaftlicher Items und Skalen (ZIS). https://doi.org/10.6102/ZIS48
von Sikorski, C. (2016). The effects of reader comments on the perception of personalized
scandals: Exploring the roles of comment valence and commenters’ social status.
International Journal of Communication, 10, 4480–4501.
Waddell, T. F. (2018). What does the crowd think? How online comments and popularity metrics
affect news credibility and issue importance. New Media & Society, 20(8), 3068–3083.
https://doi.org/10.1177/1461444817742905
(WHY) DOES COMMENT PRESENTATION ORDER MATTER? 17
Waddell, T. F., & Bailey, A. (2017). Inspired by the crowd: The effect of online comments on
elevation and universal orientation. Communication Monographs, 84(4), 534–550.
https://doi.org/10.1080/03637751.2017.1369137
Waddell, T. F., & Sundar, S. S. (2017). #thisshowsucks! The overpowering influence of negative
social media comments on television viewers. Journal of Broadcasting & Electronic
Media, 61(2), 393–409. https://doi.org/10.1080/08838151.2017.1309414
Weber, P., Prochazka, F., & Schweiger, W. (2019). Why user comments affect the perceived
quality of journalistic content. The role of judgment processes. Journal of Media
Psychology, 31(1), 24–34. https://doi.org/10.1027/1864-1105/a000217
Winter, S., Brückner, C., & Krämer, N. C. (2015). They came, they liked, they commented:
Social influence on facebook news channels. Cyberpsychology, Behavior, and Social
Networking, 18(8), 431–436. https://doi.org/10.1089/cyber.2015.0005
Xu, Q. (2013). Social recommendation, source credibility, and recency: Effects of news cues in a
social bookmarking website. Journalism & Mass Communication Quarterly, 90(4), 757–
775. https://doi.org/10.1177/1077699013503158
Zerback, T., & Fawzi, N. (2017). Can online exemplars trigger a spiral of silence? Examining the
effects of exemplar opinions on perceptions of public opinion and speaking out. New
Media & Society, 19(7), 1034–1051. https://doi.org/10.1177/1461444815625942
Zillmann, D., & Brosius, H.-B. (2000). Exemplification in communication: The influence of case
reports on the perception of issues. Erlbaum.
(WHY) DOES COMMENT PRESENTATION ORDER MATTER? 18
Figure 1
Hypothesized Model
Note: Plus signs indicate positive, minus signs negative relationships.
(WHY) DOES COMMENT PRESENTATION ORDER MATTER? 19
Table 1
Coefficient Estimates of Bayesian Serial Mediation Model Predicting Journalistic Quality
Perceptions
Path Parameter Median 89% CI LL 89% CI UL pd
Direct paths
CA BW a1 -0.01 -0.18 0.17 .53
CA DoE a2 0.10 -0.08 0.29 .83
BW JQ b1 -0.33 -0.41 -0.24 > .99
DoE JQ b2 0.01 -0.07 0.10 .61
CA JQ c' -0.15 -0.31 0.01 .93
BW DoE d21 0.03 -0.06 0.12 .71
Indirect effects
CA BW JQ a1b1 0.00 -0.06 0.06 .53
CA DoE JQ a2b2 0.00 -0.01 0.02 .58
CA BW DoE JQ
a1d21b2 0.00 0.00 0.00 .50
Total effect
c -0.14 -0.32 0.03 .91
Note: Bayesian serial mediation model as specified in Figure 1. All
values < 1.01, indicating
convergence. CI = Credibility interval; LL = Lower limit; UL = Upper limit; pd = Probability of
direction; CA = Comments presented after the article; BW = Bandwagon
perceptions; DoE = Degree of elaboration; JQ = Journalistic quality perceptions.
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
The brms package implements Bayesian multilevel models in R using the probabilistic programming language Stan. A wide range of distributions and link functions are supported, allowing users to fit – among others – linear, robust linear, binomial, Poisson, survival, ordinal, zero-inflated, hurdle, and even non-linear models all in a multilevel context. Further modeling options include autocorrelation of the response variable, user defined covariance structures, censored data, as well as meta-analytic standard errors. Prior specifications are flexible and explicitly encourage users to apply prior distributions that actually reflect their beliefs. In addition, model fit can easily be assessed and compared with the Watanabe-Akaike information criterion and leave-one-out cross-validation.
Article
Full-text available
User comments on news websites are frequently uncivil and are not supported by reasoned argumentation. These characteristics can have negative effects on the perceived quality of the commented-on journalistic content, yet to date, it remains unclear how such effects occur. We propose three mechanisms that assume that the effect of user comments depends on how deliberately and elaborately the quality of the commented-on news item is judged. We conducted an experiment (N = 633) in which we varied the level of civility and reasoning in the comments accompanying a news article and the brand of the news website on which it was presented. The results showed that a lack of reasoning in the comments decreased the perceived quality of the news item irrespective of brand awareness, but only with high elaboration during judgment. Incivility in the comments decreased the perceived quality of the journalistic content, but only with low elaboration, and only with an unknown news brand. The article discusses different psychological mechanisms that can explain this pattern of effects.
Article
A number of studies show that user comments on news websites can affect news-related judgments and perceptions. However, with news organizations increasingly shifting their comment sections to social network sites (SNS), questions arise about whether this alters previously observed effects. Instead of encountering comments “below the line,” SNS provoke a reversed direction of exposure, suggesting that comments might be read before the news article. Addressing the implications of this shift in direction of exposure, we conducted a preregistered experiment with German participants (N = 630), in which we varied comment presentation order (before vs. after the article) and comment valence (positive vs. negative) and assessed how these factors influence the way individuals perceive the journalistic quality of commented news articles. The data provide evidence for a negativity bias and presentation order effects, with negative comments showing distinct effects on quality perceptions, particularly when presented after the article.
Article
Online comments hold the potential to promote positive deliberative outcomes, although past work has also shown that comments can have undesired effects when the sentiment of the crowd turns negative. Does the presence of comments possibly bolster or interfere with the reception and traditional functions of news media? Informed by the Modality-Agency-Interactivity-Navigability (MAIN) model, an online experiment tested the effect of reader comments (positive vs negative), number of “re-tweets” and “likes” (low vs high), and coverage frequency (infrequent vs frequent) on news credibility and issue importance. Negative reader comments (relative to positive comments) decreased message credibility and issue importance through the sequential indirect pathway of bandwagon perceptions, attention, and construct accessibility. Study results suggest that the traditional functions of news media may be hindered by audience incivility.
Article
Many studies and their findings are available concerning most aspects of online user comments and their effects. But there are few data regarding the question of whether user comments have an impact on an audience’s assessment of journalistic quality, and regarding the question of how user comments and the actual journalistic quality interact. Therefore, two experimental studies were conducted. In both experiments, the quality of a journalistic product and the valence of user comments were manipulated. The results indicate that, overall, high-quality journalistic products were evaluated as (slightly) better than low-quality versions. Furthermore, users evaluated the quality of journalistic products with positive user comments as better than the quality of identical products with negative comments. Considering that, in reality, user comments are predominantly critical, it is worrying that these comments have rather negative effects on the evaluation of journalistic work.
Article
Past research on meaningful media has shown that the experience of elevation can foster increased feelings of connection to others due to the affective states that meaningful media elicit. However, what happens when inspiring media are accompanied by cues related to the opinion of others? Does the presumed opinion of the audience moderate the affective indicators or behavioral effects of elevation? An online experiment was conducted to answer these questions using a 2 (news type: inspiring vs. non-inspiring control) × 3 (comment type: civil vs. uncivil vs. no comment control) between-subjects design. Results revealed that news articles accompanied by either civil or uncivil comments elicited lower affective states associated with elevation. Furthermore, civil comments increased universal orientation through the mediating pathway of bandwagon support.
Article
Can the comments of a few viewers on social media affect viewers’ perceptions of audience sentiment or their own program enjoyment? If so, do the effects of comments vary based on their valence or placement during programming? We conducted a 2 (positive vs. negative tweets) x 2 (beginning vs. end of program) factorial experiment with an additional control condition (N = 196) to answer these questions. Negative comments undermined perceived bandwagon support for the program and reduced enjoyment, regardless of contextual or trait moderators. Findings suggest that the effects of social television may be attributable to systematic processing of bandwagon cues.
Article
Convenience samples have been a substantial driver of empirical social research for decades. Undergraduate students are still the researchers’ favorite subjects, but the importance of respondents recruited on the Internet is on the rise. This paper deals with the fuzzy concept of convenience samples, outlining their reasonable uses and limitations. To bolster the theoretical discussion on convenience samples with empirical evidence, findings from the non-commercial SoSci Panel, a large-scale volunteer respondent pool, are presented. Convenience pools allow for larger samples than traditional convenience samples, more heterogeneity, and better long-term availability of respondents. This paper discusses conditions of setting up a respondent pool and methodological and practical implications, such as software, tasks, respondent activity and panel loyalty.