Content uploaded by Miguel Barreda-Ángeles
Author content
All content in this area was uploaded by Miguel Barreda-Ángeles on Nov 03, 2020
Content may be subject to copyright.
1
On the use of implicit methods for evaluating
the impact of interactive narratives on
attitudes and behavior (Position paper)
Miguel Barreda-Ángeles
Dept. of Communication Science & Network Institute, VU University Amsterdam,
De Boelelaan 1105, 1081HV Amsterdam
Phone: +31 (0)20 5987101
e-mail: m.barredaangeles@vu.nl
Abstract. While dominant in research on user experience, the use of self-report methods such as
questionnaires or surveys involves some significant limitations for scrutinizing the transformative
potential of narratives on users’ attitudes and behavior. Here I defend the use of implicit methods,
which avoid directly asking the user, as a way to obtain a more detailed picture of such effects,
allowing researchers to understand dimensions of the transformative processes beyond what can be
verbally expressed by users.
Keywords: Interactive narratives; Implicit methods; Attitudes; Behavior
1. Introduction
The transformative aspects of an interactive narrative experience cannot be
comprehensively described without accounting for its impact on our attitudes and
behavior. This is particularly relevant in the case of narratives with an (explicit or
implicit) pro-social persuasive intention: ideally, they should not only move us
and make us think about the topics depicted but also change our attitudes and
behaviors towards them.
The measurement of attitudes and behavior brings important challenges to
researchers in the field. Self-report techniques such as questionnaires, surveys, or
interviews, offer clear advantages, including the easiness of administration and
analysis, their cheap cost, and their intuitive interpretation, which have made them
the most common methodological approach in user experience research. However,
they ground on the assumption that participants are able and willing to report the
targeted constructs, an assumption that is not always met. Self-report taps into the
introspective abilities of the subjects, and trying to access certain aspects of our
2
mind through introspection may be "rather like attempting to discern the elements
of water by looking at a drop of water." (Cummins, 2000). Indeed, consistent
evidence from social psychology research shows that we usually lack conscious
access to many of our cognitive processes, and several mental and behavioral
events occur in an automated, unconscious, way (Nisbett & Wilson, 1977; Hassin
et al., 2004). Moreover, individuals' inability to report the effects of stimuli (e.g.
narratives) may also appear in combination with some (conscious or unconscious)
motivations for inaccurate reporting (e.g. social desirability bias, acquiescence
bias, or careless responding) that may call into question the validity of the
information provided.
These issues may particularly affect the self-report of attitudes and behavior.
Indeed, theoretical models of attitudes describe the coexistence of both explicit,
consciously accessible, attitudes, together with implicit, unconscious, ones that,
despite having relevant behavioral implications, are not accessible through self-
report (Petty et al. 2007). In turn, the study of behavior is often tackled through
self-declared behavior or behavioral intentions, but the mismatch between
declared and actual behavior has been well known for a long time (Baumeister et
al. 2009). Hence, and without dismissing the utility of self-report, it is clear self-
report techniques alone are not enough to examine all the possible transformative
effects of interactive narratives.
2. Some alternatives to self-report
Given those limitations, researchers in social psychology and affine fields have
developed a battery of measures to help overcome them. With regards to the
measurement of attitudes, probably the most prominent one is the Implicit
Association Test (IAT) (see Greenwald et al. 2009 for a detailed description). The
test involves measuring the response times of participants making associations
between concepts and attributes across several trials and inferring the strength of a
certain (implicit) attitude from the variations in response times. It has been
extensively used in research on prejudice, and meta-analytical evidence shows
that it explains variability in attitudes and behavior beyond the variability
3
explained by self-report, particularly in socially sensitive topics (e.g. attitudes
towards different races or sexual orientations) (Greenwald et al. 2009).
Other alternatives for the implicit measurement of attitudinal and behavioral
change are related to brain activity measures. A particularly interesting option,
which is gaining popularity in the last years, is the use of functional near-infrared
spectroscopy (fNIRS), a relatively new, cheap, and portable system for brain
imaging. Research has shown that certain patterns of brain activity are associated
with attitudinal and behavioral change (Burns et al., 2018), making clear the
potential of fNIRS for the investigation of the transformative impact of interactive
narratives.
Finally, another option is the direct observation of participants' behavior after
being exposed to a narrative in an experimental setup. The idea is to offer the
participants the possibility to perform some action (related to the story topic) that
reveals their actual behavior, such as signing a petition or making a donation to a
NGO. Some very creative approaches have been used for this purpose: for
instance, Fonseca and Kraus (2016) showed, to different groups of participants,
narratives either on the environmental consequences of meat consumption or on
non-related topics. Then, by the end of the experiment, they offered some snacks
to the participants and analyzed how many of them in each group went for
vegetarian options.
The use of these approaches may be challenging for researchers in interactive
narratives in several ways. Among them, there is the technical expertise required,
which may make necessary the collaboration of multidisciplinary teams.
Relatedly, the design of studies and the interpretation of the results may not be
straightforward in many cases, and thus may demand additional efforts from
researchers. Finally, practical aspects such as the increase in costs (e.g. due to
equipment acquisition) and time for running studies need also to be taken into
account. However, the additional effort should pay off: methods such as the IAT
or fNIRS have been shown to predict variability in behavioral outcomes
(associated to exposure to media messages) beyond what is reported by the users
(Greenwald et al. 2009; Burns et al. 2018). Hence, a wide picture of how a
4
narrative may transform the audience can only be achieved by addressing such
methodological and practical challenges.
Conclusion
The effects of narratives on our attitudes and behavior are a central dimension of
its transformational potential. The utility of self-report methods to explore them is
undeniable, but their limitations are also clear: narratives may affect us in ways
that we are not able to express verbally. Despite the challenges they pose to
researchers, the use of implicit methods such as the IAT, fNIRS, or the direct
observation of participants behavior, should be encouraged, since it will allow
researchers to gain deeper insights on how narratives make us think and act, and,
ultimately, how they change who we are.
References
Baumeister RF, Vohs KD, Funder DC (2007) Psychology as the science of self-reports and finger
movements: Whatever happened to actual behavior?. Perspect. Psychol. Scienc. 2:396-403.
Burns SM, Barnes LN, Katzman PL, Ames DL, Falk EB, Lieberman MD (2018) A functional near
infrared spectroscopy (fNIRS) replication of the sunscreen persuasion paradigm. Soc. Cognit.
affective neuroscience, 13:628-636
Cummins DD (2000) A history of thinking. In: Cummins R, Cummins DD (eds) Minds, Brains,
and Computers: The Foundations of Cognitive Science: An Anthology, Blackwell, pp. 8-19
Fonseca D, Kraus M (2016). A comparison of head-mounted and hand-held displays for 360
videos with focus on attitude and behavior change. In: Proceedings of the 20th International
Academic Mindtrek Conference, AMD, pp. 287-296
Greenwald AG, Poehlman, TA, Uhlmann EL, Banaji, MR (2009) Understanding and using the
Implicit Association Test: III. Meta-analysis of predictive validity. J. Personalit. Soc. Psychol.
97:17-41
Hassin RR, Uleman JS, Bargh JA (Eds.) (2004) The new unconscious. Oxford University Press.
Nisbett, RE, Wilson TD (1977) Telling more than we can know: Verbal reports on mental
processes. Psychol. Rev. 84: 231-259.
Petty, RE, Briñol P, DeMarree KG (2007) The meta–cognitive model (MCM) of attitudes:
Implications for attitude measurement, change, and strength. Soc. Cognit., 25:657-686.
