Content uploaded by Eric Brangier
Author content
All content in this area was uploaded by Eric Brangier on May 28, 2014
Content may be subject to copyright.
Vol. 9, Issue 3, May 2014 pp. 105-128
Copyright © 2013-2014, User Experience Professionals Association and the authors. Permission to make digital or
hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on
the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee. URL: http://www.upassoc.org.
Set of Guidelines for Persuasive
Interfaces: Organization and
Validation of the Criteria
Alexandra Némery
User Experience Designer
Researcher
Université de Lorraine–Metz,
PErSEUs EA 7312–User
Experience Lab. Faculté des
Sciences Humaines et
Sociales–Ile du Saulcy–CS
60228–57045 METZ cedex 01,
France
SAGE–10, rue Fructidor–75017
Paris
alexandra.nemery@gmail.com
Eric Brangier
Full Professor
Université de Lorraine–Metz,
PErSEUs EA 7312–User
Experience Lab. Faculté des
Sciences Humaines et
Sociales–Ile du Saulcy–CS
60228–57045 METZ cedex 01,
France
Eric.Brangier@univ-lorraine.fr
Abstract
This study presents an attempt to organize and validate a set
of guidelines to assess the persuasive characteristics of
interfaces (web, software, etc.). Persuasive aspects of
interfaces are a fast growing topic of interest; numerous
website and application designers have understood the
importance of using interfaces to persuade and even to
change users’ attitudes and behaviors. However, research
has so far been limited by a lack of available tools to
measure interface persuasion. This paper provides a criteria-
based approach to identify and assess the persuasive power
of interfaces.
We selected164 publications in the field of persuasive
technology, and we used those publications to define eight
criteria: credibility, privacy, personalization, attractiveness,
solicitation, priming, commitment, and ascendency. Thirty
experts in human-computer interaction (HCI) were asked to
use guidelines to identify and classify persuasive elements of
15 interfaces. The average percentage of correct
identification was 78.8%, with Randolph’s kappa coefficient
= 0.61. These results confirm that the criteria for interactive
persuasion, in their current form, can be considered as valid,
reliable, and usable. This paper provides some inherent
limitations of this method and identifies potential refinements
of some definitions. Finally, this paper demonstrates how a
checklist can be used to inspect the persuasiveness of
interfaces.
Keywords
Persuasive interface, ergonomic criteria, persuasive
technology, guidelines
106
Journal of Usability Studies Vol. 9, Issue 3, May 2014
Introduction
Usability can be assessed through widely known inspection methods such as walk-throughs and
heuristic evaluations. Historically, while these methods were being developed, a set of usability
assessment metrics and criteria were defined and resulted in the International standards for
HCI and usability in 1998 (i.e., ISO 9241). In this context, these criteria have been developed
to facilitate the use of technical systems; to make interaction more useful, efficient, simple,
pleasant, etc.; and to make life easier for users.
Our research seeks to define criteria and validate a checklist of criteria to assess the persuasive
dimensions of user interfaces. Ergonomics and HCI have produced numerous recommendations
to measure the ergonomic quality of products and services, but there is a lack of knowledge and
criteria to design and to evaluate persuasive interactions.
The increase in new technologies and interactive media provides new opportunities to influence
users. Nowadays, technology offers unprecedented means to change users’ attitude or behavior
because technology is an everyday presence for a large portion of the world’s population and
can be accessed easily and quickly. This accessibility has created a new type of relationship
between users and technology; the user’s experience can be rooted in a more emotional
context. Through persuasion, technological media can take users by the hand and lead them to
perform target behavior.
Several definitions of persuasive interfaces can be found in the literature. For the purpose of
this study, we define a persuasive interface as a combination of static qualities and steps that
guide a user through a process to change his or her attitude or behavior. Many industries and
companies that have an online presence, such as social networks, e-commerce, e-health or e-
learning companies, invest in and produce persuasive interfaces. For instance, Consolvo,
Everitt, Smith, and Landay (2006) and Adams et al. (2009) observed a significant increase in
healthcare companies’ efforts that aimed to encourage safe behavior and healthier lifestyles.
While this effort provides a way to foster better health habits, which has the potential to cut
healthcare costs, it also raises some ethical issues like the manipulation of health behaviors.
With a large variety of websites, software, mobile phones, and other applications, experts need
access to effective tools to evaluate the persuasiveness of an online interface. These tools can
be found in the multidisciplinary field of ergonomics. Ergonomics has a tradition of producing
standards, checklists, guidelines, norms, and recommendations that can be used to evaluate the
persuasiveness of interfaces.
To present and validate a checklist to assess persuasion in interfaces, we start by describing the
theoretical basis of our set of guidelines. Then, we classify persuasive criteria into eight
categories. Our set of criteria aims to define and gather all aspects of persuasion present on the
interfaces. We then describe an experiment with 30 usability experts performing an evaluation
task on the persuasive human-computer interfaces. The validation of the checklist is based on a
method of categorization criteria: the more the experts categorize identical criteria, the better
the quality of the guidelines. From this point of view, our main question is to understand if the
validation says something about whether participants can tell the eight criteria apart and
whether the participants find that the eight criteria capture important aspects of the
persuasiveness of the interfaces. Finally, we analyze and discuss the results of the experiment
and provide some recommendations for the use of this set of guidelines.
Conceptual Framework
The following sections discuss the theoretical background for the concept and design of this
study, the design of the set of guidelines used for this study, and the selection of guideline
criteria for the checklist.
Theoretical Background
Fogg opened the way for the field of persuasive technology for which he coined the term
“captology,” an abbreviation for “computers as persuasive technologies” (2003, p. 5). The
concept of persuasion can cover a range of meanings, but Fogg defines it as “an attempt to
shape, reinforce, or change behaviors, feelings, or thoughts about an issue, object, or action.”
Fogg presents persuasion technology as a tool to achieve a change in the user’s behavior or
107
Journal of Usability Studies Vol. 9, Issue 3, May 2014
attitude, a media interaction that creates an experience between the user and the product, and
a social actor. The social actor deals with the technology's use of strategies for social influence
and compliance. In short, persuasion technology is a medium to influence and persuade people
through HCI. Indeed, the technology becomes persuasive when people give it qualities and
attributes that may increase its social influence in order to change users’ behavior. This notion
is at the crossroads between ergonomics, social psychology, organizational management, and
(obviously) the design of a user interface.
We chose to use and adapt some traditional persuasion techniques used in studies from fields
such as advertising, politics, and health (Chaiken, Liberman, & Eagly, 1989; Greenwald &
Banaji, 1995; McGuire, 1969; Petty & Cacioppo, 1981). We adapted the techniques used in
these earlier studies to match modern technological systems capabilities and specificities
(Oinas-Kukkonen, 2010).
This last decade has met with a growing interest for technological persuasion, as shown by
Tørning and Oinas-Kukkonen (2009), who investigated persuasive systems produced between
2006 and 2008. The large body of research on this topic clearly attests to the importance of the
field of technological persuasion. One of the recommendations for future research is to create
methods for a better measurement of successful persuasive systems (Lockton, Harrison, &
Stanton, 2010; Oinas-Kukkonen & Harjumaa, 2009; Tørning & Oinas-Kukkonen, 2009). In the
same way that Oinas-Kukkonen and Harjumaa (2009) defined four persuasive dimensions
(social support, system credibility support, primary task support, and dialogue support), we
consider that designers and ergonomists need to have a set of guidelines to evaluate
persuasion.
In this paper, we present a checklist that serves as a set of guidelines. We explain the goal of
our checklist which is to develop ergonomic practices and to integrate persuasive evaluations
into software design processes using concepts such as reduction, credibility, legitimacy,
tunneling, tailoring, personalization, self-monitoring, simulation, rehearsal, monitoring, social
learning, open persuasion, praise, rewards, reminders, suggestion, similarity, attractiveness,
liking, social role, and addictive characteristics—all of which originate from different research
(Fogg, 2003; Nemery, Brangier, & Kopp, 2009; Oinas-Kukkonen & Harjumaa, 2008).
Designing the Set of Guidelines
A lack of means to assess or to design persuasive interfaces is apparent (Tørning & Oinas-
Kukkonen, 2009). Some of the criteria gathered from earlier studies (Oinas-Kukkonen &
Harjumaa, 2009) were not validated with experimental methods that showed the relevance and
efficiency of the criteria.
Another criticism, which can be addressed against existing criteria, is the lack of consideration
for time as a structural element of social influence. Persuasion is always linked to a temporal
process that includes a beginning, followed by modifications leading to behavioral change. This
process always takes place over several hours, sometimes several months. A set of criteria
should include the moment in time to understand forms of lasting influence, such as addiction
or dependence. The concept of temporality emerged as a necessary condition to change user
behavior. In order to incur lasting changes of attitude and behavior, it was observed that
accepting an effortless initial request predisposes a user in a positive way to accept a
subsequent request asking for a greater effort; this social influence is always temporarily
structured. For example, the interface displays an invitation to play a short videogame, if you
choose to play the game, you are gradually presented with seemingly innocuous requests
designed to covertly persuade you, eventually, to give personal information or to purchase
options.
Our checklist is based on an analysis of 164 documents linked to technological persuasion (for
details of the collected documents, see Némery, 2012). These documents helped us identify the
main components of persuasion mediated by tools, organized those components, and offered a
classification framework for our guidelines. We developed a total of eight persuasive criteria and
23 sub-criteria. The set of guidelines differentiates static aspects from dynamic aspects. The
assumption is that user interfaces need a set of necessary qualities and properties. These
elements are conducive to the establishment of a commitment loop that will lead a subject from
behavior A (or attitude A) to behavior B (or attitude B).
108
Journal of Usability Studies Vol. 9, Issue 3, May 2014
New devices offer more and more interactive and innovative interfaces that use new user
interface features and technologies. The potential persuasive impact of these dynamic interfaces
is much greater than a static information display where no interaction is possible. Given a few
enabling conditions, users are able to dynamically interact with a device in a short time, which
facilitates the desired change in users’ behaviors or attitudes towards a product, service, or
informational interface. Interface properties are necessary but not sufficient to change
behaviors and attitudes. Change is also a social construct that could be planned while taking
into account user specificities.
To better understand the process of persuasion, let's compare a human to human relationship.
For instance, you would not lend money to strangers or someone you do not see as trustworthy.
Along the same line of thinking, people will not buy something on a website without critical
elements that inspire quality and security during the interaction. If websites are not deemed
trustworthy, people generally do not buy products from those sites.
A computer is a social actor (Fogg, 2003). Through repetitive interactions, a user will be more
and more emotionally involved. Computers can feel like a living entity with human-like qualities.
Through the use of formal and polite language employed in interfaces—as advice, rewards,
motivating messages, or praise—computers can serve as a social support entity or as a
teammate (Nass, Fogg, & Moon, 1996). Depending on the context, a computer can become a
coach, a cooperative partner, or a sort of a symbiotic assistant.
Selection of Criteria and Construction of the Checklist
Research to develop the list of criteria (Figure 1) is based on a review of 164 articles on
captology and persuasive design. We also used database research (PsychInfo, Springer-Kluwer,
and Science Direct) and conference proceedings presentations and articles (CHI, HCII,
Persuasive conference) to develop the criteria. These resources provide a rich collection of many
examples of persuasive technology. The publication rate in this area has increased over the past
ten years.
Figure 1. Construction of the checklist.
To select the publications for our research, we used the following indicators of relevance:
We selected papers that included research that dealt with a user’s change of attitude or
behavior through technology. We avoided articles that did not gather sufficient
empirical evidence in their methodology.
We selected papers that focused on topics of persuasive interfaces, i.e., computer-
human interactions that combined qualities and planned interactions that guide a user
through a process to change his or her attitude or behavior. The papers that did not
focus directly on these topics or were deemed less relevant to our area of study were
109
Journal of Usability Studies Vol. 9, Issue 3, May 2014
excluded (i.e., papers using persuasive concepts but dealing with the challenge of
advertising on the Internet and/or business model of e-commerce).
Two ergonomists selected or rejected articles based on reading the titles and abstracts of those
articles. For an article to be accepted for the resource list, at least one of the two ergonomists
needed to select the article. In total, 164 articles were selected. The general conclusions that
emerged from this set of publications were the following:
The publication rate in this area has clearly increased over the past ten years. From
1994 to 2005, persuasive technology was just emerging (around five papers each year
presented recommendations on persuasive interfaces), but after 2005, publications
increased until 2009 with 42 papers published on persuasive technologies.
Our findings were interdisciplinary and take into account related disciplines such as
sociology, psychology, computer science, marketing, and communication science.
The selected publications include the following fields (the percentage represents the
ratio of papers/fields used in this study): Health (28.22%), Education (23.31%),
Service (18.40%), Business (9.82%), Ecology (9.20%), Marketing (6.75%), and
Leisure (4.29%).
The following technologies are the main technology resources written about in the
selected publications: websites (31.90%), games (26.38%), software (16.56%), mobile
apps (9.20%), ambient systems (4.29%), and others (11.67%).
Previous research defined the initial checklists (Némery, Brangier, & Kopp, 2009, 2010, 2011).
After an iterative work of classification and organization, we designed the checklist to include
eight elements (Figure 2). After this checklist was finalized, a second checklist was drawn up
with the following requirements:
Simplification: Each element must be understood by ergonomists, HCI specialists,
computers engineers, and interactive designers.
Consistency: All criteria extracted from the 164 articles must be reflected in the
checklist (Figure 2). This consistency gives relevance to the checklist because all items
must be present in one form or another.
Exclusivity: Only one criterion per persuasive item must be applied. Ambiguity between
criteria should be avoided as much as possible. A persuasive item must be relevant to
the criteria, and vice versa.
Figure 2. Distribution of papers (n= 164, published from 1994 to 2011) dealing with the eight
criteria developed in the checklist for persuasive interfaces.
110
Journal of Usability Studies Vol. 9, Issue 3, May 2014
A Set of Guidelines for Persuasive Technology
A classification of the 164 papers shows that 53% of papers dealt with the dynamic aspect of
the computer's influence and 47% were more related to static dimensions of persuasive
interfaces (Figure 2).
Static Criteria
Static criteria are all the surface elements that are necessary to help launch a dynamic process
of user engagement. That is to say, in interfaces, some prerequisites are necessary to foster
acceptance of an engaging process. These criteria are based on the influence content. Mostly
not affected by time or sequence of interactions, these criteria are designed to assess the
content of the interaction. Static criteria are opposed to dynamic criteria, which incorporates
strong temporal components. We identified four static components to foster acceptance and
confidence of users: credibility, privacy, personalization, and attractiveness (Figure 3).
Figure 3. General architecture of the eight interactive persuasive criteria.
Credibility is the first general criterion (see Figure 4 for an example). It is the ability of the
interface to inspire confidence and to make the user trust the veracity of its information.
Credibility is also based on a company’s or an industry’s reputation. All the people or institutions
that provide the components that make up an industry’s or company’s systems (e.g., the
different types of technical systems) must be recognized as being honest, competent, fair, and
objective. Credibility comprises three components: trustworthiness, expertise, and legitimacy.
Much has been written about the credibility and trust of web media (e.g., Bart, Venkatesh,
Fareena, & Urban, 2005; Bergeron & Rajaobelina, 2009; Huang, 2009).
111
Journal of Usability Studies Vol. 9, Issue 3, May 2014
Figure 4. Example of credibility criterion: This website provides a list of famous clients to
inspire trustworthiness and legitimacy in their field (source: www.cooper.com, screenshot
captured on 03/15/2013).
Privacy means the protection of personal data, the preservation of personal integrity, and the
security of the interaction (Figure 5). It covers all aspects of privacy that are used in
interactions. This criterion also aims to ensure protection against loss, destruction, or
inadvertent disclosure of data (Liu, Marchewka, Lu, & Yu, 2005). Privacy concerns the
expression of perceived safety, the perception of rights, and the protection of confidentiality of
information. For instance, organizations such as Alcoholic Anonymous and Weight Watchers are
successful entities because they are very rigorous in terms of privacy (Khaled, Barr, Noble, &
Biddle, 2006).
Figure 5. Example of privacy criterion: This financial website helps users manage their budget.
Mint provides a lot of reassuring information about security and associated entities (source:
www.mint.com, screenshot captured on 03/15/2013).
Personalization refers to the ability for a user to adapt the interface to the user’s preferences
(Peppers & Rogers, 1998; Figure 6). Personalization includes all actions aimed at characterizing
a greeting, a promotion, or a context to achieve a closer approach to the user. Personalization
may include individualization and group membership. Personalization requires an analysis of the
activity beforehand. Its power is dependent on the quality of data from the user and the degree
of personalization. The latter implies that the interface gradually learns the characteristics of the
user and modifies its interaction with the user towards a higher level of personalization.
Personalization offers greater potential returns than standardized and impersonal products
112
Journal of Usability Studies Vol. 9, Issue 3, May 2014
(Pine, 1993). As customers, users want services tailored to their needs. E-commerce has
produced many strategies to customize interactions. Old business practices promoted a
homogeneous market with standardized products; this model is not adequate any longer and
does not fit each user’s needs. The proliferation of interface products pressures designers to
mass produce customizable interfaces. They provide personalized information and synthesized
solutions that match the choice of respective communities, not necessarily individuals. All these
elements combined lead to a customized user experience.
Figure 6. Example of personalization criterion: The user is called by his or her first name
throughout the interaction which creates a sense of familiarity (source: www.facebook.com,
screenshot captured on 03/15/2013).
Attractiveness is the use of aesthetics (graphic, art, design) to capture the attention of the
user, to support the interaction, and to create a positive emotion (Figure 7). The animation,
colors, menus, drawings, and video films are designed to catch and maintain the interest of the
user. Presentation of these persuasive interactive elements must consider the cognitive
perceptual characteristics of the user. Persuasive design could motivate users towards specific
actions (Redström, 2006). Attractiveness has three components: emotional appeal, call to
action, and tunneling design. Tunneling design is a kind of attractive interaction that tries to
lead the user through a predefined succession of actions to produce a desired result.
113
Journal of Usability Studies Vol. 9, Issue 3, May 2014
Figure 7. Example of attractiveness criterion: The drawings are used to create a visual identity
and unique experience that could leverage positive emotions (source: www.axure.com,
screenshot captured on 03/15/2013).
Dynamic Criteria
To encourage users to change their behavior, it is important to take the temporal aspect into
account. Design is an engaging and interactive activity that requires segmenting and planning
persuasive processes. Here, dynamics means a way to involve the user in an interactive process
that progressively engages him or her with the interface. There are four dynamic criteria:
solicitation, priming, commitment, and ascendency.
Solicitation refers to the first stage that aims to briefly attract and challenge the user to
initiate a relationship (Figure 8). We can distinguish three elements for this “invitation” stage:
allusion, suggestion, and enticement. The invitation sets up the beginning of the relationship
and the dialogue between the user and electronic media. When this approach is used often by
electronic media, the probability of initiating the desired action by the user increases. The
interface attempts, by words, graphics, or any form of dialogue, to suggest a behavior followed
by action. Solicitation represents the ability to induce an action by the user with minimal
influence from the interface. Here, the interface suggests ideas or actions that the user could
initiate or could have (Dhamija, Tygar, & Hearst, 2006).
114
Journal of Usability Studies Vol. 9, Issue 3, May 2014
Figure 8. Example of solicitation criterion: Several advertisements are displayed with a short
message that tries to catch a user’s attention with an attractive or free offer (source:
www.awwwards.com, screenshot captured on 03/15/2013).
Priming refers to elements of the media that trigger the persuasive influence (Figure 9). These
elements may take the form of piloting the first steps. Following requests from the interface,
the user's attention is captured. Users are encouraged to realize the first engaging action. With
priming, the first action is carried out without coercion or awareness. Users are caught in a
process that gradually draws them in (Yang, 2005).
Figure 9. Example of priming criterion: The first action expected in an e-commerce website is
to make the user add an item to the cart. In this interface, the message “Add to cart” on the
button is replaced by “It’s perfect” as an invitation and incitation to click. In addition, the
message is reinforced by the fact that on mouse over, the message is replaced by “I’ll take it”
to strengthen the intention (source: http://www.thefutureperfect.com, screenshot captured on
03/15/2013).
Commitment means that the interface continues to involve the user through a process
(Figure 10). This process uses action sequences or predetermined interactions to gradually
involve the user. The successful completion of these queries and actions is followed by praise,
115
Journal of Usability Studies Vol. 9, Issue 3, May 2014
encouragement, reward, and continued interaction, which induces more intensive and regular
behavior from the user (Weiksner, Fogg, & Liu, 2008).
Figure 10. Example of commitment criterion: To use this app, the person must follow a wizard
assistant that asks for more personal information through a tunneling process. The idea is not
to lose the user's attention and also to ask the user to interest his friends in the app by (a)
making the user’s use public and (b) asking the user to invite friends (source: iPhone App—
Path, screenshot captured on 03/15/2013).
Ascendency is an indicator of the completion of the engaging scenario (Figure 11). When users
enter the ascendency stage, the user has definitely accepted the logic and goals of the
electronic media. The “grip” or hold that the technology has on a user’s attention is the deepest
form of technological persuasion. At this stage, the user involvement is complete and he or she
runs the risk of addiction or at least an over-consumption of the electronic media. In these
interactions, the user behaves in such a way as to generate pleasure and maybe relieve
negative feelings.
At the interface level, the influence is apparent through various elements: irrepressible
interaction, tension release, and also consequences beyond the interaction with the media.
Immersion is often mentioned in the field of video gaming. Because these games were first
devoted to a flight simulator environment, they were realistic, and they caught the user's full
attention. In the end, the player becomes a part of the environment due to the repetition and
the regularity of interaction (Baranowski, Buday, Thomson, & Baranowski, 2008). The emotional
involvement in the story of the game could cause a risk of dependence and identification. This
does not only concern desktop applications, but also GPSs and Smartphones. Users who
developed an emotional attachment to their device cannot imagine living without these products
(Nass, Fogg, & Moon, 1996).
116
Journal of Usability Studies Vol. 9, Issue 3, May 2014
Figure 11. Example of ascendency criterion: The game challenges users throughout their
progression and offers them different types of rewards like public badges to show success or
new items to accomplish more exploits. Statistics are also displayed to encourage daily use. At
this stage, there is a risk of addiction or overconsumption (source: Video game—Starcraft 2,
screenshot captured on 3/15/2013).
To sum up, this framework presents eight persuasion criteria divided into two dimensions—
static and dynamic aspects of the interface—that seeks to improve the evaluation of persuasive
elements in interfaces. These criteria complement the traditional ergonomic criteria that focus
on the user's ability to be effective, efficient, and satisfied. Persuasion is generally not within
the scope of the inspection, and criteria can be summarized in the following sentences:
Credibility: Give enough information to users that enable them to identify the source of
information as reliable, relevant, expert, and trustworthy.
Privacy: Do not persuade users to do something that publicly exposes their private life
and which they would not consent to do.
Personalization: Consider a user as a person and consequently develop a personal
relationship.
Attractiveness: Capture the attention of a user to elicit emotion and induce favorable
action.
Solicitation: Initiate the relationship by first temptations.
Priming: Help users to do what the system wants them to do.
Commitment: Involve, engage, and adhere to the objectives of the system.
Ascendency: Provide incentives to get a user totally involved with the system.
Interface designers and ergonomic analyzers must be careful to ensure that their interfaces do
not contain intrusive aspects that would affect the ethical handling of certain domestic or
professional interactions, particularly as these factors affect the attitudes of users. By applying
knowledge about how humans work on the psychosocial level, our criteria are therefore based
on a normative approach of what should or should not be persuasive. At present, it is necessary
to validate these guidelines to demonstrate their relevance and consistency.
117
Journal of Usability Studies Vol. 9, Issue 3, May 2014
Method to Validate the Set of Guidelines for Persuasive Interfaces
The following sections discuss the internal validation process, the study participants, the study
materials and equipment, the tasks and procedures used in the study, and the data collection
and analysis methods.
Internal Validation
On the basis of the compilation of recommendations presented in the previous section, we
compiled a list of persuasive criteria to serve as a support for the classification and identification
of persuasive elements related to interfaces (examples shown in Figures 3 through 12).
In agreement with Bastien (1996), Bastien and Scapin (1992), Bach (2004), Bach and Scapin
(2003, 2010), who have validated ergonomic criteria on the pragmatic aspects of interfaces, we
used the same method to validate the criteria of persuasion. The validation method of the
checklist consists of a test, completed by experts in HCI (or study participants), identifying
persuasive elements in the interfaces using the proposed criteria. If the HCI experts identify the
problem with the right criteria, the checklist is relevant. Conversely, if experts misidentify
problems and/or criteria, the checklist is irrelevant. Correct identification is calculated and
shown on the screen interface, and subject scores are broken down into static vs. dynamic
criteria. Correct identification (or good assignment) is when the criteria defined by the
participant matches the one identified by the specialist. A high percentage of correct
identification reflects the quality of the set of guidelines; a high percentage of mismatches
indicates that the set is not efficient.
The task assignment can also identify the possible confusion between two or more criteria. An
overall score is calculated by comparing participants' responses and expected responses. This
score can include the proportion of correct identification or the proportion of incorrect
identification, which could signify that a criterion could be confused with another criterion. In
other words, the task of the test participants is to find then classify elements. Several outcomes
can happen during this process. Participants can either find an element that was defined as
persuasive (hit, correct), or they cannot find it (unidentified). Or they can find an element that
they consider persuasive, but that is not real according to the expected criteria (false). Once
something has been correctly or incorrectly identified as a persuasive element, it can be
classified either correctly, incorrectly (false), or the test participant may not know how to
classify it (unidentified). Consequently, our hypothesis is that the higher the number of correct
identifications (hits), the more the internal validation is conclusive. Respectively, the higher the
number of misallocations (false), the more the internal validation will be ineffective and the
quality of the guidelines inferior.
Participants
Thirty people participated in the study. They were all experts in the HCI field: ergonomists,
engineers, professors, researchers, project managers, R&D, and interaction designers. They
were all experienced with user interfaces evaluation and/or design. On average, participants
had 13 years of experience (SD=7.9). Twelve of the experts were from the academic field and
eighteen employed in industry, but all had a similar background in evaluation and in computer
experience. Ten women and twenty men were gathered for this study.
Materials and Equipment
The eight criteria were summarized as a set of eight paper cards (Figure 12). Each criterion card
provided a detailed definition, a rationale based on references, and some examples illustrated
from various devices and fields.
Two experts (the authors) examined and selected the interfaces used in this study. They found
84 persuasive items that included 43 static elements and 41 dynamic elements: 12 forms of
credibility, 10 forms of privacy, 13 forms of personalization, 8 forms of attractiveness, 22 forms
of solicitation, 8 forms of priming, 8 forms of commitment and 3 forms of ascendency. On
average, there were 10.5 elements per criteria (SD=5.5) and 5.6 elements per interface
(SD=2.7). These 15 screen interfaces were chosen because both authors were in total
agreement on the criteria mentioned. Screen interfaces that were disagreed upon were
discarded.
118
Journal of Usability Studies Vol. 9, Issue 3, May 2014
Figure 12. Credibility card and the seven others cards (small). Download the French version of
the cards used during the experiment at http://www.univ-metz.fr/ufr/sha/2lp-
etic/Criteres_Persuasion_Interactive-2.pdf.
We chose fifteen interfaces that contained persuasive elements for the study (Table 1 and
Figure 13). Twelve of them were websites and three were software applications. Various areas
were represented: business (3), video games (3), social networks (3), e-commerce (2),
transport (1), communication tools (1), health (1) and ecology (1). Six interfaces were
unknown, whereas nine were familiar to the participants.
119
Journal of Usability Studies Vol. 9, Issue 3, May 2014
Table 1. General Characteristics of the Tested Interfaces and Statistical Results of Correct
Identification Percentage of Criteria, Screen by Screen
Screen
Source
Field
Type
Renowned
Correct
identification
percentage
1
www.voyages-sncf.com
Utility
Website
Yes
77.14%
2
www.qualitystreet.fr
Business
Website
No
67.77%
3
www.meetic.fr
Leisure
Website
Yes
84.00%
4
www.lecoleagit.fr
Education
Video Game
No
76.00%
5
www.famicity.com
Utility
Website
No
81.53%
6
Explorer
Business
Software
No
81.33%
7
Email Yves Rocher
Advertising
Email
Yes
92.50%
8
www.skype.com
Utility
Website
Yes
87.62%
9
www.lge.com
Advertising
Website
Yes
84.00%
10
www.urban-rivals.com
Leisure
Video Game
No
65.41%
11
mail.live.com
Advertising
Website
Yes
78.33%
12
App WeightMan
Health
Mobile
No
76.66%
13
www.monster.fr
Business
Website
Yes
75.00%
14
www.facebook.com
Leisure
Website
Yes
76.66%
15
Dragon Age Origins
Leisure
Video Game
No
52.22%
Figure 13. Set of 15 interfaces.
120
Journal of Usability Studies Vol. 9, Issue 3, May 2014
Task and Procedure
The experimental task is, in fact, an identification task of persuasive criteria from examples
(classification accuracy). We individually tested each participant. The instruction began with an
introduction to the experimental procedure and information about the gathered data and
criteria. We asked the participants to provide comprehensive comments about their perceptions
of the criteria. At the beginning of the session, we asked participants about their background,
years of experience, and expertise in heuristic evaluation.
In the first phase (familiarization), we asked participants to read the criteria documents very
carefully (Figure 12) and to pay particular attention to the screenshot examples that would only
be viewed during this first phase (i.e., the examples would not be visible later on in the
experiment). The objective of this phase was for the participants to become familiar with the
criteria and its content. The first instruction was, "I will present you with a list of eight criteria to
assess interactive persuasion. I will ask you to read this list in full; it differentiates static criteria
and dynamic criteria of interfaces. I would like you to look carefully at the examples because I
won't display them again during the identification phase. Do not hesitate to ask me any
questions."
In the second phase (identification test), the aim of the task was to allow participants to
memorize the criteria and its definitions. In this second phase, we gave the participants another
version of the checklist without screenshot examples. This specific point of the method was
emphasized by Bastien (1996) who suggests that subjects should not refer to the examples that
are anecdotal, but to the criteria that are conceptual. Indeed, the presence of the screenshot
examples could have influenced the criteria identification during this phase. In fact, the goal
was not to test the ability of recognition, but to check their understanding of the criteria. In
addition, with only two or three examples per sheet, it was not possible to cover all the possible
cases for a single criterion. The 30 experts were invited to review the 15 screen interfaces; in
each interface, they had to identify persuasive elements and match them with the set of
guidelines. They could freely consult the criteria documents. At the end of each session, the
interfaces were presented again to allow the participants to confirm or not their initial choice to
assign a criterion to an item on the interfaces.
In a third phase (confirmation), the interfaces were shown again in order to confirm or refute
participants’ first choice.
At the end of the session, the researchers performed a semi-structured interview to obtain
qualitative data on the structure and content of the document containing the experimental
criteria. The questions were open and encouraged greater freedom of speech and criticism from
the participants.
To sum up, the participants were asked to evaluate 15 interfaces with the checklist for
persuasive interfaces. For each screen interface, they had to find persuasive elements and
classify them according to the set of guidelines. For each subject, interfaces were randomly
presented, and there were no time limitations for each task.
Data Collection and Analysis
Two types of data were collected. First, we were interested in recording how much time
participants spent reading the criteria. The second measure was the number of persuasive
criteria correctly identified by the participants, which was separated into dynamic vs. static
criteria. We also calculated a score for the correct identification per interface and per subject.
Identification was considered as correct if the answer matched with the expected criteria (or
theoretical answer). If the answer did not match the expected criteria, it was considered false.
Quality of the criteria definition was evaluated through this assignment task, that is, if a
criterion was defined as correct or false. Proportions of correct identifications or false
identifications (i.e., confusions) were analyzed as discriminative indicators of the relevance of
the checklist.
121
Journal of Usability Studies Vol. 9, Issue 3, May 2014
Results and Analysis
The following sections present the overall scores, details about the criteria matrix and Kappa
score, and the quality of the criteria definitions.
Overall Scores
The average time to read the criteria document was 11 minutes 45 seconds (SD=4.2). The
average session time was 1 hour 6 minutes (SD=13). The overall scores of the confusion
criteria matrix show a high level of agreement among participants to identify persuasive
elements in interfaces. Table 2 shows the classification criteria according to the proportion of
correct identification.
Correct identifications reached the score of 78.8% on average. This result is encouraging
considering the short time that participants took to learn new criteria and considering that most
of the participants lacked prior knowledge of the technological persuasion theory. With an
acceptance of 50% as a threshold to determine the quality of definition (Bach, 2004), all the
criteria reached a good to a very good level of acceptance. Overall, static criteria were better
understood than the dynamic criteria. Another possible explanation for this finding is that static
elements are easier to identify on a static user interface compared to identifying dynamic
elements on a static interface. However, ascendency appears to be difficult to assess. This can
be explained by the fact that ascendency is a criteria that does not solely depend on the
interface itself, but also on behavioral patterns of the user that are acquired over time and that
cannot be readily observed in the context of this experiment.
We calculated a percentage of correct identification per screenshot (see Table 1). Most of the
screenshots performed very well (between 65% and 85%); however, 2 screenshots stand out:
screens 7 and 15.
Screen 7 is an advertisement received by email for a birthday promotional offer. All the
elements on the screen indicate that a user's perception is used to associate persuasive
attempts in the advertising area. The awareness is developed especially when the
delivery medium is email. A score of 92.5% shows a high capacity to detect this
particular format as a persuasive element.
Screen 15 is a dashboard screen of a role-playing video game. According to the
statistics displayed, it indicates that the user played a huge amount of hours in just few
days. In this case, the persuasive elements were not easily found by the participants
(52.22%). It indicates that in the video game area, the link with persuasion is not
obvious for the participants. This media is associated with fun and a degree of freedom.
Even if the risk of dependency is quite known.
Criteria Matrix: Agreement and Disagreement of Persuasive Elements
Table 2 reports the criteria matrix for the interface elements classification. Each row of the
matrix represents the instances in a predicted criterion category. For example, we assume that
the 15 interfaces include a total of 12 items relating to credibility. These 12 items were
presented to 30 experts, representing a total of 360 items that should be properly assigned to
the criterion of credibility. In reality, participants identified 293 items correctly, committed 53
errors (2 x privacy, 5 x personalization, 12 x attractiveness, 10 x solicitation, 10 x priming, 2 x
commitment, 2 x ascendency), and provided 14 non-assignments because they could not
categorize the item. This same row also shows how many participants made mistakes (the
statistics are placed after the icon; for instance, two participants classified two items of
presupposed credibility in privacy). The following is a detailed description of the data in the
table:
The first column represents the expected answers based on the amount of each
criterion potential for a correct match, as determined by the researchers. The bottom
numbers in this column represent the maximum possible number of correct
identifications.
The diagonal table cells (boldface) show the percentage of agreement between the
participants for persuasive elements for each of the eight criteria.
The score below the percentage (diagonal cells) represents the number, rather than the
percentage, of participant agreements for that criterion.
122
Journal of Usability Studies Vol. 9, Issue 3, May 2014
All of the other percentages in the table represent disagreement between the
participants’ characterization of each criterion.
The numbers in brackets presented in conjunction with the icons represent the number
of participants who contributed to the disagreement score.
The right column (Unidentified) contains elements that could not be matched with any
element in the identified elements or criteria.
The criteria matrix reveals that most of criteria (except ascendancy, see The Quality of Criteria
Definition section) are generally found in many interfaces. The 30 experts correctly identified
the elements of persuasion and were able to categorize persuasive criteria accurately. Inter-
ranking judgments were fairly homogeneous; the same features of persuasive interfaces were
named consistently.
Table 2. Criteria Matrix: Agreement and Disagreement of Persuasive Elements
Identified criteria by test participant
Credibility
Privacy
Personalization
Attractiveness
Solicitation
Priming
Commitment
Ascendency
Uniden tified
Expected Criteria
Credibility
81.39*%
0.5% (
2)
1.4% ( 3)
3.3% ( 9)
2.8% ( 9)
0.5% ( 2)
0.5% ( 2)
0.5% ( 2)
6.7% (4)
360
293
2
5
12
10
10
2
2
24
Privacy
3.3% ( 8)
90.67%
0% ( 0)
1.7% ( 4)
1.3% ( 3)
1.7% ( 5)
0.7% ( 2)
0% ( 0)
0.7% (2)
300
10
272
0
5
4
5
2
0
2
Personalization
1.5% ( 5)
3.1% (
10)
82.82%
1.8% ( 7)
1% ( 4)
3.3% ( 8)
1.5% ( 6)
1% ( 3)
3.8% (14)
390
6
12
323
7
4
13
6
4
15
Attractiveness
1.7% ( 4)
0% ( 0)
0.4% ( 1)
79.17%
5.8% ( 12)
5.4% (
13)
0.8% ( 2)
0% ( 0)
6.7% (11)
240
4
0
1
190
14
13
2
0
16
Solicitation
2% ( 1)
0.4% (
3)
3.8% ( 15)
6.2% ( 17)
75.30%
6.1% (
17)
2.3% ( 9)
2.6% ( 10)
1.4% (7)
660
13
3
25
41
497
40
15
17
9
Priming
1.7% ( 4)
2.9% (
7)
4.2% ( 9)
3.3% ( 6)
5.8% ( 9)
74.58%
4.2% ( 9)
0.4% ( 1)
2.9% (6)
240
4
7
10
8
14
179
10
1
7
Commitment
1.7% ( 4)
0.8% (
2)
3.3% ( 7)
2.1% ( 5)
6.7% ( 12)
2.9% (
6)
75%
3.7% ( 8)
3.7% (8)
240
4
2
8
5
16
7
180
9
9
Ascendency
0% ( 0)
1.1% (
1)
6.7% ( 6)
4.4% ( 4)
4.4% ( 4)
6.7% ( 6)
5.5% ( 5)
57.78%
13.3% (9)
90
0
1
6
4
4
6
5
52
12
* The diagonal table cells (boldface) show the percentage of agreement between the participants for
persuasive elements for each of the eight criteria. The score below the percentage (diagonal cells)
represents the number, rather than the percentage, of participant agreements for that criterion.
Kappa Score
Kappa is a statistical coefficient for assessing the reliability of agreement between a fixed
number of evaluators when assigning categorical ratings to a number of criteria. We used
Randolph’s kappa (2005), instead of Cohen's Kappa or Fleiss’ Kappa, because it gives a more
accurate and precise reliability measurement of the 30 experts' agreement. We used a free-
marginal version of kappa that is recommended when raters are not restricted in the number of
cases that can be assigned to each category, which is often the case in typical agreement
studies.
Randolph’s kappa is considered to be a good statistical measure of inter-rater agreement; this
measurement tool resulted in a score of 0.61 for our study. An indication of strong agreement
123
Journal of Usability Studies Vol. 9, Issue 3, May 2014
for a measure of quality of a classification system, according to Landis and Koch (1977), is
when an overall performance score is higher than 60%.
Finally we must remember that the kappa value measures the agreement among the 30
participants, independent of the two experts who determined which interfaces had which
persuasive elements.
The Quality of Criteria Definition
Table 2 also presents the classification criteria based on their average frequency of
misidentification and the proportion of correct identification. From this point of view, seven
criteria are considered to be well defined. These criteria were privacy, personalization,
credibility, attractiveness, solicitation, priming, and commitment. Among all the elementary
criteria, only one requires an improvement in definition: ascendency. These results are
particularly interesting when taking into account the short time taken by participants during the
learning phase (M = 11min 45s; S.D. = 4.2) and the unfamiliarity of the subjects with social
psychology.
There was some slight confusion for all of the criteria; however, misclassifications are always
very low, usually below 5%. In our study the misclassification score was sometimes slightly
higher and in the case of the ascendancy criteria was 13%. Ascendency is the criterion that
caused the most confusion and that needs the most adjustments. In fact, the ascendancy score
is linked both to confusion with other criteria (personalization, 6.7%; priming, 6.7%;
commitment, 5.5%) and to the lack of attribution; experts prefer to classify forms of
ascendancy as an unidentified criterion (13.3%). Other criteria show confusion too, due to the
high number of persuasive elements. Note also that the static criteria, which are directly
identifiable on the screens, cause less confusion than the dynamic criteria, which involve a time
aspect of persuasion.
Discussion and Conclusion
Computer ergonomics have produced a number of guidelines to measure the ergonomic quality
of products, technologies, and services. We seek to establish a tool—a checklist that is sound,
reliable, useful, relevant, and easy to use—to focus on the persuasive dimensions of interfaces
and their effects.
While many criteria appear in the literature, it is very important to structure them within a set
of guidelines or some other forms. Too many suggested criteria are not supported by further
information on its potential use and application by HCI experts. The fundamental advantage of
this checklist is its remarkable internal validation. The criteria for interactive persuasion in its
current form can be considered valid, reliable, and usable. The use of criteria enables the
identification and classification of elements at the source of social influence in interfaces. We
designed an experimental test to assess the relevance of our checklist by measuring confusion
scores—the lower the confusion score is, the better the set of guidelines. In addition to
improving the rate of elements found, the checklist also covers a larger number of elements
(regardless of the type of software, website, game, or mobile device to be evaluated.)
Let us summarize our main findings during the successive phases of refinement of the criteria
and the findings from the experiments. Firstly, the criteria were based on the selection and
classification of 164 publications published between 1994 and 2011, covering many sectors
(health, services, business, ecology, education, leisure, and marketing), and based on all types
of technologies (website, video games, phones, software, and ambient systems). Secondly, the
content analysis of these publications identified two dimensions of persuasion (static and
dynamic) and eight criteria that we have defined, explained, and exemplified. The criteria for
interactive persuasion emphasize the social and emotional dimensions of interfaces and, as
such, enhance the usual inspection criteria (clarity, consistency, homogeneity, compatibility,
usability, etc.). Thirdly, an experimentation of an internal validation of the criteria should show
that the criteria can be effectively identified in a realistic setting. In an allocation task, 30
specialists in ergonomic software correctly matched 78.8% of a series of 15 interfaces with 84
forms of persuasion (12 forms of credibility, 10 forms of privacy, 13 forms of personalization, 8
forms of attractiveness, 22 forms of solicitation, 8 forms of priming, 8 forms of commitment,
and 3 forms of ascendency). Accuracy measures are a little bit higher than those found by
Bastien (1996) and Bach (2004) when studying the ergonomic quality of inspection criteria.
124
Journal of Usability Studies Vol. 9, Issue 3, May 2014
Fourthly, the criterion of ascendency has a good score, but is lower than other criteria. Not
surprisingly ascendency is the last step of the engaging scenario. In many cases, ascendency is
not visible on the interface as it is the user who reflects the result of the persuasive influence.
Ascendency is a hard criterion to allocate because it also has a strong temporal incidence.
The influence of technology is a long-term process that does not only depend on the interface
but also on the user's personal characteristics. In regards to ascendency, the problem is to
identify the possible effects that go over and beyond interactions with the media: the user
persists in attitude or behavior, even outside his or her interaction with technology. The way to
identify ascendency implies making assumptions about links between elements of interfaces and
opportunities for acquiring and engaging the user. These diagnoses are not obvious to
ergonomists and interactive designers. For some of them, psychosocial aspects and notions
were not easily understood. Obviously, ascendency is the one that gets the lowest score in
terms of correct assignment. It represents the most difficult dynamic criterion because it is the
last of the engaging scenario. It marks the final step, the one where we obtain the change in
behavior or attitude expected. However, in most cases of persuasive processes, this result is not
directly observable through the interface, but in the real environment of the subject and
sometimes a long time after interactions with the media. We can imagine that this criterion
could be improved by combining it with a more clinical analysis. But, the objective for this
checklist is only to be integrated into the practice of HCI experts.
Criteria allow rapid identification of the most important problems; they can also identify a wide
variety of problems. That said, the experiment task implies that the identified criteria are
correlated with scenarios. This may be a drawback of the method that we used. Indeed, this
presupposes the existence of a relatively generic model of participant. Nevertheless, the
diversity of people and context of use (tested interfaces came from games, mobile phones,
business software, social networks, e-commerce, health, and transport) belies this narrow
conception of the human. Consequently, the support provided by persuasive criteria can
certainly be improved further by producing complementary methods. It seems important that
specialists’ inspection methods refine ways to integrate these persuasive criteria in assessment
practices (Mahatody, Sagar, & Kolski, 2010).
As with other guidelines, our persuasive set of guidelines could be useful in several effective
ways. An interface, for example, could receive a score of absence/presence per criterion (0 if
the criterion is not present, 1 if the interface matches the criterion) or a score on a scale (e.g.,
0 if the criterion is not present, 5 if the interface totally matches the criterion). A general score
could be calculated for several interfaces in order to compare them. The checklist could also
help the designer to detect strengths and weaknesses in his or her interface among the eight
criteria. For example, Brangier and Desmarais (2013, 2014) attempted to design a more
motivating and engaging e-learning applications through persuasive guidelines. They conducted
a persuasiveness assessment with our set of guidelines of an existing e-learning application
intended for self-regulated learning of middle school mathematics (Figure 14). It showed that
the inspection could reveal factors that explained the low engagement observed as the
e-learning application was used and usage data was recorded.
Becoming a social actor, technology is not only shaping information, but it also reshapes our
behaviors, perceptions, and attitudes towards desired goals, often without the awareness of the
user. Thus, persuasion is effectively inseminating the field of HCI and thereby renewing the
concepts, methods, and tools of interactions analysis, leading us to develop a new set of criteria
for the design and evaluation of persuasive interactions, which aim to provide user experience
designers with more knowledge to develop persuasive interfaces, while respecting users’ ethics.
125
Journal of Usability Studies Vol. 9, Issue 3, May 2014
Figure 14. Example of a screen inspected with the set of guidelines for persuasiveness in
interfaces. (Source: http://www.polymtl.ca/, Brangier and Desmarais 2013, 2014).
Tips for Usability Practitioners
Our approach is to define guidelines to measure and assess the persuasive dimensions of user
experience. The following tips are recommended for those who are studying persuasive
systems:
In addition to accessibility, usability, and emotional criteria, HCI experts can provide
efficient and effective guidance to gauge how interfaces could be persuasive. A set of
persuasive guidelines should be used by UX designers in interface evaluation and
design to improve the psychosocial aspect of the interfaces.
The set of eight criteria is well identified by HCI experts. Consequently, it encourages
the use of credibility, privacy, personalization, attractiveness, solicitation, priming,
commitment, and ascendency criteria to understand how persuasive elements work in
interfaces. The importance of persuasive elements in interface design can be an
important factor in developing or evaluating a study that relies on that technology.
In addition to the criteria of clarity, consistency, brevity, adaptability, etc., persuasive
guidelines should/could also be included in the training of UX designers.
Acknowledgments.
The authors would like to thank SAP (Front de Seine - 157-159 rue Anatole France - 92309
Levallois-Perret cedex) and especially Steve Kopp, Christophe Favart, and Chahab Nastar, PhD
for their involvement and for their numerous insightful comments on this research.
Special thanks to the three reviewers for their constructive comments on this paper.
126
Journal of Usability Studies Vol. 9, Issue 3, May 2014
References
Adams, M.A., Marshall, S.J., Dillon, L., Caparosa, S., Ramirez, E., Phillips, J., & Norman, G.J.
(2009). A theory-based framework for evaluating exergames as persuasive technology.
Proceedings of the 4th International Conference on Persuasive Technology. New York, NY :
ACM.
Bach, C. (2004). Élaboration et validation de Critères Ergonomiques pour les Interactions
Homme-Environnements Virtuels. Thèse de Doctorat, Université Paul Verlaine, Metz.
Bach, C., & Scapin D. L. (2003). Adaptation of ergonomic criteria to human-virtual
environments interactions. In Interact’03 (pp. 880-883). Zürich, Switzerland): IOS Press.
Bach, C., & Scapin, D.L. (2010). Comparing inspections and user testing for the evaluation of
virtual environments. International Journal of Human-Computer Interaction, 26(8), 786-
824.
Baranowski, T., Buday, R., Thomson, D., & Baranowski J. (2008). Playing for real video games
and stories for health-related behavior change. Am J Prev Med, 34(1), 74-82.
Bart, Y., Venkatesh, S., Fareena, S., & Urban, G.L. (2005). Are the drivers and role of online
trust the same for all web sites and consumers? A large-scale exploratory empirical study,
Journal of Marketing, 69, 133-52.
Bastien, J. M. C (1996). Les critères ergonomiques: Un pas vers une aide méthodologique à
l’évaluation des systèmes interactifs. Thèse de Doctorat, Université René Descartes, Paris.
Bastien, J. M. C., & Scapin, D. L. (1992). A validation of ergonomic criteria for the evaluation of
human-computer interfaces. International Journal of Human-Computer Interaction, 4 (156),
183-196.
Bergeron, J., & Rajaobelina, L. (2009). Antecedents and consequences of buyer-seller
relationship quality in the banking sector. The International Journal of Bank Marketing, 27
(5), 359-380.
Brangier, E., & Desmarais, M. (2013). The design and evaluation of the persuasiveness of e-
learning interfaces. International Journal of Conceptual Structures and Smart Applications.
Special issue on Persuasive Technology in Learning and Teaching. 1(2), 38-47.
Brangier, E., & Desmarais, M. (2014). Heuristic inspection to assess persuasiveness: a case
study of a mathematics e-learning program. In A. Marcus (Ed.): DUXU 2014, Part I.
Springer International Publishing. LNCS 8517, pp. 425–436.
Chaiken, S. Liberman, A., & Eagly, A.H. (1995). Heuristic and systematic processing within and
beyond the persuasion context. In J.S. Uleman, J.A. & Bargh,(Eds.), Attitude Strength:
Antecedents and Consequences (pp. 387-412). NJ: Erlbaum.
Consolvo, S., Everitt, K., Smith, I., & Landay, J.A. (2006). Design requirements for technologies
that encourage physical activity. Proceedings of the ACM Conference on Human Factors in
Computing systems , CHI 2006 (pp. 457-466). New York, NY: ACM.
Dhamija, R., Tygar, J.D., & Hearst, M. (2006). Why phishing works. Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems, CHI 2006 (pp. 581-590). New York,
NY: ACM.
Fogg, B. J. (2003). Persuasive technology: Using computers to change what we think and do.
San Francisco: Morgan Kaufmann.
Greenwald, A.G., & Banaji, M.R. (1995). Implicit social cognition: Attitudes, self-esteem and
stereotypes. Psychological Review, 102, 4-27.
Huang, X. (2009). A review of credibilistic portfolio selection. Journal Fuzzy Optimization and
Decision Making archive, 8(3), 263-281
Khaled, R., Barr, P., Noble, J., & Biddle, R. (2006). Investigating social software as persuasive
technology. Proceedings of the First International Conference on Persuasive Technology for
Human Well-Being, Persuasive06. Berlin Heidelberg: Springer.
127
Journal of Usability Studies Vol. 9, Issue 3, May 2014
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical
data. Biometrics, 33, 159–174.
Liu, C., Marchewka, J., Lu, J., & Yu, C. (2005). Beyond concern: A privacy-trust-behavioral
intention model of electronic commerce, Information and Management, 42, 289-304.
Lockton, D., Harrison, D., & Stanton, N. (2010). The design with intent method: A design tool
for influencing user behaviour. Applied Ergonomics, 41(3), 382-392.
Mahatody, T., Sagar M., & Kolski C. (2010). State of the art on the cognitive walkthrough
method, its variants and evolutions. International Journal of Human-Computer Interaction,
26(8), 741-785.
McGuire, W.J. (1969). Attitude and attitude change. In G. Lindzey & E. Aronson (Eds.),
Handbook of social psychology (2nd. ed., pp. 136-314) Reading, MA: Addison-Wesley.
Nass, C., Fogg, B.J., & Moon, Y. (1996). Can computers be teammates? International Journal of
Human-Computer Studies, 45(6), 669-678.
Némery, A. (2012). Élaboration, validation et application de la grille de critères de persuasion
interactive. Thèse de Doctorat, Université de Lorraine, Metz. http://tel.archives-
ouvertes.fr/tel-00735714
Némery, A., Brangier, E., & Kopp, S. (2009). How cognitive ergonomics can deal with the
problem of persuasive interfaces: Is a criteria-based approach possible? In L. Norros, H.
Koskinen, L. Salo, & P. Savioja. Designing beyond the product: understanding activity an
user experience in ubiquitous environments (pp. 61-64). ECCE’2009, European Conference
on Cognitive Ergonomics. New York, NY: ACM.
Némery, A., Brangier, E., & Kopp, S. (2010). Proposition d’une grille de critères d’analyses
ergonomiques des formes de persuasion interactive In B. David, M. Noirhomme et A. Tricot
(Eds), Proceedings of IHM 2010, International Conference Proceedings Series (pp. 153-
156). New York, NY: ACM.
Némery, A., Brangier, E., & Kopp, S. (2011). First validation of persuasive criteria for designing
and evaluating the social influence of user interfaces: Justification of a guideline. In A.
Marcus (Ed), Design, User Experience, and Usability (pp.616-624), LNCS 6770.
Oinas-Kukkonen, H. (2010). Behavior change support systems: Research agenda and future
directions. Lecture Notes for Computer Science, Persuasive2010, 6137, 4-14.
Oinas-Kukkonen, H., & Harjumaa, M. (2009). Persuasive systems design: Key issues , process
model, and system features. Communications of the Association for Information Systems,
24(1), 485-500.
Peppers P., & Rogers M. (1998). Better business one customer at a time. The Journal For
Quality and Participation, Cincinnati, 21(2), 30-37.
Petty, R.E., & Cacioppo, J.T. (1981). Personal involvement as a determinant of argument-based
persuasion. Journal of Personality and Social Psychology, 41, 847-855.
Pine II, B.J. (1993). Mass Customization. Boston: Harvard Business School Press.
Randolph, J. J. (2005, October 14-15). Free-marginal multirater kappa: An alternative to Fleiss'
fixed-marginal multirater kappa. Paper presented at the Joensuu University Learning and
Instruction Symposium 2005, Joensuu, Finland.
Redström, J. (2006). Persuasive design: Fringes and foundations. Persuasive 2006, LNCS 3962,
pp. 112 – 122.
Tørning, K., & Oinas-Kukkonen, H. (2009). Persuasive system design: State of art and future
directions. ACM International Conference Proceeding Series, Proceedings of the Fourth
International Conference on Persuasive Technology, Claremont, CA, USA. New York, NY:
ACM.
Weiksner, G. M., Fogg, B. J., & Liu, X. (2008). Six patterns for persuasion in online social
networks. In Proceedings of the 3rd international conference on Persuasive Technology.
Persuasive '08. Berlin Heidelberg: Springer.
128
Journal of Usability Studies Vol. 9, Issue 3, May 2014
Yang, K.C.C. (2005). The influence of humanlike navigation interface on users’ responses to
Internet advertising. Telematics and Informatics, 23, 38-55.
About the Authors
Alexandra Némery
Dr. Némery completed a
PhD in Ergonomics of
Persuasive Systems. She
is working as a User
Experience Manager at
Sage Company in Paris.
She’s also teaching
Human Computer
Interaction at a Computer
Engineering School. She
is an expert in human-
computer interaction, web
usability, strategy and
marketing.
Eric Brangier
Pr. Dr. Brangier is Full
Professor at the
University of Lorraine.
His field of expertise is
in the domains of
Cognitive Ergonomics,
Human-Computer
Interactions, Prospective
Ergonomics and
Psychology of the user.
He has also been
working as an expert for
different institutions in
France, Canada and
Belgium.
