Understanding of Return to Player messages: Findings from user testing.

Full text

Understanding of
Return to Player
messages
Findings from user testing
Authors: Debbie Collins, Sophie Green, Jo d’Ardenne , Heather Wardle & Shauna-Kaye
Williams
Date: October 2014
Prepared for: xxx

At NatCen Social Research we believe
that social research has the power to
make life better. By really understanding
the complexity of people’s lives and what
they think about the issues that affect
them, we give the public a powerful and
influential role in shaping decisions and
services that can make a difference to
everyone. And as an independent, not for
profit organisation we’re able to put all
our time and energy into delivering social
research that works for society.
NatCen Social Research
35 Northampton Square
London EC1V 0AX
T 020 7250 1866
www.natcen.ac.uk
A Company Limited by Guarantee
Registered in England No.4392418.
A Charity registered in England and Wales (1091768) and Scotland (SC038454)

Contents
Acknowledgements ........................................................... 1
Executive summary ........................................................... 2
1 Background ................................................................. 5
1.1 Background and purpose ........................................................................ 5
1.2 Policy Context ......................................................................................... 6
1.3 Gambling machines in Great Britain ....................................................... 7
2 Study Methodology ..................................................... 9
2.1 Data Collection ........................................................................................ 9
2.1.1 Ethical approval and protocol ................................................................... 9
2.1.2 Recruitment ........................................................................................... 10
2.1.3 Fieldwork protocols ................................................................................ 13
2.1.4 Analysis ................................................................................................. 15
2.2 Study limitations .................................................................................... 16
2.3 Report structure .................................................................................... 17
3 Research context ...................................................... 18
4 Return to player message ......................................... 19
4.1 The message tested ............................................................................. 19
4.2 Understanding of the return to player message .................................... 19
4.2.1 Those who understood the message correctly ....................................... 20
4.2.2 Those who misunderstood the message ................................................ 21
4.2.3 Those who did not understand the message ......................................... 23
4.3 Perceived utility of the return to player message .................................. 23
5 RTP messages for different stakes ........................... 26
5.1 The message tested ............................................................................. 26
5.2 Comprehension of the message on different rates for different stakes . 26
5.2.1 Interpretation of the message ................................................................ 26
5.2.2 Understanding the term ‘stake’ .............................................................. 27
5.3 Perceived utility of the message on different rates for different stakes . 27
6 Messages on ‘compensated’ and ‘random’ games ... 28
6.1 The messages tested ............................................................................ 28
6.2 Comprehension of ‘this game is compensated’..................................... 28
6.3 Comprehension of ‘this game is random’ .............................................. 29

6.4 Differentiation between ‘compensated’ and ‘random’ games ................ 30
6.5 Perceived utility of messages on ‘compensated’ versus ‘random’ games31
7 Game features and the odds of winning message .... 32
7.1 The message tested ............................................................................. 32
7.2 Comprehension of message ................................................................. 32
7.3 Perceived utility of the messages .......................................................... 33
8 Player’s views on alternative messaging .................. 35
8.1 What types of messages, if any, do you think should be on machines to
explain win chances to player? ............................................................. 35
8.2 The odds of winning any prize per game .............................................. 36
8.3 The odds of winning the jackpot prize per game ................................... 37
8.4 The odds of winning prizes of different sizes ........................................ 38
8.5 The average hourly loss rate................................................................. 38
8.6 The total amount of money you have put in the machine in your gaming
session ........................................................................................... 39
8.7 Should money put in include prize money that has been re-staked using
credit transfer or not .............................................................................. 40
8.8 A record of the amount you have put in a single gaming session minus the
money you have in the bank ................................................................. 40
8.9 Other types of message about win chances that could be useful for game
players ........................................................................................... 40
9 Discussion and implications ...................................... 42
9.1 Understanding return to play messaging .............................................. 42
9.2 Use of the message .............................................................................. 43
9.3 Alternative messaging ........................................................................... 43
9.4 Implications for responsible gambling messaging ................................. 44
9.5 Recommendations for further research ................................................. 45
References .................................................................... 47
Appendix A. Recruitment documents .............................. 49
Appendix B. Interviewer protocol ..................................... 52
Appendix C. Showcards .................................................. 59
Tables
Table 1.1 Gaming machine types and prizes ........................................................... 8
Table 2.1 Split of interviews by recruitment activity ................................................ 11
Table.2.2 Sex, age and highest education qualification of study participants ........... 12
Table.2.3 Sex, age and highest education qualification of study participants ........... 12
Figure 6.1 Differences in understanding between ‘compensated’ and ‘random’ games
among participants ................................................................................ 30
Figure 6.2 Suggestions to improve the clarity of the ‘compensated’ and ‘random’ game
descriptions ........................................................................................... 31

NatCen Social Research | Understanding of Return to Player messages
1
Acknowledgements
The authors are grateful to Stuart McPhee, Peter Rangeley, Nigel Owen and
Amanda Fox at the Gambling Commission and Jonathan Parke at the
Responsible Gambling Trust for their support and advice on technical matters,
and to Professor Ladouceur and Professor Blaszczynski for their comments on
an earlier draft of this report. We are also grateful to Coral, Praesepe and Rank
for allowing us access to their venues and helping us with recruitment of study
participants. Most importantly we would like to express our gratitude to all
those people who took part in this research.

2
NatCen Social Research | Understanding of Return to Player messages
Executive summary
This research, commissioned by the Responsible Gambling Trust, explored
machine players’ understanding of the ‘return to player’ (RTP) messages
displayed on gaming machines. These messages advertise what proportion of
money is returned to players in prizes and form part of a package of measures
to promote responsible gaming. The Gambling Commission has expressed
concern that these messages are not understood by players. A small-scale,
qualitative study, using cognitive interviewing methods was undertaken to
explore players’ understanding and perceived utility of RTP information,
involving face-to-face interviews with 25 players.
Understanding of RTP messages
Evidence from this study suggests that current RTP messages are not well
understood for a number of reasons, supporting the Gambling Commission’s
concerns:
 Messages use technical language that does not hold the same meaning
for the general population as industry specialists.
 Messages use complex terms that have ambiguous or unclear meaning.
 The provision of messages in English only adds to difficulties with
understanding them for those for whom English is a second language.
 The use of mathematical concepts and language such as ‘average’
payout, ‘random’ payout schedule and the expression of win chance as
a percentage assume a level of mathematical literacy that some players
do not possess.
For some, this lack of understanding promoted confusion and or mistrust of
both the industry and regulator. Moreover there was some evidence to suggest
that the current messages are being interpreted by some as meaning players
are going to win. This is of concern as it may indicate that current messages
rather than encouraging responsible gambling behaviour are in fact
encouraging some people to continue to play beyond their limits.
Perceived utility of RTP messages
Participants in this study tended not to have seen or noticed the RTP
messages prior to interview and no participant reported that RTP messages
influenced their choice of machine. Perceived utility, post study exposure to
the messages, was influenced by player’s views on whether they thought the
messages would have an influence on their own behaviour or that of others.
Participants were largely split between those who felt RTP messages imparted
useful information and those who did not feel they were useful. Typically, those
who felt that messages were not useful were participants who thought that the
RTP messages were unclear and therefore unhelpful. Those who did not
understand the messages tended to express that they felt them to be
misleading. Some participants were concerned that the messages would be

NatCen Social Research | Understanding of Return to Player messages
3
misinterpreted by players and that this could encourage them to gamble more
than intended. This was especially true for the message around how RTP
varies by stake, with some participants expressing concern that this would
lead to players increasing their stake.
However, some participants did say they felt the messages could be useful in
helping them to decide which machine or game to play. These were people
who tended to understand the meaning of the messages.
Alternative messaging
The Gambling Commission considers that players may benefit from greater
transparency around a range of other game and machine features. To explore
this, participants were presented with a range of alternative messages ranging
from the odds of winning any prize/jackpot/different prize levels to the average
hourly loss rate and variations showing the amount of money spent.
Views were mixed with regards to whether these messages would be useful or
not. There was no clear consensus over which messages were felt to be most
important or most useful. Participants were split as to whether they felt
monitoring how much money they had spent in a session was useful or not.
Learning points and further research
This study provides some useful insights that could be taken forward.
 Messages should be clear, more direct and use less complex
terminology.
 Both the regulator and industry should think more carefully about how
they communicate with those for whom English is not their first
language. For example, if venues are in areas with high proportions of
minority ethnic groups, then signage in appropriate languages should be
considered.
 This study suggests people may not notice or use the RTP messaging.
For any kind of responsible gambling messaging to be effective, people
have to see it, understand it, reflect on the information conveyed in the
message and then, hopefully, translate this into changed behaviour (in
the direction intended).
 RTP messages may not influence players’ behaviour at all or in the way
intended. For example, whilst some participants in this study felt that
knowing more about the total amount of money spent would be useful,
others felt this could encourage chasing behaviour - the direct opposite
of what this feature intends. The wording of messages needs to be
carefully formulated and tested to ensure that they influence players

4
NatCen Social Research | Understanding of Return to Player messages
behaviours in the ways intended and do not have unintended
consequences.
However this research was small-scale and further research is necessary to
corroborate this study’s findings. Specifically research should further:
 establish the extent and nature of misunderstandings, and how these
vary across the wider population and between different subgroups;
 explore whether players actually look at messages in a ‘real’ play
environment and if so in what circumstances, and identify the features of
message presentations that encourage participants to read them; and
 further test alternative wordings of RTP messages, to assess their
performance in terms of being of interest to players, being correctly
understood, and in influencing players’ behaviour as intended.

NatCen Social Research | Understanding of Return to Player messages
5
1 Background
1.1 Background and purpose
This project was commissioned by the Responsible Gambling Trust (RGT) in
April 2014 to explore machine players’ understanding of the ‘return to player’
(RTP) messages displayed on gaming machines. RTP messages are required
to be displayed on all gaming machines in Great Britain. These messages
provide information to players as to the proportion of money paid to use the
machine that will, on average, be returned to players by way of prizes over a
large number of games. These messages are intended to help consumers
choose which machine or game to play, within the consumer choice paradigm
(Blaszczynsk et al, 2008). There is, therefore, an implicit acknowledgement that
these messages should influence behaviors by helping consumers better
understand the potential outcomes of their gambling engagement.
Gaming machines have to include information about the proportion of total
stakes returned in prizes on a machine, see section 1.2 for more details. The
percentage RTP is generally measured over 10,000 or more game plays for
compensated machines and over 100,000 or more game plays for random
machines
1
. With this concern in mind this research was designed, in
agreement with the RGT, to explore players’ understanding of the RTP
information.
The specific aims of this research were to:
 explore players’ understanding of key concepts used in the RTP
information;
 explore the factors that may affect players’ understanding of the RTP
message; and
 collect information that will inform alternative forms of wording on RTP.
This research forms part of the RGTs broader research into category B gaming
machines. Therefore, only category B machines were included in this project
(see section 1.3 for an overview of machine categories in Great Britain).
However, similar messages are used on all other categories of machines and
therefore findings from this project should be applicable to them also. In Great
Britain, two types of gaming machine are available: one based on random
1
Machines operating on a compensated payout schedule vary the chance of winning based on
the outcome of previous play. However, the prize distribution is still determined by chance. In
contrast, on machines operating on a random payout schedule, the odds of winning remain
constant, and are not affected by previous wins or losses on machines.

6
NatCen Social Research | Understanding of Return to Player messages
probability event outcomes and one based on compensated payout methods.
Both types of machines were included in this research.
The project design focused on exploring machine players’ understanding of
RTP messages with actual machine players. The need for research into
machines to better replicate real play, using real life gamblers, was noted by
the Responsible Gambling Strategy Board in their submission to the
Department of Culture, Media and Sport’s triennial review of machine stakes
and prizes (RGSB, 2013).
1.2 Policy Context
As mentioned in section 1.1 gambling machines have to include information on
the proportion of total stakes returned in prizes, or the odds of winning prizes
from play on a machine known as the ‘return to player’ (RTP). This is required
by regulation
2
, and the Gambling Commission’s gaming machine technical
standards require the provision of such information in standard forms, through
statements relating to the ‘return to player’ (RTP) or ‘average payout
percentage’
3
. The typical form in which the information is provided is as
follows:
‘This machine has an average percentage payout of at least 80%’
The percentage quoted in this message varies depending on the
characteristics of the game. The Gambling Commission’s technical standards
also mandate a range of other information to be provided to the player, based
on type of machine/game being played. This includes what category of
machine it is (or where there are different categories of game within one
machines, what category the game is) and whether the machine is based on
random or compensated payout schedules.
Currently players are presented with gaming machines which look similar and
may be described in an almost identical fashion but operate in a very different
manner, depending on game design. This variability in game design poses a
challenge in terms of how best to provide information on the RTP. The
Gambling Commission consider that players may benefit from greater
transparency around:
 the ‘cost of using the machine’;
 the (hourly) loss rate;
2
The Gaming Machines (Circumstances of Use) Regulations 2007. Regulation 2 requires a gaming machine to
display information about the proportion of amounts paid to use the machine that is returned by way of
prizes; or about the odds of winning prizes from use of the machine. Regulation 3 requires a notice to be
displayed on the machine at all times when it is available for use, indicating where the above
information can be found, and that the information is readily accessible by a person using the machine.
3
Gambling Commission gaming machine technical standard 8.3, Display Notice Requirements

NatCen Social Research | Understanding of Return to Player messages
7
 the volatility of the game;
 win/loss possibilities over a typical gambling session
4
;
 greater transparency in respect to game features in terms of how
they could affect the RTP; and
 RTP – more accurately reflected by the above in combination.
To date, no research has been undertaken in the UK to explore how players, in
real gambling settings, understand the current RTP messaging. This needs to
be explored before further changes, such as those suggested by the Gambling
Commission, are made. This research aims to fill this gap, providing practical
and useful information about what players do and do not understand about
RTP messaging and providing suggestions for improvements.
1.3 Gambling machines in Great Britain
Great Britain has a complex array of different types of gambling machines
available to the public. They are categorised into 11 different types by the
Gambling Commission (the industry regulator). This information is summarised
in table 1.1.
4
‘Typical gambling session’ in this context may need to be specific to the group at which the machine
would generally be aimed as it will vary across different venues

8
NatCen Social Research | Understanding of Return to Player messages
Table 1.1 Gaming machine types and prizes
Category
Current
max stake
Current
max prize
New
max stake
New
max prize
B1
£2
£4000
£5
£10000*
B2
£100
£500
£100
£500
B3
£2
£500
£2
£500
B3A
£1
£500
£2
£500
B4
£1
£250
£2
£400
C
£1
£70
£1
£100
D non-money prize (not
crane grab)
30p
£8
30p
£8
D non-money prize (crane
grab)
£1
£50
£1
£50
D money prize
10p
£5
10p
£5 N
D combined money and
non-money prize (coin
pusher/penny falls)
10p
£15 (of which
no more than
£8 may be
money prize)
20p
£20 (of which no
more than £10
may be a money
prize)
D combined money and
non-money prize (other
than coin pusher or
penny falls)
10p
£8 (of which no
more than £5
may be a
money prize)
10p
£8 (of which no
more than £5
may be a money
prize
*with option of max £20,000 linked progressive jackpot on premises basis only
Different categories of machine are available in different venues, according to
the licensing regulations set out by the Gambling Commission. In bookmakers,
adult gaming centres and casinos (the venues used for this research, see
Section 2.1.2) machine categories B to D are permissible. In practice,
bookmakers tend to have machines which offer B2 and B3 games, casinos
tend to have B1 and B3 machines and AGCs have a range of B3 to category D
machines (the precise composition can vary from venue to venue).

NatCen Social Research | Understanding of Return to Player messages
9
2 Study Methodology
This study employed a qualitative method – that of cognitive interviewing – to
explore real machine players’ understanding of the RTP message. Cognitive
interviewing methods explore what participants think about as they complete a
task or review information - in this case the RTP message. Information on
cognitive processes is gathered using a combination of think aloud and
probing techniques (see for example, Beatty and Willis, 2007; Collins, 2003;
Willis, 2005). The sampling strategy used for this study was purposive and
findings are based on a thematic analysis of the interview data using the
Framework method (Spencer et al, 2003).
The purpose of this study is to explore machine players’ understanding of the
RTP message and to elicit in-depth accounts regarding why any confusion
occurred from a player’s perspective. This study is not designed to address
how often players actually read RTP messages or estimate the prevalence of
misunderstandings that occur in practice.
In this report we document the number of study participants who
misunderstood the RTP messages or who encountered particular types of
misunderstanding, as a means of describing the patterns observed. However,
these numbers are not indicative of how frequently problems will occur in real
life as the purposive sampling methods we used for this study are not designed
to produce a statistically representative sample from which robust population
estimates can be derived. In the rest of this chapter we describe in more detail
the methodology used for this study.
2.1 Data Collection
2.1.1 Ethical approval and protocol
All stages of this research were approved by NatCen’s Research Ethics
Committee (REC), which includes external experts and specialists in
methodology. In obtaining ethical approval for this study, a pre-specified
research protocol was set out and agreed. Any subsequent amendments to
this protocol were resubmitted and approved by the REC.
On the day of the research, a full description of what the research involved was
given to participants prior to interview. Participants had the opportunity to ask
any questions about the research and were only included in the project if they
provided verbal consent (all did so).

10
NatCen Social Research | Understanding of Return to Player messages
2.1.2 Recruitment
Participants were recruited via gambling venues as this offered a cost effective
means of recruitment. We also felt that testing the RTP messages within a
gambling environment familiar to players would provide a more realistic test
setting, albeit with interviews taking place in a private location within the venue.
Gambling venues were recruited first. Once this was done, participants who
actively use these venues were then recruited.
Recruiting venues
Venues were recruited through our network of contacts with industry members.
This involved liaising with a number of operators, explaining the purpose of the
research and outlining our requirements to negotiate access to the venues.
There was a great deal of willingness to co-operate but, for some operators,
other considerations prohibited full support when deciding whether to grant
access to venues or not. Coral, Praesepe and Rank were the operators who
agreed to grant access to their venues and ultimately supported this research.
They provided the actual venues in which the research was conducted,
supported us in recruiting participants and provided general oversight and
points of clarification about their population of machine players. These
operators, whilst providing support when needed, also understood that the
research should be conducted independently and that our design should not
be influenced by commercial considerations. They respected the views and
directives of NatCen’s Research Ethics Committee and adhered to these
protocols.
The venues offered by Coral, Praesepe and Rank were two bookmakers in city
centre locations, an adult gaming centre at a busy town centre location and an
edge-of town casino respectively. All venues include category B machines. The
findings from this research are based of machine players at these four venues.
No other venues were approached.
Recruiting participants
We used the same recruitment method in all venues to make contact with
potential participants who played gaming machines. The process involved the
research team working closely with venue staff, who acted as gatekeepers.
The venue staff informed their customers about the opportunity to participate
by putting up posters, handing out flyers and telling customers about the
research. This method meant that the research team was dependent on the
venue staff advertising the research. Would-be research participants were
asked to contact the research team using a freephone number or email, who
provided them with some background information about the study and asked
them a short screening questionnaire to ensure eligibility for the study. Copies
of the recruitment materials are included in Appendix A. In addition, on the day

NatCen Social Research | Understanding of Return to Player messages
11
the research took place the research team was, with the prior agreement of the
venue manager, allowed to attempt to recruit machine players to boost
numbers where this was necessary. This additional recruitment activity
occurred in all three organisations’ venues. Table 2.1 shows the split of
interviews across venues by recruitment activity.
Table 2.1 Split of interviews by recruitment activity
Recruitment method
Bookmakers
AGC
Casino
Pre-booked
4
5
2
On the day
5
6
3
Total
9
11
5
Qualitative sampling approach
This study involved a qualitative sampling approach. Unlike quantitative
sampling methods, which are concerned with producing statistical estimates of
the prevalence of characteristics or phenomena of interest to the study in the
wider target population, qualitative sampling methods involve the study of far
fewer people, but explore in more depth those individuals, settings,
subcultures, and scenes, so as to generate a deeper understanding of
individual perspectives, understandings and behaviours.
In contrast to the probability sampling techniques used in quantitative studies,
qualitative studies, including those using cognitive interviewing methods,
deploy purposive sampling approaches which involve the development of a
framework of the characteristics that might influence an individual's
contribution. The choice of framework characteristics is based on the
researcher's practical knowledge of the research area and available literature
and evidence, and participants are recruited based on these characteristics
(see for example, Bryman, 2012; Ritchie et al, 2003).
Profile of participants
A purposive sample of 25 participants was recruited from all three venues. The
recruitment strategy was designed to include the diversity of machine players
in each venue with respect to age, sex, highest qualification, length and
frequency of play. However, recruitment was limited by a) the venues
conducting the recruitment process, b) clientele frequenting the four venues
during the recruitment and fieldwork period, and c) those who agreed to
participate. The profile of participants is detailed in the following tables.

12
NatCen Social Research | Understanding of Return to Player messages
Table.2.2 Sex, age and highest education qualification of study participants
Venue
Bookmakers
AGC
Casino
Total
Sex
Male
8
5
1
14
Female
1
6
4
11
Age
18-30
1
1
0
2
31-50
5
6
2
13
51+
2
3
3
8
Not asked
1
1
0
2
Highest qualification
A level or above
5
3
2
10
GCSE or below
3
7
3
13
Not asked
1
1
0
2
Total
9
11
5
25
Table.2.3 Sex, age and highest education qualification of study participants
Venue
Bookmakers
AGC
Casino
Total
Time been playing
Less than 6 months
0
1
0
1
More than 6 months
9
10
5
24
Frequency of play
At least once a week
7
10
4
21
At least once a month
1
1
1
3
Less often than once a
month
1
0
0
1
Total
9
11
5
25
As can be seen from table 2.2 we interviewed a mix of men and women with
different levels of educational attainment. However, our study did not include
many younger players – those aged 18-30 and particularly young men, who we
know from survey evidence to be at greatest risk of having problem gambling
behaviours.
5
Another limitation of our study is that it included only one player
who had been playing for less than 6 months, see table 2.3. Our participants
tended to be experienced players, who in some cases had been playing for
decades. These limitations are discussed in section 2.2 and should be borne in
mind when considering the study’s findings.
5
The Health Survey for England 2012 showed that problem gambling rates among men aged 16-24 who
played fruit machines were 2.2% falling to 0.3% for men aged 75 and over (Wardle & Seabury, 2012).

NatCen Social Research | Understanding of Return to Player messages
13
2.1.3 Fieldwork protocols
Before each interview, a member of the research team spoke to the participant
individually about the study and answered any questions he or she had about
the study. Once the participant fully understood the study and what taking part
involved, verbal consent was gained. If the team member believed that the
person did not understand the study or felt coerced or influenced in any way to
take part in the study they were not interviewed. Some basic demographic
information was collected at the recruitment stage. Everyone who took part in
the interviews was given a £20 high street voucher, a thank you letter, and a
leaflet providing them with details of organisations that they could approach,
should they wish to, for further information about problem gambling and
support services.
The research involved face-to-face in-depth interviews with one of four
members of the NatCen research team
6
. All the interviews took place in
designated areas in the gambling venue to ensure privacy and confidentiality at
all times, for example in office spaces. The interviews were audio recorded,
with the participant’s consent, using encrypted digital recorders. At the start of
the interview, the interviewer explained the structure and the purpose of the
interview. The interviewer then showed the participant each card, containing an
RTP message, in turn. The interviewer used an interview protocol, which
contained instructions on when to use think aloud and when to probe, using
scripted probes, to follow up on specific elements. Interviews varied in length,
from 20- 50 minutes depending on how effusive the participant was and the
time available for the interviews.
Interview protocol
An interview protocol was developed and used with all participants. In the
interview, participants were shown a set of five different messages on win
chances, which are included in different RTP messages.
 Message 1 showed a ‘basic’ RTP message.
 Messages 2a and 2b showed information indicating whether a machine
was compensated or random.
 Message 3 showed an explanation that the odds of winning are not
indicated by the game features or display.
 Message 4 showed a variation of an RTP message where the average
percentage pay out varied depending on the stake size.
6
The fieldworkers for this study were Debbie Collins, Sophie Green and Jo d’Ardenne, who have
between them 30 years’ experience in undertaking cognitive interviews, and Kate Green a student
working on a 12 month placement at NatCen, who had received training in cognitive interviewing
methods and worked on several previous studies. All fieldworkers met to walk through the interview
protocol in advance of fieldwork commencing and followed a written protocol.

14
NatCen Social Research | Understanding of Return to Player messages
Each message was displayed on a card and shown to the participant in turn.
Following an introductory exercise to explain think-aloud techniques,
participants were asked to think aloud when reading each message.
Interviewers then asked a series of follow up cognitive probes following the
think aloud. These interviewing techniques are described in more detail in the
next section.
Appendix A contains a copy of the interview protocol and the probes used.
Cognitive interviews are semi-structured and interviewers were encouraged to
supplement the protocol with spontaneous probes in order to fully explore any
issue that arise.
Think aloud
Think aloud, or verbal protocol, is a method that requires participants to talk
aloud while solving a problem or performing a task. It was originally developed
by Ericsson and Simon (1980) and has been widely applied in cognitive
psychology research. Think aloud has become an established means of
observing different forms of behaviour requiring individuals to verbalise their
thought processes and actions (Gray & Wardle, 2013).
Think aloud can be carried out in two ways:
 concurrently: at the time the subject is solving the problem or
completing the task (known as a ‘live’ report), or
 retrospectively: after the event.
Concurrent think aloud requires minimal input from the investigator (Ericsson
and Simon, 1993) whilst retrospective think aloud data can either involve
uninterrupted accounts of the event or can be facilitated by retrieval or other
cues such as video recordings of the event or specific questions about what
happened (Taylor & Dionne, 2000). Concurrent think aloud has previously been
used within gambling research (Brochu et al, 2010; Gabour & Ladouceur,1989;
Griffiths, 1994; Husain et al 2013; Walker, 1992).
In this research concurrent think aloud was used to explore participants’ initial
reactions and thoughts on being presented with each RTP message.
Cognitive probing
Immediately after the think aloud, participants were asked a series of cognitive
probes to explore in more detail their understanding of each RTP message to
explore:
 their comprehension of each message;
 whether or not they had noticed the message (or a similar message)
when playing machines;

NatCen Social Research | Understanding of Return to Player messages
15
 their view on whether the message was useful to game players or not;
and
 their views on whether the messaged could be improved (or made
clearer) or not.
Probing is a qualitative technique, involving both scripted and spontaneous
probes that aim to elicit further information about participants thought
processes, including understanding of information and phrases contained
therein (see Beatty, 2004; d’Ardenne, 2014). Probes were asked concurrently,
after each message had been viewed. After participants had been shown all
the messages they were asked further questions on what other types of
messages, if any, would be useful for games players. Work by Conrad and
Blair (2009) suggests that respondent-driven cognitive interviewing techniques,
in which think aloud takes place prior to any probing, improves the reliability of
error detection.
2.1.4 Analysis
A Framework approach to data management and analysis was used.
Framework is a matrix approach where data is summarised into cells with a
row representing an individual case and a column representing a common
theme across the data. The advantage of this approach is that it facilitates the
analysis of different aspects of an individual’s processing of information and
the information features that influence understanding as well as enabling
analysis of particular themes across different cases (see Collins,,2014).
All notes and transcripts were ‘summarised’ into an analytical framework set
up in Excel. This analytical framework consisted of a number of descriptive and
analytical categories, reflecting the interview protocol. The framework included
a summary of the characteristics of participants: such as their sex, age,
frequency they played on the machines
7
, highest educational qualification and
interview location. The framework was organised by each of the five RTP cards
shown to participants during the interview. Under each card, a summary was
made of each interview’s findings pertinent to the think aloud and each
cognitive probe. An additional ‘other’ category was included for each card, to
capture any other comments made by the participant that had not been
anticipated in the design of the probes. Thus, data could be read horizontally
as a complete case record for an individual, or vertically by statement, looking
across all cases.
7
The following frequency of play question was asked at the end of the interview: In the last 12 months,
how often have you spent money on fruit/slot machines? Answer options: Everyday/ almost every day; 4-5
days a week; 2-3 days a week; About once a week; 2-3 days a month; About once a month; 6-11 times a
year; 1-5 times a year.

16
NatCen Social Research | Understanding of Return to Player messages
2.2 Study limitations
This study has a number of limitations that should be born in mind when
considering its findings and these are discussed below.
Limitations of cognitive interviewing
Cognitive interviews are qualitative in nature, involving an in-depth interviewing
approach and typically small, purposive samples. In these interviews we can
explore the cognitive processes that go on while a participant is attempting to
make sense of the information provided. The method helps us identify different
types of problem that participants encounter and provides us with evidence
about why these problems occur. However, cognitive interviewing cannot
provide quantitative information on the extent of the problem or the size of its
impact on survey estimates. Samples are usually too small, and cases not
selected with a known and equal chance to support such extrapolation.
The method relies on participants’ ability to articulate their thought processes;
however not everyone can do this (well). Moreover it is thought that cognitive
interviewing favours people with a higher level of education, as they find it
easier to articulate their thoughts, particularly when using think aloud (Fowler,
1995). A further point to note is that not all thought is conscious and therefore
capable of being articulated. Our interview protocol was designed to use a
range of cognitive interviewing techniques to ameliorate these problems.
The method may also change participants’ usual behaviour, for example
participants may spend more time reading and considering information
presented than they would in a real life setting and this could lead to spurious
findings (see for example Wilson et al, 1995). In our study we were not
principally concerned with participants’ reading behaviour rather we assessed
understanding of the messages using a range of cognitive techniques and then
asked participants if they ever read the messages.
Limitations of our sample coverage
As noted in section 2.1.2, our sample included few young men or
inexperienced machine players (those who had been player machines for less
than 6 months). Young men aged between 16 and 34 are more likely to have
played slot machines than women and older men and are more likely to exhibit
problem-gambling behaviours (Wardle and Seabury, 2012). It is possible that
these groups may have interpreted the return to player messages differently to
those included in our study. However it seems reasonable to hypothesise that
younger players and/or those with less gaming experience would exhibit the
same level, if not less, understanding of RTP than older, and or more
experienced players and this is something that should be empirically tested,
(see section 9.5 for more details). Moreover, the concept of RTP is applicable

NatCen Social Research | Understanding of Return to Player messages
17
to slot machines and thus younger players are likely to be familiar with the
concept.
Limitations of the recruitment method
The lack of younger machine players, particularly young men, and less
experienced players is in part likely to be a result of the recruitment strategy we
used for this study: that of recruiting through gaming venues. This limited our
pool of potential participants to those who frequented the selected venues,
saw our recruitment materials/ were approached by venue staff, were available
during our fieldwork session and volunteered to take part. Recruitment through
other methods, such as following up survey participants who are machine
players and have given their consent to be recontacted about future research
may have provided a wider pool of players.
2.3 Report structure
In the rest of this report we describe the main findings from the cognitive
interviewing. Chapter 3 provides contextual information about the factors
influencing machine choice that participants mentioned and the role of the RTP
message in this choice.
Chapters 4 to 7 explore participants’ understanding of the ‘simple’ RTP
message (chapter 4), RTP messages for different stake values (chapter 5),
messages for compensated and random games (chapter 6), game features
(chapter 6) and the odds of winning (chapter 7). Each chapter also look at
participants perceptions of the utility of each message. Chapter 8 looks at
participant’s views on what types of messages they think would be useful, and
on the views on the utility of a number of alternative messages. Finally, in
chapter 9 we provide a summary of the main findings from this study, their
implications for gambling policy, suggest what the next steps might involve in
developing new messages and make recommendations for further research.

18
NatCen Social Research | Understanding of Return to Player messages
3 Research context
Prior to being shown the RTP messages participants were asked how they
decided which machine or game to play. The aim of asking about machine
choice in this context was to see whether participants spontaneously
mentioned using RTP messages (or other messages explaining win chances)
when making decisions about which games to play.
Collectively participants mentioned a range of factors in deciding which game
to play. These were:
 whether the game is familiar to the participant – one that he or she
knows how to play;
 features of the game, such as how noisy or colourful it is;
 how complicated the game looks to play;
 how enjoyable the game is to play;
 the type of game: whether it is new; has screens, reels, roulette; is a
community game; and or has special features such as the chance to
have free spins and goes;
 the cost per play;
 the size of the maximum payout or jackpot;
 how the machine is playing – whether the jackpot has been won
recently; and
 whether the participant has won playing the game before.
The features mentioned by participants as influencing their choice of game
varied and reflected personal preferences, such as whether the participant
liked playing ‘reels’ or roulette, higher or lower stake games, or games that
were noisy or quieter. RTP messages, or odds displayed, were not mentioned
spontaneously by any participants as a factor influencing game choice.

NatCen Social Research | Understanding of Return to Player messages
19
4 Return to player message
4.1 The message tested
The first message shown to participants on a card was a standard RTP
message including a single percentage value (see Appendix C). The message
read as follows: “This machine has an average pay-out of at least 90%”. We
explored how players understood terms in the message using a four-step
process:
1) Participants were first asked to think aloud as they reviewed the
message.
2) The interviewer then asked some general cognitive probes to establish,
qualitatively, their understanding of the message, the terms ‘average’
and ‘average payout’, and whether they use similar messages when
deciding which machines to play.
3) Participants were then given a multiple-choice question and asked to
select the answer they thought was closest to what the RTP message
means, thinking aloud as they did so, (see insert).
The purpose of this question was to help facilitate further qualitative
exploration of participants’ understanding of the RTP message.
4) Finally, further probing followed, in which the correct answer was
revealed and participants were asked to comment on the clarity of the
message, to suggest ways in which it could be made clearer and what
they thought of the usefulness of the message. In this section we
discuss the qualitative findings from this process.
4.2 Understanding of the return to player
message
Participants’ understanding of this message was variable and can be
summarised in terms of those who:
Multiple choice question presented to participants:
This machine has an average percentage payout of at least 90%
Which of the following best describes what the message means:
 90% of people who play this machine will win something
 This machine will give out a prize 9 times in 10
 If you put £1 in this machine you are guaranteed to win 90p
 For every £100 put in to this machine about £90 is paid out in prizes

20
NatCen Social Research | Understanding of Return to Player messages
 understood the message correctly;
 misunderstood the message, interpreting it in a different way to that
intended; or
 did not understand it at all.
Understanding of the RTP message was judged on the basis of what
participants said during the think aloud and subsequent probing, their
response to the multiple choice question and their rational for their choice of
answer. We developed a typology of participants’ understanding of the RTP
message based on reviewing the aforementioned qualitative data. The lead
researcher developed the initial typology, which was then used to code each
case. Coding was undertaken by the lead researcher and another member of
the team independently. Codes were then compared and discrepancies
discussed and reviewed. The initial typology was revised in light of this process
to clarify the boundaries between each category and cases were subsequently
recoded independently again by the same two coders, who discussed and
agreed on an appropriate ‘final’ code, where necessary.
This is a small-scale, qualitative study, not designed to estimate the prevalence
of different interpretations of the RTP message. We include the number of
participants who understood the RTP message in a particular way merely to
indicate the pattern of interpretations encountered among those who took part
in this study. The numbers presented have no statistical significance. Among
those who took part in this study it was more common for the message to be
misunderstood than for it to be understood as intended; 18 of the 25
participants interviewed misunderstood the meaning of the message and this
pattern of understanding /misunderstanding persisted across the other RTP
messages we tested.
In the rest of this section we look in more detail at these different
interpretations.
4.2.1 Those who understood the message correctly
A minority of participants (six) were judged to understand the RTP message as
intended. They understood that winning was based on chance, that this
chance was determined by the machine, and that ‘90 per cent’ referred to the
proportion of takings the machine would pay out over a (unspecified) period of
time.
“Although it has an average payout of at least 90% that doesn’t
mean that if you spin the wheel 100 times 90 times out of those
you will in, or even that if you put £100 in £90 will come out
because it’s random.”
Female, 51 +, A level or higher, plays every day

NatCen Social Research | Understanding of Return to Player messages
21
This group included men and women of different ages and educational
attainment. All had been playing gaming machines for many years and set
themselves limits
8
of either the amount of time they would play for or the
amount of money they would play with.
4.2.2 Those who misunderstood the message
Misunderstanding of the RTP message was common among participants in
this study: 18 participants were judged to misunderstand the meaning of the
message, interpreting it in a different way to that intended (i.e. to the guidance
set out by the Gambling Commission). Two distinct types of misunderstanding
were identified.
Type 1 - Participants interpreting the message as meaning ‘you are
going to win’
This type of misunderstanding involved participants mistakenly inferring that
the RTP message was telling them that they were going to or were very likely
to win if they played the machine. They did not appear to understand that the
90% average payout referred to the proportion of the total value of money
staked by all players that will be returned in prizes over a large number of
games and that winning a prize was subject to chance. Rather they thought
that the payout was pretty much guaranteed and would occur within their play
session.
“…guaranteed to win really…well pretty much.”
Male, 18-30, played for 6 months or more, at least
once a week
“[the machine] will tend to pay out quite a lot. Out of the times
I play it I'm more than likely to win on it.”
Female, 31-50, played for 6 months or more at least
once a week
“Is it true that any customer is showing to be winning at least
90% average payout?”
Male, 31-50, played for 6 months or more,at least once a week,
English not first language
This group included men and women of different ages, with differing levels of
education. They all described themselves as people who had been playing for
more than 6 months, and who played several times a week. This group
included people who did not have English as their first language.
8
These were not machine-based limits, but strategies that participants employed themselves.

22
NatCen Social Research | Understanding of Return to Player messages
Type 2 - Participants being confused about the relationship between
how much you (the player) put in to the machine and your chance of
winning
This type of misunderstanding was more common among those in our study,
although as a small scale study we make no claims regarding its prevalence in
the wider player population. It involves misunderstanding the message as
referring to a player’s own individual chance of winning during a play session
rather than it referring to the proportion of the total value of money staked by
all players that will be returned in prizes over a large number of games.
“I couldn’t understand average payout here as I never win
£500… I spend a lot of money here but I don’t win nothing;
just £50, £40, I win. Yesterday I put £100 in but I didn’t win
nothing. I can’t understand this payout average.”
Male, 18-30, GCSEs or lower, played for less than 6 months,
English not first language
In addition there was a lack of understanding of what an ‘average’ payout was
among this group. Some participants thought that it meant that you might get
“a bit more or a bit less”
than 90% or that you would get a proportion of 90%
-
“you get 50% of 90%”
or that out of
“£500”
you put in you would
“get back
£150-£200”
.
In other cases participants took the 90 per cent average payout information to
mean that they would get back exactly 90
9
per cent of what they put into the
machine in any one play session. In some cases participants spontaneously
described
10
this in terms of for every £1 or £100 they put into the machine they
would get back 90p or £90 within their play session.
There was some scepticism about the ‘truth’ of the RTP message. Where this
was expressed it took two forms:
 those who did not believe it because in their experience they rarely or
never won (90% of what they put in);r
“If you put say £100 in, you should get £90 back, within
playing that £100 …although its total rubbish.”
Male, 31-50, A levels or higher, plays at least once a week
 or those who thought that the machines at the venue(s) they played at
had been
“turned back”
and were actually operating at a lower RTP than
9
This was the % quoted in the message shown to participants. It is likely that they would think they
would get back whatever % was quoted in the message.
10
This occurred during the think aloud or when probed on their understanding of the message, prior to
being asked the multiple choice question.

NatCen Social Research | Understanding of Return to Player messages
23
advertised. In these cases participants believed the machines were set
to a 70% RTP.
In the case of the first form of scepticism, the dissonance between
participants’ gaming experience and their understanding of the RTP message
exacerbated their confusion.
Confusion about the meaning of the RTP message was found among men and
women of different ages and educational attainment who took part in our
study. This group had all been playing for 6 months or more – in some case for
many years - and played at least once a week. Among this group were
participants who had English as a second language and some players who
described chasing their losses.
4.2.3 Those who did not understand the message
Only one participant said he did not understand the message. He had been
playing gaming machines for more than six months but played less than once
per month now as he had lost
“lots of money”
playing roulette in the past. This
participant did not speak English as his first language and had, by his own
admission, difficulty reading and understanding English. It is possible that this
affected his ability to understand the message and to articulate what the
message meant in English. This participant did not understand any of the
subsequent messages tested and we make no further reference to him in
describing participants’ understanding of the test messages in this report.
4.3 Perceived utility of the return to player
message
At the end of stage 2, after the initial think aloud and probing on the RTP
message, players were asked whether they used the RTP message when
deciding whether to play particular games. Most reported they did not use it
but two players said they did use this information. In one case the participant
said he used the RTP information routinely to select games to play that had
high payouts. This participant, a male aged 31-50, casino player, understood
the message correctly. In the other case the participant, a female aged over 50,
used the information to select games to play where she thought she had a
chance of winning the jackpot. She wrongly assumed that the message was
telling her that someone else had been playing the game beforehand and that
she might not therefore win the jackpot if she played. She had been confused
by the RTP message and misunderstood it.
There were two other cases where participants said they used the RTP
information sometimes to select a game, although how they used the
information was unclear. Both were participants who had misunderstood the

24
NatCen Social Research | Understanding of Return to Player messages
RTP message – one thought it meant she was going to win and the other was
confused by the message.
After further probing on the multiple choice question, see section 4.1,
participants were asked how clear and useful they thought the message was.
Those who thought the message to be clear and useful were in the minority.
They thought the message useful because, with one exception, they had
misunderstood the message as intended. All those who misunderstood the
message, thinking it meant that they were guaranteed to win, thought the
message was clear. In addition some of those who were confused by the
message thought that, if the message was described as in option 4 of the
multiple choice question, that it would be clear and could encourage people to
play. The exception was the case of the aforementioned gentleman who used
the RTP information to select games with a high payout, who understood the
message correctly.
The majority of participants in this study thought the RTP message was unclear
and therefore unhelpful. Reasons for these views were:
 they had misunderstood the message;
 the use of percentages and averages was confusing to those with self-
reported low levels of mathematical literacy;
 the message was seen to be misleading because:
o it didn’t specify a time period over which the average payout
would happen; or
o because participants didn’t believe the RTP was 90% based on
their experience or view that the machines operated at a lower
RTP;
 they played for fun and enjoyment; and
 whether you win or not depends on other factors, such as whether
someone has just won the jackpot, how long you play for, how the
machine is playing or how lucky you are.
In addition, those who were sceptical about the RTP message felt it was only
displayed because there was a legal requirement to do so.
Various suggestions were made at this point as to how the RTP message could
be improved. These were:
 use the wording in option 4 of the multiple choice question;
 don’t use percentages but instead describe the RTP in pounds and
pence;
 add a time period to the message, such as ‘
over a month or a week the
typical RTP is…’
;
 Change the message to say ‘
The more money you put in the more
chance you have of [winning] it
”;

NatCen Social Research | Understanding of Return to Player messages
25
 add the word
‘overall’
to average; and
 state that the RTP is independently verified.
It should be noted that some of the suggestions for improving the RTP
message were provided by participants who misunderstood the original RTP
message.
Chapter 8 looks in more detail at alternative forms of wording of the RTP
message and participants views on these and their own suggestions.

26
NatCen Social Research | Understanding of Return to Player messages
5 RTP messages for different stakes
5.1 The message tested
Later in the interview, after discussing the simple RTP message, participants
were shown a more complex RTP message which described how the average
percentage pay out varied depending on the amount staked. This message
was tested because some machines in Great Britain have a number of different
games that players can choose and the RTP may vary based on the individual
game characteristics. RTP can also vary within games based on the features
accessed and used, and the staking level. This makes the RTP messaging
more complex in reality as one machine can house different games with
different RTP payouts. The messages, shown to participants on a single card,
were as follows.
At 50p stake, this game has an average percentage pay-out of at least 88%.
At £1 stake, this game has an average percentage pay-out of at least 90%.
At £2 stake, this game has an average percentage pay-out of at least 92%.
Participants were asked to think aloud as they reviewed the messages and
were then asked some scripted probes to further explore their understanding
and their views on the utility and clarity of the messages.
5.2 Comprehension of the message on
different rates for different stakes
5.2.1 Interpretation of the message
This message was interpreted in a number of ways.
 The higher the stake the greater the payout.
 The more you put in the greater the chance of winning.
 That there is little difference in the percentage payout by stake value.
The second interpretation was more commonly cited by participants in this
study and occurred among those who had been confused by the simple RTP
message, discussed in chapter 4. The first interpretation (the higher the stake
the greater the payout) was provided exclusively by participants who had
correctly understood the simple RTP message, see chapter 4. The third was
provided by two female participants with lower level qualifications (GCSE or
below) who had been confused by the simple RTP message. In addition, a few
participants who had been confused by the original RTP message and

NatCen Social Research | Understanding of Return to Player messages
27
expressed scepticism about the accuracy of it spontaneously expressed their
scepticism about the truth of this message.
5.2.2 Understanding the term ‘stake’
Specific probes were asked to explore what participants understood by the term
‘stake’ and this was understood in one of two ways.
 The cost per game, such as the
“price per spin, if you are playing a one
pound spin, each stake is one pound.”
(Male, aged 31-50), or for
Roulette the odds being offered, such as 36 to one.
 The money gambled, bet or invested in trying to win - the “
money you
put in. The money you are risking.”
(Male, aged 30-50).
5.3 Perceived utility of the message on
different rates for different stakes
This message was seen as being clear and no improvements were suggested.
The perceived utility of this message varied and was generally linked to
whether participants thought that the message could influence their own and
or other players’ behaviour. Those who thought the message could influence
behaviour thought that the message was useful. However some participants
expressed concern that the message could be a marketing ploy designed to
encourage people to up their stake and that might, in some cases, encourage
people to bet more than they could afford.
“I think it’s to encourage you to try to increase your likelihood
of winning by upping your stake.”
Male, over 50, A levels or higher, plays at least once a week
In addition, there were two participants who did not think the message would
change their or others’ behaviour but who felt this information was useful to
know. These were the two participants mentioned in section 5.3.1, who
interpreted the message as meaning that there was little difference in the
percentage payouts between the three stakes. Both these participants said
they would play the lowest stake.
Those who did not think the message would influence their own or other
players’ behaviour thought the message was not useful.
It is worth noting that a few participants questioned whether it was fair to offer
different average percentage payouts based on the size of the stake. They felt
that the size of payout should be the same irrespective of the cost per play.

28
NatCen Social Research | Understanding of Return to Player messages
6 Messages on ‘compensated’ and
‘random’ games
6.1 The messages tested
As mentioned in section 1.2, the Gambling Commission’s technical standards
mandate that information on whether a machine uses a compensated or
random payout schedule must be provided to players and it affects the odds of
winning. This research study covered both types of payment schedule.
Participants were shown a message explaining that a game was
‘compensated’ or influenced by previous play. This message is as follows:
“This game is compensated and may be influenced by previous play.”
Participants were then shown a message about random play. The message is
as follows: “This game is random”. They were asked to think aloud as they
were shown each message on a card, and were then asked more specific
probes to elicit their understanding of the messages and their perceived utility.
6.2 Comprehension of ‘this game is
compensated’
This message was not always understood. Those who spontaneously said they
did not understand the message were unsure what ‘compensated’ meant,
“I don’t understand what this means, compensated…this game
is compensated. What does this mean?”
Female, aged over 50,
no qualifications, confused by the initial RTP message
“I don’t know what that means, am I stupid?”
Female, aged
31-50, GCSEs or lower, was confused by the initial RTP
message
In addition, one participant questioned what previous play means, asking
whether it means
“it encourages you to go on it because it’s paying out, you
mean?”
Female, aged over 50, confused by initial RTP message.
Those who did not understand what the message meant included participants
who did not speak English as a first language.
Where participants thought they understood the message, in most cases they
interpreted it as meaning that the chance of winning is influenced by the

NatCen Social Research | Understanding of Return to Player messages
29
outcome of previous play. However, they did not always grasp the fact that the
prize distribution is determined by chance.
“My interpretation of this message is that if the last guy has
lost you are more likely to win.”
Male, 31-50, A levels or
higher, was confused by the initial RTP message
“
If it has paid out a lot of money already, it may not pay out as
much
.” Female, over 50, A levels or higher, understood the
initial RTP message.
Those who understood it in this way included participants of different ages,
levels of education and those who did and those who did not understand the
initial RTP message, discussed in chapter 4.
However, there was one case where the participant considered that the
message was saying
“you could win some money playing this game”
. This
participant, a female aged 31-50 with GCSEs or lower, had been confused by
the initial RTP message.
6.3 Comprehension of ‘this game is random’
On machines operating on a random payout schedule, the odds of winning
remain constant, and are not affected by previous wins or losses on machines.
Only one participant in our study interpreted the message in this way. This
participant was a male aged 18-30, who had been confused by the initial RTP
message, discussed in chapter 4.
Occasionally participants said that they were uncertain what the statement
meant and not having English as a first language was a factor here. However,
unlike with the message ‘this machine is compensated’, see section 6.2, the
‘random’ message elicited a wide range of interpretations that were not in line
with the intended meaning. The difficulties participants had with understanding
the message related to the term ‘random’, which caused confusion. The range
of interpretations was as follows.
 Winning is down to
“chance”
or
“luck”
- you could put a pound in and
win the jackpot or put in £100 and win nothing
 It’s unpredictable – there’s no way of knowing what is going to happen
next
 Some games will payout the big prize
 The odds of winning are not fixed
 Staff haven’t
“fixed”
the odds of winning

30
NatCen Social Research | Understanding of Return to Player messages
 If you play it at the
“right time”
, the machine will pay out in
“a
sequence”
. If you pay out at the
“wrong time”
, you will keep loosing
 It means you are winning
 It means it’s an
“exclusive”
machine or game that is a speciality of each
gaming venue
 That the game is
“different”
.
In terms of random being interpreted as meaning
“different”
this interpretation
was provided by a young, male respondent aged 18-30 who used the analogy
“you might share a photo and call it random because it is different”
. This usage
illustrates how language can change over time and that different generations
may interpret the same word or phrase in very different ways.
6.4 Differentiation between ‘compensated’ and
‘random’ games
There was some uncertainty about the difference between random and
compensated games, which stemmed from players not knowing what either or
both ‘random’ and ‘compensated’ meant. However others were clear that the
messages were different, although as we have discussed in sections 6.2 and
6.3, participants’ interpretations of these two statements were not always in
line with the intended meaning. Figure 6.1 presents the differences cited by
participants who felt the messages were different.
Figure 6.1 Differences in understanding between ‘compensated’ and ‘random’
games among participants
Understanding of term compensated
Understanding of term random
Chance of winning affected by previous
play
Chance of winning not affected by
previous play
Game is exclusive to a particular gaming
establishment
Chance of winning and size of win
affected by previous play
The game can do what it likes / Game
programmed to include some ‘random’,
unpredictable elements
Player has more influence over the odds
of winning
Player has no influence of the odds of
winning

NatCen Social Research | Understanding of Return to Player messages
31
6.5 Perceived utility of messages on
‘compensated’ versus ‘random’ games
Views were mixed about the utility of these messages and whether or not
participants understood them influenced their views on their usefulness.
Moreover, as reported in earlier chapters of this report for the other messages,
other factors influencing views on utility were whether participants:
 read such messages before playing the game; and/or
 whether they felt that the message would influence their, or another
player’s choice of game.
In addition, there was a view expressed by some that the information about the
game being compensated would (only) be useful if you knew the outcome of
the previous play. There was some doubt expressed over whether operators
would be willing to include this type of information on machines, as this might
affect players’ decisions about whether to play a particular game, for example,
if the jackpot had recently been won.
Participants made some specific suggestions about how these messages
could be made clearer and thus potentially more useful. These are summarised
in Figure 6.2.
Figure 6.2 Suggestions to improve the clarity of the ‘compensated’ and ‘random’
game descriptions
Compensated
Random
If this machine has already paid out then
future payouts may be influenced
This game is random and is not affected
by previous payouts
Don’t play this machine because it has
just paid out
This game is random and you stand an
equal chance of winning regardless of
previous wins

32
NatCen Social Research | Understanding of Return to Player messages
7 Game features and the odds of
winning message
7.1 The message tested
The percentage RTP average payout can be affected by the features of
particular games. For example, on a reel gaming machine ‘three cherries’, three
orange’ and ‘three apples’ may contribute 40%, 40% and 10% to the overall
90% RTP in one game and 30%, 40% and 40% to the overall 90% RTP in
another. Contribution to the overall RTP of each feature is a combination of the
odds of achieving the prize multiplied by the prize value (e.g. if the contribution
of ‘three cherries’ is 40% and the prize value is £1, that would mean that the
chance of ‘three cherries’ occurring would be 40 in every 100 games, e.g. for
each £100 wagered I would on average receive £40 back as prizes). Since the
distribution of prizes is random there is no guarantee you will win £40 for every
£100 gambled. Assuming that the prize value for ‘three cherries’ remains the
same, then the chance of winning the prize changes if the contribution is
altered from 40% to 30%. Hence it would not be possible to deduce the odds
of winning a prize from the RTP unless the game is either very simple and/or
the odds can be calculated from game format (e.g. roulette). Since gaming
machines use virtual reels and do not display true odds (in most cases) they
are required to display the notice ‘the outcome of any game or feature is not
necessarily that shown by the odds displayed’. We tested the following
message, which can appear on some games.
‘The outcome of any game or feature is not necessarily that shown by the odds
displayed’.
Testing was carried out in the same way as for other messages, described in
earlier chapters. Participants were asked to think aloud as they were shown the
message on a card, before being asked more specific probes to elicit their
understanding of the message and its perceived utility.
7.2 Comprehension of message
This message was particularly problematic and participants struggled to
understand it when it was first presented. There were several factors that
contributed to participants’ difficulty.
 Terms such as ‘necessarily’ and ‘outcome’ were confusing, particularly
for those people in our study who did not have English as a first
language.
 The message was seen as being complex and “
long-winded
” and this
was off-putting.

NatCen Social Research | Understanding of Return to Player messages
33
 There was some confusion about how the 90% average payout
information related to this particular message.
Some participants thought that they understood the message but in fact it
transpired in probing that they misunderstood it. Earlier misunderstandings of
what the simple RTP message meant coloured participants’ understanding of
this message, see section 4.2.2. The following quotes illustrate these
misunderstandings.
“Putting £100 in and getting 90% of your money back [is] so
remote now.”
Female, over 50, GCSEs or lower
“So you’re displaying odds of 90% payout…and the outcome
of any game is not featured…oh yeah clearly, everyone knows
it’s not.”
Male, aged 31-50, A levels or higher
“It’s basically saying it doesn’t matter what it’s saying up there
[on the screen], the chances are you might not win…we won’t
accept any responsibility”
Male, over 50, education level unknown
Only one participant understood the message correctly, that is that the odds of
a particular game feature can vary from the overall RTP of the game. He had
correctly understood the earlier messages.
7.3 Perceived utility of the messages
There were some suggestions about how the message could be made clear.
These were:

"If a feature is awarded on a game, the feature may not necessarily
return a 90% rate of play"
Male, 31-50, A level or higher, understood the simple RTP message
 the information displayed in the message on the machine may not
reflect the actual odds for the specific game being played; and
 different machines work in different ways so the RTP may vary from that
advertised.
It’s worth noting that these suggestions were provided by people who did not
understand the message. In the case of the latter two suggestions, these were
influenced by the discussion of the message during probing immediately
beforehand.

34
NatCen Social Research | Understanding of Return to Player messages
Views on the utility of the message were mixed and were influenced by
whether participants could understand the message (even when explained by
the interviewer) and whether they thought players would use it.
If participants did not understand the message then they were inclined to think
that the message was not that useful.
Some participants thought this information would be useful because they or
other players would use it when selecting what game to play. However, others
thought it was not useful because people would not use this information. This
was because either:
 the information was felt to be too long-winded and complex for people
to understand; or
 that people do not read these messages: they know it is a game of
chance, with risks and they just want to get on and enjoy playing it.

NatCen Social Research | Understanding of Return to Player messages
35
8 Player’s views on alternative
messaging
At the end of the interview we asked participants for their views on the types of
messages they thought should be on machines and for their views on the utility
of providing the following, additional or alternative information.
 The odds of winning any prize per game
 The odds of winning the jackpot prize per game
 The odds of winning prizes of different sizes for example whether small
prizes are given frequently or large prizes are given infrequently
 The average hourly loss rate
 The total amount of money you have put in the machine in your gaming
session
In this chapter we discuss participants’ reactions to each of these.
8.1 What types of messages, if any, do you
think should be on machines to explain win
chances to player?
Views were divided about whether there should be messages on games that
explain win chances between those in favour and those opposed. Those
opposed, played for fun and the thrill. They knew the risks and did not think
there should be any messages. They do not want to have to read a ‘prologue’
before playing. Mostly these were players who did not read these messages.
Those in favour of messages felt that they could inform either their own, or
others’ choice of game. However there was no real consensus on which
messages would be the most useful. Figure 8.1 illustrates the types of
messages participants suggested should be on machines.

36
NatCen Social Research | Understanding of Return to Player messages
Participants were then read a list of suggested alternative messages and asked
to say if they thought these messages would be useful or not and why. In the
rest of this chapter we look at participants comments on these alternative
messages.
8.2 The odds of winning any prize per game
Views on this message varied and are summarised below.
Those who did not understand what this would entail
This group did not understand what was meant by this suggestion or thought
this was confusing. For example, if a machine has more than one game and
the RTP message said the average percentage payout was 92% players may
assume that the 92% applies to all of the games on the machine.
Those who thought it would be useful
Many participants in this study felt that this information would be useful
because it would provide information on your chance of winning. However the
misunderstandings about what the average percentage payout meant
persisted, see chapter 4. Some participants suggest that the information would
be more useful if:
 it was provide for different stakes;
Figure 8.1 Messages that should be included that explain win chances
Current information
 The basic RTP message providing the percentage average payout
information
 Whether the game is compensated or random, and if compensated,
whether the jackpot has been won already
 RTP information for different stakes, making it clear that there are no
guaranteed payouts e.g., all percentages are not guaranteed all of
the time
 Odds of winning the features
New/ additional information
 The maximum prize which can be won/ jackpot prize, this can attract
people to play
Improvements to the presentation/ accessibility of information
 A help button for new players which explains how to play or interact
with the game
 A panel, which displays the messages separately. Information needs
to be shown at the beginning of each game.

NatCen Social Research | Understanding of Return to Player messages
37
 talked about the odds of winning rather than an average percentage
payout; and
 included information on whether the machine is compensated or
random.
Those who did not think it would be useful
Those who felt the message would not be useful expressed concerns that it
would encourage people to put more money in the machines because they
think they are going to win. This misplaced belief that you are going to win
stems, in this group’s opinion, from people misunderstanding the percentage
average payout as referring to your chance of winning in that play session
rather than it being based on a much longer period, perhaps a year.
Undecided
There were a few participants who felt this message might be useful but
queried how it could be displayed if each game has different prizes/ how you
can compare one game to another. There was a view that the message would
raise players’ expectations that they would win. It should not be displayed
unless there was a guarantee that the game would pay out as promised.
8.3 The odds of winning the jackpot prize per
game
As with the previous message, view about the utility of this message varied.
Those who thought it would be useful
This information was considered useful because. winning the jackpot is rare.
This information would provide a sense of how hard it is and participants
thought that providing this information might temper people’s play. However
the odds of winning any prize per game were considered, in some cases, to be
more useful.
Those who thought it would not be useful
Some participants felt that this information could put people off playing or
encourage players to watch each other playing so that you can see if they have
won the jackpot. This would be off-putting. Other participants also felt that the
message should simply say that winning the jackpot was impossible’!
Undecided
These participants felt there were both advantages and disadvantages to
displaying the odds of winning the jackpot. They felt that knowing the chance

38
NatCen Social Research | Understanding of Return to Player messages
of winning the jackpot may decrease your enjoyment of playing the game as
the odds will be very high. Conversely, they felt that it may be useful to identify
which games you might have more of a chance on than others.
8.4 The odds of winning prizes of different
sizes
Views were split regarding the utility of this message.
Those who thought it would be useful
Those who thought this information would be useful wanted to know: if large
prizes are given and how often small prizes are given, as it was thought that
there is more of a chance of winning a smaller prize. One participant described
having recently seen
“roller coaster graphics”
on a machine to show whether it
gives lots of frequent small payouts or big payouts less frequently. He stated
that he found this very useful as in the past he played blindly, whereas now he
chooses games based on the payout.
Those who thought it would not be useful
A variety of reasons were given as to why this information would not be that
useful based on players beliefs, which are summarised below.
 No one is interested in small prizes only large ones so this information is
not that useful:
 Winning money is not the only reason for playing, participants played for
enjoyment, the lights and colours.
 Having lots of information about the odds of winning different prizes
would be confusing or too complicated to understand, knowing the
average payout all that is needed.
 This information is unnecessary because:
o most players know this anyway; or
o winning is down to luck.
Undecided
One participant was unsure if people would read this amount of information.
“you’ve got a limited timeframe in which to get this information
across. it could take me a number of visits or days to read all of
the different instructions.” Male, 18-30., GCSEs or lower,
confused by the initial RTP message.
8.5 The average hourly loss rate
Those who thought this would be useful

NatCen Social Research | Understanding of Return to Player messages
39
This was seen by some participants as a potentially useful addition. The
rationale given was that sometimes players can lose a lot and this might make
people realise that the longer you play the more you will lose.
Those who thought this would not be useful
Where this was seen as not being useful this was because:
 participants felt that people wouldn’t read the information: they just want
to play the game;
 that the information could be misleading. For example, if people think
that if it has a high loss rate the may think that it will eventually have a
high payout; or
 because they simply do not want to know this information: it would
deter participants from playing.
Undecided
Where participants were unsure as to whether this would be useful or not, it
was mainly because they felt this information might be misleading, noting that
there can be so much variation between stakes it would be confusing.
8.6 The total amount of money you have put in
the machine in your gaming session
Views on the usefulness of this message were split.
Those who thought this would be useful
Where this was seen as useful participants thought it may stop people going
out of control with their spending. Participants talked about their own loss of
control, of
“getting carried away”
and not keeping track of what you are
spending, particularly now that you can use debit cards to buy stakes.
“That is definitely needed…so we know how much we spend. Last week
I got upset as in the last two days I put £200 in the machines and my
mind every few minutes is ‘freezing’ - my £200 is gone”
Male, 18-30,
GCSEs or lower
Those who thought this would not be useful
Those who felt this message would not be useful stated that they did not really
want to know how much they spent, or said they knew (or believed they knew)
how much money they put into the machine. There was also a concern that it
might encourage people to gamble more to try and re-coup their losses if they
saw how much they were spending.

40
NatCen Social Research | Understanding of Return to Player messages
One participant, a male aged 31-50 with GCSEs or lower, did not understand
what was meant by this message.
8.7 Should money put in include prize money
that has been re-staked using credit
transfer or not
This message, on the whole, was not seen as being that useful. The consensus
was that only the money you, the player, put into the machine should be
included as ‘most people’ do not count re-staked money but rather just the
amount they put in the machine. Where participants felt this may be of use was
for those who wanted to know how much money has ‘churned’ through the
game compared to how much a player has won.
8.8 A record of the amount you have put in a
single gaming session minus the money
you have in the bank
Views on the utility of this message were evenly split.
Those who felt that this would be useful
The arguments put forward in favour of this information being provided were
that it would remind people of how much they are losing; it helps with control
to show what you have put in and what you have won; and that knowing if you
are winning or losing can help you make decisions about whether to cut your
losses or not.
Those who felt that this would not be useful
The arguments put forward against it being useful were that people already
know this information so it is not necessary to provide it and that
“when you
have the fever, you have the fever”
and that having this information would
make little difference to gambling behaviour.
8.9 Other types of message about win
chances that could be useful for game
players
Participants were asked what other messages about win chances they thought
might be useful for machine players. A number of suggestions were made.
These were as follows:

NatCen Social Research | Understanding of Return to Player messages
41
 providing a clear example of the odds you get back from a £10 stake;
 creating a statistics button, which displays all the winning ratios and
odds, if you want to see them;
 showing how much you lose. This was felt to be important to make
people realise the costs associated with playing the game; and
 a message saying that a payout is not guaranteed. One participant felt
the current message was misleading as it can make you think you will
win. This participant also felt that a message saying ‘this is just for
entertainment and it is not a way of subsidising your income’ would be
helpful.
In addition, some participants wanted to be given information about how much
a machine had paid out recently as this would influence their machine choice:
“when did they pay their last big win, good to know the last time
somebody took 3 grand out of that machine”
Male, over 50, A levels or higher
Other participants wanted to know what machines had paid out over a longer
time frame. For example, one stated that it would be useful to know whether
major prizes which have been won recently e.g., over the last 7 days ‘
because
you can lose track of what’s happening if you don’t go into the venues
frequently’
. It is not clear whether this participant was referring to random or
compensated machines but for some there is clearly an entrenched
association between recent payouts and their own estimations of winning.
Finally, one participant stated that no message would be useful because
gambling should be fun and you should only gamble what you can afford to
lose. This was supported by another participant who suggested that the
message should be simply
"be prepared to lose your money... only put in what
you can afford to lose".

42
NatCen Social Research | Understanding of Return to Player messages
9 Discussion and implications
9.1 Understanding return to play messaging
The findings of this research support the Gambling Commission’s concern that
return to player (RTP) messages are not well understood. This study found that
the meaning of the percentage average payout and how this information
relates to an individual player’s win chance in any one given play session are
particularly problematic. Dissonance between participants’ gaming experience,
i.e. not winning (90%) of what they put into the machine, and their
understanding of the RTP message exacerbated their confusion. For a group of
participants, this confusion led to the perception that the RTP message was
telling them that were going to win. This is particularly concerning as this might
lead some players to play more than they intended.
There was some cynicism about the ‘truth’ of the RTP information being
presented. This scepticism was fuelled by the aforementioned dissonance
between individuals’ past experience and their understanding of the RTP
messages and/or a wider distrust of the venue/gambling industry and its
motives for displaying the RTP messages. Again, this is concerning as this is
an unintended consequence of these messages. RTP messages are intended
to provide players with greater transparency about the proportion of the total
value of money staked by all players that will be returned in prizes, on average,
by a particular machine or game. However, misunderstanding of the messages
is leading some to question how reliable the information provided is and could
promote mistrust of this type of messaging.
Understanding of technical terms such as compensated or random was found
to be variable. Some players did not understand what the term compensated
meant or the clarification that the outcome of the game may vary based on
previous play. A common mistake was thinking that outcomes would (rather
than may) vary based on previous play. The term random was particularly
problematic and was interpreted in several different ways. This was confused
with the payout schedules attached to ‘compensated’ machines and in some
cases the term random was not understood in statistical or probabilistic terms
but rather as meaning something different, strange or odd. This highlights the
risk of using technical language used by industry and regulators alike in
messages aimed at the general population. What sector specialists understand
by certain terms is not always the same as the broader general population.
Finally, the message focusing on how the outcome of any game or feature is
not necessarily that shown by the odds displayed was particularly poorly
understood. It was viewed as long winded, the meaning of the terms ‘outcome’
and ‘necessarily’ in this message were often unclear and participants
struggled to explain what this message meant.

NatCen Social Research | Understanding of Return to Player messages
43
9.2 Use of the message
We explored how useful participants found RTP messages. Typically
participants in this study had not seen or noticed the RTP messages prior to
interview and no participant reported that RTP messages influenced their
choice of machine. Therefore, findings about the utility of these messages
need to be contextualised based on this understanding: for most participants
what was tested was perceived utility now that they had been made aware of
these messages. Perceived utility was influenced by player’s views on whether
they thought the messages would have an influence on their own behaviour or
that of others. Participants were largely split between those who felt RTP
messages imparted useful information and those who did not feel they were
useful. Typically, those who felt that messages were not useful were
participants who thought that the RTP messages were unclear and therefore
unhelpful. Those who did not understand the messages tended to express that
they felt them to be misleading. Some participants were concerned that the
messages would be misinterpreted by players and that this could encourage
them to gamble. This was especially true for the message around how RTP
varies by stake, with some participants expressing concern that this would
lead to players increasing their stake.
However, some participants did feel that the messages could be useful in
helping them to decide which machine or game to play. These were people
who tended to understand the meaning of the messages.
This research did not look at how and if people use the messages in practice,
so the evidence is limited to perceptions of usefulness or concerns about
impact on other people. We suggest further research in this area, see section
9.5
9.3 Alternative messaging
The Gambling Commission considers that players may benefit from greater
transparency around a range of other game and machine features. To explore
this, participants were presented with a range of alternative messages ranging
from the odds of winning any prize/jackpot/different prize levels to the average
hourly loss rate and variations showing the amount of money spent.
Participants were mixed as to whether they felt these messages would be
useful or not. Some players seemed keen to have better information that they
could access about game statistics, including the odds of winning different
prizes or winning jackpot prizes. Others were concerned about the impact this
might have on their play experience but also because they felt that these
messages would still be misunderstood by most. One suggestion was that a

44
NatCen Social Research | Understanding of Return to Player messages
game statistics feature could be included in machines to display all the relevant
information for those who wanted to use it.
There was no clear consensus over which messages were felt to be most
important or most useful. Interestingly, participants were split as to whether
they felt monitoring how much money they had spent in a session was useful
or not. There was clearly a group of players who felt this could be effective in
helping to control machine expenditure. Conversely, some participants felt this
may have an unintended consequence of prompting people to chase their
losses. This deserves further attention, see section 9.5.
9.4 Implications for responsible gambling
messaging
This project aimed to assess understanding of RTP messages, the factors that
might affect players’ understanding of them and to provide insight into
alternative forms of wording. Evidence from this study has shown that current
messages are not well understood for a number of reasons. These include:
 use of technical language that does not hold the same meaning for the
general population as industry specialists;
 use of complex terms that have ambiguous or unclear meaning;
 provision of messages in English only making these difficult to
understand by those for whom English is a second language; and
 use of mathematical terms, especially percentages, average and the
term ‘random’, which are difficult for less mathematically literate people
to understand.
For some, this lack of clarity promoted confusion, misunderstanding and
mistrust of both the industry and regulator. The confusion and
misunderstanding are particularly important as this could have unintended
consequences in terms of impact on play. Of concern are those who
interpreted the message to mean that they were going to win. It is of interest
that some participants themselves were concerned about the potential impact
on behaviour of these messages. These themes suggest clear learning points
when thinking about responsible gambling messaging more broadly.
Firstly, messages should be clear, more direct and use less complex
terminology. For example, of the options presented to participants, the
message which said ‘For every £100 put into this machine about £90 is paid
out in prizes’ was typically judged to be clearer than the original message
which used percentages. Some participants noted that this could be qualified
with a direction that if you play this machine, you should be prepared to lose.
A second suggestion is that both the regulator and industry should think more
carefully about how they communicate with those for whom English is not their

NatCen Social Research | Understanding of Return to Player messages
45
first language. For example, if venues are in areas with high proportions of
minority ethnic groups, then signage in appropriate languages may be
considered. Evidence from the Health Surveys and the BGPS has shown that
those from minority ethnic backgrounds are at greater risk of harm from
gambling. These groups are more likely to read and speak English as a second
language. Therefore, further attention should be given to how best to
communicate with these groups.
A third learning point is that many people included in this study had not noticed
or used the RTP messaging to date. For any kind of responsible gambling
messaging to be effective, people have to see it, understand it, reflect on the
information conveyed in the message and then, hopefully, translate this into
changed behaviour (in the direction intended). As evidence from evaluations of
health warning messages for similar products, like cigarettes, tells us, this is
easier said than done (Wardle et al, 2010). This study focused only on
comprehension and his can be improved but more work needs to be done
around how best to present and situate these messages so that they are
noticed in the first place.
A final learning point, and arguably the most important, is that RTP messages
may not influence players behaviour at all or in the way intended. For example,
whilst some participants in this study felt that knowing more about the total
amount of money spent would be useful, others felt this could encourage
chasing behaviour - the direct opposite of what this feature intends. The
wording of messages needs to be carefully formulated and tested to ensure
that they influence players behaviours in the ways intended and do not have
unintended consequences.
9.5 Recommendations for further research
This research highlights a number of issues with the current RTP messages
that call into question their effectiveness in providing players with clear
information that they can understand. Further research is necessary to
corroborate this study’s findings. Specifically research should further:
 establish the extent and nature of misunderstandings, and how these
vary across the wider population and between different subgroups;
 explore whether players actually look at messages in a ‘real’ play
environment and if so in what circumstances. Identify the features of
message presentations that encourage participants to read them; and
 further test alternative wordings of RTP messages, to assess their
performance in terms of being of interest to players, being correctly
understood, and in influencing players’ behaviour as intended.
The latter point is particularly important: even if RTP messages are produced
that are clear and understood correctly, and are situated in a way that

46
NatCen Social Research | Understanding of Return to Player messages
facilitates them being seen and read by players, this may not mean that they
will have any influence on player behaviour. If the purpose of the RTP
messages is to encourage ‘responsible gambling’ through informed consumer
choice then the effectiveness of such messages needs to be tested through a
program of quantitative experiments.

NatCen Social Research | Understanding of Return to Player messages
47
References
Beatty, P. (2004) The Dynamics of Cognitive Interviewing. In S. Presser, J.M. Rothgeb,
M.P. Couper, J.T. Lessler, E. Martin, J. Marting, E. Singer. Methods for Testing and
Evaluating Survey Questionnaires. John Willey & Sons: Hoboken, New Jersey: pp445-
66.
Beatty, P. and Willis, G. (2007) ‘Research synthesis: the practice of cognitive
interviewing’,
Public Opinion Quarterly
, 71: 287–311.
Blaszczynski, A., Ladouceur, R., Nower, L., Shafferk, H. (2008) ‘Informed choice and
gambling: principles for consumer protection’.
The Journal of Gambling Business and
Economics.
2:1:103–118
Brochu, P., Dufour, J., Giroux, I. & Sévigny, S. (2010).
Comparison of Internet Texas
Hold’em players’ perceptions and reported behaviours according to their level of
gambling related problems: a step toward adapting treatment.
Poster presented at
11th Annual NCRG (National Center for Responsible Gaming Conference on Gambling
and Addiction, Las Vegas.
Bryman, A. (2012)
Social Research Methods
(4th edition). Oxford: Oxford University
Press.
Collins, D. (2014) Analysis and Interpretation in D. Collins (ed)
Cognitive Interviewing
Practice
. Sage: London
Collins D. (2003) ‘Pretesting survey instruments: an overview of cognitive methods’,
Quality of Life Research
,12(3): 219–27.
Conrad, F., Blair, J. (2009) Sources of error in cognitive interviews.
Public Opinion
Quarterly
; 73 (1): 32-55.
d’Ardenne, J. (2014) Developing test materials and interviewing protocols. In D. Collins
(ed) Cognitive Interviewing Practice. Sage: London.
Ericsson, K.A., and Simon, H.A. (1980) ‘Verbal Reports as Data’, Psychological
Review, 87:215-51.
Ericsson, K A., Simon, H.A. (1993) Protocol Analysis: Verbal Reports as Data. MIT
Press, Cambridge, MA.
Fowler, F.J. (1995) Improving Survey Questions: Design and Evaluation (Vol. 38).
Thousand Oaks, CA: Sage.
Gabour, A,. Ladouceur, R. (1989) Erroneous perceptions and gambling.
Journal of
Behaviour and Personality
4:411–20.
Gray M., Wardle H. (2013) Observing gambling behaviour using think aloud and video
technology: a methodological review. NatCen Social Research. Available at:
www.natcen.ac.uk.
Griffiths, M.D. (1994). The role of cognitive bias and skill in fruit machine gambling.
British Journal of Psychology
, 85: 351-369.

48
NatCen Social Research | Understanding of Return to Player messages
Husain, F., Wardle, H., Kenny, T., Balarajan, M., Collins, D. (2013)
Examining Machine
Player Behaviour: a qualitative exploration
. NatCen Social Research. Available at:
www.natcen.ac.uk
Responsible Gambling Strategy Board (2013).
RGSB advice to the Commission on the
Triennial Review consultation
, June 2013. Available at:
http://www.rgsb.org.uk/publications.html
Ritchie, J. Lewis, J., and Ellam, G. (2003) ‘Designing and selecting samples’ in J.
Ritchie and J. Lewis (eds)
Qualitative Research Practice: A guide for social research
students and researchers
. Sage: London. pp 77-108.
Spencer, L., Ritchie, J. and O’Connor, W. (2003) ‘Analysis: practices, principles and
processes’, in J. Ritchie and J. Lewis (eds)
Qualitative Research Practice
(1st edition).
London: Sage Publications. pp. 199–218.
Taylor, K., Dionne, J.P. (2000) Assessing Problem-Solving Strategy Knowledge: The
Complimentary use of Concurrent Verbal Protocols and Retrospective Debriefings,
Journal of Educational Psychology
92(3): 413-425.
Wardle H, Pickup D, Lee L, Hall J, Pickering K, Moodie C, Mackintosh A (2010).
Evaluating the impact of picture health warnings on cigarette packets
. Public Health
Research Consortium: York.
Walker, M. (1992) Irrational Thinking Among Slot Machine Players.
Journal of
Gambling Studies
8(3).
Willis, G. (2005)
Cognitive Interviewing
. Thousand Oaks, CA: Sage.
Wilson, T.D., LaFleur, S.J.and Anderson, D.A. (1995) ‘The validity and consequences
of verbal reports about attitudes’, in N.Schwarz and S. Sudman (eds),
Answering
Questions: Methodology for Determining Cognitive and Communicative Processes in
Survey Research
. San Francisco: Jossey-Bass. pp. 91–114.

NatCen Social Research | Understanding of Return to Player messages
49
Appendix A. Recruitment documents
This appendix contains a copy of the screening questionnaire used by NatCen’s
Telephone Unit to recruit participants.
Return to Player Cognitive Testing
Screening questionnaire
Introduction
My name is [AS APPROPRIATE] and I work for NatCen Social Research, an
independent research institute.
We are looking for people who play games machines to help us with a piece of
research. In particular we are looking for people who play ‘category B’ machines.
These are also known as slot machines, reels or fruit machines.
By law these machines have to have certain messages on them. The
Responsible Gambling Trust has commissioned us to talk with players about
these messages and how clear they are.
If you would like to/ are eligible for an interview then during the interview you
will be shown these messages and asked for you opinion on whether they
could be made clearer. By doing this we can see if they could be improved in
the future.
We would like to talk to some people who play these games regularly and
some people who only play these games occasionally.
What will taking part involve?
Taking part in this study will involve talking to one of our researchers.
The interview, which would take place at a gaming machine venue, would last
about half an hour, and, if you take part, you will be given a 20 pound high
street voucher for your time. All information will be treated in the strictest
confidence and used for research purposes only.
Taking part is entirely voluntary. With your consent, we would like to audio
record the interview. This will allow the researcher to focus completely on
what you are saying. No one outside the research team will have access to the
audio recording or to any information that could identify that you took part in
the study. The audio recording will be deleted at the end of the project.
IF WILLING TO TAKE PART GO ON. IF NOT, THANK THEM AND END

50
NatCen Social Research | Understanding of Return to Player messages
Screening Questions
Can I just ask you a few questions to see if you are eligible to take part in the
interview?
Q1 Do you use ever play category B game machines, such as slot
machines, reels or fruit machines?
o [If the participant is unsure]: The types of machine we are interested
are called ‘type B’ machines. There are lots of variations between the
machines and the names of the games you can play on them. They
typically have a top prize of £500 although the prize can be bigger for
machines in casinos.
Q2 Through which of the following venues did you hear about the
research?
 List of participating venues [not shown to protect confidentiality]
Q3 Would you be available to take part in an interview at ….? We can
do daytime and evening slots.
Q4 Sex of participant (do not ask this, just record)
Male 
Female 
Q5 Are you?
18-30 
31-50 
Or 51+ 
Q6 What is your highest qualification?
A-level or higher 
GCSE or lower 

NatCen Social Research | Understanding of Return to Player messages
51
Q7 How often do you use gaming machines?
At least once a week
At least once a month
Less than once a month
Less than once every 6 months
Q8 How long have you been using gaming machines for?
Less than 6 months 
More than 6 months 
Check your individual quota sheets to decide if the participant is eligible to take part in an
interview or is a reserve.
If the participant is not eligible, thank them and ask if we can contact them to take part if
someone else drops out on the day. If participant eligible ask them what time appointment they
would like.
Collect contact details for ALL screened in people::
Explain that you will be sending the written note to the participant confirming details of the study
and the time/ location of the interview (this can be sent by post or email depending on their
preference).
Thank the participant

52
NatCen Social Research | Understanding of Return to Player messages
Appendix B. Interviewer protocol
This appendix contains the interviewer protocol used during the interviews.
P11125 Return to Player: User-testing probe sheet
Stage 1: Introduction
Aims: To introduce the study and collect informed consent.
 Thank participant
 Introduce yourself and NatCen Social Research. We are an independent,
not for profit, research organisation.
 Explain we are doing this study on behalf of the Responsible Gambling
Trust. The project is about how clear the messages on games machines
are (such as slot machines and fruit machines).
o The types of machine we are interested are called ‘category B’ machines.
There are lots of variations between the types of games you get on these
machines. They typically have a top prize of £500 although the prize can be
bigger for machines in casinos.
o INTERVIEWER: Double-check that the respondent plays ‘category B’
machine [if required can show example picture or describe where these
machines are in the venue].
 Explain that different messages, by law, have to be displayed on games
machines. During the interview they will be shown these messages and
asked how clear they think the messages are and whether they could be
improved.
 Taking part is entirely voluntary – emphasise that this isn’t a ‘test’ and
there are no ‘right or wrong’ answers. If they find the messages unclear
it is really important that they say so, because if they do other people
will too. We want to improve the messages if people don’t find them
helpful.
 Stress confidentiality. The findings from all the interviews will be written
up as report on what people think if the messages. We never include the
names of the people who take part in our reports.
 Explain that you will be recording the interview so that you don't have to
make lots of notes during the interview.
o Recorder is encrypted and only the research team at NatCen will have
access to the recordings.
o Check this is OK with the respondent.
 The interview will last around 30 minutes. Everyone who takes part
receives a £20 high street voucher to thank them for their time.
 Ask whether they have any questions before you start.

NatCen Social Research | Understanding of Return to Player messages
53
Stage 2: Context of using venue/ gaming
Aims:
 To ‘warm participant up’.
 To double-check screening details are correct.
 To get background information on why the participant comes to the
venue.
 To get background information describing the situations in which the
participant plays game machines.
 To get background information on how participant selects their gaming
machine.
 To start to explore what information (if any) the participant looks for
when selecting a machine.
 Explain before we show the messages we would like to collect a little bit
of background about you
 Double check details from screening.
 Explore use of venue:
o Length of time since they started coming to venue/other venues of this type.
o Frequency of attending.
o Who they come with (alone/ with others).
o Activities they like to do at the venue.
 Use of games machines
o Length of time since they started playing play games machines.
o Frequency of playing
 How often they use machines
 How long they spend on them once they start playing
o When do they play games machines.
 Other activities they do [if any] whilst gaming
 Who do they play with (alone/ with others)
 Factors that impact choice of gaming machine.
o PROMPT: What else?

54
NatCen Social Research | Understanding of Return to Player messages
Stage 3: Think aloud training and placement of RTP message 1
Aim:
 To explain the ‘Think Aloud’ process and to encourage thinking aloud
behaviours.

Explain that you are about to show the participant some messages that are
shown on games machines. Whilst they look at these you want to know
their first impressions of these messages and what they are thinking about.

Explain the ‘Think Aloud’ technique using a scenario of your choice (the
windows example is given below but you are welcome to use an
alternative). Part of this is to get the participant used to talking.

Encourage think aloud through rest of interview…
Stage 4: Understanding of RTP message 1
Aims:
 To get first impressions of a standard RTP message
 To explore whether the participant has seen these messages before
and whether they use this information when selecting a machine.
 To explore understanding of the RTP message in more detail using a
quiz item.
 Show the participant message 1: “THIS MACHINE HAS AN AVERAGE
PERCENTAGE PAYOUT OF AT LEAST 90%”
 Give them time to read this… PROMPT: Tell me what you are thinking?….
General probes on RTP messages
 Have you seen a message like this before?
 In your own words what do you think this message means?
 What does the word ‘average’ mean?
 What does ‘average payout’ mean?
Windows example: I would like you to have a practice at ‘thinking aloud.’ I am
going to ask you a question and I want you to tell me what you are thinking as
you work out your answer…
‘How many windows are there in your home?’ Please describe what you are
thinking as they work out your answer….

NatCen Social Research | Understanding of Return to Player messages
55
 Do you use this message, or similar messages when deciding which
machines to play? Why/ Why not?
 Explain that the message you have just shown them is called a ‘return to
player’ message. Explain you would now like to explore their understanding
of the message in more detail. This is important as we want to check
whether the messages could be made clearer.
 Explain that you are going to give them a multiple choice question on a
piece of paper. You would like them to read the options and tick the box
which is closest to what the message means.
 As they read each option you would like them to think aloud about why they
are selecting it or not selecting it.
 Give the respondent the quiz.
Follow up probes on RTP messages
 How easy or difficult did you find it to pick a statement? Why?
 Explain that the closest answer to what the message means is OPTION 4.
[Now you know this] how clear do you think the message is?
 Could the message be more clear and, if so, how?
 How useful do you think this message is for game players? Why do you say
that?
Stage 5: Understanding of RTP message 2
 Aim: To explore whether participants understand messages on
compensated play versus random play and whether these could be
made clearer.
 Show the participant message 2a: THIS GAME IS COMPENSATED AND MAY BE
INFLUENCED BY PREVIOUS PLAY
 Give them time to read this… PROMPT: Tell me what you are thinking….
Probes on compensated play
 Have you seen a message like this before?
 In your own words what do you think this message means?
 Show the participant message 2b: THIS GAME IS RANDOM

56
NatCen Social Research | Understanding of Return to Player messages
Probes on random play
 In your own words what do you think this message means?
 What do you understand by the word ‘random’ in this context?
 What do you think is the difference between a machine that is
‘compensated’ and a machine that is ‘random’?
 Explain the different between a compensated game and a random game:
 READ OUT: A ‘compensated game’ is when the odds of winning a prize are
influenced by how much money the game has already given out. For
example, if the game has just given out a prize the likelihood of winning a
prize may decrease. A ‘random’ game is not influenced by previous games
at all (the odds of winning are the same regardless of whether or not the
game has just given out a prize).
Further probes on compensated play and random play
 How useful do you think this message is for game players? Why do you say
that?
 Do you use these messages, or similar messages, when deciding which
machines to play? Why/ Why not?
 Could message 2a (on compensated play) be made clearer and, if so, how?
 Could message 2b (on random play) be made clearer and, if so, how?
Stage 6: Understanding of RTP message 2
Aim: To explore whether participants understand variations and caveats
that are shown with RTP messages.
 Show the participant message 3: THE OUTCOME OF ANY GAME OR FEATURE
IS NOT THAT NECESSARILY SHOWN BY THE ODDS DISPLAYED
 Give them time to read this… PROMPT: Tell me what you are thinking….
 Interviewer note: This message is trying to convey the fact that visual display
of a game [e.g. the number of symbols per reel] is not indicative of the odds
of winning. For example, there may be 12 symbols per reel but the odds of
any symbol occurring is NOT 1 in 12].
Probes on how games have variable odds dependant on game features
 In your own words what do you think this message means?
 [Explain the meaning of the message to participants] Could this message
be made clearer? How?
 How useful do you think this message is for game players? Why do you say
that?

NatCen Social Research | Understanding of Return to Player messages
57
 Would you personally want any information on
how
game features influence
the likelihood of winning a prize? What sort of information would you want?
 Show the participant message 4: AT 50P STAKE, THIS GAME HAS AN
AVERAGE PERCENTAGE PAYOUT OF AT LEAST 88%, AT £1 STAKE,
THIS GAME HAS AN AVERAGE PERCENTAGE PAYOUT OF AT LEAST
90% etc
 Give them time to read this… PROMPT: Tell me what you are thinking….
Probes on how games have variable odds dependant on stake size
 In your own words what do you think this message means?
 What does the term ‘stake’ mean?
 Could this message be made clearer? How?
 How useful do you think this message is for game players? Why do you say
that?
Stage 7: What messages would be most useful for gamers going forward
Aims: To explore what information participants think should be displayed
on machines in relation to the likelihood of winning a prize. This includes:
 The odds of winning any prize/ the jackpot.
 The volatility of the game (e.g. whether small prizes are given
frequently or large prizes are given infrequently).
 Average hourly loss rate.
 The total amount of money bet per session.
 The cost of a gaming session (the money put in minus the money in
the bank).

Explain you are now going to talk to them about what alternatives there
might be to Return to Player messages.
Probes on alternative messages on win chances
 What types of messages, if any do you think should be on machines to
explain win chances to players?
 How should these messages be phrased?
 I am now going to read you a list of things that other people have
suggested may be useful. For each suggestion please say whether or not
you think this type of message would be useful to have on gaming
machines:
1. The odds of winning
any
prize per game [Why do you think this would/
wouldn’t be useful?]
2. The odds of winning the jackpot prize per game [Why do you think this
would/ wouldn’t be useful?]
3. The odds of winning prizes of different sizes for example whether small
prizes are given frequently or large prizes are given infrequently. [Why
do you think this would/ wouldn’t be useful?]

58
NatCen Social Research | Understanding of Return to Player messages
4. The average hourly loss rate. [Why do you think this would/ wouldn’t be
useful?]
5. The total amount of money you have put in the machine in your gaming
session. [Why do you think this would/ wouldn’t be useful?]
Explore with
participants whether ‘money put in’ should include prize money that has
been re-staked using credit transfer or not.
6.
A record of the amount you have put in a single gaming session minus
the money you have in the bank [Why do you think this would/ wouldn’t
be useful?]
Explore with participants whether money put in should include
prize money that has been re-staked using credit transfer or not.
 Are there any other types of messages about win chances that you think
could be useful for game players?
End of interview

Thank the participant for their time and give them their incentive voucher.

Provide the ‘further resources’ leaflet. Explain that you will be giving this to
everyone who takes part.

Reassure them about confidentiality

Answer any questions they may have

NatCen Social Research | Understanding of Return to Player messages
59
Appendix C. Showcards
The following showcards were presented to participants at appropriate points during
the interview.
Message 1
“THIS MACHINE HAS AN
AVERAGE PAYOUT OF AT
LEAST 90%”

60
NatCen Social Research | Understanding of Return to Player messages
Message 2a
“THIS GAME IS
COMPENSATED AND MAY BE
INFLUENCED BY PREVIOUS
PLAY”

NatCen Social Research | Understanding of Return to Player messages
61
Message 2b
“THIS GAME IS RANDOM”

62
NatCen Social Research | Understanding of Return to Player messages
Message 3
“THE OUTCOME OF ANY
GAME OR FEATURE IS NOT
NECESSARILY THAT SHOWN
BY THE ODDS DISPLAYED.”

NatCen Social Research | Understanding of Return to Player messages
63
Message 4
AT 50P STAKE, THIS GAME HAS AN AVERAGE
PERCENTAGE PAYOUT OF AT LEAST 88%
AT £1 STAKE, THIS GAME HAS AN AVERAGE
PERCENTAGE PAYOUT OF AT LEAST 90%
AT £2 STAKE, THIS GAME HAS AN AVERAGE
PERCENTAGE PAYOUT OF AT LEAST 92%

64
NatCen Social Research | Understanding of Return to Player messages