Content uploaded by Jacqueline Douglas
Author content
All content in this area was uploaded by Jacqueline Douglas on Dec 15, 2015
Content may be subject to copyright.
The TQM Journal
Mystery shoppers: an evaluation of their use in monitoring performance
Jacqueline Douglas
Article information:
To cite this document:
Jacqueline Douglas , (2015),"Mystery shoppers: an evaluation of their use in monitoring
performance", The TQM Journal, Vol. 27 Iss 6 pp. 705 - 715
Permanent link to this document:
http://dx.doi.org/10.1108/TQM-04-2015-0052
Downloaded on: 08 December 2015, At: 04:08 (PT)
References: this document contains references to 27 other documents.
To copy this document: permissions@emeraldinsight.com
The fulltext of this document has been downloaded 61 times since 2015*
Users who downloaded this article also downloaded:
Chih-Hsing Liu, Sheng-Fang Chou, Bernard Gan, Jin-Hua Tu, (2015),"How “quality” determines
customer satisfaction: Evidence from the mystery shoppers’ evaluation", The TQM Journal, Vol. 27
Iss 5 pp. 576-590 http://dx.doi.org/10.1108/TQM-01-2013-0004
Alan M. Wilson, (1998),"The role of mystery shopping in the measurement of service
performance", Managing Service Quality: An International Journal, Vol. 8 Iss 6 pp. 414-420 http://
dx.doi.org/10.1108/09604529810235123
Margaret Erstad, (1998),"Mystery shopping programmes and human resource management",
International Journal of Contemporary Hospitality Management, Vol. 10 Iss 1 pp. 34-38 http://
dx.doi.org/10.1108/09596119810199318
Access to this document was granted through an Emerald subscription provided by emerald-
srm:371169 []
For Authors
If you would like to write for this, or any other Emerald publication, then please use our Emerald
for Authors service information about how to choose which publication to write for and submission
guidelines are available for all. Please visit www.emeraldinsight.com/authors for more information.
About Emerald www.emeraldinsight.com
Emerald is a global publisher linking research and practice to the benefit of society. The company
manages a portfolio of more than 290 journals and over 2,350 books and book series volumes, as
well as providing an extensive range of online products and additional customer resources and
services.
Downloaded by Liverpool John Moores University At 04:08 08 December 2015 (PT)
Emerald is both COUNTER 4 and TRANSFER compliant. The organization is a partner of the
Committee on Publication Ethics (COPE) and also works with Portico and the LOCKSS initiative for
digital archive preservation.
*Related content and download information correct at time of
download.
Downloaded by Liverpool John Moores University At 04:08 08 December 2015 (PT)
Mystery shoppers:
an evaluation of their use
in monitoring performance
Jacqueline Douglas
Liverpool Business School, Liverpool John Moores University, Liverpool, UK
Abstract
Purpose – The purpose of this paper is to examine what mystery shopping is, why it is used and how
mystery customers are trained and how the information collected is fed back to the client organisation.
Design/methodology/approach – The approach was to use an online survey of mystery shoppers
compares the reality of the situation with the best practice identified from the extant literature.
Findings – The main outcome was that results identify good and bad practices in all areas of the
process and guidelines for the recruitment, training and monitoring of mystery shoppers are proposed
including in-depth training in all aspects of the job.
Research limitations/implications – A sample of 85 mystery shoppers was used and only in the
UK. It would be interesting to widen this out internationally.
Practical implications – Mystery shoppers are used worldwide by services to evaluate the
performance of their front-line people and processes but are their evaluations valid and reliable? This
research identifies good and bad practice which should help managers to design their training for
mystery shoppers.
Originality/value – The paper addresses a gap in the literature on the perceptions of mystery shoppers.
Keywords Service quality, Mystery customers, Validity and reliability
Paper type Research paper
1. Introduction
Service managers have traditionally used service standards to manage the service
delivery process. First they set the standard and then they measure performance
against that standard, taking action when standards have not been met. The main
difficulties with monitoring performance are associated with the heterogeneous nature
of services, their perishability, their blend of tangible and intangible elements and the
fact that consumption takes place simultaneously with production (Fitzsimmons, 2012).
It is this simultaneity of production and consumption in particular that has added to
the complexity of monito ring what is now commonly referred to as the moment of
truth. This means that the customer is present and therefore part of the delivery
process and able to interact to varying degrees with the service provider.
One method for monitoring this moment of truth and the processes and standards
that support it that has increased in popularity in recent years is the use of mystery
shoppers or mystery customers as they are sometimes called. This study focuses on
what the experience of mystery shoppin g is like for the mystery shopper. It examines
recruitment and training, the evaluation visit, data collection, reporting methods and
feedback, as well as the reliability and validity of this particular method of obtaining
customer intelligence in order to determine its effectiveness. These aims will be
achieved by examining the literature on mystery shopping and by reporting the results
and analysis of a survey of mystery shoppers undertaken to gather their views on what
they do and how they do it.
The TQM Journal
Vol. 27 No. 6, 2015
pp. 705-715
© Emerald Group Publishing Limited
1754-2731
DOI 10.1108/TQM-04-2015-0052
Received 17 April 2015
Revised 3 August 2015
Accepted 7 August 2015
The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com/1754-2731.htm
Appreciation is given to S. Afzal for the collection and analysis of the primary data.
705
Mystery
shoppers
Downloaded by Liverpool John Moores University At 04:08 08 December 2015 (PT)
2. Mystery shopping
2.1 What is mystery shopping
Wilson (1998a) defines mystery shoppers as researchers who “act as customers or
potential customers to monitor the quality of processes and procedures used in the
delivery of a service (p. 414). Xu and He (2014) found that this method has been in use
since the 1940s. The emphasis is on the service experience as it unfolds, looking at which
activities and procedures which do or do not happen rather than gathering opinions”
(p. 415). The aim is to obtain facts rather than perceptions. In their research into mystery
shopper protocol, Peterman and Young (2015) observed that mystery shoppers
documented their experiences in order to provide a unique perspective on the situation.
Mystery shopping is an example of concealed participant observation in a public
setting (Calvert, 2005). It uses a structured approach of checklists and codes to
gather and measure specific information about service performance in everyday
encounters (Grove and Fisk, 1992). Mystery shopping data are not used to evaluate
individual performance and so findings are always reported on an anonymous basis
(Kehagias et al., 2011).
Mystery shopping involves a mystery shopper making exact checks against specified
criteria (usually standards of service) whilst being provided with a service, whereas,
retrospective customer research relies on what the customer can recall, making it more
subjective and less reliable and accurate. It could be argued that real customers surveyed
at a later date cannot recall their experiences exactly and will therefore, offer only
subjective perceptions of them (Finn and Kayande, 1999). Mystery shopping may take a
number of forms such as branch visits, telephone calls and e-mail checks in order to
measure compliance with processes, systems and standards of service.
2.2 Why use mystery shopping
Mystery shopping is seen as an efficient and effective instrument to gain in-depth
knowledge of the customer’ s perception of service delivery (Finn, 2001). If an
organisation is communicating the expectations of management an d customers
through standards of service then there is a need to measure performance against thes e
standards so that an organisation can ensure that it is delivering what it promises and
maintaining its competitive position. This is particularly true when dealing with the
performance of sales staff whose job it is to sell products and services to customers
(Ramesh, 2010). Myster y shopping is a technique that can do this as it aims to collect
facts rather than perceptions. These facts can relate to almost any aspect of the service
transaction covering topics such as number of rings before the telephone was
answered, length of time in a queue, number of checkouts open and the form of greeting
used. They can include more complex encounters such as a loan application or a
request to open a bank account where the procedures adopted can be assessed in terms
of service quality and financial compliance (Wilson, 1998b). It is the quality of the
collected data that differentiates this technique from other means of evaluation.
The extant literature on mystery shopping has identified a number of other
advantages associated with the technique. Mystery shopping results can be used as a
diagnostic tool to identify failings or weaknesses in procedures and processes,
to encourage, develop and motivate staff and to assess an organisation’s competitiveness
by benchmarking them against others in the same sector (Wilson, 1998b). Erstad (1998,
p. 34) linked mystery shopping techniques to “building a team spirit, evaluating/
identifying training needs, providing feedback, and linking performance to incentives
and rewards”. It can bring immediate service improvements with continuous
706
TQM
27,6
Downloaded by Liverpool John Moores University At 04:08 08 December 2015 (PT)
improvement possibilities (Cabinet Office, 2004). It can also be used to measure the
effectiveness of training programmes (Morrison et al., 1997) and checking whether all
customers are treated equally (Morrall, 1994; Tepper, 1994). Thus, what Berry et al. (1988)
describe as the service-quality loop of service standards, employee performance, training
and rewards can all be linked by using mystery shopper evaluations in a positive way.
However, there are some disadvantages to using mystery shoppers, such as
employees may view the evaluation process as threatening in that they may perceive it as
the management “spying” upon them, with a view to instigating disciplinary action for
any mistakes they make, rather than as a trigger to improve staff training (Erstad, 1998).
It reviews processes not their outcomes (Wilson, 1998a) and this can be problematic for
many services, for example, the service was excellent but the meal was inedible.
However, some organisations have now started to examine outcomes, particularly where
there is a tangible element involved. For example, fast food chains are now evaluating the
meal – the outcome of the service. If used as an ongoing or regular method of evaluation
it will result in a constant need for mystery shoppers. This will inevitably utilise staff
time, impact on training and also on finances particularly where an external agency is
used. Furthermore, staff can become complacent about the level of service they provide
after the novelty of being mystery shopped wears off (Wilson, 1998a). Memory demands
placed on assessors could affect the accuracy of the information obtained (Morrison et al.,
1997). As is the case with all sampling techniques it offers a snapshot of the service
process, which may not be representative (Cabinet Office, 2004). It has been suggested
that mystery shopping violates the principle of informed consent, as staff are not aware
at any stage that they are being observed and their behaviours monitored. According to
good research practice subjects should be protected through the practice of informed
consent (Silverman, 2000). Subjects have the right to know they are being observed.
Another potential problem is staff attempting to play “spot the mystery shopper”.
However, this can be alleviated to a certain extent by making the assignment brief as
credible as possible and by not using the same shoppers every time. These issues can
affect the reliability of mystery shopping as a technique to measure customer service. It is
the reliability and validity issues that are examined in the next section.
2.3 Validity and reliability
Mystery shopping is a form of participant observation, albeit that the observation is
carried out secretly (Calvert, 2005). Jesson (2004) with particular reference to pharmacy
practice questioned whether mystery shopping techniques were methodologically
sound ways of collecting data on performance. Specifically, the method and sample
sizes have been criticised from reliability and validity aspects. In order for the data to
be valid and reliable Hesselink and Van der Wiele (2003) argued that the mystery
shopping study had to be well-designed with particular attention on the process itself,
the data gathering process, the person doing the study (the mystery shopper) and the
reporting process. Finn and Kayende (1999) examined the psychometric quality of
mystery shopping data and concluded that the data collected held up to reliability and
validity tests particularly when compared to data collected from customer surveys.
With regards to the generalisability of the data Collins and Turner (2005) argued that
although sample sizes were small, each observation was valid in its own right since it
was a “snapshot” of the service experience at one moment in time and as such was not
trying to represent the “population” of all such experiences.
A mystery shopper will visit an establishment, note whether standards are being met
and then complete an evaluation feedback form later and in private. This procedure relies
707
Mystery
shoppers
Downloaded by Liverpool John Moores University At 04:08 08 December 2015 (PT)
on the assessors’ memory, and two problems can arise from memory failures on their
part. First, an assessor may forget to check attainment of one or more standards on the
list, i.e. not complete the assignment brief fully, and second, the assessor may record
information incorrectly. Factors associated with encoding, storage and retrieval of
information are likely to influence the accuracy of the results reported by mystery
shoppers. Apart from problems related to memory other psychological processes can
exert an influence. The time of day at which the assessor visits the premises may affect
the encoding accuracy due to fatigue which can affect perceptions, especially when a
large number of observations are required (Guerrien et al.,1993).
A mystery shopper feedback evaluation reports the perceptions of an individual
rather than a representative interaction and is influenced by the behaviour and
appearance of both participants. The effects of individual differences between
assessors can be reflected in ass essments. For example, in depar tment stores men tend
to get priority over women (Morrison et al. , 1997). Also, the style of dress chosen and
gender interaction can further influence service priority (Stead and Zinkham, 1986).
Women are also more likely to provide more accurate mystery shopper reports than
men and the age of the assessor can affect mystery shopper reports with young adults
being the most reliable (Morrison et al., 1997).
2.4 Recruitment and training
Recruitment of the right assessor is vital as they need to be typical of the organisations’
customer base or they will be easily identified as a mystery shopper. This makes
selection more difficult the more comple x the situation to be evaluated (Wilson, 1998b).
The nature of mystery shopping requires shoppers who are “objective, smart and able
to think on their feet” (Leeds, 1992, p. 25). Wilson (1998b, p. 155) argues that for most
scenarios the adopted persona of a mystery shopper should be “neutral rather than
aggressive” and recruits should be selected on the basis of having such a personality.
Further research indicates mystery shoppers should be independent, critical, objective
and anonymous.
Therefore the recruitment process for mystery shoppers should be robust. Intensive
training should also be given so that they have a thorough understanding of
their assignments (Leeds, 1992). The type of training received can also impact the
reliability of the results (Wilson, 1998a). Training in data collection skills is important
and shoppers should receive memory testing and training because sometimes the
information collected has to be memoris ed until the service encounter is complete
such as an interview with a bank manager concerning the opening of an account
(Wilson, 1998a).
2.5 Feedback
Whatever the finding s of mystery shopper exercises, employees need to be given
feedback on the results in a positive manner (Burnside, 1994). Management must be
informed of what value has been obtaine d from mystery shopping programmes
(Morrell, 1994) and any changes necessary to improve customer service. Results in
some organisations can be linked to bonuses, awards and prizes (Dorman, 1994).
However the mystery shoppers should also receive feedback on their performance and
the quality of their reports from their employing organisation.
To summarise, mystery shopping has been defined as an objective method of
monitoring the service as it unfolds. It is a documented, systematic process that has
strengths and weaknesses associated with it.
708
TQM
27,6
Downloaded by Liverpool John Moores University At 04:08 08 December 2015 (PT)
2.6 Research method, data and analysis
A questionnaire was developed based on the issues arising from the literature and
previous studies on mystery shopping. The questionnaire contained a mix of quantitative
and qualitative questions. Question types included self-selection tick box, ranking and
five-point Likert scales asking participants the extent to which they agreed or disagreed
withaseriesofstatementsaswellasaseriesofopenquestionsthatrequiredmorelengthy
answers and explanations. The instrument was pre-tested and based on comments
received some minor amendments were made to the questionnaire. Where appropriate,
responses were analysed using SPSS v 20. Participants were recruited via advertising the
questionnaire as a web link on UK mystery shopping message boards on the internet.
A sample of 85 mystery shoppers participated in the study consisting of 22 males and
63 females. The participants were from all regions of the UK and from a range of social
backgrounds. The “professional” category included nurses, teachers, actors, accountants
and managers. The “other” category consisted of mothers and housewives with caring
responsibilities, child minders, carers and retired people. The respondents’ ages ranged
from 21 to over 60. The mode age range was 31-40 and comprised 43.5 per cent of the
sample. The mean age was 35. There were no participants less than 21 years old in the
sample studied. The length of mystery shopping experience varied. The mean length of
experience was two to three years, with 12.9 per cent of sample having over five years
experience. The most experienced respondents had at least ten years experience.
2.7 Key findings
2.7.1 Approach to mystery shopping. A qualitative analysis of the responses indicated
that mystery shoppers tend to integrate their evalua tion visits into their daily lives and
make it par t of their shopping experience. A typical day involved completion of
between one and ten assessments and involved meticulous planning to ensure that they
understood the requirements of the scenarios. After the mystery shop was conducted
notes were taken, usually in their car, and the report was submitted immediately if no
further assignments were to be conducted. Other duties involved, checking e-mails,
submitting reports, preparing for visits and answering an y queries raised on
assignment reports previously submitted. The preparation for visits was anything
from 30 minutes to two hours if it involved reading briefs and making up false details.
2.7.2 Deception and guilt. Analysis indicated that mystery shoppers did not view
adopting the role of a customer as deception since staff employed by companies would
normally have been made aware that mystery shoppers would assess them and this would
form part of their contract of employment. They felt it was vital to capture a true account of
the customer experience and staff performance and this could only be done covertly. If it
werenotcovertthenstaffwouldbehavedifferently.Theyfelttheywereactingonbehalfof
consumers generally. They argued that if staff became complacent in their work and that
was not reported to corporate head office then the company image would suffer. Mystery
shopping was perceived as a realistic way to gain information on how a company operated.
They felt uncomfortable in giving negative feedback especially if a staff member
was being helpful but not following company policy. However, overall mystery
shoppers felt that theirs’ was an objective method of capturing information and was the
only way to get a true reflection of customer service levels.
With regards to being challenged, 89.4 per cent of respondents reported not being
challenged by staff when conducting visits. Analysis indicated sex and age of the
mystery shopper was not a significant factor for those who did report being challenged.
709
Mystery
shoppers
Downloaded by Liverpool John Moores University At 04:08 08 December 2015 (PT)
2.7.3 Qualities of a good mystery shopper. Respondents were asked to rank the eight
qualities needed to be a good mystery custome r. Table I ind icates their rankings.
Analysis indicated that there was a significant difference regarding the qualities
required to be a good mystery shopper depending on the experience of the respondent.
Those with over three years experience identified “able to think on your feet” as a key
quality. Sex and age did not appear to have a significant effec t on qualities reported.
2.7.4 Reporting methods are their perceived effectiveness. Table II shows which
methods respondents used to report the results of their evaluations and their views on
the effectiveness of those methods.
The above table clearly shows the popularity of the internet for giving fast feedback
on a mystery visit and it was considered to be the most effective method. Video
mystery shopping is clearly still in its infancy. However of the nine (10.5 per cen t)
respondents who had used video as a reporting method eight (88.8 per cent) rated it as
the most effective method. Analysis indicated that where a particular method was used
it was rated highly as an effective reporting method. Analysis indicated that the age
and sex of the assessor had no influence on the reporting method used. However, the
same analysis indicated an association between the length of mystery shopping
experience and the reporting method used. Those respondents with under three years
mystery shopping experience reported a higher usage of paper-based reporting
methods compared to those with over three years experience.
2.7.5 Post evaluation issues. With regards to giving negative feedback on a mystery
shop, 27.1 per cent of respondents reported that they felt guilty whilst 72.9 per cent said
they did not. Regarding uncertainty of what to include in reports, over half of
respondents (51.8 per cent) admitted feeling unsure at times what to inc lude. A majority
(59.2 per cent) of respondents relied on checklists to recall the information required for
completing the report. When it came to reporting the visit 48.1 per cent of respondents
found the deadlines they had been given difficult to meet. Howeve r, the vast majority of
Quality Ranking
Ability to follow briefs 1
Ability to think on one’s feet 4
Critical 7
Anonymous 5
Independent 6
Objective 2
Smart 8
Work to deadlines 3
Table I.
Qualities of a good
mystery shopper
Reporting method Used Not used Most effective (%) Least effective
a
(%)
Web/internet 84 1 60.5 4.9
Telephone 69 16 2.5 22.2
Paper 71 14 2.5 27.2
Video 9 76 34.5 43.2
Note:
a
2.5 per cent of respondents did not answer this question
Table II.
Mystery shopper
reporting methods
710
TQM
27,6
Downloaded by Liverpool John Moores University At 04:08 08 December 2015 (PT)
mystery shoppers in this sample (82.7 per cent) believed that their report was a true
reflection of the customer experience.
The majority of mystery shoppers tended to complete their reports within four
hours of a mystery shop having taken place. The mean time taken to complete reports
was 2.16 hours – see Table III.
2.7.6 Training and its perceived effectiveness. Table IV shows the main types of
training undertaken by respondents.
The most popular training method was web based which was an interactive training
package and nearly 90 per cent of mystery shoppers in the study had received this. The
second most popular training method was paper-based manuals with nearly 80 per cent
of mystery shoppers having received this type of training. Nearly two-thirds of
respondents had received mock mystery shop training that involved following a brief
and doing a trial shop and reporting the results to the client company. A very small
number (2.5 per cent) had received no training at all. The least used training method was
that of being accompanied by an experienced assessor. Other ad hoc training methods
included telephone conferencing or using audio-visual materials.
2.7.7 Receiving feedback . Nearly 60 per cent of mystery shoppers received feedback
from the employing mystery shopping agency on their completed evaluation reports.
The type of feedback received varied. Nearly 47 per cent of respondents had been
praised for the excellent quality of their report, 7.1 per cent had corrections made to
their grammar, 4.7 per cent had recorded information in the wrong sections of the
report, 22.4 per cent had had the accuracy of their report challenged by the client and
5.9 per cent had been criticised for non compliance with the brief. Other feedback
reported included the grading of submitted reports. However from the sample surveyed
no one reported receiving feedback on his or her report not being submitted on time.
2.7.8 Changes in staff behaviour. Nearly two-thirds of respondents (63.3 per cent)
reported changes in staff behaviour during their visit indicating that staff had been
alerted to their mystery shop. Respondents noticed briefing notes/posters on staff
notice boards, computers and shop windows announcing that a mystery shopper visit
was due and for staff to be alert. Scores from previous visits were also prominently
Time after mystery shop evaluation had taken place when report
was completed
Less than 2 hours 2-4 hours 4-6 hours 6-8 hours Over 8 hours
Number of respondents 29 28 17 7 4
Table III.
Length of time
to submit reports
after mystery
shop evaluation
has taken place
Type of training Yes (%) No (%)
Accompanied by an experience assessor on visits and observed 6.2 93.8
Online web-based training 88.9 11.1
Paper-based manuals/briefs 79 21
In-house specialist training course 18.5 81.5
Mock mystery shop with feedback 60.5 39.5
Table IV.
Main types of
training methods
711
Mystery
shoppers
Downloaded by Liverpool John Moores University At 04:08 08 December 2015 (PT)
displayed and staff had been overheard discussing upcoming mystery shopping
visits. When an enquiry was made the perception was that staff were waiting for the
“trigger question”. The subsequent following of the checklist would mean that the
mystery shopper was easily distinguishable from a “normal” customer. Some staff
became nervous and gave too much information. Others used gestures to try and warn
colleagues that they were dealing with a potential mystery shopper.
3. Discussion
The results indicate that mystery shoppers are highly effective in measuring customer
service elements as 89 per cent of those surveyed reported not being challenged by staff
whilst conducting an assessment. The age and sex of the assessor had no effect on this.
However, subtle behaviours were detected in some workplaces that could have influenced
staff behaviours when faced with a potential mystery shopper. This supports findings by
other researchers (Marketing News, 1987) that in some cases employees are warned about
forthcoming visits. This could have an adverse affect on the validity of the findings.
Leeds (1992) identified being able to “think on feet” as a key quality of a mystery
shopper whereas Van der Wiele et al. believed mystery shoppers should be anonymous,
independent, critical and objective. This study ranks the contention of the former as
more important.
The use of reporting methods influences how the information is recorded on the
report that in turn influences how the report is scored by the mystery shopping company.
The web-based reporting forms proved to be highly popular and were rated as effective.
However, there were several problems raised with their design including web-based tick
box forms where if multiple selections were made there was no space provided for
comments and hence mystery shoppers were not able to describe a particular aspects of
their experiences. This posed a problem where an unusual event occurred. Mystery
shoppers felt this restriction on web-based forms hindered them in providing potentially
valuable feedback to the client. Another problem was not having an “other” option box if
none of the responses available on the report form were applicable.
Despite these problems the benefits of web reporting methods were reported as
numerous and included the ability to upload results quickly which allowed feedback to
be given to the client and the mystery shopping company within 48 hours of the
mystery shop having taken place. Suggestions for improvements to the web-based
reporting process included having the ability to upload photographs of problem areas
such as faulty toilets or poor displays in outlets.
The experience of video reporting methods was rate d as very effective due to the
covert nature of the equipment and the ability to capture non-verbal behaviour, body
language and style of communication used in comparison to other methods. The use of
video placed less demand on an assessor’s memory to retain and recall information
over long time periods.
The difference in attitudes on effectiveness of mystery shopping was found to be
highly significant in this study. “Guilty feelings” were identified as an important
attribute when giving negative feedback. This links with how assessors may be subject
to subtle social pressures and may wish to give favourable reports of customer service
because of empathy with people working in those establishments and hence would find
it difficult to maintain objectivity. Women relied heavily on using checklists to recall
information and men were unsure at times what to include in the report. These findings
partially support the contention of Morrison et al. (1997) in that women provide more
accurate feedback due to their use of memory testing techniques. Men, in comparison,
712
TQM
27,6
Downloaded by Liverpool John Moores University At 04:08 08 December 2015 (PT)
appear uncertain of exactly what to include in reports and so are at greater risk of the
report being challenged by the mystery shopping client if it was deemed not to be a fair
reflection of customer experiences (as perceived by the employee if om issions and
errors were identified).
The most popular training received was web based due to accessibility and
convenience. The web-based assessments had to b e completed successfully before the
mystery shopper was allowed to conducted “real” evaluations. There were variations
in the type of training received, some respondents receiving none or were simply
given a m anual to read. Others felt that the “ability to think and act quickly” was
more important than training. The recomm endations for improving training included
clearer briefing notes, realistic in-depth scenarios and being able to examine samples
of correctly completed evaluation reports. Other recommendations included having to
work for a trial period where all asse ssments w ere monitored and checked for
report quality. From the sa mple surv eyed those who had accomp anied an
experienced assessor as part of their training rated this m ethod as highly effective
as it gave a more accurate assessment of a candidate’s suitability. Respondents
identified preparing for visits, completion of reports and common errors as
important elements of their trainin g and these factors sho uld be incorpora ted into all
training programmes.
4. Conclusions
This paper has identified both good and bad practices in the recruitment, training and
use of mystery shoppers. Cle arly bad practice needs to be eliminated. Responsibility for
this is with the mystery shopper employment agencies. The findings indicated that
there are varying approaches to the visits and the method of reporting back. There
appears to be no feelings of guilt or deception, although some discomfort can be felt
when reporting on negative behaviours. The type of training offered also varied.
Agencies need to follow some straightforward guidelines regarding recruitment and
use of mystery shoppers, including:
•
Recruit the right people and match people profile with the client brief.
•
Train recruits in all areas of the job including different monitoring techniques
memory and observational skills, interviewing skills, avoiding detection,
understanding ass ignments briefs, the use of checklists and completing and
submitting reports.
•
Give all mystery shoppers periodic performance appraisals.
•
Ensure that client assignment briefs are robust and will not lead to the easy
identification of the mystery customer. Be prepared to refuse to take on a client if
the briefs are of poor quality.
Overall, the findings from this study concur with the literature that the use of mystery
customers, as a covert method of measuring performance, appears useful and effective.
However, there are many companies, particularly via the internet, offering mystery
shopping services and they are not as selective when it comes to recruitment and offer
no training to new recruits, despite Leeds (1992) suggestions that training should
be intensive. Client briefs are delivered via email and reports are submitted online.
This research suggests that the validity and reliability of data so gathered may be
questionable.
713
Mystery
shoppers
Downloaded by Liverpool John Moores University At 04:08 08 December 2015 (PT)
References
Berry, L.L., Parasuraman, A. and Zeithaml, V. (1988), “The service-quality puzzle”, Business
Horizons, Vol. 31 No. 5, pp. 35-43.
Burnside, A. (1994), “In store spies snuff out poor service”, Marketing, 28 April, pp. 32-33.
Cabinet Office Regulatory Impact Unit (2004), Code of Practice on Consultation, Cabinet Office,
London.
Calvert, P. (2005), “It’s a mystery: mystery shopping in New Zealand’s public libraries”, Library
Review, Vol. 54 No. 1, pp. 24-35.
Collins, J. and Turner, H. (2005), “The new truth about mystery shoppers”, The Wise Marketer,
March, available at: www.thewisemarketer.com (accessed 1 November 2014).
Dorman, K. (1994), “Mystery shopping results can shape your future”, Bank Marketing, Vol. 26
No. 8, pp. 17-21.
Erstad, M. (1998), “Mystery shopping programmes and human resource management”,
International Journal of Contemporary Hospitality Management, Vol. 10 No. 1, pp. 34-38.
Finn, A. (2001), “Mystery shopper benchmarking of durable-goods chains and stores”, Journal of
Service Research, Vol. 3 No. 4, pp. 310-320.
Finn, A. and Kayende, U. (1999), “Unmasking a phantom: a psychometric assessment of mystery
shopping”, Journal of Retailing, Vol. 75 No. 2, pp. 195-217.
Fitzsimmons, J.A. (2012), “Department of management University of Texas at Austin”,
Perspectives in Operations Management: Essays in Honor of Elwood S. Buffa, pp. 101-188.
Grove, S.J. and Fisk, R. (1992), “Observational data collection methods for services marketing:
an overview”, Journal of the Academy of Marketing Science , Vol. 20, pp. 217-224.
Guerrien, A., Leconte-Lambert, C. and Leconte, P. (1993), “Time of day effects on attention and
memory deficiency: is chronopsychology a method for studying the function of the human
subject?”, Psycholgica Belica, Vol. 33 No. 3, pp. 159-170, cited in Morrison et al. (1997).
Hesselink, M. and Van der Wiele, T. (2003), “Mystery shopping: in-depth measurement of customer
satisfaction”
, ERIM Report Series Research in Management, ERS-2003-20-ORG, March.
Jesson, J. (2004), “Mystery shopping demystified: is it a justifiable research method”,
The Pharmaceutical Journal, Vol. 272 No. 7299, pp. 615-617.
Kehagias, J., Rigopoulou, I. and Vassilikopoulou, A. (2011), “Linked mystery shopping inventory
to customer-seller encounters”, Journal of Customer Behaviour, Vol. 10 No. 1, pp. 7-34.
Leeds, B. (1992), “Mystery shopping offers clues to quality service”, Bank Marketing, Vol. 24
No. 11, pp. 24-26.
Marketing News (1987), “Mystery shoppers provide check on customer-service experience”,
Vol. 21, Marketing News, p. 12.
Morrall, K. (1994), “Mystery shopping tests service and compliance”, Bank Marketing, Vol. 26
No. 2, p. 13.
Morrison, L.J., Colman, A.M. and Preston, C.C. (1997), “Mystery customer research: cognitive
processes affecting accuracy”, Journal of the Market Research Society, Vol. 39, pp. 349-361.
Peterman, K. and Young, D. (2015), “Mystery shopping: an innovative method for observing
interactions with scientists during public science events”, Visitor Studies, Vol. 18 No. 1,
pp. 83-102.
Ramesh, M.A. (2010), “Mystery shopping with reference to durable goods – a Hyderabad
experience”, Journal of Marketing and Communication, Vol. 5 No. 3, pp. 37-45.
Silverman, D. (2000), Doing Qualitative Research: A Practical Handbook, Sage, London.
714
TQM
27,6
Downloaded by Liverpool John Moores University At 04:08 08 December 2015 (PT)
Stead, B. and Zinkham, G.M. (1986), “Service priority in department stores: the effects of customer
gender and dress”, Sex Roles, Vol. 15 Nos 11-12, pp. 601-611.
Tepper, G. (1994), “The merits of self-testing ”, Mortgage Banking, Vol. 54 No. 8, p. 76.
Wilson, A.M. (1998a), “The role of mystery shopping in the measurement of service performance”,
Managing Service Quality, Vol. 8 No. 6, pp. 414-420.
Wilson, A.M. (1998b), “The use of mystery shopping in the measurement of service delivery”,
The Service Industries Journal, Vol. 18 No. 3, pp. 148-163.
Xu, L. and He, S. (2014), “Analysis on the survey method of mystery shopping in hospitality
management”, E-Commerce, E-Business and E-Service, CRC Press, Taylor and Francis,
London, pp. 221-225.
Corresponding author
Jacqueline Douglas can be contacted at: j.a.douglas@ljmu.ac.uk
For instructions on how to order reprints of this article, please visit our website:
www.emeraldgrouppublishing.com/licensing/reprints.htm
Or contact us for further details: permissions@emeraldinsight.com
715
Mystery
shoppers
Downloaded by Liverpool John Moores University At 04:08 08 December 2015 (PT)
