Content uploaded by Ewan Ferlie
Author content
All content in this area was uploaded by Ewan Ferlie on Feb 16, 2016
Content may be subject to copyright.
An open letter to The BMJ editors on qualitative
research
Seventy six senior academics from 11 countries invite The BMJ’s editors to reconsider their policy
of rejecting qualitative research on the grounds of low priority.They challenge the journal to develop
a proactive, scholarly, and pluralist approach to research that aligns with its stated mission
Trisha Greenhalgh professor of primary care health sciences, Nuffield Department of Primary Care
Health Sciences, University of Oxford, UK, Ellen Annandale professor, Sociology, University of
York, UK, Richard Ashcroft professor of bioethics, Queen Mary University London, UK, James
Barlow professor of technology and innovation management–healthcare, Imperial College Business
School, UK, Nick Black professor of health services research, London School of Hygiene and
Tropical Medicine, UK, Alan Bleakley emeritus professor of medical education, University of Plymouth,
UK, Ruth Boaden professor of service operations management, Manchester Business School, UK
, Jeffrey Braithwaite professor of health systems research, Australian Institute of Health Innovation,
Sydney, Australia, Nicky Britten professor of applied healthcare research, University of Exeter
Medical School, UK, Franco Carnevale professor, Ingram School of Nursing, McGill University,
Canada, Kath Checkland professor of health policy and primary care, Centre for Primary Care,
University of Manchester, UK, Julianne Cheek professor, Faculty of Business, Languages and Social
Sciences, Ostfold University College, Norway, Alex Clark professsor, Faculty of Nursing, University
of Alberta, Canada, Simon Cohn reader in anthropology, London School of Hygiene and Tropical
Medicine, UK, Jack Coulehan professor emeritus, Department of Preventative Medicine, Stony
Brook University, NY, USA, Benjamin Crabtree professor, Department of Family Medicine and
Community Health, Rutgers University, NJ, USA, Steven Cummins professor of population health,
London School of Hygiene and Tropical Medicine, UK, Frank Davidoff executive editor, Institute for
Healthcare Improvement, Cambridge, MA, USA, Huw Davies professor of healthcare policy and
management, University of St Andrews, UK, Robert Dingwall professor of sociology, Nottingham
Trent University, UK, Mary Dixon-Woods professor of medical sociology, Department of Health
Sciences, University of Leicester, UK, Glyn Elwyn professor, Dartmouth Institute for Health Policy
and Clinical Practice, Dartmouth, NH, USA, Eivind Engebretsen professor, Institute for Health and
Society, University of Oslo, Norway, Ewan Ferlie professor of public services management, Kings
College London, UK, Naomi Fulop professor of healthcare organisation and management, University
College London, UK, John Gabbay emeritus professor of public health, University of Southampton,
UK, Marie-Pierre Gagnon professor, Faculty of Nursing, Université Laval, Quebec, Canda, Dariusz
Galasinski professor of discourse and cultural studies, University of Wolverhampton, UK, Ruth
Garside senior lecturer in evidence synthesis, University of Exeter, UK, Lucy Gilson professor of
health policy and systems, University of Cape Town, South Africa, Peter Griffiths professor of health
services research, University of Southampton, UK, Penny Hawe professor of public health, University
of Sydney, Australia, Jan-Kees Helderman associate professor of public administration, Radboud
University Nijmegen, Netherlands, Brian Hodges professor, Faculty of Medicine, University of
Toronto, Canada, David Hunter director, Centre for Public Policy and Health, Durham University,
UK, Margaret Kearney professor, University of Rochester, NY, USA, Celia Kitzinger codirector,
Coma and Disorders of Consciousness Research Centre, University of York, UK, Jenny Kitzinger
codirector, Coma and Disorders of Consciousness Research Centre, University of Cardiff, UK,
Ayelet Kuper assistant professor, Department of Medicine, University of Toronto, Canada, Saville
Kushner professor of public education, University of Auckland, New Zealand, Andree Le May emerita
professor of nursing, University of Southampton, UK, France Legare Canada research chair in
implementation of shared decision making in primary care, University of Laval, Canada, Lorelei
Lingard professor, Schulich School of Medicine and Dentistry, University of Western Ontario, Canada,
Louise Locock director of applied research, Health Experiences Research Group, University of
Oxford, UK, Jill Maben professor of nursing, King’s College London, UK, Mary Ellen Macdonald
associate professor, Faculty of Dentistry, McGill University, Canada, Frances Mair professor of
primary care research, University of Glasgow, UK, Russell Mannion professor of health systems,
For personal use only: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe
BMJ 2016;352:i563 doi: 10.1136/bmj.i563 (Published 10 February 2016) Page 1 of 4
Analysis
ANALYSIS
University of Birmingham, UK, Martin Marshall professor of healthcare improvement, University
College London, UK , Carl May professor of healthcare innovation, Faculty of Health Sciences,
University of Southampton, UK, Nicholas Mays professor of health policy, London School of Hygiene
and Tropical Medicine, UK , Lorna McKee professor of management, University of Aberdeen, UK,
Marissa Miraldo associate professor of health economics, Imperial College London, UK, David
Morgan professor, Department of Sociology, Portland State University, OR, USA, Janice Morse
professor, College of Nursing, University of Utah, UT, USA, Sarah Nettleton professor of sociology,
University of York, UK, Sandy Oliver deputy director, EPPI-Centre, UCL Institute of Education, UK,
Warrren Pearce Institute for Science and Society, University of Nottingham, UK, Pierre Pluye director,
Methodological Developments, Department of Family Medicine, McGill University, Montreal, Canada,
Catherine Pope professor of medical sociology, University of Southampton, UK, Glenn Robert
professor of healthcare quality and innovation, King’s College London, UK, Celia Roberts emerita
professor of linguistics, King’s College London, UK, Stefania Rodella Regional Agency for Health
and Social Care, Bologna, Italy, Jo Rycroft-Malone professor of implementation, University of Bangor,
UK, Margarete Sandelowski professor, School of Nursing, University of North Carolina at Chapel
Hill, NC, USA, Paul Shekelle director, Southern California Evidence-Based Practice Center, RAND
Corporation, CA, USA, Fiona Stevenson senior lecturer in medical sociology, University College
London, UK, Sharon Straus director, Division of Medicine, University of Toronto, Canada, Deborah
Swinglehurst senior clinical lecturer in primary healthcare, Queen Mary University of London, UK,
Sally Thorne professor, University of British Columbia School of Nursing, Vancouver, Canada,
Göran Tomson senior professor in international health systems research, Karolinska Institutet,
Stockholm Sweden, Gerd Westert professor of health services research and quality of care, Scientific
Institute for Quality of Care, Nijmegen, Netherlands, Sue Wilkinson honorary professor, Department
of Sociology, University of York, UK, Brian Williams dean of reseach enhancement, University of
Stirling, UK, Terry Young associate dean of health partnerships, Brunel University, UK, Sue Ziebland
director, Health Experiences Research Group, University of Oxford, UK
We are concerned that The BMJ seems to have developed a
policy of rejecting qualitative research on the grounds that such
studies are “low priority,” “unlikely to be highly cited,” “lacking
practical value,” or “not of interest to our readers” (box). Here,
we argue that The BMJ should develop and publish a formal
policy on qualitative and mixed method research and that this
should include appropriate and explicit criteria for judging the
relevance of submissions. We acknowledge that (as with all
methods) some qualitative research is poor quality, badly
written, inaccessible, or irrelevant to the journal’s readership.
We also acknowledge that many of The BMJ’s readers (not to
mention its reviewers and editors) may not have been formally
trained to read, conduct, or evaluate qualitative studies. We see
these caveats as opportunities not threats.
The BMJ’s mission is method agnostic
The BMJ says its mission is to lead the debate on health and to
engage, inform, and stimulate all doctors, researchers, and other
health professionals in ways that enable them to make better
decisions and improve outcomes for patients.
Some clinical and policy questions are best answered by the
results of randomised controlled trials or other quantitative
approaches, but other decisions and outcomes are more usefully
informed by qualitative studies. Qualitative studies help us
understand why promising clinical interventions do not always
work in the real world, how patients experience care, and how
practitioners think. They also explore and explain the complex
relations between the healthcare system and the outside world,
such as the sociopolitical context in which healthcare is
regulated, funded, and provided, and the ways in which
clinicians and regulators interact with industry.
Some of The BMJ’s top papers have been
qualitative
The BMJ recently celebrated 20 years of online presence by
asking experts to name the most influential paper published in
that period.1The 20 nominated papers included 11 commentaries
or editorials (highlighting the journal’s important role in
publishing papers that contextualise and interpret research),
three randomised controlled trials, three qualitative studies, two
surveys, and one methodological paper.
The three qualitative papers explored how primary care
clinicians develop and use collective “mindlines” instead of
written guidelines2; what worries parents when their preschool
children are acutely ill3; and the nature of collusion in the
doctor-patient relationship when death is imminent.4They have
been cited by 572, 197, and 114 subsequent papers respectively
(Google Scholar data). In contrast, the three nominated
randomised trials have been cited by 321,578,6and 387
subsequent papers.
We are not claiming that citation rates for these nominated
papers are statistically representative. But they do show that
good qualitative research with a clear and important clinical
message can be highly cited, is popular with readers, and
enriches The BMJ’s overall contribution to the knowledge base.
Different study designs provide
complementary perspectives
Few research topics in clinical decision making and patient care
can be sufficiently understood through quantitative research
alone. Take patient safety, for example, in which quantitative
studies have examined the effect size of interventions to improve
safety and qualitative ones have examined equally important
Correspondence to: Trisha Greenhalgh trish.greenhalgh@phc.ox.ac.uk
For personal use only: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe
BMJ 2016;352:i563 doi: 10.1136/bmj.i563 (Published 10 February 2016) Page 2 of 4
ANALYSIS
Excerpt from rejection letter tweeted by McGill Qualitative Health Research Group (@MQHRG), 30 September 2015
Thank you for sending us your paper. We read it with interest but I am sorry to say that qualitative studies are an extremely low priority for
The BMJ. Our research shows that they are not as widely accessed, downloaded, or cited as other research.
We receive over 8000 submissions a year and accept less than 4%. We do therefore have to make hard decisions on just how interesting
an article will be to our general clinical readers, how much it adds, and how much practical value it will be.
questions such as why the observed effect occurred and, in some
cases, why the predicted effect did not occur.
The surgical safety checklist is a revealing case in point. A
controlled before and after study published in the New England
Journal of Medicine showed that in 3733 patients having
non-cardiac surgery, the introduction of a surgical safety
checklist was associated with a highly significant reduction in
perioperative mortality (from 1.5% to 0.8%) and complication
rate (from 11% to 7%).8
But attempts to replicate these impressive improvements have
sometimes failed dramatically.9 10 Eighteen qualitative studies,
summarised in a recent qualitative systematic review, help
explain why.11 The operating theatre is a complex social space
with established hierarchies and routines. Far from being a
simple “technical” procedure, the checklist demands new forms
of cooperation and communication between surgeons,
anaesthetists, and nurses. Depending on a host of contextual
factors, safety checks may substantially disrupt team routines
and be resented rather than welcomed. When (and to the extent
that) the checklist is treated as a tick-box exercise, it will fail
to generate benefits and may even lead to harms.
From the policy maker’s perspective, qualitative studies of the
professional, organisational, and political context of nationally
driven checklist based patient safety initiatives can help explain
both successes and failures.12 13
The BMJ has a long tradition of educating
its readers about less familiar research
methods
Statistics is a closed book to many jobbing clinicians. “Bite
sized” methodological commentaries, often linked to exemplar
papers published in the research section of The BMJ, have
enabled its readers to grasp important concepts such as why
continuous variables should not be dichotomised14 or why some
apparent improvements are explained by regression to the
mean.15 Through the journal’s Statistics Notes and Economics
Notes series (of which over 100 have been published in the past
20 years), the quantitative research literacy of its clinician
readership has significantly improved.
The BMJ has not yet introduced a comparable ongoing
educational approach for qualitative research. It is 20 years since
Pope and Mays edited the original BMJ Education and Debate
series on qualitative methods, which covered interviews, focus
groups, ethnography, case study, and criteria for assessing
quality and establishing rigour.16-24 Their 2000 paper on how to
analyse qualitative data remains The BMJ’s 12th most highly
cited paper ever (Web of Science data).24 In 2008, The BMJ
published a further series updating and extending the range of
qualitative research methodologies and emphasising the
importance of theory in interpreting evidence.25-29
An opportunity exists to supplement these popular series on
qualitative theory and method with an occasional series of
“qualitative notes” accompanying exemplars of empirical studies
in qualitative research. Through such a series, the journal’s
readership would gain in qualitative research literacy.
New challenges
The inclusion of qualitative research as a mainstream theme
will undoubtedly raise new methodological, philosophical, and
ethical questions. For example, the laudable principle of data
archiving and sharing is supported by some but not all qualitative
funding bodies (see, for example, the Qualidata archive, part of
the UK Data Service https://discover.ukdataservice.ac.uk/?
q=qualidata). A requirement to share data may generate tricky
challenges in the trade-off between transparency and informant
confidentiality, especially in the digital age when anonymisation
of interview data may not be possible.30 31
We offer no simple solutions to such complex issues, but suggest
that (as with comparable questions in quantitative research) The
BMJ could provide a forum for methodological commentaries
or online discussion.
A proposal
We believe it is time for a prospective study to assess whether
The BMJ can come to value and be proud of qualitative research
as part of its mission to lead the debate on health, inform clinical
decision making, and improve outcomes for patients. We
challenge The BMJ to allocate one slot a month for one year to
a “landmark” qualitative paper along with an accompanying
methodological commentary from an international expert. We
offer to assist The BMJ to appoint an appropriate team of
reviewers, guest editors, and commentators. We can also advise
on training to build capacity and confidence of editorial staff to
distinguish good from poor qualitative research and identify
which of the many submissions it receives holds promise as
“qualitative paper of the month.”
Conclusion
As pointed out by its editors in response to an earlier draft of
this letter, The BMJ is by no means an outlier in its current
policy on qualitative research. Many leading US journals
(including JAMA and the New England Journal of Medicine)
also consider such research low priority. We believe all such
journals would benefit from revisiting their policies.
The BMJ, with its history of supporting qualitative research, is
in a unique position to lead the field by ensuring that all types
of research relevant to its mission are considered for publication;
we believe its reputation as an international academic journal
will be strengthened if it does so. Some qualitative papers will
be highly cited and contribute directly to the journal’s impact
factor. With others, the reputational benefit will be indirect and
result from introducing the new ways of thinking that are
essential to scientific progress.
Both the International Cochrane Collaboration and the UK
Health Technology Assessment Programme, though initially
predominantly focused on the quantitative, were persuaded to
include qualitative and mixed methods research where
appropriate.32 33 The Health Technology Assessment
Programme’s monograph on qualitative methods33 subsequently
became the most downloaded of its more than 700 online
publications by a considerable margin. These organisations have
For personal use only: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe
BMJ 2016;352:i563 doi: 10.1136/bmj.i563 (Published 10 February 2016) Page 3 of 4
ANALYSIS
decided that “quantitative versus qualitative” is yesterday’s war.
We encourage The BMJ to join them.
Contributors and sources: TG wrote the first draft of the letter, which
was modified by ST, AK, and LLingard and then circulated to all other
authors, many of whom suggested further amendments. All authors
have seen and approved the final manuscript.
Competing interests: We have read and understood BMJ policy on
declaration of interests and declare that two of us have received
consultancy income from qualitative research and some of us have
received royalties for books or book chapters on qualitative research.
Our only other conflict of interest is that we value the contribution of
qualitative research to medicine.
Provenance and peer review: Not commissioned; not externally peer
reviewed.
1 Payne D. Twenty top papers to mark The BMJ’s two digital decades. 2015. BMJ
2015;351:h3660.
2 Gabbay J, le May A. Evidence based guidelines or collectively constructed “mindlines?”
Ethnographic study of knowledge management in primary care. BMJ 2004;329:1013.
3 Kai J. What worries parents when their preschool children are acutely ill, and why: a
qualitative study. BMJ 1996;313:983-6.
4Hak T, Koëter G, van der Wal G. Collusion in doctor-patient communication about imminent
death: an ethnographic study. BMJ 2000;321:1376-81.
5 Baqui AH, Black RE, El Arifeen S, et al. Effect of zinc supplementation started during
diarrhoea on morbidity and mortality in Bangladeshi children: community randomised trial.
BMJ 2002;325:1059.
6 Kaczorowski J, Chambers LW, Dolovich L, et al. Improving cardiovascular health at
population level: 39 community cluster randomised trial of Cardiovascular Health
Awareness Program (CHAP). BMJ 2011;342:d4442.
7 Seehusen DA, Johnson DR, Earwood JS, et al. Improving women’s experience during
speculum examinations at routine gynaecological visits: randomised clinical trial. BMJ
2006;333:171.
8 Haynes AB, Weiser TG, Berry WR, et al. A surgical safety checklist to reduce morbidity
and mortality in a global population. N Engl J Med 2009;360:491-9.
9 Urbach DR, Govindarajan A, Saskin R, et al. Introduction of surgical safety checklists in
Ontario, Canada. N Engl J Med 2014;370:1029-38.
10 Reames BN, Krell RW, Campbell DA, et al. A checklist-based intervention to improve
surgical outcomes in Michigan: evaluation of the keystone surgery program. JAMA Surg
2015;150:208-15.
11 Bergs J, Lambrechts F, Simons P, et al. Barriers and facilitators related to the
implementation of surgical safety checklists: a systematic review of the qualitative evidence.
BMJ Qual Saf 2015 Jul 21. doi:bmjqs-2015-004021.
12 Dixon-Woods M, Bosk CL, Aveling EL, et al. Explaining Michigan: developing an ex post
theory of a quality improvement program. Milbank Q 2011;89:167-205.
13 Dixon-Woods M, Leslie M, Tarrant C, et al. Explaining matching Michigan: an ethnographic
study of a patient safety program. Implement Sci 2013;8:70.
14 Altman DG, Royston P. The cost of dichotomising continuous variables. BMJ
2006;332:1080.
15 Bland JM, Altman DG. Statistics notes: some examples of regression towards the mean.
BMJ 1994;309:780.
16 Pope C, Mays N. Qualitative research: reaching the parts other methods cannot reach:
an introduction to qualitative methods in health and health services research. BMJ
1995;311:42-5.
17 Mays N, Pope C. Qualitative research: rigour and qualitative research. BMJ
1995;311:109-12.
18 Mays N, Pope C. Qualitative research: Observational methods in health care settings.
BMJ 1995;311:182.
19 Britten N. Qualitative interviews in medical research. BMJ 1995;311:251.
20 Keen J, Packwood T. Case study evaluation. BMJ 1995;311:444.
21 Bland JM, Altman DG. Statistics notes: calculating correlation coefficients with repeated
observations. Part 1—correlation within subjects. BMJ 1995;310:446.
22 Mays N, Pope C. Qualitative research in health care: assessing quality in qualitative
research. BMJ 2000;320:50.
23 Kitzinger J. Qualitative research. Introducing focus groups. BMJ 1995;311:299.
24 Pope C, Ziebland S, Mays N. Qualitative research in health care: analysing qualitative
data. BMJ 2000;320:114.
25 Reeves S, Albert M, Kuper A, et al. Why use theories in qualitative research? BMJ
2008;337:a949.
26 Hodges BD, Kuper A, Reeves S. Discourse analysis. BMJ 2008;337:a879.
27 Reeves S, Kuper A, Hodges BD. Qualitative research methodologies: ethnography. BMJ
2008;337:a1020.
28 Kuper A, Reeves S, Levinson W. An introduction to reading and appraising qualitative
research. BMJ 2008;337:a288.
29 Kuper A, Lingard L, Levinson W. Critically appraising qualitative research. BMJ
2008;337:a1035.
30 Saunders B, Kitzinger J, Kitzinger C. Participant anonymity in the internet age: from theory
to practice. Qual Res Psychol 2015;12:125-37.
31 Saunders B, Kitzinger J, Kitzinger C. Anonymising interview data: challenges and
compromise in practice. Qual Res 2014:1468794114550439.
32 Noyes J. Never mind the qualitative feel the depth! The evolving role of qualitative research
in Cochrane intervention reviews. J Res Nurs 2010:1744987110381696.
33 Murphy E, Dingwall R, Greatbatch D, et al. Qualitative research methods in health
technology assessment: a review of the literature. Health Technol Assess (Winchester,
England) 1998;2:iii.
Accepted: 30 December 2015
Cite this as: BMJ 2016;352:i563
© BMJ Publishing Group Ltd 2016
For personal use only: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe
BMJ 2016;352:i563 doi: 10.1136/bmj.i563 (Published 10 February 2016) Page 4 of 4
ANALYSIS