Do higher primary care practice performance scores predict lower rates of emergency admissions for persons with serious mental illness? An analysis of secondary panel data

Abstract

Serious mental illness (SMI) is a set of chronic enduring conditions including schizophrenia and bipolar disorder. SMIs are associated with poor outcomes, high costs and high levels of disease burden. Primary care plays a central role in the care of people with a SMI in the English NHS. Good-quality primary care has the potential to reduce emergency hospital admissions, but also to increase elective admissions if physical health problems are identified by regular health screening of people with SMIs. Better-quality primary care may reduce length of stay (LOS) by enabling quicker discharge, and it may also reduce NHS expenditure.

Objectives
We tested whether or not better-quality primary care, as assessed by the SMI quality indicators measured routinely in the Quality and Outcomes Framework (QOF) in English general practice, is associated with lower rates of emergency hospital admissions for people with SMIs, for both mental and physical conditions and with higher rates of elective admissions for physical conditions in people with a SMI. We also tested the impact of SMI QOF indicators on LOS and costs.

Data

We linked administrative data from around 8500 general practitioner (GP) practices and from Hospital Episode Statistics for the study period 2006/7 to 2010/11. We identified SMI admissions by a main
International Classification of Diseases
, 10th revision (ICD-10) diagnosis of F20–F31. We included information on GP practice and patient population characteristics, area deprivation and other potential confounders such as access to care. Analyses were carried out at a GP practice level for admissions, but at a patient level for LOS and cost analyses.

Methods
We ran mixed-effects count data and linear models taking account of the nested structure of the data. All models included year indicators for temporal trends.

Results
Contrary to expectation, we found a positive association between QOF achievement and admissions, for emergency admissions for both mental and physical health. An additional 10% in QOF achievement was associated with an increase in the practice emergency SMI admission rate of approximately 1.9%. There was no significant association of QOF achievement with either LOS or cost. All results were robust to sensitivity analyses.

Conclusions
Possible explanations for our findings are (1) higher quality of primary care, as measured by QOF may not effectively prevent the need for secondary care; (2) patients may receive their QOF checks post discharge, rather than prior to admission; (3) people with more severe SMIs, at a greater risk of admission, may select into practices that are better organised to provide their care and which have better QOF performance; (4) better-quality primary care may be picking up unmet need for secondary care; and (5) QOF measures may not accurately reflect quality of primary care. Patient-level data on quality of care in general practice is required to determine the reasons for the positive association of QOF quality and admissions. Future research should also aim to identify the non-QOF measures of primary care quality that may reduce unplanned admissions more effectively and could potentially be incentivised.

Funding
The National Institute for Health Research Health Services and Delivery Research programme.

Full text

HEALTH SERVICES AND DELIVERY RESEARCH
VOLUME 3 ISSUE 16 APRIL 2015
ISSN 2050-4349
DOI 10.3310/hsdr03160
Do higher primary care practice performance scores
predict lower rates of emergency admissions for
persons with serious mental illness? An analysis of
secondary panel data
Rowena Jacobs, Nils Gutacker, Anne Mason, Maria Goddard, Hugh Gravelle,
Tony Kendrick, Simon Gilbody, Lauren Aylott and June Wainwright

Do higher primary care practice
performance scores predict lower rates of
emergency admissions for persons with
serious mental illness? An analysis of
secondary panel data
Rowena Jacobs,
1
*
Nils Gutacker,
1
Anne Mason,
1
Maria Goddard,
1
Hugh Gravelle,
1
Tony Kendrick,
2
Simon Gilbody,
3
Lauren Aylott
4
and June Wainwright
4
1
Centre for Health Economics, University of York, York, UK
2
Primary Care and Population Sciences, University of Southampton,
Southampton, UK
3
Department of Health Sciences, University of York, York, UK
4
Service user
*Corresponding author
Declared competing interests of authors: Simon Gilbody is a member of the HTA Clinical Evaluation
and Trials Board. Tony Kendrick’s MD thesis provided evidence of the potential benefit of regular
assessments of people with SMI which informed the Quality and Outcomes Framework performance
indicator. He has been a member of the NICE national Quality and Outcomes Framework Advisory
Committee since 2009.
Published April 2015
DOI: 10.3310/hsdr03160
This report should be referenced as follows:
Jacobs R, Gutacker N, Mason A, Goddard M, Gravelle H, Kendrick T, et al. Do higher primary care
practice performance scores predict lower rates of emergency admissions for persons with serious
mental illness? An analysis of secondary panel data. Health Serv Deliv Res 2015;3(16).

Health Services and Delivery Research
ISSN 2050-4349 (Print)
ISSN 2050-4357 (Online)
This journal is a member of and subscribes to the principles of the Committee on Publication Ethics (COPE) (www.publicationethics.org/).
Editorial contact: nihredit@southampton.ac.uk
The full HS&DR archive is freely available to view online at www.journalslibrary.nihr.ac.uk/hsdr. Print-on-demand copies can be purchased from
the report pages of the NIHR Journals Library website: www.journalslibrary.nihr.ac.uk
Criteria for inclusion in the Health Services and Delivery Research journal
Reports are published in Health Services and Delivery Research (HS&DR) if (1) they have resulted from work for the HS&DR programme
or programmes which preceded the HS&DR programme, and (2) they are of a sufficiently high scientific quality as assessed by the
reviewers and editors.
HS&DR programme
The Health Services and Delivery Research (HS&DR) programme, part of the National Institute for Health Research (NIHR), was established to
fund a broad range of research. It combines the strengths and contributions of two previous NIHR research programmes: the Health Services
Research (HSR) programme and the Service Delivery and Organisation (SDO) programme, which were merged in January 2012.
The HS&DR programme aims to produce rigorous and relevant evidence on the quality, access and organisation of health services including
costs and outcomes, as well as research on implementation. The programme will enhance the strategic focus on research that matters to the
NHS and is keen to support ambitious evaluative research to improve health services.
For more information about the HS&DR programme please visit the website: http://www.nets.nihr.ac.uk/programmes/hsdr
This report
The research reported in this issue of the journal was funded by the HS&DR programme or one of its preceding programmes as project
number 10/1011/22. The contractual start date was in April 2012. The final report began editorial review in October 2013 and was accepted
for publication in April 2014. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up
their work. The HS&DR editors and production house have tried to ensure the accuracy of the authors’report and would like to thank the
reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising
from material published in this report.
This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by
authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HS&DR
programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the
interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the
HS&DR programme or the Department of Health.
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning
contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and
study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement
is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre,
Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.
Published by the NIHR Journals Library (www.journalslibrary.nihr.ac.uk), produced by Prepress Projects Ltd, Perth, Scotland
(www.prepress-projects.co.uk).

Health Services and Delivery Research Editor-in-Chief
Professor Ray Fitzpatrick Professor of Public Health and Primary Care, University of Oxford, UK
NIHR Journals Library Editor-in-Chief
Professor Tom Walley Director, NIHR Evaluation, Trials and Studies and Director of the HTA Programme, UK
NIHR Journals Library Editors
Professor Ken Stein Chair of HTA Editorial Board and Professor of Public Health, University of Exeter Medical
School, UK
Professor Andree Le May Chair of NIHR Journals Library Editorial Group (EME, HS&DR, PGfAR, PHR journals)
Dr Martin Ashton-Key Consultant in Public Health Medicine/Consultant Advisor, NETSCC, UK
Professor Matthias Beck Chair in Public Sector Management and Subject Leader (Management Group),
Queen’s University Management School, Queen’s University Belfast, UK
Professor Aileen Clarke Professor of Public Health and Health Services Research, Warwick Medical School,
University of Warwick, UK
Dr Tessa Crilly Director, Crystal Blue Consulting Ltd, UK
Dr Peter Davidson Director of NETSCC, HTA, UK
Ms Tara Lamont Scientific Advisor, NETSCC, UK
Professor Elaine McColl Director, Newcastle Clinical Trials Unit, Institute of Health and Society,
Newcastle University, UK
Professor William McGuire Professor of Child Health, Hull York Medical School, University of York, UK
Professor Geoffrey Meads Professor of Health Sciences Research, Faculty of Education, University of Winchester, UK
Professor John Powell Consultant Clinical Adviser, National Institute for Health and Care Excellence (NICE), UK
Professor James Raftery Professor of Health Technology Assessment, Wessex Institute, Faculty of Medicine,
University of Southampton, UK
Dr Rob Riemsma Reviews Manager, Kleijnen Systematic Reviews Ltd, UK
Professor Helen Roberts Professor of Child Health Research, UCL Institute of Child Health, UK
Professor Helen Snooks Professor of Health Services Research, Institute of Life Science, College of Medicine,
Swansea University, UK
Please visit the website for a list of members of the NIHR Journals Library Board:
www.journalslibrary.nihr.ac.uk/about/editors
Editorial contact: nihredit@southampton.ac.uk
NIHR Journals Library www.journalslibrary.nihr.ac.uk

Abstract
Do higher primary care practice performance scores predict
lower rates of emergency admissions for persons with
serious mental illness? An analysis of secondary panel data
Rowena Jacobs,
1
*
Nils Gutacker,
1
Anne Mason,
1
Maria Goddard,
1
Hugh Gravelle,
1
Tony Kendrick,
2
Simon Gilbody,
3
Lauren Aylott
4
and June Wainwright
4
1
Centre for Health Economics, University of York, York, UK
2
Primary Care and Population Sciences, University of Southampton, Southampton, UK
3
Department of Health Sciences, University of York, York, UK
4
Service user
*Corresponding author rowena.jacobs@york.ac.uk
Background: Serious mental illness (SMI) is a set of chronic enduring conditions including schizophrenia
and bipolar disorder. SMIs are associated with poor outcomes, high costs and high levels of disease
burden. Primary care plays a central role in the care of people with a SMI in the English NHS. Good-quality
primary care has the potential to reduce emergency hospital admissions, but also to increase elective
admissions if physical health problems are identified by regular health screening of people with SMIs.
Better-quality primary care may reduce length of stay (LOS) by enabling quicker discharge, and it may also
reduce NHS expenditure.
Objectives: We tested whether or not better-quality primary care, as assessed by the SMI quality
indicators measured routinely in the Quality and Outcomes Framework (QOF) in English general practice, is
associated with lower rates of emergency hospital admissions for people with SMIs, for both mental and
physical conditions and with higher rates of elective admissions for physical conditions in people with a
SMI. We also tested the impact of SMI QOF indicators on LOS and costs.
Data: We linked administrative data from around 8500 general practitioner (GP) practices and from
Hospital Episode Statistics for the study period 2006/7 to 2010/11. We identified SMI admissions by a main
International Classification of Diseases, 10th revision (ICD-10) diagnosis of F20–F31. We included
information on GP practice and patient population characteristics, area deprivation and other potential
confounders such as access to care. Analyses were carried out at a GP practice level for admissions, but at
a patient level for LOS and cost analyses.
Methods: We ran mixed-effects count data and linear models taking account of the nested structure of
the data. All models included year indicators for temporal trends.
Results: Contrary to expectation, we found a positive association between QOF achievement and
admissions, for emergency admissions for both mental and physical health. An additional 10% in QOF
achievement was associated with an increase in the practice emergency SMI admission rate of
approximately 1.9%. There was no significant association of QOF achievement with either LOS or cost.
All results were robust to sensitivity analyses.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
v

Conclusions: Possible explanations for our findings are (1) higher quality of primary care, as measured by
QOF may not effectively prevent the need for secondary care; (2) patients may receive their QOF checks
post discharge, rather than prior to admission; (3) people with more severe SMIs, at a greater risk of
admission, may select into practices that are better organised to provide their care and which have better
QOF performance; (4) better-quality primary care may be picking up unmet need for secondary care; and
(5) QOF measures may not accurately reflect quality of primary care. Patient-level data on quality of care in
general practice is required to determine the reasons for the positive association of QOF quality and
admissions. Future research should also aim to identify the non-QOF measures of primary care quality that
may reduce unplanned admissions more effectively and could potentially be incentivised.
Funding: The National Institute for Health Research Health Services and Delivery Research programme.
ABSTRACT
NIHR Journals Library www.journalslibrary.nihr.ac.uk
vi

Contents
List of tables ix
List of figures xi
Glossary xiii
List of abbreviations xvii
Plain English summary xix
Scientific summary xxi
Chapter 1 Introduction 1
Chapter 2 Measures of quality of primary care for people with serious mental illness 5
The Quality and Outcomes Framework 5
The mental health domain of the Quality and Outcomes Framework 5
Exception reporting in the Quality and Outcomes Framework 7
Chapter 3 Empirical analysis 9
Does better primary care reduce hospital admissions? 9
Overview 9
Data 9
Empirical approach 12
Results 13
Does better primary care reduce inpatient length of stay? 30
Overview 30
Data 30
Analytical model 31
Results 32
Sensitivity analyses 35
Does better primary care reduce cost of care? 36
Overview 36
Data 36
Analytical model 39
Results 42
Sensitivity analyses 47
Chapter 4 Discussion 51
Chapter 5 Conclusions 53
Implications for research 53
Implications for practice 54
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
vii

Acknowledgements 57
References 59
Appendix 1 Patient and public involvement 65
Appendix 2 Further results 67
CONTENTS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
viii

List of tables
TABLE 1 Evidence for the impact of the QOF on admissions 3
TABLE 2 Hypothesised direction of association between better quality of care
and outcomes 3
TABLE 3 Overview of the four SMI QOF indicators included in the study 6
TABLE 4 International Classification of Diseases, 10th revision codes used to
identify patients with SMI 10
TABLE 5 General practitioner-level analyses: descriptive statistics for the samples 14
TABLE 6 General practitioner-level analyses: QOF achievement and exception rates 15
TABLE 7 General practitioner-level analyses: descriptive statistics for the variables 18
TABLE 8 General practitioner-level analysis 1: regression results (achievement
includes exception-reported patients) 19
TABLE 9 General practitioner-level analysis 1: approximate switching
point –percentage valid exception reporting 24
TABLE 10 General practitioner-level analysis 2: regression results (achievement
includes exception-reported patients) 25
TABLE 11 General practitioner-level analysis 2: regression results (achievement
excludes exception-reported patients) 25
TABLE 12 General practitioner-level analysis 2: switching point –percentage
valid exception reporting 26
TABLE 13 General practitioner-level analysis 3: regression results (achievement
includes exception-reported patients) 29
TABLE 14 General practitioner-level analysis 3: regression results (achievement
excludes exception-reported patients) 30
TABLE 15 General practitioner-level analysis 3: switching point –percentage
valid exception reporting 30
TABLE 16 Length-of-stay analysis: descriptive statistics 33
TABLE 17 Length-of-stay analysis: regression results 34
TABLE 18 Derivation of the regression samples 39
TABLE 19 Covariates used in the MHMDS analyses 41
TABLE 20 Mental Health Minimum Data Set analysis: overview of regression models 42
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
ix

TABLE 21 Mental Health Minimum Data Set analysis: descriptive statistics for the
base case (all individuals) 43
TABLE 22 Mental Health Minimum Data Set analysis: descriptive statistics for the
sensitivity analyses (working-aged individuals) 44
TABLE 23 Mental Health Minimum Data Set analysis: regression results for the
QOF indicators (all models, base case) 45
TABLE 24 Mental Health Minimum Data Set analysis: regression results from the
base case (model 1), all individuals 46
TABLE 25 Mental Health Minimum Data Set analysis: regression results for the
QOF indicators (all models, sensitivity analysis) 47
TABLE 26 Mental Health Minimum Data Set analysis: regression results from the
sensitivity analyses, working age individuals 48
TABLE 27 Cross-sectional models: admissions for patients with a SMI 67
TABLE 28 Cross-sectional models: admissions for patients with bipolar disorder 67
TABLE 29 Results for covariates on SMI admissions model 68
LIST OF TABLES
NIHR Journals Library www.journalslibrary.nihr.ac.uk
x

List of figures
FIGURE 1 Overview of performance measures constructed for the regression analyses 8
FIGURE 2 Average number of admissions per GP practice 14
FIGURE 3 Distribution of QOF achievement rates across practices in 2010/11 16
FIGURE 4 Change in IRR for the number of annual SMI admissions per practice
when the percentage of ‘valid’exceptions ranges from 0% to 100% 20
FIGURE 5 Change in IRR for the number of annual physical (emergency)
admissions per practice when the percentage of ‘valid’exceptions ranges from
0% to 100% 21
FIGURE 6 Change in IRR for the number of annual physical (elective) admissions
per practice when the percentage of ‘valid’exceptions ranges from 0% to 100% 22
FIGURE 7 Change in IRR for the number of annual bipolar admissions per
practice when the percentage of ‘valid’exceptions ranges from 0% to 100% 23
FIGURE 8 Use of R69 as primary diagnoses for psychiatric admissions 27
FIGURE 9 Length-of-stay analysis: histogram of patient LOS 31
FIGURE 10 Stylised example of a patient record in MHMDS 36
FIGURE 11 Mental Health Minimum Data Set analysis: histograms of total
per-patient cost, before and after log transformation 40
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
xi

Glossary
Attribution data set General practitioner practice data which include information on the age and gender
and number of registered patients who are resident in each lower super output area (small-area level).
Black and minority ethnic groups Patients in the black and minority ethic groups are at higher risk of
compulsory admissions.
Care Programme Approach assessments A needs assessment for people with serious mental disorders
that involves the development of a care plan which is regularly reviewed.
Chronic obstructive pulmonary disorder A collection of lung diseases, including chronic bronchitis and
emphysema, that are often caused by smoking.
Community Mental Health Team Community Mental Health Teams provide community-based services
to people who are experiencing mental health problems. They are multiagency teams consisting of mental
health professionals such as community mental health nurses, social workers, occupational therapists,
psychiatrists and psychologists.
Continuous inpatient spell The time between initial admission and final discharge. Patients are tracked
over time, so a continuous inpatient spell can cover transfers between hospitals when this is part of a
care pathway.
Crisis resolution and home treatment services Developed to provide care for service users living
in the community and experiencing a crisis requiring emergency admission to secondary care services.
The services often have a gatekeeping function.
Electroconvulsive therapy A treatment for severe depression.
Finished consultant episode Period of care within a particular consultant specialty at a single hospital
provider. This is the way in which Hospital Episode Statistics data are typically recorded.
General Medical Services contract An alternative contract to Personal Medical Services contract status
for general practitioner practices. The majority of practices have General Medical Services contracts.
General practitioner Doctors who provide primary care services to patients who are registered with a
general practice.
Health of the Nation Outcome Scale A tool that assesses the health and social functioning of people
with serious mental illness.
Health Services and Delivery Research programme Part of the National Institute for Health Research
funding regime (it subsumed the Service Delivery and Organisation programme).
Hospital Episode Statistics Hospital Episode Statistics is an administrative data set that contains records
of all patients in England who receive inpatient, outpatient or accident and emergency care. The inpatient
data set contains around 18 million records annually. Hospital Episode Statistics patient (anonymised)
identifiers allow tracking of patients over time and provide data on patients’demographic and clinical
characteristics, the area they live in and their registered general practitioner practice.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
xiii

Incidence rate ratio A way of expressing the coefficients in the regression model which is easier to
interpret. This is typically the number of new cases of a condition in a defined (specified) group
or population.
Index of Multiple Deprivation Measure of deprivation which has subdomains such as education, crime,
housing and health, available at lower super output area level.
International Classification of Diseases, 10th revision Diagnostic categorisation system which is used
in Hospital Episode Statistics to provide primary and secondary diagnosis codes.
Length of stay Length of stay for inpatients, calculated from Hospital Episode Statistics data; proxy
measure of resource use.
Lower super output area Census-defined small-area level with an average population size of
approximately 1500 individuals.
Mental health Mental well-being, good mental functioning or having no particular problems in thinking,
feelings or behaviour. The term ‘mental health problem’or ‘mental disorder’denotes the opposite.
Mental Health Minimum Data Set Mandatory data collection for mental health provider hospitals
(secondary care), patient-level data for people with severe and enduring mental health problems on
hospitalisations, community and outpatient services.
Mental Health Research Network Part of the National Institute for Health Research; helps research
teams to recruit service user participants.
National Institute for Health Research Funds health research through the Department of Health.
Office for National Statistics Source of census data on population, demographics which are available at
lower super output area level (small area).
Patient and public involvement Getting patients or members of public involved in research; crucial
underpinning of research activity.
Personal Medical Services contract Forty per cent of practices have Personal Medical Services contracts.
Personal Medical Services general practitioners earn about 10% more than General Medical Services
general practitioners. Personal Medical Services practices take part in the Quality and Outcomes Framework,
but, because they are already paid for some of the services counting towards the Quality and Outcomes
Framework, they have points deducted from their Quality and Outcomes Framework score.
Primary care trust Until March 2013 primary care trusts were responsible for commissioning primary,
community and secondary health services from providers. They were responsible for spending around 80%
of the total NHS budget. When primary care trusts were abolished in 2013 their work was taken over by
clinical commissioning groups.
Quality and Outcomes Framework A voluntary pay-for-performance incentive scheme for general
practitioner practices, who earn points for achieving clinical targets for chronic conditions (including mental
health problems). Points are also given depending on how well the practice is organised, the extra services
offered and how patients view their experience.
GLOSSARY
NIHR Journals Library www.journalslibrary.nihr.ac.uk
xiv

Quality Management and Analysis System A computer system used by the NHS for Quality and
Outcomes Framework data.
Resource Allocation for Mental Health and Prescribing A project commissioned by the English
Department of Health to calculate resource allocation for mental health services.
Serious mental illness Defined within the Quality and Outcomes Framework as those with
schizophrenia, psychoses and bipolar disorder.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
xv

List of abbreviations
ADS attribution data set
CI confidence interval
CIPS continuous inpatient spell
CMHT Community Mental Health Team
COPD chronic obstructive pulmonary
disorder
CPA Care Programme Approach
CRHT crisis resolution and home
treatment
ECT electroconvulsive therapy
FCE finished consultant episode
GMS General Medical Services
GP general practitioner
HES Hospital Episode Statistics
HESG Health Economists Study Group
HoNOS Health of the Nation Outcome
Scale
ICD-10 International Classification of
Diseases, 10th revision
IMD Index of Multiple Deprivation
IQR interquartile range
IRR incidence rate ratio
LIT local implementation team
LOS length of stay
LSOA lower super output area
MHMDS Mental Health Minimum Data Set
MHRN Mental Health Research Network
NICE National Institute for Health and
Care Excellence
NIHR National Institute for Health
Research
ONS Office for National Statistics
P4P pay for performance
PCT primary care trust
PMS Personal Medical Services
PPI patient and public involvement
QOF Quality and Outcomes Framework
RAMP Resource Allocation for Mental
health and Prescribing project
SD standard deviation
SMI serious mental illness
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
xvii

Plain English summary
Serious mental illness (SMI) such as schizophrenia and bipolar disorder can have a devastating impact.
General practitioners (GPs) provide both mental and physical care for people with SMIs. If GPs provide
high-quality care it is possible that their patients with a SMI, in addition to experiencing improvements
in the health and well-being, will have fewer unplanned hospital admissions (for mental and physical
health problems) and, if they are admitted, they may have shorter lengths of stay (LOSs) as they can be
discharged sooner.
The UK Quality and Outcomes Framework (QOF) pays GP practices for providing good-quality care to
patients with a SMI through having a regular review and a care plan. Records are kept of how well
practices carry out these tasks. Using these records, we investigated whether or not better quality of
primary care, as measured in the QOF, is linked to (a) lower levels of unplanned (emergency) admissions
for people with a SMI; (b) higher levels of planned admissions for physical care; (c) a shorter length of
hospital stay; and (d) lower public sector costs. We found that, contrary to our expectations, better care is
associated with higher rates of unplanned and planned admissions for people with a SMI, for both mental
and physical health problems, and has no impact on LOS or costs.
It is possible that higher admissions reflect GPs finding previously unmet need. To investigate this further,
research needs to look at the care received by individual patients and the way QOF reflects this care.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
xix

Scientific summary
Background
Serious mental illness (SMI) encompasses a set of chronic enduring conditions such as schizophrenia,
bipolar disorder and psychoses. Despite a lifetime prevalence of >1%, considerable disease burden, poor
outcomes and costs, there has been little empirical research on the processes of care for people with SMIs
in primary care. Primary care plays a central role in the provision of care for people with SMIs, with around
31% treated solely by their general practitioner (GP).
Good-quality primary care management of patients with a SMI should reduce complications of a SMI and
comorbidities and should, therefore, be associated with lower unplanned admission rates. Conversely,
better quality of care may result in more health problems being identified as part of regular screening
activities and more frequent GP–patient contacts, thereby leading to more planned (elective) admissions
for hospital care. If better-quality primary care leads to reduced emergency admissions, it may also be
associated with lower NHS expenditure. Length of stay (LOS) for patients with a SMI is typically much
longer than for other patients and better management in primary care could shorten their lengths of stay
in hospital.
Quality indicators for the management of SMIs have been routinely measured in English primary care as
part of the Quality and Outcomes Framework (QOF) since its introduction in 2004. The QOF is a voluntary
incentive scheme for primary care practices, which offers financial rewards for good-quality care and one
domain of the QOF focuses specifically on the management of people with SMIs.
Our study used four SMI QOF indicators. MH6 and MH9 relate to patients receiving a review and having a
care plan in place, while two indicators, MH4 (record of thyroid and renal function) and MH5 (lithium
levels in appropriate range), relate only to the subset of SMI patients who have bipolar disorder.
Objectives
Our research questions are:
1. Is better general practice performance on SMI QOF indicators associated with:
i. lower rates of emergency hospital admissions for SMIs for practice patients with a diagnosis of
aSMI?
ii. lower rates of emergency admissions for a SMI for practice patients with a diagnosis of
bipolar disorder?
iii. lower rates of emergency admissions for physical conditions for practice patients with a current or
previous diagnosis of a SMI?
iv. higher rates of elective admissions for physical conditions in patients with a current or previous
diagnosis of a SMI?
2. Is better general practice performance on SMI QOF indicators associated with shorter LOS for practice
patients with SMI following admission for a SMI?
3. Is better performance on SMI QOF indicators associated with lower secondary care expenditure for
mental health services for practice patients with a SMI?
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
xxi

Data
To answer the first set of research questions (1i to iv), we merged practice-level QOF data from around 8500
GP practices in England with admissions data for practice patients from Hospital Episode Statistics (HES) data
for the study period 2006/7–10/11. We took account of baseline admissions for the financial years 2003/4–4/5.
This pre-sample baseline picks up unobserved practice confounding characteristics which are time invariant.
We identified SMI admissions by a main International Classification of Diseases, 10th revision, diagnosis of
F20–F31 and bipolar admissions as F30–F31. We dropped practices with a list size <1000 patients. We
excluded practices if they did not report a SMI register in QOF or if the number of patients on the SMI QOF
register was below 5. Only adult patients (aged 18 years and over) were included in the analyses.
Practices can ‘exception report’patients from achievement on QOF indicators for various reasons including
the patient is deemed to be unsuitable for treatment, is newly registered with the practice or newly
diagnosed or that the patient makes an informed dissent. Data on excluded individuals are removed from
the achievement calculation for the purposes of determining the QOF payments due to practices. However,
we included all SMI patients (those recorded as eligible plus those who were exception reported) in the
denominator for calculating achievement rates for QOF indicators, since we could not distinguish whether
or not an admitted patient had been exception reported.
The HES and QOF data were linked to information on GP practice characteristics, characteristics of their
patient populations and to population characteristics such as deprivation and other potential confounders
that are recorded at small-area level [i.e. lower super output areas (LSOAs)]. We also controlled for
measures of access to care such as distance to nearest hospital and availability of crisis resolution and
home treatment teams. All analyses were carried out at GP practice level.
To answer the second research question, the same data sources were used as described above, but
(1) admissions were not aggregated to practice level, (2) we excluded patients admitted primarily for
physical conditions, and (3) we excluded outlier patients who remained in hospital for more than 180 days
to reduce the effect of unusually long-stay patients.
To examine the third research question looking at the relationship between practice QOF performance and
subsequent mental health expenditure, we used individual-level data from the Mental Health Minimum
Data Set (MHMDS), which was costed using data from NHS Reference Costs for 2006/7 and 2007/8.
Variables included demographic information and resource use data for hospital inpatient and outpatient
care and community care provided by specialist mental health teams. MHMDS data were not structured
in complete spells (episodes of care) and so we estimated a total cost per year for each individual rather
than using spells as the unit of analysis. Owing to the absence of activity volume data for local authority
services, we were unable to attach costs to all the resource use variables in MHMDS. This meant that
the total annual cost was missing for around 20% of patients. As MHMDS contained no diagnostic or
procedure codes we focused only on overall SMI measures in the QOF and excluded measures that applied
specifically to people with bipolar disorder. Area characteristics were incorporated at practice level using
weighted average values based on the LSOAs in which practice patients resided.
Methods
For the first research question (the relationship between practice QOF performance and admission rates),
we estimated mixed-effects count models that take account of the nested structure of annual counts of
admissions for each GP practice. We estimated separate models for each of the four admission types and
allowed the two set of QOF indicators (MH6 and MH9) and (MH4 and MH5; for bipolar admissions) to
enter separately or jointly. We ran sensitivity analyses to account for the fact that some patients with a SMI
SCIENTIFIC SUMMARY
NIHR Journals Library www.journalslibrary.nihr.ac.uk
xxii

are admitted repeatedly within a short period of time. We therefore also counted the number of patients
admitted at least once in a year, as an alternative to the number of admissions of practice patients in the
year. We also tested the inclusion of patients with an unspecified main diagnosis to account for poor
coding of diagnoses in some providers.
For the second research question, (relationship between practice QOF performance and LOS) we estimated
mixed-effects linear regression models. We transformed LOS using a logarithmic transformation. We
analysed the number of days spent in hospital and included day cases, rather than just analysing the
number of nights, since admissions with no nights still consume resources. We estimated models for the
two QOF indicators (MH6 and MH9) both separately and jointly. We ran sensitivity analyses, using a model
without either patient-level covariates or hospital fixed effects, and a model with patient-level covariates
and without hospital fixed effects.
The third research question (relationship between practice QOF performance and annual patient costs) was
investigated using a multilevel mixed-effects linear regression model, with a logarithmic transformation of
total annual cost per patient. We estimated models for each of the two QOF indicators (MH6 and MH9)
separately and jointly. Given the lack of diagnostic information in MHMDS, we ran a sensitivity analysis
excluding individuals aged 65 years and over who may have had dementia rather than a psychotic disorder.
We carried out further robustness checks for all three research questions estimating various levels of exception
reporting to test sensitivity to assumptions around the specification of QOF achievement. The absence of
individual-level data on QOF achievement and exceptions means we do not know what percentage of
exceptions is valid. We therefore ran a series of regressions in which the percentage of exceptions deemed
valid ranged from 0% to 100%, with increments of 10% in each regression. All models included year
indicators to allow for temporal trends. We reported GP-level analysis coefficients as incidence rate ratios (IRRs).
Services users and carers were actively involved throughout the project, with representation on our
steering group.
Results
The data set for the first research question resulted in a sample of 8223 GP practices for analyses with
SMI admissions and 8042 practices for the bipolar sample. The association between QOF achievement
and admissions was generally positive, implying better quality of primary care is associated with more
admissions. The estimated IRRs suggested that, for the average practice, an additional 10 percentage
points in QOF achievement was associated with an increase in the practice SMI admission rate of
approximately 1.9% (95% confidence interval 1.0% to 2.9%). The strength of the effect varied across
indicators and admission types. We found statistically significant associations between QOF achievement
on MH9 (review of SMI patients) and both mental health and physical admissions. In contrast, while always
positive, the effect of MH6 (care plan) on admissions was only statistically significant for physical
emergency admissions. Results were not significant for elective admissions, although these were always
positive. Of the two lithium indicators, results were statistically significant for MH4 (thyroid and renal
function record). The significance of results depends on the way in which we specify the percentage of
valid exception reporting. Results were robust to sensitivity analyses for the number of patients admitted at
least once in a year and the inclusion of patients with an unspecified main diagnosis.
For the LOS analysis we had a data set of 98,993 individuals in 7912 practices. Longer LOS was associated
with a primary diagnosis of schizophrenia, a higher number of comorbidities, older age, male gender,
formal detention status, and Asian and black ethnicity. The quality of primary care, as measured by the
QOF scores of the patient’s practice, had no significant effect on LOS. Results were robust to sensitivity
analyses for model specification and valid exception reporting.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
xxiii

For the analysis of costs using the MHMDS we had a sample of 981,373 observations for 711,820 adults.
The mean annual per patient cost was £3159. The covariates had the anticipated signs in the regressions
and suggested higher costs are associated with middle age, black or mixed ethnicity and formal detention.
Across all analyses, a higher prevalence of informal carers in the residential area covered by the practice
population was strongly associated with lower cost. Results from the regression analyses found the QOF
indicators for annual review (MH9) and care planning (MH6) had no significant effect on total annual patient
costs in any of the models, whether tested alone or jointly. Results were robust to sensitivity analyses that
excluded individuals aged 65 years and over and varied the assumed validity of exception reporting.
Conclusions
The positive association we found between higher QOF achievement, particularly for annual health checks
(MH9), and higher rates of emergency admissions for both mental and physical health admissions, was
contrary to expectation. There are a range of possible explanations: (1) higher quality of primary care, as
measured by QOF, may not effectively prevent the need for secondary care; (2) patients may receive their
QOF checks post discharge, rather than prior to admission, as we do not know whether individuals who
were admitted had received QOF checks or not; (3) SMI patients may select into practices that are more
receptive to them or better organised to provide their care, and such practices would report carrying out
more QOF checks but also have more emergency admissions; (4) better-quality primary care may be
picking up unmet need for secondary care; and (5) the QOF measures may not reflect accurately the
quality of primary care.
Further research would require patient-level data, in addition to practice-level data, to examine a number
of research priorities: (1) the patient pathway and the timing of QOF checks in relation to admissions to
determine causality; (2) which QOF measures might effectively prevent secondary care admissions among
this patient group; (3) whether or not there are other (non-QOF) measures of primary care quality and
management of people with a SMI that could reduce unplanned admissions and could potentially be
incentivised; (4) the specific conditions and indications for admission among people with a SMI, to determine
how they could be prevented; (5) which types of admissions are potentially avoidable for SMI care;
(6) how comprehensive care plans are developed and documented for people with a SMI and their families
and carers; (7) the level of unmet need for people with a SMI, particularly at GP practice level; and
(8) how the supply-side capacity constraints impact on the ability of GPs to admit patients with a SMI.
There are a number of implications for practice: First, assess value for money of QOF health checks for people
with a SMI. One possible conclusion from our results is that the QOF is not effective at reducing the use of
secondary care services and should therefore be abandoned. However, the QOF was not specifically designed
to reduce unplanned admissions. Many of the emergency admissions may be appropriate and represent
good-quality care by GPs and may pick up and address unmet need. It would therefore be premature to
draw conclusions about whether or not regular checks of people with a SMI should continue to be
incentivised through the QOF. QOF checks, specifically those that focus on physical care may still be effective
in promoting patient health and may be valued by service users. Second, factor in resource requirements for
likely increase in referrals following QOF checks for SMIs. Practitioners and commissioners should be aware
that carrying out regular checks on people with SMIs will have implications for the organisation and funding
of mental health care. Third, improve diagnostic coding quality in secondary care, and finally, improve data
coverage and quality of the MHMDS. A general observation from the study is the need for better-quality
mental health data to enable important questions about quality of care to be addressed. Data quality could
be incentivised particularly around the collection of accurate diagnostic information.
Funding
The National Institute for Health Research Health Services and Delivery Research programme.
SCIENTIFIC SUMMARY
NIHR Journals Library www.journalslibrary.nihr.ac.uk
xxiv

Chapter 1 Introduction
Serious mental illness (SMI) encompasses a set of chronic enduring conditions such as schizophrenia,
bipolar disorder and other psychoses. Although some people make a full recovery, most will develop a
lifelong illness.1Schizophrenia is a psychotic disorder marked by severely impaired thinking, emotions and
behaviours. People with schizophrenia are typically unable to filter sensory stimuli and may have altered
perceptions of their environment including delusions and hallucinations. If untreated, people with
schizophrenia may gradually withdraw from interactions with other people and lose their ability to take
care of their personal needs. Schizophrenia is a disease that usually begins in early adulthood and the
average age at onset is 18 years in men and 25 years in women.2Psychosis is a symptom or feature of
SMI, typically characterised by radical changes in personality, impaired functioning, and a distorted or
non-existent sense of objective reality exhibited by delusions and hallucinations. Bipolar disorder is a mood
disorder that causes dramatic emotional changes and mood swings, whereby individuals experience
alternating episodes of mania, or hypomania, and depression.
The prevalence of bipolar disorder is about 1–2% of the UK population, although bipolar spectrum
disorder may affect as many as 8%.3The point prevalence of schizophrenia is around 0.7%4and the
lifetime prevalence around 1%. A systematic review of the incidence and prevalence of schizophrenia and
other psychotic disorders in England found an overall (pooled) annual incidence for all psychotic disorders
of 32 cases per 100,000 people,5with much higher rates for psychotic disorders in young adults, in men,
in black and minority ethnic groups and in more deprived neighbourhoods.
The total annual economic burden of schizophrenia (non-affective psychoses) in 2009 was estimated at
£8.8B, of which service costs contributed 40%, informal care 13% and lost employment 47%, while for
affective psychoses (bipolar disorder) the total cost to UK services and society per annum was estimated
at £5.0B, with 80% coming from NHS costs, 3% from informal care costs and 16% costs of lost
employment.5Therefore, SMI creates a high cost to society as well as to NHS services. Mental health is the
single biggest programme budget expenditure area in the NHS out of 23 main programmes of care in
England,6bigger than cancer or cardiovascular disease, and schizophrenia and psychoses are a key driver
of length of stay (LOS), bed-days and resource use in the NHS.7
Life expectancy for people with schizophrenia and bipolar disorder is usually around 20 years less than for
the general population,8–13 and people with a SMI die prematurely, the majority from preventable causes.
People with a SMI are at higher risk of physical ill-health and thus hospitalisation.14–19 Compared with the
general population, people with a SMI have double the risk of diabetes, two to three times the risk of
hypertension and three times the risk of dying from coronary heart disease,20,21 and experience a 10-fold
increase in deaths from respiratory disease.2,22 Owing to much higher smoking rates than the general
population, smoking-related diseases, heart disease and premature death are more common in people
with SMI.23 People with SMI are at much higher risk of obesity because the atypical antipsychotic
medications they take are associated with weight gain6and their illness reduces their activities and impairs
their ability to exercise. Poor compliance with medication is well recognised among people with these
diagnoses and this may lead to relapse, poorer outcomes and admissions. Schizophrenia and bipolar
disorder rank among the top 10 causes of disability in developed countries worldwide.5
Despite its prevalence, considerable disease burden, poor outcomes and costs, there has been little
empirical research on the processes of care for people with SMI. They are often disenfranchised and
marginalised and experience stigma and thus do not receive the same priority as other chronic
disease conditions.2,24
In the English NHS, a number of different services provide care for people with a SMI. There has been a
general trend away from long hospital stays in favour of shorter-term pharmacological stabilisation in
hospital, followed by longer multidisciplinary follow-up in the community or primary care setting.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
1

In secondary care, patients may be seen by crisis resolution and home treatment (CRHT) services which
provide intensive home-based care for individuals in crisis as an alternative to hospital treatment.25
However, most people with a SMI are treated in primary care by their general practitioner (GP). People
with a SMI consult their GPs more frequently26 and are in contact with primary care services for a longer
cumulative time than people without mental health problems.27,28 Recent evidence from the UK finds that
around 31% of patients with a SMI are treated solely by their GP or other primary-care clinician and the
estimated national rate is around 57% for schizophrenia and 38% for bipolar disorder.29 Primary care is
therefore central in the care of people with a SMI. The GP oversees care, prescribes medication and
provides both mental and physical health services.
If we accept that high-quality provision of primary care may allow people to be cared for in a more
comprehensive and proactive way, reducing the chance that they will be admitted to hospital as an
emergency, we may find higher-quality primary care is associated with fewer unplanned admissions. In the
UK, unplanned hospital admissions have risen steadily over the past 10 years and approximately 35% of
all hospital admissions are unplanned, which places an increasing source of pressure on health system
resources, costing the English NHS around £11B per annum.30 Policy-makers are increasingly seeking ways
to reduce unplanned hospital admissions, especially for people with long-term conditions.
A number of interventions have been proposed in primary care as a means to reduce unplanned hospital
admissions. The evidence for an association between higher quality of primary care as an intervention and
reduced rates of admission is however mixed.31 Lower rates of admission for asthma were found in
practices whose prescribing patterns suggested better preventative care.32 Provision of diabetes clinics in
primary care was significantly associated with reduced admission rates for diabetes, but the provision of
asthma clinics was not associated with a similar reduction in admissions.33 Conversely, a systematic review
showed that high standards of diabetes care in primary care did not necessarily lead to reduced
hospital admissions.34
General practitioners are incentivised through the NHS Outcomes Framework to improve quality of care
and outcomes for patients, with a stronger emphasis on mental health.35 Quality indicators for mental
health are routinely measured in English primary care as part of the Quality and Outcomes Framework
(QOF) which was introduced in 2004.36 The QOF is a voluntary incentive scheme for primary care practices
which offers financial rewards for good-quality care such as meeting targets on clinical, organisational and
patient experience indicators. In practice, nearly all GP practices participate. This may be related to the
generous financial incentives attached to achievements on the QOF and the ease with which many
practices have fulfilled the requirements. By encouraging the provision of better-quality care, the QOF has
the potential to reduce unplanned, preventable hospital admissions, but evidence for this effect is mixed.
Table 1 shows a summary of the evidence on the association between the QOF and admissions for a range
of disease conditions. No association has been found between admission rates and primary care quality
indicators for coronary heart disease, asthma or chronic obstructive pulmonary disorder (COPD).37–40
However, other studies have found a significant association between lower levels of achievement and
higher emergency admissions for diabetes40,41 and a small effect for stroke.42 Most of these studies are,
however, based on cross-sectional data rather than longitudinal panel data. There has been no evidence
to date on the relationship between primary care quality and admission rates for SMIs.
Effective primary care can have an important preventative role, and could therefore be associated with a
reduction in emergency admissions. Conversely, better quality of care may result in more health problems
being identified as part of regular screening activities and more frequent GP–patient contacts, thereby
leading to more elective admissions for hospital care. If better-quality primary care can reduce costly
emergency hospital admissions it may have knock-on effects for NHS expenditure and resource use.
Better-quality primary care may also reduce LOS, if patients are effectively cared for outside hospital.
LOS for patients with a SMI is typically much longer than for other patients and better management in
primary care could shorten their LOS in hospital.16
INTRODUCTION
NIHR Journals Library www.journalslibrary.nihr.ac.uk
2

1. Our first research question therefore is whether or not better primary care practice performance on
specific mental health QOF indicators is associated with:
i. lower rates of emergency hospital admissions for SMIs for practice patients with a diagnosis of
aSMI.
ii. lower rates of emergency admissions for SMIs for practice patients with a diagnosis of
bipolar disorder.
iii. lower rates of emergency admissions for physical conditions for practice patients with a current or
previous diagnosis of a SMI.
iv. higher rates of elective admissions for physical conditions in patients with a current or previous
diagnosis of a SMI.
2. Our second research question relates to whether or not better-quality primary care as measured by
specific mental health QOF indicators is associated with reduced resource use in terms of shorter LOS
for people with a SMI following admission for a SMI.
3. Our third research question is whether or not better-quality primary care as measured by specific mental
health QOF indicators is associated with reduced resource use in terms of lower secondary care
expenditure for mental health services for people with SMIs.
Our null hypotheses are that there is no association between primary care quality and either admissions,
LOS or costs. Our alternative hypotheses, as presented in Table 2, are that preventative care could lower
emergency hospital admissions, reduce LOS and reduce mental health expenditure; and that regular
screening could increase elective admissions.
TABLE 1 Evidence for the impact of the QOF on admissions
Study Clinical area Methodology Results
Downing et al., 200737 Asthma, cancer, COPD,
CHD, diabetes, stroke
2004/5 (2 PCTs) Small and inconsistent
Bottle et al., 200838 CHD (coronary angioplasty
and CABG)
2004/5 No association
Bottle et al., 200839 Diabetes 2004/5 Significant, but weak negative
association (patients over 60 years)
Purdy et al., 201140 CHD (angina and MI) 2005/6 No association, CHD (negative
association, angina)
Dusheiko et al., 201141 Diabetes 2004/5–2006/7 Significant negative association
Soljak et al., 201142 Stroke (transient
ischaemic attack)
QOF 2008/9,
admissions 2006/7–2008/9
Small negative association
(cholesterol)
CABG, coronary artery bypass surgery; CHD, chronic heart disease; MI, myocardial infarction; PCT, primary care trust.
TABLE 2 Hypothesised direction of association between better quality of care and outcomes
Research question Expected association
1: Mental health admissions –emergency Negative
1: Physical health admissions –emergency Negative
1: Physical health admissions –elective Positive
2: LOS Negative
3: Mental health expenditure Negative
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
3

The remainder of the report is structured as follows. Chapter 2 describes the QOF and the SMI domain
within the QOF as well as how some cases are excluded from the calculation of the achievement rates in
QOF (exception reporting). Chapter 3, Does better primary care reduce hospital admissions?, gives the data,
empirical approach, results and sensitivity analysis for the first research question; Chapter 3,Does better
primary care reduce inpatient length of stay?, provides the data, empirical approach, results and sensitivity
analysis for the second research question and Chapter 3, Does better primary care reduce cost of care?,
provides the data, empirical approach, results and sensitivity analysis for the third research question.
Chapter 4 discusses the findings, while Chapter 5 includes discussion on the implications for research and
practice (see Chapter 5, Implications for research,andChapter 5,Implications for practice).
INTRODUCTION
NIHR Journals Library www.journalslibrary.nihr.ac.uk
4

Chapter 2 Measures of quality of primary care for
people with serious mental illness
The Quality and Outcomes Framework
Pay-for-performance (P4P) programmes have been widely adopted as a method for improving the quality
of care and incentivising efficiency.43,44 In April 2004, the QOF was introduced as part of a new General
Medical Services (GMS) contract for British primary care. This major P4P scheme seeks to reward
higher-quality primary care by offering financial incentives to general practices, and participation is
voluntary.45 The QOF has targets on clinical, organisational and patient experience indicators against which
practices score points according to their level of achievement. The indicators are based on clinical evidence
and designed to support NHS policies; they are regularly reviewed and revised. Points are not directly
proportional to performance or performance improvement; rather, achievement is triggered at lower and
upper target thresholds of attainment for each performance indicator.46 Total points are adjusted for
practice size and disease prevalence relative to national average values.47
When the QOF was introduced in 2004/5, the price per point was £75, which translated into per-patient
payments ranging from just £0.13 for an indicator on chronic kidney disease, to almost £88 for the mental
health indicator on lithium (MH5).46 By 2013/14, the price per point had risen to £157.48
The mental health domain of the Quality and Outcomes Framework
One of the clinical domains of the QOF is severe mental illness, the focus of our study. There have been
regular revisions to the QOF since its introduction in 2004/5, but the subset of four indicators we examine
have remained unchanged over the study period April 2006–March 2010.
We considered several other indicators in QOF that may also be relevant for people with SMI. These
included two organisational domain indicators, Education 7, which requires practices to undertake
‘significant event reviews’including suicide or sections under the Mental Health Act 198349 and Medicines 7,
which requires practices to have a system to identify and follow up non-attenders for injectable neuroleptic
medication. We also considered the indicators for depression, for smoking and the patient experience
indicator, PE01. None of these indicators proved suitable for our analyses; some indicator definitions varied
too much over our study period (smoking, patient experience), while others were binary measures that
captured limited between-practice variation when achievement was high (Education 7 and Medicines 7).
We considered the indicators for depression as a marker of practice quality. However, the National
Institute for Health and Care Excellence (NICE) QOF Indicator Advisory Committee had recommended the
depression indicators should be retired because they were not shown to be effective in improving
processes of care or health outcomes for people with depression, and encouraged ‘a bureaucratic
approach to identifying depression at the expense of more engaged screening’.7Instead, we derived a
measure of patient experience from the annual GP patient survey and included this in all the relevant
analyses (see Chapter 3, Data sets used to generate other covariates).
Each GP practice is required to record the number of SMI patients on its practice list, and the practice’s
achievement on five SMI-related QOF indicators. Indicators are described in Table 3, along with their
clinical motivation rational and payment thresholds.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
5

TABLE 3 Overview of the four SMI QOF indicators included in the study
Indicator
number Variable description Rationale
Payment
threshold
Review indicators
MH9 The percentage of patients with schizophrenia,
bipolar affective disorder and other psychoses
with a review recorded in the preceding
15 months. In the review there should be
evidence that the patient has been offered
routine health promotion and prevention
advice appropriate to their age, gender and
health status
Patients with serious mental health problems
are at considerably increased risk of physical
ill-health, are less likely to be offered health
promotion advice and far more likely to
smoke than the general population;
premature death and smoking-related
diseases (e.g. respiratory disorders and heart
disease), are more common among people
with a SMI who smoke than in the general
population of smokers. People with
schizophrenia appear to be at increased risk
of impaired glucose tolerance and diabetes,
and this is independent of treatment with the
newer atypical antipsychotic drug
40–90%
Lithium indicators
MH4 The percentage of patients on lithium therapy
with a record of serum creatinine and TSH in
the preceding 15 months
There is a much higher than normal incidence
of hypercalcaemia and hypothyroidism in
patients on lithium, and of abnormal renal
function tests
40–90%
MH5 The percentage of patients on lithium therapy
with a record of lithium levels in the
therapeutic range within the previous
6 months
The therapeutic range for patients on lithium
therapy is normally 0.6–1.0 mmol/l. Levels
below 0.6 mmol/l may be acceptable,
depending on the clinical circumstances of
the patient
40–90%
Care plan indicators
MH6 The percentage of patients on the register
who have a comprehensive care plan
documented in the records agreed between
individuals, their family and/or carers
as appropriate
This indicator reflects good professional
practice. The plan will include information on
the patient’s current health status and social
care needs including how needs are to be
met, by whom, and the patient’s
expectations; how socially supported the
individual is; co-ordination arrangements with
secondary care and/or mental health services
and a summary of services are received;
occupational status; early warning signs
(relapse signature); the patient’s preferred
course of action (discussed when well) in the
event of a clinical relapse (who to contact and
medication preferences)
25–50%
TSH, thyroid-stimulating hormone.
MEASURES OF QUALITY OF PRIMARY CARE FOR PEOPLE WITH SERIOUS MENTAL ILLNESS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
6

Our study focuses on four indicators: MH4, MH5, MH6 and MH9. Two indicators, MH9 and MH6, are
applicable to all people with a SMI. MH9 is an annual review of the patient’s physical health. Relative to
the general population, people with SMIs are at a higher risk of physical illness and are more likely
to smoke. If they do smoke, they are more likely than other smokers to suffer premature death and
smoking-related diseases. The review should cover use of alcohol, drugs and smoking behaviour and offer
appropriate checks for blood pressure, cholesterol, body mass index and drug-related diabetes risk.
The review may also include checks for cervical screening and medication review. MH6 requires a
comprehensive care plan to be documented and agreed with individuals and their families or carers.
It is designed to reflect good professional practice, and should cover the patient’s current health and social
care needs and how these are met. Co-ordination arrangements with secondary care, occupational status
and patient preferences in the event of a clinical relapse are also to be set out. If the patient is treated
under the Care Programme Approach (CPA), this care plan can be used for the QOF.20
The two lithium-related indicators, MH4 and MH5, relate to admissions for patients specifically with bipolar
and mood affective disorder, a subset of all people with SMIs. This can be justified by the observation that
lithium therapy is indicated for the treatment of bipolar disorder but rarely used in other people with SMI.
MH4 involves an annual check of thyroid and renal function, as the risk of hypothyroidism and of
abnormal renal function tests is elevated in people on lithium. MH5 requires that people on lithium have
regular tests to ensure serum lithium levels are within the therapeutic range.
The indicator MH7 is excluded from our study as it does not apply to all practices on a regular basis.
It has been acknowledged by NICE as an anomaly indicator: practices can only achieve MH7 if some
patients did not attend the annual review meeting. If all patients attend all reviews, which in itself would
be an indicator of good process quality by reviewing ‘hard-to-reach patients’, no achievements can be
made on MH7.
All QOF mental health indicators have upper payment thresholds of between 50% and 90%. This means
that practices can earn the maximum points on an indicator without necessarily achieving the target for all
patients on the register.
Two of our four QOF indicators, MH6 and MH9, apply to all patients on the practice SMI register, that is
the number of patients at risk of admission. For the remaining two SMI QOF indicators, MH4 and MH5,
the relevant denominator is the number of patients on lithium therapy, which forms a subsample of the
patients on the SMI register.
Exception reporting in the Quality and Outcomes Framework
Practices can ‘exception report’patients from specific indicators.50 The GMS contract sets out valid
exception reporting criteria, such as the patient is deemed unsuitable for treatment, is newly registered
with the practice or newly diagnosed, or that the patient exercises informed dissent. This means that data
on these individuals are removed from the achievement calculation for payment purposes. An analysis of
the prevalence and reasons for exception reporting in 2008/9 found wide variation between practices
and between indicators although relatively few patients were excluded for informed dissent.51 Exception
reporting boosted practice income by an average of £3834. One-quarter of this amount was explained by
2 of the 62 indicators studied, one of which was MH9, the annual review indicator. On average, practices
exception reported 14% of eligible patients for this indicator, which is high relative to other indicators.
Reporting cases as ‘exceptions’may reflect good-quality care –GPs are carefully reviewing cases to
establish their eligibility for treatment –but could alternatively reflect ‘gaming’by GPs, who can increase
the number of points they earn by reducing the eligible population inappropriately,52 and so the legitimacy
of exception reporting is ambiguous. As a conservative approach to assessing performance, we therefore
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
7

include all patients in the denominator for each QOF indicator: those recorded as eligible, plus those who
are potentially eligible but who were ‘exception reported’. Achievement is calculated as:
achievement =A
(A+NA+E), (1)
where Adenotes the number of patients for whom the indicator was achieved, NA denotes the number of
patients for whom the indicator was not achieved and Edenotes the number of patients exception
reported. For simplicity, we refer to the sum of (A+NA +E) as the ‘register’for this specific indicator.
The exception report rate is derived as:
exception rate =E
(A+NA+E). (2)
The QOF incentive regime rewards GPs on the basis of their achievement adjusted for exceptions, which is
calculated as:
adjusted achievement =A
(A+NA). (3)
We chose to analyse our data based on the achievement, not the adjusted achievement. This approach is
justified by the fact that, given our data, we cannot distinguish whether an admitted patient did not
receive the ‘QOF treatment’or was exception reported (E). We conduct sensitivity analysis to ascertain
whether or not our results are sensitive to this analytical choice.
A diagrammatic overview of how these performance indicators are constructed is provided in Figure 1.
Not achieved (NA)
Achieved (A)
Exception
reported (E)
FIGURE 1 Overview of performance measures constructed for the regression analyses.
MEASURES OF QUALITY OF PRIMARY CARE FOR PEOPLE WITH SERIOUS MENTAL ILLNESS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
8

Chapter 3 Empirical analysis
Does better primary care reduce hospital admissions?
Overview
This section focuses on our first research question, namely whether or not better-quality primary care is
associated with reduced unplanned hospital admissions for both mental and physical care. Good-quality
primary care might be expected to reduce emergency hospital admissions for people with SMI. Conversely,
better quality of care may result in more health problems being identified as part of regular screening
activities and more frequent GP–patient contacts, thereby leading to more planned (elective) admissions for
hospital care. Our null hypothesis is that better primary care, as measured by the SMI QOF indicators, has
no effect on the number of hospital admissions, both elective and emergency, for both physical and
mental health conditions. Our alternative hypothesis is that better primary care is associated with the
number of hospital admissions, and we expect this association to be negative for emergency admissions,
for both physical and mental health conditions, and positive for elective admissions for physical
health problems.
We combined patient- and GP practice-level data for the study period April 2006–March 2011. The unit of
analysis was the GP practice.
We considered three types of admissions and analysed these separately:
1. admissions for a mental health crisis –as indicated by a main diagnosis of a SMI [International
Classification of Diseases, 10th revision (ICD-10) codes F20–F31]
2. admissions for non-mental health-related care (‘physical care’)–admitted as an emergency patient
3. admissions for non-mental health-related care (‘physical care’)–admitted as an elective patient.
Identification of patients was based on ICD-10 diagnosis codes. Physical care admissions included
admissions for all primary diagnoses excluding R69 (‘Unknown and unspecified causes of morbidity’)
and excluding all diagnosis codes in the range F00–F99 (‘Mental and behavioural disorders’).
Furthermore, because some QOF indicators specifically apply to people with bipolar disorders, we
also considered:
4. admissions for a mental health crisis –for people with a diagnosis of bipolar disorder only
(ICD-10 codes F30–F31).
All SMIs and bipolar admissions were deemed to be emergency admissions (irrespective of how they are
coded by the hospital), which is in line with expert policy and clinical guidance provided by our steering
group. In practice, there is inconsistency of coding by providers and it makes sense to combine admissions.
Our interpretation is that considering all admissions as emergencies is a crude, but sensitive, metric of
avoidable admissions.
In the analysis of bipolar admissions, we had data from 8042 practices; the remaining three analyses were
based on data from 8223 practices.
Data
We merged QOF data from around 8500 GP practices in England with admissions data from Hospital
Episode Statistics (HES) for the study period April 2006–March 2011 using a unique GP practice identifier.
These data were linked to publicly available information on GP practice characteristics, characteristics of
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
9

their patient population, such as disease prevalence, and to population characteristics such as deprivation
and other potential confounders that are recorded at small-area level [i.e. lower super output areas
(LSOAs)]. We also controlled for measures of access to care. All analyses were carried out at GP practice
level and data were aggregated accordingly. We provide details about the individual data sets and the
linkage process in the following subsections.
Hospital Episode Statistics
The HES database records all inpatient and outpatient activity in England that is funded under the NHS.
The inpatient component of the HES data warehouse consists of approximately 18 million records per year,
each of which provides detailed information about the patient’s demographic characteristics, medical
condition, care pathway, as well as the GP practice with which the patient is registered. HES data are
reported at the level of finished consultant episodes (FCEs) and a new FCE is created every time a patient is
discharged from the care of one consultant to another consultant. To capture the entire care pathway and
derive correct admission numbers, we converted FCEs to continuous inpatient spells (CIPSs) which cover
the entire period from admission to final discharge. CIPSs also allow for transfers between providers of
inpatient care.
We extracted information on all NHS-funded inpatient activity for people aged 18 years or over and
diagnosed with a SMI. To identify people who have been diagnosed with a SMI, we searched all primary
and secondary diagnosis fields for the relevant ICD-10 diagnosis codes. Although the QOF uses diagnosis
to define patients eligible for practices’SMI registers, ICD-10 is not used. Instead, GP practices record
diagnoses using Read codes. We used a cross-mapping provided by the NHS Information Centre to
translate Read codes to ICD-10 codes. We considered diagnosis codes that cover two large groups of SMI:
schizophrenia, schizotypal and delusional disorders (F20–F25, F28–F29), and bipolar and mood affective
disorders (F30–F31) (Table 4).
For some episodes of physical care, a diagnosis of SMI or bipolar affective disorder may not have been
made or recorded in the inpatient records even though the patient suffers from this condition. This may
occur if (1) the diagnosis is not deemed clinically relevant for the physical care provided, that is, the patient
was treated for an unrelated medical problem and the psychiatric comorbidity did not interfere with the
treatment; (2) the diagnosis is not important for reimbursement purposes; (3) the condition is not apparent
at the time of assessment; (4) clinical coding is poor; or (5) to avoid any stigma associated with the
diagnosis. As a result, people with SMIs (including bipolar disorder) who are admitted for physical care may
not be identified as such when detection is based solely on diagnostic information contained in the current
inpatient record.
TABLE 4 International Classification of Diseases, 10th revision codes used to identify patients with SMI
ICD-10 code Description
F20 Schizophrenia
F21 Schizotypal disorder
F22 Persistent delusional disorders
F23 Acute and transient psychotic disorders
F24 Induced delusional disorder
F25 Schizoaffective disorders
F28 Other non-organic psychotic disorders
F29 Unspecified non-organic psychosis
F30 Manic episode
F31 Bipolar affective disorder
EMPIRICAL ANALYSIS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
10

To capture all relevant activity, we therefore linked patient records across time, based on the patient’s
unique identifier, and identified all secondary care provided to this patient on or after a diagnosis of a
SMI/bipolar has been made, whether or not the diagnosis is recorded in this specific inpatient record.
This identification strategy can be justified on the grounds that SMI is an enduring illness that may increase
or reduce in burden over time but rarely resolves. To ensure that activity at the beginning of our study
period is identified correctly, we extended our search period retrospectively by 5 years to April/March
2011. Hence, if a patient was diagnosed with bipolar affective disorder (ICD-10: F31) in 2002 and received
inpatient care for a physical condition in 2008, we will count the activity in 2008 even if no diagnosis of
bipolar affective disorder was recorded then.
All admissions for patients who changed GP practices within a financial year were excluded from the
data set because (1) it is not possible to assign the admission to a practice and (2) because it is unclear
whether the patient changed practices before or after an admission. The GP practice identifier in HES
refers to the practice to which the patient is discharged. We assumed that patients were already under
the care of the same GP at admission if no change in practice association is evident from the data.
Quality and Outcomes Framework data set
We extracted data on practice quality performance from the QOF data set. For each GP practice, we
obtained information on the number of patients on the SMI register, that is, the number of patients at risk
of admission, and the practice’s achievement on four SMI-related QOF indicators discussed previously.
We linked these data to the aggregate practice-level admissions data derived from HES through the unique
practice-year identifier.
Practices were excluded from our sample if they did not report a SMI register or if the number of patients
on this register was below 5. The latter exclusion criterion is justified by the potentially noisy measure of
practice performance that would be derived if achievement were calculated on a very small number of
patients. Owing to the typically small number of bipolar patients in each GP practice, we did not apply this
exclusion criterion to the analysis of bipolar admissions, that is, all practices were retained that had at least
one bipolar patient registered with them. Furthermore, we excluded practices for which the SMI or bipolar
registers were inconsistent across indicators, for example when the denominator of MH6 was different from
the denominator of MH9 even though both refer to the same set of patients with SMI in the practice.
Data sets used to generate other covariates
We controlled for a number of GP and practice characteristics from the GMS data set and attribution data
set (ADS). These include the 2-year moving average practice list size as well as the average age of GPs,
proportion of male GPs, whether or not the practice operates single-handed, and whether or not the
practice is contracted under the Personal Medical Services (PMS) scheme. We dropped observations
(practice-year) with a practice list size with fewer than 1000 patients because these are deemed unusually
small and uncharacteristic of the way in which primary care is normally organised in the English NHS.
To control for local population characteristics, we linked data from the Neighbourhood Statistics Census
(2001)53 and the Index of Multiple Deprivation (IMD; 2004) which is available at the LSOA level. LSOAs are
defined geographic units that cover an average population of 1500 individuals. There are 32,482 LSOAs
in England. GP practices typically care for people who reside in multiple LSOAs and the ADS provides a
breakdown of the practice population by LSOA. Based on this information, we derived a weighted average
of the local population characteristics of the practice and assigned this to the practice. From the ADS data
set we obtained the average age and male proportions of each practice’s registered patients. We also
derived a measure of deprivation based on the proportion of the population claiming incapacity benefit
for mental health disorders –this variable is part of the IMD employment domain. Finally, we derived
measures of ethnicity (percentage non-white) and rurality (percentage living in urban areas). ADS data are
collected at the beginning of each financial year, but QOF data are collected at the end of the financial
year. We therefore adjusted the estimates based on ADS by taking moving averages across 2 years of data.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
11

We constructed several measures of access to care. First, we derived a measure of access to secondary
care based on the distance between GP practice and the nearest (1) acute hospital and (2) mental health
hospital. Distance was calculated on the basis of postcodes and grid co-ordinates. We also controlled
for the availability of CRHT teams that provide alternative home care in an emergency and play a
‘gatekeeping’role in admissions to hospital.25,54,55 Data on CRHTs were collected as part of the Mental
Health Services Mapping Data between 2000 and 2009 at the level of ‘local implementation teams’(LITs)
which partly cover the geography of local authority social services.56 There is an almost one-to-one
correspondence between LITs and the approximately 150 commissioning organisations in England at that
time, primary care trusts (PCTs). Since we were only able to obtain service mapping data for two of our
five study years (2008 and 2009), we instead used PC-level fixed effects to model differences in service
provision by CRHTs. These 156 dummy variables capture all time-invariant differences between PCTs in
terms of their resource capabilities. We tested the inclusion of LIT data for the 2 years that we had
available and assuming the rest of the period’s data constant. This made little difference from the results
using PCT-level fixed effects.
Finally, to reflect differences between practices and regions in terms of supply and access to care, we
recorded the catchment population prevalence of NHS community psychiatric residential beds, the
percentage of practice patients able to book an appointment within 48 hours (measured in the GP patient
survey) and a measure of informal care provision (% of the catchment population providing informal care)
based on census data; the last is intended to acknowledge that the level of informal care provided is often
high for people with SMIs and may be considered a substitute for inpatient care.57
Empirical approach
The aim of this empirical analysis is to relate the number of patients admitted to hospital from a GP
practice to the practice’s quality performance, controlling for other factors that may drive admissions but
are unrelated to the quality of care provided. The number of admissions per GP practice is a non-negative
integer (i.e. count variable). We therefore estimated mixed-effects count models that acknowledge the
data generating process and the nested structure of annual counts of admissions reported for each
GP practice.58,59 We estimated separate models for each admission type and allowed the two QOF
indicators MH6 and MH9 (MH4 and MH5 for bipolar admissions) to enter separately (individually) or
simultaneously (jointly).
Let adm
it
be the number of hospital admissions from GP practice i=1,. . ., I within the year t=2006,. . .,
2010. The number of SMI (or bipolar) patients at risk of admission is denoted as risk
it
and enters the model
as an offset variable. We specified the Poisson regression model as follows:
admit =riskit ∗γi∗exp (Q′
itδ+X′
itβ+T′
tλ), (4)
where Q
it
is the measure of GP practice quality as measured by the QOF, X
it
is a vector of covariates that
capture differences in the practice patient population and the supply of and access to other mental
health-care resources as well as an overall intercept term, and T
t
is a vector of time indicator variables to
control for general trends in admissions. We also introduced a GP practice-specific effect γ
i
that captures
unobserved, time-invariant differences between practices in terms of their admission propensity.
The GP practice effects γ
i
are assumed to be randomly drawn from a gamma distribution with mean 1 and
variance αand assumed to be uncorrelated with the other covariates. Alternatively, one can also assume a
normal distribution for the random effects. However, the model with gamma distributed effects has a
closed form solution and is therefore typically preferred.59 We did not model the GP practice effects as
fixed effects using indicator variables because (1) this would preclude the estimation of PCT fixed effects,
and (2) because many of the dependent and independent variables of interest vary little over time,
that is, there would be insufficient within-GP practice variation to estimate the model. In order to reduce
any potential bias from unobserved practice-specific confounders and make the assumptions underlying
EMPIRICAL ANALYSIS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
12

the random-effects model more tenable, we included pre-sample baseline admission numbers per GP
practice as an additional regressor.60 These are taken as the average number of hospital admissions within
the financial years 2003/4 and 2004/5.
As it is common to all Poisson models, the conditional variance, V(adm
it
|Q
it
,X
it
,T
t
,γ
i
), is constrained to be
equal to the conditional mean, E(adm
it
|Q
it
,X
it
,T
t
,γ
i
). This property is known as equidispersion and is often
found to be violated in empirical data. To allow for over- or underdispersed data, we derived bootstrapped
standard errors for all parameter estimates using 200 replications and sampling with replacement.
We ran three sets of GP-level analyses. In the first analysis, our base case, the response variable is the
number of admissions per practice per year. In the second GP-level analysis, we run a sensitivity analysis,
in which our response variable is the number of people admitted at least once per practice per year. In the
third GP-level analysis, we run a sensitivity analysis in which we include an unspecified main diagnosis code
for SMI admissions. The sensitivity analyses are further described in Chapter 3,Sensitivity analyses.
Although the results from panel data models are regarded as more robust, we also estimated separate
cross-sectional models for each of the years 2006–10 individually. We adapted the empirical specification
presented above by dropping both the GP practice effects γ
i
and the time indicator variables T
t
, and report
robust standard errors.
All coefficient estimates are presented as incidence rate ratios (IRRs). The IRR represents the estimated event
rate under one scenario over the estimated event rate under a different scenario. For example, the IRR on a
covariate indicating whether or not the practice is reimbursed under PMS measures the ratio of expected
event rates under PMS over the expected event rate not under PMS. Values greater than one indicate
that increases in the value of the covariate are associated with an increased number of admissions; this
relationship is considered to be statistically significant (i.e. not a chance finding) if the lower confidence
interval (CI) is also above 1. Similarly, the covariates with estimated IRRs smaller than one are expected to
have a protective effect on admissions (with an upper CI below 1).
The presented IRRs reflect the percentage change in admissions for a unit change in the explanatory
variable. This relationship is non-linear and IRRs need to be rescaled for changes that are smaller/larger
than one unit using the following formula:
IRRx=IRR^(1=x). (5)
Here, xis the number of points in the new scale. Hence, if a unit change in a continuous variable is
associated with 20% more admissions (i.e. IRR =1.2), a 5% increase is associated with 0.92% more
admissions (IRR10%=1:2(1
20)=1:0092). This transformation does not affect inferences, that is, the assessment
of statistical significance. All models are estimated using the xtpoisson and Poisson commands in
Stata 12.0 (release 12; StataCorp LP, College Station, TX, USA).
Results
Descriptive statistics
Our sample consists of 8223 GP practices that have reported treating patients with a SMI during the 5-year
study period. The panel is unbalanced (mean t=4.7) because for some years some practices either (1) do
not report or participate in the QOF, (2) report having fewer than five patients with a SMI on their patient
register, (3) report inconsistent QOF registers or (4) are yet to be established or have ceased to exist.
The overall panel consists of 38,774 practice-year observations. Note that the number of practices and
practice-year observations is somewhat lower for the bipolar sample because not all practices that treat
people with a SMI (and produce a valid register) also treat people with bipolar disorder. The median
number of people on the SMI register in a GP practice is 40 [interquartile range (IQR) 22–64] and the
median number of people on the bipolar register is 6 (IQR 3–10).
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
13

Table 5 presents the total number of admissions, number of GP practices, number of GP practice-year
observations, as well as the mean and median annual number of admissions per GP practice, all broken
down by admission type. As expected from a count variable, the empirical distribution of the number of
admissions is highly skewed. The median number of admissions per year and practice ranges from
6 (IQR 3–12) physical (emergency) admissions to 1 (IQR 1–2) bipolar admission. The mean number of
admissions is generally higher. On average, the number of annual admissions per practice for SMIs was
3.52 admissions, and this ranged from 1.12 admissions [standard deviation (SD) 1.61 admissions]
for bipolar admissions to 8.86 admissions (SD 9.24 admissions) for ‘physical’emergency admissions.
Figure 2 shows the development of the average number of admissions per GP practice over time.
There was a marked increase in the average number of admissions for physical care over time with the
number of emergency admissions nearly doubling over the course of the 5-year period. This is in line with
national trends which between 2001 and 2011 saw an increase in the number of emergency admissions
per year for ambulatory care sensitive conditions of 40% and for all other conditions of 34%.61 In contrast,
the average number of admissions with a main diagnosis of SMI or bipolar disorder remained relatively
stable over time.
12
10
8
6
4
2
2006 2007 2008
Admissions per GP practice
2009 2010
HES
y
ear
SMI
Bipolar
Physical (emergency)
Physical (elective)
FIGURE 2 Average number of admissions per GP practice.
TABLE 5 General practitioner-level analyses: descriptive statistics for the samples
Admission
type
Number of
GP practices
Number of GP
practice-years
Total
number of
admissions
Annual admissions per GP practice
Mean SD
50th
percentile
25th
percentile
75th
percentile
SMI 8223 38,774 136,507 3.521 3.919 2 1 5
SMI with
R69
8223 38,774 161,858 4.174 4.405 3 1 6
Physical
elective
8223 38,774 128,382 3.311 4.628 2 1 5
Physical
emergency
8223 38,774 343,486 8.859 9.244 6 2 12
Bipolar 8042 37,037 41,372 1.117 1.606 1 0 2
EMPIRICAL ANALYSIS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
14

Table 6 presents the achievement and exception rates for the four QOF SMI indicators. On average, GP
practices report to have a comprehensive care plan in place for 76% of their patients with a SMI (MH6),
and have reviewed 81% of their patients with a SMI during the last 15 months (MH9). However, 12% of
patients were excluded from these SMI QOF indicators and practices did not achieve the indicators for the
remaining 12% (MH6) and 7% (MH9) of eligible individuals. The achievement rates were higher (95% and
83%) (exception and failure rates were lower –3% and 9%, respectively) for the QOF indicators MH4 and
MH5, which are specific to bipolar patients. Figure 3 displays the empirical distribution of achievement
scores for each of the four indicators across all practices in 2010/11. The distribution is approximately
representative for the other 4 years. Table 7 presents descriptive statistics for all dependent and
independent variables in our regression model.
Main regression results
Table 8 presents the calculated IRRs and the 95% CIs for the estimated panel data models. Achievement
scores are unadjusted, that is, all patients who have been flagged as unsuitable for this indicator are
included in the denominator (see Chapter 2, Exception reporting in the Quality and Outcomes Framework).
The first sets of results (columns 2–4) are derived from estimations in which both indicators were included
simultaneously (joint modelling). The second and third sets of results (columns 5–7 and 8–10) show the
results for models in which only one indicator was included. The first set of results (joint modelling)
comprises the base case, as the statistical approach explicitly accounts for correlation between the two
measures of QOF achievement. Because the focus of our study is on the association between QOF
achievement and admissions, we refrain from reporting the effects of the various control variables that
were included in all analyses. The interested reader is referred to Table 29 in Appendix 2, Further results.
Full results and fit statistics are available on request from the authors.
Several important findings emerge from these results. First, the association between QOF achievement and
admission is generally positive (IRR >1), implying that better QOF performance is associated with more
admissions, not fewer. The IRR for MH9 (1.210) suggests that a change in QOF achievement of 10% is
associated with an increase in the practice SMI admission rate of 1.9% (95% CI 1.0% to 2.9%). The strength
of the effect varies across indicators and admission types, but may have important clinical and economic
implications. Second, we find statistically significant associations between QOF achievement on MH9 and
both SMI admissions and physical emergency admissions. Results are not significant for elective admissions,
although these are still all positive. In contrast, the effect of MH6 on admissions is only statistically significant
(i.e. the IRR is different from zero) for physical (emergency) admissions when modelled jointly. Of the two
lithium indicators, results are statistically significant for MH4 when modelled jointly. Third, the estimated IRRs
are generally larger and more often statistically significant when the association between admissions and only
TABLE 6 General practitioner-level analyses: QOF achievement and exception rates
QOF indicator nMean SD Median 25th percentile 75th percentile
SMI
MH6 achievement 38,774 0.76 0.17 0.79 0.66 0.88
MH6 exception rate 38,774 0.12 0.11 0.09 0.04 0.17
MH9 achievement 38,774 0.81 0.13 0.83 0.75 0.90
MH9 exception rate 38,774 0.12 0.11 0.10 0.04 0.18
Bipolar
MH4 achievement 37,190 0.95 0.12 1.00 0.93 1.00
MH4 exception rate 37,190 0.03 0.09 0.00 0.00 0.00
MH5 achievement 37,190 0.83 0.21 0.89 0.75 1.00
MH5 exception rate 37,190 0.09 0.15 0.00 0.00 0.14
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
15

80
60
40
Per cent
20
0
0.0 0.2 0.4 0.6 0.8 1.0
Achievement
(a)
40
30
20
Per cent
10
0
(b)
0.0 0.2 0.4 0.6 0.8 1.0
Achievement
FIGURE 3 Distribution of QOF achievement rates across practices in 2010/11. (a) MH4 –serum creatinine and
thyroid-stimulating hormone check <15 months, 1 point; (b) MH5 –lithium within range, <6 months, 2 points;
(c) MH6 –comprehensive care plan documented, 6 points; and (d) MH9 –percentage reviewed <15 months,
23 points. (continued )
EMPIRICAL ANALYSIS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
16

(c)
10
8
6
4
2
0
0.0
Achievement
Per cent
0.2 0.4 0.6 0.8 1.0
15
10
5
0
Per cent
0.0 0.2 0.4 0.6 0.8 1.0
Achievement
(d)
FIGURE 3 Distribution of QOF achievement rates across practices in 2010/11. (a) MH4 –serum creatinine and
thyroid-stimulating hormone check <15 months, 1 point; (b) MH5 –lithium within range, <6 months, 2 points;
(c) MH6 –comprehensive care plan documented, 6 points; and (d) MH9 –percentage reviewed <15 months,
23 points.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
17

TABLE 7 General practitioner-level analyses: descriptive statistics for the variables
Variable description Source n
Mean
(or % where
indicated) SD Median Min. Max.
Number of SMI admissions within a year HES 38,774 3.52 3.92 2.00 0.00 55.00
Proportion of GP practices reimbursed
under PMS
GMS 38,774 43% –– – –
Proportion of male GPs in GP practice GMS 38,774 61% –– – –
Percentage of foreign GPs in GP practice GMS 38,774 33% –– – –
Mean age of GPs in GP practice (years) GMS 38,774 48.05 7.65 46.33 28.00 76.00
Practice list size ADS 38,774 6707 4008 5908 1040 40,082
Patient population: average age (years) ADS 38,774 38.91 4.15 39.33 21.56 56.43
Proportion male patients ADS 38,774 50% _ _ _ _
Proportion claiming incapacity benefit for
mental health, practice catchment area
ONS 38,774 2% _ _ _ _
Proportion providing informal care, practice
catchment area
ONS 38,774 10% _ _ _ _
NHS psychiatric residents per 1000 population,
practice catchment area
ONS 38,774 0.19 1.12 0.00 0.00 63.57
Proportion of non-white ethnicity, practice
catchment area
ONS 38,774 11% _ _ _ _
Proportion living in urban setting, practice
catchment area
ONS 38,774 82% _ _ _ _
Proportion of practice patients able to access
care within 48 hours
GP survey 38,774 84% _ _ _ _
Distance (in miles) to closest acute hospital HES 38,774 4.74 4.91 2.95 0.00 59.44
Distance (in miles) to closest mental
health hospital
HES 38,774 10.55 8.27 8.21 0.00 74.05
Mean number of admissions between
April 2004 and March 2006
HES 38,774 4.37 4.29 3.00 0.00 63.00
Max., maximum; min., minimum; ONS, Office for National Statistics.
EMPIRICAL ANALYSIS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
18

one QOF indicator is modelled instead of both QOF indicators. Both MH6 and MH9, as well as MH4 and
MH5, tend to be highly correlated. Failing to account for this correlation may therefore lead to false
conclusions. The cross-sectional models do not result in qualitatively different findings and are therefore not
reported here (see Appendix 2,Tables 27 and 28).
Sensitivity analyses
We perform a range of additional analyses to test the sensitivity of our findings to different modelling
assumptions or inclusion criteria.
Percentage of valid exception reporting
Practices vary both in the number and proportion of SMI patients they exception report (Table 6). Our base
case makes the assumption that all patients who were exception reported could have been ‘treated’,
that is they could have received an annual review and/or could have had a care plan developed. This
assumption means that achievement scores are lower than they would be if exception-reported individuals
are counted in the denominator) (see Chapter 2,Exception reporting in the Quality and Outcomes
Framework). However, we know that some people are exception reported because they decline to attend
the GP practice surgery (‘informed dissent’), and these exceptions are valid or ‘correct’–individuals are
free to choose whether or not they attend the practice for review and GPs should not be incentivised
to use inappropriate pressure to encourage attendance. The absence of individual-level data on QOF
achievement and exceptions means that we do not know what percentage of the exceptions is valid.
To test the effect of exception reporting on our results, we therefore ran a series of regressions in which
the percentage of exceptions deemed valid ranged from 0 (as in the base case) to 100, with the variable
increasing by 10% in each regression.
In all analyses, as the percentage of exceptions considered to be ‘valid’increases the magnitude of the
effect on the number of admissions decreases. For example, in the analysis on SMI admissions that
includes both indicators (‘estimated jointly’), the results that are significant in the base case (0% exceptions
assumed valid) become statistically insignificant when all (100%) exceptions are considered to be ‘valid’
(Table 8 and Figure 4). Figures 5–7show data for other admission types. This is also the case when
indicators are modelled individually, aside from in one case: admissions for physical emergency care remain
TABLE 8 General practitioner-level analysis 1: regression results (achievement includes exception-reported patients)
Indicator and admission type
Joint modelling MH6/4 only MH9/5 only
IRR 95% CI IRR 95% CI IRR 95% CI
Admissions for SMI
MH6 1.020 0.944 to 1.102 1.113 1.039 to 1.192 ––
MH9 1.210 1.104 to 1.327 –– 1.226 1.129 to 1.330
Admissions for physical (elective) care
MH6 1.135 0.979 to 1.315 1.220 1.084 to 1.374 ––
MH9 1.179 0.969 to 1.435 –– 1.270 1.082 to 1.491
Admissions for physical (emergency) care
MH6 1.180 1.087 to 1.281 1.269 1.183 to 1.362 ––
MH9 1.189 1.084 to 1.304 –– 1.303 1.203 to 1.411
Admissions for bipolar disorder a
MH4 1.171 1.018 to 1.347 1.226 1.079 to 1.393 ––
MH5 1.089 0.994 to 1.194 –– 1.124 1.033 to 1.222
a These models did not converge when including PCT fixed effects and we therefore dropped PCT fixed effects.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
19

Exceptions assumed valid (%)
100
80
60
40
200
0.9
1.0
IRR
1.1
1.2
1.3
(a)
Exceptions assumed valid (%)
100
80
60
40
200
IRR
1.1
1.0
0.9
1.2
1.3
(b)
FIGURE 4 Change in IRR for the number of annual SMI admissions per practice when the percentage of ‘valid’
exceptions ranges from 0% to 100%. (a) MH6; and (b) MH9.
EMPIRICAL ANALYSIS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
20

(a)
1.3
1.2
1.1
IRR
1.0
0.9
020 40 60 80 100
Exceptions assumed valid (%)
1.3
(b)
1.2
1.1
IRR
1.0
0.9
020 40 60 80 100
Exceptions assumed valid (%)
FIGURE 5 Change in IRR for the number of annual physical (emergency) admissions per practice when the
percentage of ‘valid’exceptions ranges from 0% to 100%. (a) MH6; and (b) MH9.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
21

1.6
(a)
1.4
1.2
IRR
1.0
0.8
0 20 40 60 80 100
Exceptions assumed valid (%)
1.6
(b)
1.4
1.2
IRR
1.0
0.8
020
40 60 80 100
Exceptions assumed valid (%)
FIGURE 6 Change in IRR for the number of annual physical (elective) admissions per practice when the percentage
of ‘valid’exceptions ranges from 0% to 100%. (a) MH6; and (b) MH9.
EMPIRICAL ANALYSIS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
22

1.4
1.3
1.2
IRR
1.1
1.0
0.9
020 40 60 80 100
Exceptions assumed valid (%)
(a)
1.4
1.3
1.2
IRR
1.1
1.0
0.9
020 40 60 80 100
Exceptions assumed valid (%)
(b)
FIGURE 7 Change in IRR for the number of annual bipolar admissions per practice when the percentage of ‘valid’
exceptions ranges from 0% to 100%. (a) MH4; and (b) MH5.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
23

significant and positively associated with practice achievement on both MH6 (the care plan indicator)
and MH9 (annual review indicator), regardless of how we treat exceptions.
This approach enables us to identify the approximate point at which the QOF ceases to have a statistically
significant impact on admissions (i.e. the ‘switching point’). For the jointly estimated equations, the
switching point range varies between 70% and 80% (e.g. for the effect of MH9 on SMI admissions) to
between 90% and 100% [for the effect of MH6 on physical (emergency) admissions]. Table 9 summarises
the switching points for each base case analysis. By combining these estimates with information on the
SMI practice register size and exception rates, one can derive an approximate estimate of the maximum
number of patients allowed to be ‘incorrectly’exception reported for the result to hold. For example, given
the median SMI practice register of 40 patients, the median MH9 exception rate of 0.12 (see Table 6),
and a switching point of 80%, we calculate that approximately 0.96 patients per practice and year
[=40 × 0.12 × (1 –0.8)] need to be ‘incorrectly’exception reported for the positive effect of MH9 on the
annual number of SMI admissions per GP practice to be statistically significant.
Admitted at least once
Some patients with SMI are admitted repeatedly within a short time period, a behaviour known as the
‘revolving door’phenomenon.62 The observed admission pattern may be genuinely related to the quality of
primary care, but may also be the result of the hospitals’discharge management, the patients’personal
circumstances, or the availability of alternative care resources in the patient’s locality [e.g. access to CRHT
teams or Community Mental Health Teams (CMHTs) or community nurses, availability of an informal carer,
access to appropriate housing]. Owing to the high frequency of admissions, these individual patients have
the potential to distort the admission rates observed at the GP practice level. In order to test whether or
not our previous findings reflect this phenomenon, we conduct a sensitivity analysis on the number of
patients being admitted at least once within a financial year. Hence, the dependent variable now reflects
the number of patients who have required secondary care in a year, rather than the number of admissions
in a year. Results are reported in Table 10 (assuming 0% exceptions are ‘valid’) and Table 11 (100%
exceptions are ‘valid’).
TABLE 9 General practitioner-level analysis 1: approximate switching point –percentage valid exception reporting
Indicator and admission type Joint modelling MH6/4 only MH9/5 only
Admissions for SMI
MH6 NA –always insignificant 40–50% –
MH9 70–80% –70–80%
Admissions for physical (elective) care
MH6 NA –always insignificant 80–90% –
MH9 NA –always insignificant –70–80%
Admissions for physical (emergency) care
MH6 90–100% NA –always significant –
MH9 80–90% –NA –always significant
Admissions for bipolar disorder
MH4 70–80% 90–100% –
MH5 NA –always insignificant –40–50%
NA, not applicable.
EMPIRICAL ANALYSIS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
24

TABLE 11 General practitioner-level analysis 2: regression results (achievement excludes
exception-reported patients)
Indicator and admission type
Joint modelling MH6/4 only MH9/5 only
IRR 95% CI IRR 95% CI IRR 95% CI
Admissions for SMI
MH6 0.980 0.921 to 1.043 0.984 0.926 to 1.047 ––
MH9 1.015 0.908 to 1.135 –– 1.001 0.899 to 1.115
Admissions for physical (elective) care
MH6 0.991 0.921 to 1.066 1.023 0.957 to 1.094 ––
MH9 1.125 1.005 to 1.259 –– 1.118 1.009 to 1.240
Admissions for physical (emergency) care
MH6 1.019 0.967 to 1.075 1.031 0.979 to 1.086 ––
MH9 1.040 0.966 to 1.120 –– 1.053 0.978 to 1.133
Admissions for bipolar disorder
MH4 1.110 0.946 to 1.303 1.136 0.969 to 1.332 ––
MH5 1.043 0.946 to 1.150 –– 1.060 0.962 to 1.167
TABLE 10 General practitioner-level analysis 2: regression results (achievement includes
exception-reported patients)
Indicator and admission type
Joint modelling MH6/4 only MH9/5 only
IRR 95% CI IRR 95% CI IRR 95% CI
Admissions for a SMI
MH6 1.014 0.952 to 1.079 1.127 1.064 to 1.194 ––
MH9 1.253 1.156 to 1.359 –– 1.265 1.175 to 1.362
Admissions for physical (elective) care
MH6 1.060 0.981 to 1.146 1.181 1.100 to 1.269 ––
MH9 1.260 1.163 to 1.366 –– 1.309 1.214 to 1.411
Admissions for physical (emergency) care
MH6 1.091 1.030 to 1.155 1.190 1.131 to 1.252 ––
MH9 1.212 1.139 to 1.290 –– 1.280 1.211 to 1.352
Admissions for bipolar disorder
MH4 1.107 0.976 to 1.256 1.170 1.040 to 1.315 ––
MH5 1.102 1.009 to 1.204 –– 1.126 1.037 to 1.222
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
25

Findings from the analysis of people admitted at least once a year were broadly similar to results from
base case analyses (number of admissions per year). With one exception, the effects were significant and
positive in all analyses in which indicators were modelled individually; in these cases, an assumption that
up to 60% of exceptions were valid was sufficient for the relationship between QOF and admissions to be
statistically significant. For the jointly estimated equations, the switching point for the percentage of valid
exceptions lay between 20% and 30% for the effect of MH5 on bipolar admissions, but was between
90% to 100% for the effect of MH9 on physical (emergency) admissions. Table 12 summarises the
approximate ‘switching point’for these sensitivity analyses.
Patients with unspecific main diagnosis
Inspection of HES data at local area level revealed that SMI admissions in some regions were far below
levels that would be expected given the demographic characteristics of the area. In some years, some
mental health hospitals had coded over 90% of their cases with a primary diagnosis ICD-10 code R69
(‘Unknown and unspecified causes of morbidity’).63 The use of R69 varied across providers and across
years, but was neither unusual nor confined to a small number of providers (Figure 8).
We do not know how many of these ‘R69’admissions were for individuals with SMI, but it is very unlikely
that none of them were. Therefore, we undertook a further sensitivity analysis to test the relationship
between QOF achievement scores and a revised measure of SMI admission that included records with a
diagnostic code from our base case analyses (ICD-10 codes F20–F31), and records with a primary diagnosis
of R69. The latter code was used if, and only if, the treatment specialty for the admission was one of the
following: adult mental illness; forensic psychiatry; psychotherapy; or old age psychiatry (i.e. excluding
learning disability, child and adolescent psychiatry and other types of service such as those for eating
disorders). The HES field is ‘tretspef’and defines the specialty in which the consultant was working during
the period of care. We investigated the effect on both of our response variables (the number of admissions
and the number of individuals admitted at least once) and varied the percentage of exceptions assumed to
be valid, as previously (see Chapter 3, Percentage of valid exception reporting).
TABLE 12 General practitioner-level analysis 2: switching point –percentage valid exception reporting
Indicator and admission type Joint modelling MH6/4 only MH9/5 only
Admissions for SMI
MH6 NA –always insignificant 60–70% –
MH9 80–90% –80–90%
Admissions for physical (elective) care
MH6 NA –always insignificant 80–90% –
MH9 NA –always significant –NA –always significant
Admissions for physical (emergency) care
MH6 60–70% 90–100% –
MH9 90–100% –90–100%
Admissions for bipolar disorder
MH4 NA –always insignificant 60–70% –
MH5 20–30% –60–70%
NA, not applicable.
EMPIRICAL ANALYSIS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
26

(a)
Frequency
6
4
2
0
0.0 0.2 0.4 0.6 0.8 1.0
Provider admissions (%)
(b) 5
4
3
2
1
0
0.0
Frequency
Provider admissions (%)
0.2 0.4 0.6 0.8 1.0
FIGURE 8 Use of R69 as primary diagnoses for psychiatric admissions. Percentage of provider admissions for SMI
patients with primary diagnosis of R69. (a) 2006; (b) 2007; (c) 2008; (d) 2009; and (e) 2010. (continued )
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
27

8
6
Frequency
4
2
0
0.0 0.2 0.8 1.0
0.4 0.6
Provider admissions (%)
(c)
(d)
Frequency
8
6
4
2
0
0.0 0.2 0.4 0.6 0.8 1.0
Provider admissions (%)
FIGURE 8 Use of R69 as primary diagnoses for psychiatric admissions. Percentage of provider admissions for SMI
patients with primary diagnosis of R69. (a) 2006; (b) 2007; (c) 2008; (d) 2009; and (e) 2010. (continued )
EMPIRICAL ANALYSIS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
28

Findings are presented in Table 13 (assuming 0% of exceptions are ‘valid’) and Table 14 (assuming 100%
of exceptions are ‘valid’). The effect of varying the percentage of valid exceptions did not alter the
statistical significance of results from the base case analysis. Findings were also robust to the way that SMI
admissions were defined, with the effect of a 10% change in QOF performance on practice SMI admission
rates approximately 0.1 percentage point lower when R69 codes were included in the definition compared
with the base case analysis that excluded R69 codes. In the jointly estimated model, the ‘switching point’
at which MH9 ceased to have a statistically significant impact on admissions lay between 80% and 90%
(Table 15). Therefore, provided the percentage of invalid exceptions is no more than 10–20%, the positive
association between practice achievement on the QOF review indicator, MH9, and the number of annual
admissions is statistically significant.
TABLE 13 General practitioner-level analysis 3: regression results (achievement includes
exception-reported patients)
Indicator and admission type
Joint modelling MH6/4 only MH9/5 only
IRR 95% CI IRR 95% CI IRR 95% CI
Frequency
MH6 1.019 0.949 to 1.095 1.116 1.049 to 1.188 ––
MH9 1.220 1.118 to 1.332 –– 1.235 1.144 to 1.334
Admitted at least once
MH6 1.019 0.963 to 1.080 1.128 1.071 to 1.188 ––
MH9 1.242 1.151 to 1.341 –– 1.258 1.175 to 1.347
10
8
(e)
6
4
2
0
Frequency
0.0 0.2 0.4 0.6 0.8 1.0
Provider admission (%)
FIGURE 8 Use of R69 as primary diagnoses for psychiatric admissions. Percentage of provider admissions for SMI
patients with primary diagnosis of R69. (a) 2006; (b) 2007; (c) 2008; (d) 2009; and (e) 2010.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
29

Does better primary care reduce inpatient length of stay?
Overview
This section focuses on our second research question, namely the effect of better-quality primary care on
LOS in hospital. Better-quality primary care may also reduce LOS if patients are effectively cared for outside
hospital. Our null hypothesis is of no association between QOF and LOS and our alternative hypothesis is
that better management in primary care could shorten LOS in hospital for people with SMI.
We use the same data set as for the previous analyses on admissions, but instead of aggregating the
admissions data to the practice level we keep the original patient-level structure. The analysis is limited to
admissions with a non-zero LOS and practices with no SMI admissions are dropped from the data set.
As before, QOF data are available only at practice level.
The data set includes 98,993 admissions for patients registered with 7912 practices.
Data
The analysis is based on the same data set as derived for the previous analysis (see Chapter 3,Data) but
differs in several respects. First, we retain the original patient-level structure of the data and do not
aggregate admissions to practice level. Second, we focus only on patients admitted with a primary
diagnosis of a SMI, that is, admissions for physical care are excluded from the analysis. Third, we exclude
patients who remained in hospital for more than 180 days (approximately 6 months) or were admitted
after the 2 October 2010, that is, 180 days before the end of our study period. The latter exclusion
criterion is applied to ensure that patients spent sufficient time outside the hospital to be able to be seen
by their GP and interact with the primary care sector and thus potentially be subject to QOF activities.
TABLE 15 General practitioner-level analysis 3: switching point –percentage valid exception reporting
Indicator and admission type Joint modelling MH6 only MH9 only
Frequency
MH6 NA –always insignificant 50–60% –
MH9 80–90% –80–90%
Admitted at least once
MH6 NA –always insignificant 60–70% –
MH9 80–90% –80–90%
NA, not applicable.
TABLE 14 General practitioner-level analysis 3: regression results (achievement excludes
exception-reported patients)
Indicator and admission type
Joint modelling MH6/4 only MH9/5 only
IRR 95% CI IRR 95% CI IRR 95% CI
Frequency
MH6 0.980 0.912 to 1.053 1.000 0.936 to 1.069 ––
MH9 1.068 0.952 to 1.199 –– 1.054 0.947 to 1.173
Admitted at least once
MH6 0.978 0.922 to 1.038 0.991 0.937 to 1.049 ––
MH9 1.046 0.940 to 1.164 –– 1.030 0.931 to 1.139
EMPIRICAL ANALYSIS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
30

Fourth, we exclude patients with invalid discharge information, that is, where it is possible that the patient
is still in hospital or it is unclear when the patient was discharged. Last, practices with no SMI admissions
are no longer included in the data set.
The HES data warehouse contains detailed information on patient characteristics and characteristics of the
hospital stay. We derive a range of variables that describe the medical and socioeconomic characteristics of
the patient. These are age (categorised in 10-year age bands), gender, ethnicity (coded as white, Asian,
black, mixed or other/unknown), proportion of people in a small area (LSOA) claiming mental health
benefits (coded as quintiles), primary diagnosis (see Table 4), number of non-duplicate comorbidity codes
recorded throughout the hospital stay, whether or not the patient had a carer, whether or not the patient
was detained under the Mental Health Act 198349 during the inpatient stay, and whether or not the
patient has a psychiatric history, that is, was previously admitted under the care of a psychiatric consultant.
We also derive two variables that describe the hospital stay. These are discharge type (coded as discharge
on clinical advice, self-discharge or died in hospital) and year of admission.
Analytical model
The aim of this empirical analysis is to relate the length of inpatient stay for people admitted to hospital
with a main diagnosis of a SMI to the quality performance of their GP practices. In order to isolate the
effect, the analysis also controls for other patient factors that may drive LOS but are unrelated to
the quality of care provided in primary care, as well as the general efficiency of the hospital provider.
Length of stay is a continuous, non-negative variable that is non-normally distributed with large kurtosis
and substantial skew (Figure 9). We used a mixed-effects linear regression model to analyse these data.
In order to reduce the skewness and make the assumption of normality underlying the linear regression
model more tenable, we transformed LOS using the logarithmic transformation; an approach frequently
used in the analysis of cost and LOS data.64 Some patients were admitted and discharged within the same
day, that is, had a recorded LOS of no days. For these observations the logarithm of LOS would not be
defined and the observation would thus be excluded from the analysis even though they still consumed
hospital resources. Following this argument, we measured LOS as the number of days spent in hospital,
rather than the number of nights, thereby avoiding the problem of undefined logarithms.
10
8
6
Per cent
4
2
0
050
100 150 200
Inpatient days
FIGURE 9 Length-of-stay analysis: histogram of patient LOS.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
31

Let LOS
ijk
be the LOS for admission i=1, . . ., I, where the patient is registered with practice j=1,. . ., Jand
treated in hospital k=1, . . . , K. We acknowledge the potential clustering of admissions within patients
but do not specifically model it. However, most patients are admitted only once within each financial year.
We also assume, at least in notation, that GP practices are clustered in hospitals, that is, submit patients
exclusively to one hospital. However, we model hospital effects using dummy variables (‘hospital fixed
effects’) so that this simplification in notation is not consequential. The empirical model is specified
as follows:
LOSijk =αþQOF′
jkγþX′
ijkβþπkþμjk þεijk,(6)
where X
ijk
is a set of patient characteristics and QOF
jk
measures the QOF achievement of practice j. The
parameter αdenotes the common intercept, whereas the parameters ε
ijk
and μ
jk
are random error terms at
the admission and GP practice level and are assumed to be independently distributed as N∼ð0, σ2
c) with
c∈[i,j]. The parameters π
k
are modelled as fixed effects and capture variation in LOS between providers
owing to variation in efficiency and clinical practice. All models include year effects to account for common
temporal trends.
The model is estimated via maximum likelihood using xtmixed in Stata 12.0. The reported standard errors
are clustered at the GP practice level. Coefficient estimates can be interpreted as semi-elasticities, that is, a
one-unit increase in a continuous variable is expected to increase/decrease LOS by a percentage equal to
the estimated coefficient. Given the large number of observations in this analysis, the likelihood of finding
a statistically significant but small and potentially clinically unimportant effect is large. We therefore
caution the reader to consider the absolute size of the effect. Our discussions of the results are based at
the highest level of significance (p<0.001).
We ran several models, all of which tested the QOF indicators individually and jointly and all of which
included year dummies. In the base case, we calculated achievement to include exception-reported cases,
included a set of patient-level covariates and hospital fixed effects. We also ran a model without either
patient-level covariates or hospital fixed effects, and a model with patient-level covariates and without
hospital fixed effects. We ran the same set of models for QOF achievement measures calculated without
exception-reported cases.
Results
Descriptive statistics
Table 16 presents the descriptive statistics for the LOS analysis. All variables are at an individual level, except
for QOF achievement. The data set includes 98,993 individuals with a primary diagnosis of SMI and whose
inpatient stay is 180 days (6 months) or less. The mean LOS was 42.3 days (SD 39.5 days) and the median LOS
was30days(IQR12–60 days). As shown from the descriptive statistics in Table 16, the mean age of these
individuals is 45.22 years (SD 16.30 years) and 52% are male. Most are of white ethnicity (76%), with 2% of
mixed ethnicity, 7% Asian and 9% black. Just less than one in five (17%) of these hospitalised individuals is
formally detained under the Mental Health Act 1983.49 On average, practices perform well on the QOF
indicators for both the care plan indicator MH6 (mean 74%, SD 16%) and the annual review indicator MH9
(mean 80%, SD 12%). For both MH6 and MH9, around 12% of people with SMI are exception reported.
Regression results
Table 17 presents the regression results of the LOS analysis. The reference categories in this analysis describe a
patient with a primary diagnosis of schizophrenia, aged 24 years or younger, female, white, not detained, no
carer, discharged on clinical advice, no previous psychiatric history, no comorbidities, and residing in an area
with a low (first quintile) proportion of residents claiming mental health benefits. QOF achievement was
calculated so that all exceptions were deemed invalid, that is, the number of exception reported patients was
included in the denominator (see Chapter 2,Exception reporting in the Quality and Outcomes Framework).
EMPIRICAL ANALYSIS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
32

TABLE 16 Length-of-stay analysis: descriptive statistics
Variable description n
Mean
(or % where
indicated) SD Median Min. Max.
Length of inpatient stay –excluding admissions
>180 days (days)
98,993 42.29 39.50 30.00 1.00 180.00
Achievement on MH6 QOF indicator 98,993 0.74 0.16 0.77 0.00 1.00
Achievement on MH9 QOF indicator 98,993 0.80 0.12 0.82 0.00 1.00
Proportion with schizophrenia 98,993 41.3% –– ––
Proportion with schizotypal disorder 98,993 0.2% –– ––
Proportion with persistent delusional disorders 98,993 4.9% –– ––
Proportion with acute and transient psychotic disorders 98,993 7.8% –– ––
Proportion with induced delusional disorder 98,993 0.1% –– ––
Proportion with schizoaffective disorders 98,993 8.5% –– ––
Proportion with other non-organic psychotic disorders 98,993 0.4% –– ––
Proportion with unspecified non-organic psychosis 98,993 4.0% –– ––
Proportion with manic episode 98,993 3.3% –– ––
Proportion with bipolar affective disorder 98,993 29.4% –– ––
Patient age (years) 98,993 45.22 16.30 43.00 18.00 104.00
Proportion male patients 98,993 52.0% –– ––
Proportion detained patients 98,993 17.5% –– ––
Proportion white ethnicity 98,993 75.8% –– ––
Proportion mixed ethnicity 98,993 1.9% –– ––
Proportion Asian ethnicity 98,993 7.4% –– ––
Proportion black ethnicity 98,993 9.5% –– ––
Proportion other or undefined ethnicity 98,993 5.4% –– ––
Proportion patients who have a carer 98,993 6.3% –– ––
Discharge type: discharged on clinical advice 98,993 97.1% –– ––
Discharge type: self-discharged against clinical advice 98,993 2.4% –– ––
Discharge type: died in hospital 98,993 0.5% –– ––
Proportion claiming incapacity benefit for mental
health, practice catchment area
98,993 2.26 1.62 1.87 0.00 20.41
Proportion patients with previous psychiatric admission 98,993 42.2% –– ––
Number of non-duplicate comorbidities recorded for
the patient
98,993 0.98 3.10 0.00 0.00 288.00
Max., maximum; min., minimum.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
33

TABLE 17 Length-of-stay analysis: regression results
Variable description
Joint modelling MH6 only MH9 only
Beta SE p-value Beta SE p-value Beta SE p-value
Achievement on MH6
QOF indicator
–0.055 0.037 ––0.034 0.029 ––––
Achievement on MH9
QOF indicator
0.042 0.044 ––––0.003 0.034 –
Financial year 2007/8 –0.022 0.013 ––0.024 0.013 ––0.027 0.013 <0.05
Financial year 2008/9 0.021 0.014 –0.019 0.013 –0.015 0.013 –
Financial year 2009/10 –0.019 0.015 ––0.022 0.014 ––0.028 0.013 <0.05
Financial year 2010/11 –0.060 0.016 <0.001 –0.063 0.015 <0.001 –0.070 0.014 <0.001
Schizotypal disorder –0.062 0.064 ––0.062 0.064 ––0.062 0.064 –
Persistent delusional disorders –0.109 0.017 <0.001 –0.109 0.017 <0.001 –0.109 0.017 <0.001
Acute and transient
psychotic disorders
–0.308 0.014 <0.001 –0.308 0.014 <0.001 –0.308 0.014 <0.001
Induced delusional disorder –0.229 0.123 ––0.229 0.123 ––0.229 0.123 –
Schizoaffective disorders 0.026 0.015 –0.026 0.015 –0.026 0.015 –
Other non-organic
psychotic disorders
–0.314 0.055 <0.001 –0.314 0.055 <0.001 –0.314 0.055 <0.001
Unspecified non-organic
psychosis
–0.193 0.021 <0.001 –0.193 0.021 <0.001 –0.193 0.021 <0.001
Manic episode –0.167 0.018 <0.001 –0.167 0.018 <0.001 –0.167 0.018 <0.001
Bipolar affective disorder –0.132 0.011 <0.001 –0.132 0.011 <0.001 –0.132 0.011 <0.001
Aged 25–34 years 0.074 0.027 <0.01 0.074 0.027 <0.01 0.074 0.027 <0.01
Aged 35–44 years 0.035 0.026 –0.035 0.026 –0.035 0.026 –
Aged 45–54 years 0.101 0.027 <0.001 0.101 0.027 <0.001 0.101 0.027 <0.001
Aged 55–64 years 0.261 0.027 <0.001 0.261 0.027 <0.001 0.261 0.027 <0.001
Aged 65–74 years 0.512 0.033 <0.001 0.512 0.033 <0.001 0.512 0.033 <0.001
Aged 75 years and over 0.773 0.031 <0.001 0.773 0.031 <0.001 0.773 0.031 <0.001
Male gender 0.093 0.027 <0.001 0.093 0.027 <0.001 0.093 0.027 <0.001
Patient was detained 0.465 0.012 <0.001 0.465 0.012 <0.001 0.465 0.012 <0.001
Ethnicity: mixed 0.086 0.033 <0.01 0.086 0.033 <0.01 0.086 0.033 <0.01
Ethnicity: Asian 0.089 0.018 <0.001 0.089 0.018 <0.001 0.089 0.018 <0.001
Ethnicity: black 0.179 0.018 <0.001 0.179 0.018 <0.001 0.178 0.018 <0.001
Ethnicity: other or undefined 0.035 0.020 –0.035 0.020 –0.035 0.020 –
Detained and ethnicity: mixed –0.089 0.083 ––0.089 0.083 ––0.089 0.083 –
Detained and ethnicity: Asian –0.066 0.031 <0.05 –0.066 0.031 <0.05 –0.065 0.031 <0.05
Detained and ethnicity: black –0.079 0.028 <0.01 –0.079 0.028 <0.01 –0.079 0.028 <0.01
Detained and ethnicity: other –0.040 0.036 ––0.040 0.036 ––0.040 0.036 –
Patient had a carer 0.074 0.023 <0.01 0.074 0.023 <0.01 0.074 0.023 <0.01
Discharge type: self–discharged –0.802 0.027 <0.001 –0.802 0.027 <0.001 –0.803 0.027 <0.001
EMPIRICAL ANALYSIS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
34

The quality of primary care, as measured by the QOF scores, has no significant effect on LOS in any of the
analyses. Longer LOS is associated with some primary diagnoses, older age and male gender. People with
a primary diagnosis of persistent delusional disorder (F22), acute and transient psychotic disorder (F23),
non-organic psychoses (F28 or F29), manic episode (F30) or bipolar affective disorder (F31) have
significantly shorter stays than people with schizophrenia (F20). Formal detention under the Mental Health
Act 198349 significantly increases LOS, and longer stay is associated with black and Asian ethnicity,
compared with white ethnicity. People of one of these ethnic groups who were also detained are likely
to stay even longer than those with just one of these characteristics, but the overall effect is smaller than
the sum of the two individual effects. People with an informal carer were likely to have longer inpatient
stays than those who did not have a carer, but people with a history of psychiatric admission tended to
have a shorter LOS. The former may be an indicator of severity, whereas the latter may reflect clinical
knowledge about the previous psychiatric history of the patient and a familiarity with the care
requirements of the patient. Unsurprisingly, patients who decided to leave the hospital on their own
responsibility, that is, self-discharged against clinical advice, had shorter stays than those who were
discharged following clinical advice. Deprivation scores relating to the patient’s residential area did not
explain LOS, but having a higher number of comorbidities did increase duration of stay in hospital.
Sensitivity analyses
Percentage of valid exception reporting
In previous analyses, we sought to identify the ‘switching point’at which the relationship between QOF
achievement and admissions became statistically insignificant. In the LOS analysis, we found that the way
in which achievement was measured, that is, whether or not exception reported cases were included in
the measures, had no influence on the statistical significance of the relationship between achievement and
LOS. Results remained insignificant for whether we assumed 0% of exceptions were ‘valid’or 100% of
exceptions were assumed ‘valid’. Therefore, there is no ‘switching point’to identify in this analysis. All
covariates were approximately equal to those reported in Table 17 and are therefore not reported here.
TABLE 17 Length-of-stay analysis: regression results (continued )
Variable description
Joint modelling MH6 only MH9 only
Beta SE p-value Beta SE p-value Beta SE p-value
Discharge type: died in
hospital
–0.048 0.053 ––0.048 0.053 ––0.048 0.053 –
Patient had previous
psychiatric admission
–0.050 0.009 <0.001 –0.050 0.009 <0.001 –0.050 0.009 <0.001
MH claimants –second
quintile
–0.007 0.012 ––0.007 0.013 ––0.007 0.012 –
MH claimants –third quintile 0.006 0.013 –0.006 0.013 –0.006 0.013 –
MH claimants –forth quintile 0.011 0.014 –0.011 0.014 –0.011 0.014 –
MH claimants –fifth quintile 0.022 0.015 –0.022 0.015 –0.022 0.015 –
Number of comorbidities 0.051 0.004 <0.001 0.051 0.004 <0.001 0.051 0.004 <0.001
Constant term 2.916 0.180 <0.001 2.937 0.178 <0.001 2.917 0.179 <0.001
GP-level variance 0.211 0.012 <0.001 0.211 0.012 <0.001 0.211 0.012 <0.001
Admission-level variance 1.033 0.004 <0.001 1.033 0.004 <0.001 1.033 0.004 <0.001
Number of observations 98,993 –– 98,993 –– 98,993 ––
MH, mental health; SE, standard error.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
35

Does better primary care reduce cost of care?
Overview
This section focuses on our third research question, namely whether or not better-quality primary care as
measured by specific mental health QOF indicators is associated with reduced resource use in terms of
lower secondary care expenditure for mental health services for people with SMI. If better-quality primary
care can reduce costly emergency hospital admissions it may have knock-on effects for NHS expenditure
and resource use. Our null hypothesis is one of no association between QOF achievement and secondary
care expenditure while our alternative hypothesis proposes that preventative care could reduce mental
health expenditure.
This set of analyses used the Mental Health Minimum Data Set (MHMDS) to investigate the relationship
between QOF performance and the total per-patient cost of care. The MHMDS was available for 2 years:
2006/7 and 2007/8. The base case included 981,373 records on 711,820 individuals, clustered within
8064 practices. As there were no diagnostic codes in the database, we were unable to identify individuals
with SMI. As far as possible we tried to match the LOS analysis in Chapter 3, Does better primary care
reduce inpatient length of stay?
Data
The MHMDS was available for 2 years: 2006/7 and 2007/8. The data set contained details of community
care, social care and hospital care received by people with mental illness in England, and so had a broader
scope than HES which covers only hospital care.
The data comprised a series of records for each ‘spell’of care received by an individual over the 2-year
period. The MHMDS spell differs from that in HES: it covers a period of care that may or may not include a
hospital admission. Unlike HES spells, MHMDS spells are not necessarily ‘closed’, or finished, before
another spell is initiated –even if the spell is with the same provider. An individual patient in MHMDS may
therefore have multiple spells running concurrently. A stylised example of a patient’s record in MHMDS is
shown in Figure 10.
Spells
20082007200620052004
Spell A Spell B Spell C Spell D overlaps previous spell C,
and is nested within spell B,
which is nested within spell A
MHMDS data 2006/7
MHMDS data 2007/8
Finished 2006/7 spell
Unfinished 2006/7 spell
Finished 2007/8 spell
Unfinished 2007/8 spell
Spell E and spell F have identical start/end dates but the
resource use recorded for each is unique. The total LOS is
the sum of the two; they are not duplicates.
If we apply a 180-day cut-off, we would drop the latter and
keep the former
Year
FIGURE 10 Stylised example of a patient record in MHMDS.
EMPIRICAL ANALYSIS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
36

Variables in the data set included demographic information (such as age, gender, ethnicity and marital
status). Although two-thirds of the cases recorded marital status, the categories of ‘married’and
‘separated’were concatenated. The project steering group agreed that this undermined the value of
the variable sufficiently to render it unusable. Variables included resource use data such as community
care (contacts with social workers, community psychiatric nurses, physiotherapists, psychotherapists,
psychiatrists and occupational therapists) and hospital care (outpatient visits, the number of days spent in
inpatient care, in medium-secure units and in psychiatric intensive care units). Medium-secure hospitals/
units are for people who are detained under mental health legislation and pose a serious danger to the
public.65 There were binary indicators for receipt of local authority services such as residential care, day
care, home help and sheltered work. MHMDS also records use of electroconvulsive therapy (ECT) which
is a treatment for severe depression. We use this variable to exclude patients from the data set, as the
treatment is sometimes used for patients with bipolar disorder and we do not examine the QOF indicators
for bipolar disorder (MH4/MH5). Aside from the variable for ECT, the MHMDS data sets contained no
diagnostic and procedure codes. There was no variable indicating whether or not death had occurred,
and there were no dates of admission and discharge to hospital.
The MHMDS also reported the code for the GP practice with which the individual was registered, the
electoral ward of the patient and several PCT codes. There was no LSOA code or statistical ward code.
Some MHMDS data were too poorly populated to be used in the regressions:
lThere were several variables relating to the outcome measure Health of the Nation Outcome Scale
(HoNOS) that assesses the health and social functioning of people with a SMI. However, data were
missing in over 90% of records.
lFewer than 5000 (0.12%) records in the data set had a non-zero value for attendance at NHS day-care
facilities: the majority attended for a single day, and the maximum use was 9 days (for two patients).
These numbers appeared implausibly low and were too unreliable to inform the analyses. This
judgement was supported by the NHS Reference costs for 2007/8, which recorded over 1.38 million
regular adult attendances at NHS Trusts Mental Health Day Care Facilities.66
lSimilarly, MHMDS data on acute home-based care and stays in NHS residential homes (community
bed-days) were insufficiently robust for inclusion in the regressions.
lThe CPA is a needs assessment for people with severe mental disorders, and involves development
of a care plan that is regularly reviewed. Therefore, we would expect almost everyone in the data set
to have had at least one CPA review, but 40% of individuals had no record of having had a
CPA assessment.
lThe type of community team caring for each individual (e.g. substance misuse team, early intervention
in psychosis team) is a potentially very useful indicator, but was missing from 82% of records.
Costing
The dependent variable was the total annual cost of care per patient. The MHMDS data sets did not
include costs (prices), but there were data on resource use (activity) for hospital (inpatient and outpatient)
care and community care provided by specialist mental health teams. There were no data on care provided
by GP practices (visits, prescriptions) and only binary variables for most social care services. We applied the
costing methodology used for the Resource Allocation for Mental health and Prescribing (RAMP) project.67
Owing to the way the MHMDS data were structured in spells, we could not use completed spells as the
unit of analysis as we did for HES. Instead, we estimated a total cost per year for each individual.
We used national NHS Reference costs66 to assign costs to mental health inpatient days, medium secure
days, psychiatric intensive days and outpatient attendances. We used a single price year (2007/8) to find
‘real’quality effects. For individuals of working age, inpatient costs were derived as weighted averages of
the mental health adult inpatient costs for intensive care, acute care and rehabilitation. For patients aged
65 years and over, we assigned the mental health inpatient cost for the ‘elderly’. The unit costs of medium
secure days and of psychiatric intensive days were taken from the NHS trusts’Mental Health Secure Units
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
37

sheet66 and separate adult and elderly outpatient visit unit costs were derived as weighted averages of
first/follow-up attendances for face-to-face and non-face-to-face contacts.
For community and social care, we used the unit cost values from the RAMP report, adjusted for price
inflation using the Personal Social Services Research Unit (PSSRU) Hospital and Community Health
Services indices.68
Owing to the absence of data on activity volume for local authority services, we could not attach costs to all of
the resource use variables in the MHMDS and, as these were the only services recorded for a subset of the
patients, this meant that our response variable (total annual cost) was missing for around 20% of patients in
the MHMDS and these patients could, therefore, not be included in the cost analyses. As these patients had
no record of hospital care, or care provided by members of the CMHT, they probably represent a group with
less severe illness. It is, therefore, likely that our analysis focuses on individuals with more severe mental illness,
but the lack of diagnostic information is a serious limitation of the analysis and introduces potential bias.
Data sets used to generate other covariates
The MHMDS included codes for the individual’s GP practice. We used practice codes to merge the
MHMDS with data on QOF performance and GP practice variables. The QOF data were sourced from the
Health and Social Care Information Centre (see Chapter 3, Quality and Outcomes Framework data set).
We calculated practice achievement scores as the number of patients who achieved the indicator as a
fraction of all patients who were reported eligible –including those who had been exception reported
(see Chapter 2,Exception reporting in the Quality and Outcomes Framework).
For the MHMDS analysis, we included two of the QOF indicators, MH6 and MH9, as there was no
diagnostic information in the MHMDS with which to identify people with bipolar disorder. As QOF
performance data were available only at practice level, we did not know whether or when individual
patients had received care under the QOF. We return to this limitation in our discussion section.
The MHMDS contained codes for the individual’s electoral ward, but the boundaries differ from those
of the statistical wards used by the Office for National Statistics (ONS). The match between electoral wards
in the MHMDS and the statistical wards used by the ONS was too poor to use ward-level characteristics.
Unlike HES, the MHMDS did not contain a small-area code (such as a LSOA code) at the patient level and
so practice-level variables were used as controls instead. We incorporated measures of area characteristics
at practice level using weighted average values based on the LSOAs in which registered practice patients
reside (see Chapter 3, Data sets used to generate other covariates). These included deprivation, rurality,
the number of NHS psychiatric residents per 1000 population and the prevalence of informal care.
Eligibility criteria used for the Mental Health Minimum Data Set analysis
As far as possible, we derived covariates that matched those of the LOS analysis and followed the same
inclusion criteria (e.g. practice list size of at least 1000). The criteria used to identify records eligible for
inclusion in the MHMDS analyses are:
lnon-zero cost in year
lno ECT
lvalid GP code
lvalid ethnicity code
laged 18 years and over
llearning disability was not the main treatment specialty
lLOS of no more than 180 days
lGP practice has at least 1000 registered patients
lGP practice has at least five eligible patients with a SMI
lGP practice has consistent records of numbers of eligible patients.
EMPIRICAL ANALYSIS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
38

The base case analysis included 981,373 observations, which represents around 39% of the total records
in the MHMDS. In the sensitivity analysis of people aged 18 to 64 years, the corresponding number was
685,154 (27% of the total).
Table 18 lists the reasons why records were excluded. The largest single reason for excluding a record was
that there were no data on receipt of social care (almost 26% missing). Of cases with non-missing data,
only 1% were recorded as having received social care. The other key reasons for excluding records were
that we could not assign a cost, the record either did not meet the eligibility criteria or the MHMDS GP
practice code did not match codes in the QOF data set.
Analytical model
In all analyses, the (log of) total cost per patient per year was the dependent (response) variable. We took
the natural logarithm because costs were positively skewed (i.e. characterised by a long right-hand tail).
This helped to normalise the response variable (Figure 11).
The analytic model mirrored the approach adopted in the individual-level LOS analysis (see Chapter 3,Does
better primary care reduce inpatient length of stay?). We merged the MHMDS data with QOF data and
other data on practice characteristics from around 8000 GP practices in England. We restricted the sample
of patients to those aged 18 years and over, used the LOS analysis categories for age and ethnicity, for
area measures such as deprivation and rurality, and used the same criteria to define eligible GP practices in
the LOS analysis. As there was no diagnostic information in the MHMDS, the two lithium QOF indicators
TABLE 18 Derivation of the regression samples
Reasons for record exclusion Loss (n) Total remaining (n) Percentage loss (of total)
Total records in the MHMDS –2,537,324 –
Excluded recordsa243,662 –9.6
Unmatched recordsb174,902 –6.9
Excluded/unmatched records 292,683 2,244,641 11.5
Zero cost 482,594 1,762,047 19.0
Practice list size <1000 or MH register <5 10,974 1,751,073 0.4
Invalid ethnicity code 3352 1,747,721 0.1
No social care datac652,690 1,095,031 25.7
PCT code missing 8644 1,086,387 0.3
Duplicate patient records 85,836 1,000,551 3.4
Missing at least one covariate 19,178 981,373 0.8
Patients aged over 65 years 296,219 685,154 11.7
Sample for base case analysis (% total) –981,373 (39%) –
Sample for sensitivity analysis (% total) –685,154 (27%) –
MH, mental health.
a Reasons for exclusion: age <18 years; annual LOS >180 days; learning disability as main speciality of treatment;
received ECT; invalid GP code; changed GP within year.
b Matching: MHMDS practice code did not match codes in practice data set (QOF, GMS, GP survey, ONS variables).
c Social care data: binary indicators for use of home help, social work, sheltered work, local authority day centre care,
local authority residential care.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
39

(MH4 and MH5) could not be tested against the subgroup of people with bipolar disorder. We therefore
tested only two QOF indicators, MH6 and MH9, in our analysis, individually and in combination (jointly)
(see Table 19). We used a trim point of 180 days for combined LOS over the year (i.e. as an inpatient in
any type of setting); the rationale for this was that a longer inpatient stay would reduce opportunities for
primary care to affect cost. However, we could not fully replicate the methodology for the LOS analysis:
the MHMDS did not include admission and discharge dates for hospital care, so individuals whose
inpatient stay began <180 days before the end of the study period could not be identified.
The MHMDS contains four variables indicating that an individual has been detained under the Mental
Health Act 198349 and these were used to derive a dummy variable for detention.
80
60
40
30
00
Per cent
(a)
Total annual cost per patient (£)
50,000 100,000 150,000
(b)
Total annual cost per patient (£)
050,000 100,000 150,000
80
60
40
30
0
Per cent
Per cent
(c)
4681012
Lo
g
total annual cost per patient
10
5
0
(d)
4681012
Lo
g
total annual cost per patient
Per cent
10
5
0
FIGURE 11 Mental Health Minimum Data Set analysis: histograms of total per-patient cost, before and after log
transformation. (a) 2006/7; (b) 2007/8; (c) 2006/7; and (d) 2007/8.
EMPIRICAL ANALYSIS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
40

We use xtmixed in Stata 12 (as used in the LOS analyses) to run mixed-effects continuous response models
that took account of the clustering of patients within practices and practices within PCTs. Around 39% of
individuals in the data set contributed data for both years, with 28% and 33% contributing 1 year’s
data for 2006/7 and 2007/8, respectively. We included year indicators to allow for temporal trends,
and controlled for individual- and practice-level factors. Table 19 lists the covariates used in the cost
regression analyses.
Around 30% of the individuals in the data are aged 65 years and over. There is no diagnostic information
in the MHMDS, and at least some of these patients may have dementia rather than a psychotic disorder.
We therefore conduct a sensitivity analysis that excludes people aged 65 years and over, as the prevalence
of dementia is lower in people aged between 18 years and 64 years.69
We ran four models in both the base case and sensitivity analyses, and tested the two QOF achievement
indicators individually and jointly. A summary of the models is provided in Table 20.
TABLE 19 Covariates used in the MHMDS analyses
Category Variables Sources
Individual-level characteristics:
‘need’variables
Age group, gender, ethnicity, formal detention MHMDS
Individual-level characteristics:
‘supply’variables
Inpatient, day care, social work, residential care, home help,
and sheltered work
MHMDS
GP-level characteristics QOF achievement indicators:
MH6 (care plan)
MH9 (annual review)
QMAS data
Percentage of practice patients able to access primary care
within 48 hours
GP survey
PMS practice (binary variable) GMS data
Mean age of GPs within a practice
% male GPs within a practice
% of non-UK qualified GPs
Size of GP practice (small, medium, large) ADS data set
Mean age of practice patients
% male practice patients
Weighted average deprivation score of practice population
(percentile categories), based on the number of people
claiming incapacity benefit for mental health
ONS data
Weighted average proportion of practice population providing
informal care
ADS data
Weighted average proportion of practice population
living in urban areas
Census data
Weighted average number of NHS psychiatric residents
per 1000 practice population
Other Year dummies MHMDS
QMAS, Quality Management and Analysis System.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
41

The equation for specifying the models is:
log costijkt =αþQOF′
jktγþX d
′
ijktβþX s′
ijktδþvkþujk þeijk, (7)
where log_cost
ij
is the log of annual cost of treatment for patient i=1, . . . , I, registered with practice
j=1, . . . , Jand based in PCT k=1,...,Kin year t=1,2; X_d
ijk
is a set of patient need characteristics,
X_s
ijk
is a set of individual supply characteristics and QOF
jkt
measures the QOF achievement of practice j.
The parameter αdenotes the random intercept and the parameters e
ijk
,u
jk
and v
k
are random effects at
the patient, GP practice and PCT levels, respectively; these are assumed to be independent of one another
and normally distributed with zero means and constant variances.
We ran multilevel mixed-effects linear regressions in Stata 12.0 (xtmixed) and fitted the models via
maximum likelihood. For characteristics that entered as dummy variables, their proportionate influence was
calculated as p =[exp(β)–1].70 Given the large number of observations in this analysis, the likelihood of
finding a statistically significant but small and potentially clinically unimportant effect is large. We therefore
caution the reader to consider the absolute size of the effect.
Results
When the 2 years of MHMDS data were merged, the data set contained over 2.5 million records.
After excluding ineligible records, the base case included 981,373 observations on 711,820 individuals in
8064 practices. The sensitivity analysis of individuals aged 18–64 years included 685,154 observations on
494,647 individuals in 8055 practices.
Descriptive statistics of the individual-level data for the two samples are provided in Tables 21 and 22.
In the data set for the base-case (sensitivity) analysis, the mean annual per-patient cost was £3159
(SD £7278) (£3289, SD £7414), mean age was 52 years (41) and 44% (48%) of the patients were male.
With regard to ethnicity, 83% (80%) were white, 4% (5%) were of mixed race, 4% (5%) Asian and 1%
(1%) were black. Six per cent (7%) of patients are formally detained, 12% (12%) were hospitalised during
the period and 6% (7%) had a social worker assigned to their care. Within any 1 year, <1% of the
patients used local authority day care, and in both analyses similar percentages applied for other local
authority services such as home help and residential care. In both data sets, the use of sheltered work was
rare (0.12% and 0.13%, respectively).
In total, 8064 practices contributed data for the base case. For each of the QOF indicators, practice
achievement ranged from 0% to 100%. Average (mean) achievement was 69% for MH6 (SD 19%) and
81% (SD 14%) for MH9. When achievement was adjusted to exclude exception-reported individuals, the
TABLE 20 Mental Health Minimum Data Set analysis: overview of regression models
Variables included Model 1 Model 2 Model 3 Model 4
Dependent variable Log of annual per-patient cost
QOF achievement measure A
(AþNAþE)
A
(AþNAþE)
A
(AþNA)
A
(AþNA)
Individual need variables ✓✓✓✓
Individual supply variables ✗✓✗✓
GP-level characteristics ✓✓✓✓
Year dummy variable ✓✓✓✓
✓, included in the model; ✗, not included in the model; A, the number of patients for which the indicator was achieved;
E, the number of patients who were exception reported; NA, the number of patients for which the indicator was
not achieved.
EMPIRICAL ANALYSIS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
42

TABLE 21 Mental Health Minimum Data Set analysis: descriptive statistics for the base case (all individuals)
Variable description n
Mean (or %
where indicated) SD Median Min. Max.
Total annual cost per patient 981,373 3159 7278 731 100 155,914
Natural log of annual cost 981,373 6.81 1.48 6.59 4.61 11.96
Financial year 2006/7 981,373 47.7% –– ––
Financial year 2007/8 981,373 52.3% –– ––
Patient age at start of spell/record period (years) 981,373 52.31 20.50 49.00 18.00 110.00
Aged 18–25 years 981,373 7.6% –– ––
Aged 25–34 years 981,373 15.0% –– ––
Aged 35–44 years 981,373 19.8% –– ––
Aged 45–54 years 981,373 15.7% –– ––
Aged 55–64 years 981,373 11.8% –– ––
Aged 65–74 years 981,373 9.4% –– ––
Aged 75 and over 981,373 20.8% –– ––
Proportion male patients 981,373 44.0% –– ––
Ethnicity, revised 981,373 1.50 1.21 1.00 1.00 5.00
Proportion white ethnicity 981,373 82.5% –– ––
Proportion mixed ethnicity 981,373 3.8% –– ––
Proportion Asian ethnicity 981,373 3.7% –– ––
Proportion black ethnicity 981,373 0.9% –– ––
Proportion other or undefined ethnicity 981,373 9.0% –– ––
Indicator for multiple spells in-year 981,373 10.4% –– ––
Indicator for learning disability speciality
(any time)
981,373 0.0% –– ––
Indicator for patient being detained (year) 981,373 6.0% –– ––
Indicator for inpatient (year) 981,373 11.6% –– ––
Indicator for whether or not receiving local
authority organised day centre care (year)
981,373 0.9% –– ––
Indicator for whether or not receiving local
authority domiciliary care support (year)
981,373 0.7% –– ––
Indicator for whether or not receiving local
authority organised residential care (year)
981,373 0.6% –– ––
Indicator for using sheltered work (year) 981,373 0.1% –– ––
Indicator for involvement of social worker (year) 981,373 6.2% –– ––
Max., maximum; min., minimum.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
43

TABLE 22 Mental Health Minimum Data Set analysis: descriptive statistics for the sensitivity analyses
(working-aged individuals)
Variable description nMean SD Median Min. Max.
Total annual cost per patient 685,154 3289 7414 815 100 155,914
Natural log of annual cost 685,154 6.88 1.49 6.70 4.61 11.96
Financial year 2007/8 685,154 48.2% –– ––
Financial year 2008/9 685,154 51.8% –– ––
Patient age at start of spell/record period (years) 685,154 40.93 12.18 41.00 18.00 64.00
Aged 18–25 years 685,154 10.8% –– ––
Aged 25–34 years 685,154 21.5% –– ––
Aged 35–44 years 685,154 28.3% –– ––
Aged 45–54 years 685,154 22.5% –– ––
Aged 55–64 years 685,154 16.9% –– ––
Gender: male 685,154 47.9% –– ––
Ethnicity, revised 685,154 1.56 1.25 1.00 1.00 5.00
Proportion white ethnicity 685,154 79.7% –– ––
Proportion mixed ethnicity 685,154 4.8% –– ––
Proportion Asian ethnicity 685,154 4.7% –– ––
Proportion black ethnicity 685,154 1.3% –– ––
Proportion other or undefined ethnicity 685,154 9.6% –– ––
Indicator for multiple spells in-year 685,154 10.4% –– ––
Indicator for if learning disability speciality (any time) 685,154 0.0% –– ––
Indicator for patient being detained 685,154 7.1% –– ––
Indicator for inpatient 685,154 12.1% –– ––
Indicator for whether or not receiving local authority
organised day centre care
685,154 0.8% –– ––
Indicator for whether or not receiving local authority
domiciliary care support
685,154 0.6% –– ––
Indicator for whether or not receiving local authority
organised residential care
685,154 0.5% –– ––
Indicator for using sheltered work 685,154 0.1% –– – –
Indicator for involvement of social worker 685,154 6.6% –– ––
Max., maximum; min., minimum.
EMPIRICAL ANALYSIS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
44

corresponding figures were 80% (SD 18%) for MH6 and 93% (SD 10%) for MH9. The exception
reporting rates were 15% (range 0–88%) for MH6 and 13% (range 0–83%) for MH9. In the sensitivity
analysis of individuals aged from 18 years to 64 years, these practice-level statistics were almost identical
as almost all practices are represented (8055 practices).
The QOF indicators for annual review (MH9) and care planning (MH6) have no significant effect on total
annual patient cost in any of the models, whether tested individually or jointly (Table 23). Results from the
base case for model 1 are presented in Table 24 while the sensitivity regression analyses results can be
found in Sensitivity analyses (see Table 26).
The relationship between general practices’performance on the QOF indicators and annual per-patient
cost is not statistically significant at the 5% level in any of the models. Within each model, the relationship
is insignificant when the indicators, MH9 and MH6, were modelled jointly and also when they were each
tested individually. This relationship is robust to alternative calculations of the QOF performance measure
(i.e. the inclusion/exclusion of data on exception-reported individuals), and to the inclusion/exclusion of
patient supply variables.
Across the four models, patient-level factors most consistently associated with higher cost include middle
age, being of black or Asian ethnicity, and being formally detained under the Mental Health Act 1983.49
Male gender is associated with significantly higher cost, but this relationship is not statistically significant
when supply-level factors such as use of inpatient care are accounted for. The relationship between age
and cost is bell-shaped: relative to the reference group of people aged 25–34 years, costs are lower for
people aged under 25 years, about 3–4% higher for those aged 35–55 years and similar for people
aged over 55 years. Costs for people aged 75 years and over are around 20% lower than those of the
reference group. Higher cost is also associated with greater deprivation scores for the practice population.
Interestingly, the variable measuring the prevalence of informal care within the practice population
residency is strongly and negatively related to annual per-patient cost.
TABLE 23 Mental Health Minimum Data Set analysis: regression results for the QOF indicators (all models,
base case)
Model/indicator
Joint modelling MH6 only MH9 only
Beta 95% CI Beta 95% CI Beta 95% CI
Model 1
MH6 0.016 –0.026 to 0.058 0.018 –0.015 to 0.051 ––
MH9 0.005 –0.049 to 0.059 –– 0.018 –0.025 to 0.060
Model 2
MH6 0.018 –0.019 to 0.056 0.017 –0.011 to 0.044 ––
MH9 –0.003 –0.052 to 0.045 –– 0.011 –0.025 to 0.047
Model 3
MH6 0.013 –0.026 to 0.053 0.008 –0.027 to 0.042 ––
MH9 –0.019 –0.086 to 0.047 –– –0.008 –0.067 to 0.051
Model 4
MH6 0.010 –0.025 to 0.045 0.007 –0.023 to 0.037 ––
MH9 –0.010 –0.067 to 0.047 –– –0.002 –0.051 to 0.047
Note
For model definitions, see Table 20.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
45

TABLE 24 Mental Health Minimum Data Set analysis: regression results from the base case (model 1), all individuals
Variable description
Joint modelling MH6 only MH9 only
Beta SE p-value Beta SE p-value Beta SE p-value
Achievement on MH6 QOF indicator 0.016 0.022 –0.018 0.017 ––––
Achievement on MH9 QOF indicator 0.005 0.028 ––––0.018 0.022 –
Gender: male 0.026 0.006 <0.001 0.026 0.006 <0.001 0.026 0.006 <0.001
Aged 18–25 years –0.101 0.012 <0.001 –0.101 0.012 <0.001 –0.101 0.012 <0.001
Aged 35–44 years 0.032 0.007 <0.001 0.032 0.007 <0.001 0.032 0.007 <0.001
Aged 45–54 years 0.042 0.009 <0.001 0.042 0.009 <0.001 0.042 0.009 <0.001
Aged 55–64 years 0.020 0.012 –0.020 0.012 –0.020 0.012 –
Aged 65–74 years 0.043 0.019 <0.05 0.043 0.019 <0.05 0.043 0.019 <0.05
Aged 75 years and over –0.215 0.023 <0.001 –0.215 0.023 <0.001 –0.215 0.023 <0.001
Ethnicity: mixed 0.222 0.019 <0.001 0.222 0.019 <0.001 0.222 0.019 <0.001
Ethnicity: Asian –0.022 0.016 ––0.022 0.016 ––0.022 0.016 –
Ethnicity: black 0.137 0.019 <0.001 0.137 0.019 <0.001 0.137 0.019 <0.001
Ethnicity: other or undefined –0.678 0.023 <0.001 –0.678 0.023 <0.001 –0.678 0.023 <0.001
Indicator for learning disability
speciality (any time)
0.275 0.216 –0.275 0.216 –0.275 0.216 –
Indicator for patient being
detained (year)
1.943 0.051 <0.001 1.943 0.051 <0.001 1.943 0.051 <0.001
Financial year 2007/8 0.032 0.019 –0.032 0.019 –0.033 0.019 –
Proportion of practice patients able
to access care within 48 hours
–0.032 0.029 ––0.032 0.029 ––0.032 0.029 –
Indicator for whether or not GP
practice is reimbursed under PMS
0.002 0.007 –0.002 0.007 –0.002 0.007 –
Proportion of male GPs in
GP practice
0.027 0.012 <0.05 0.027 0.012 <0.05 0.027 0.012 <0.05
Proportion of foreign GPs in
GP practice
–0.007 0.010 ––0.007 0.010 ––0.006 0.010 –
Mean age of GPs in GP practice 0.000 0.001 –0.000 0.001 –0.000 0.001 –
MH claimants –second quintile 0.032 0.011 <0.01 0.032 0.011 <0.01 0.033 0.011 <0.01
MH claimants –third quintile 0.058 0.012 <0.001 0.058 0.012 <0.001 0.059 0.012 <0.001
MH claimants –fourth quintile 0.063 0.013 <0.001 0.063 0.013 <0.001 0.063 0.013 <0.001
MH claimants –fifth quintile 0.061 0.018 <0.001 0.061 0.018 <0.001 0.061 0.018 <0.001
Proportion providing informal care,
practice catchment area
–1.995 0.570 <0.001 –1.994 0.571 <0.001 –1.989 0.570 <0.001
NHS psychiatric residents per
1000 population, practice
catchment area
0.003 0.003 –0.003 0.003 –0.003 0.003 –
Proportion living in urban setting,
practice catchment area
0.046 0.019 <0.05 0.046 0.019 <0.05 0.046 0.019 <0.05
GP practice list size (second tertile –
medium)
0.006 0.007 –0.006 0.007 –0.006 0.007 –
GP practice list size (third tertile –
large)
0.008 0.008 –0.008 0.008 –0.008 0.008 –
EMPIRICAL ANALYSIS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
46

In the two models that include covariates to control for supply-side factors (not shown), findings are
broadly consistent. Having multiple spells within a year (see Figure 10) is associated with significantly lower
annual cost, whereas higher cost is associated with receipt of inpatient care, and some local authority
services such as day centre attendance, and social worker involvement.
Sensitivity analyses
As in the base case, the sensitivity analysis of individuals aged 18–64 years found no statistically significant
relationship between general practices’performance on the QOF indicators and annual per-patient cost
in any of the four models (Table 25). For the other explanatory factors, findings were broadly similar to
those of the base case (Table 26).
TABLE 25 Mental Health Minimum Data Set analysis: regression results for the QOF indicators (all models,
sensitivity analysis)
Model/indicator
Joint modelling MH6 only MH9 only
Beta 95% CI Beta 95% CI Beta 95% CI
Model 1
MH6 0.024 –0.022 to 0.069 0.024 –0.011 to 0.059 ––
MH9 0.000 –0.057 to 0.057 –– 0.019 –0.024 to 0.062
Model 2
MH6 0.022 –0.018 to 0.063 0.023 –0.007 to 0.053 ––
MH9 0.001 –0.051 to 0.054 –– 0.019 –0.019 to 0.057
Model 3
MH6 0.021 –0.022 to 0.065 0.007 –0.029 to 0.044 ––
MH9 –0.046 –0.117 to 0.025 –– –0.028 –0.089 to 0.033
Model 4
MH6 0.017 –0.022 to 0.055 0.008 –0.024 to 0.041 ––
MH9 –0.029 –0.092 to 0.035 –– –0.014 –0.068 to 0.040
Note
For model definitions, see Table 20.
TABLE 24 Mental Health Minimum Data Set analysis: regression results from the base case (model 1),
all individuals (continued )
Variable description
Joint modelling MH6 only MH9 only
Beta SE p-value Beta SE p-value Beta SE p-value
Patient population: average age 0.001 0.001 –0.001 0.001 –0.001 0.001 –
Patient population: proportion of
male patients
0.234 0.170 –0.234 0.170 –0.233 0.170 –
Constant term 6.776 0.110 <0.001 6.778 0.109 <0.001 6.776 0.110 <0.001
PCT-level variance 0.320 0.018 <0.001 0.320 0.018 <0.001 0.320 0.018 <0.001
GP-level variance 0.153 0.006 <0.001 0.153 0.006 <0.001 0.153 0.006 <0.001
Patient-level variance 1.337 0.008 <0.001 1.337 0.008 <0.001 1.337 0.008 <0.001
Number of observations 981,373 –– 981,373 –– 981,373 ––
SE, standard error.
–,p>0.05
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
47

TABLE 26 Mental Health Minimum Data Set analysis: regression results from the sensitivity analyses,
working age individuals
Variable description
Joint modelling MH6 only MH9 only
Beta SE p-value Beta SE p-value Beta SE p-value
Achievement on MH6 QOF indicator 0.024 0.023 –0.024 0.018 ––––
Achievement on MH9 QOF indicator 0.000 0.029 ––––0.019 0.022 –
Gender: male 0.034 0.008 <0.001 0.034 0.008 <0.001 0.034 0.008 <0.001
Aged 18–25 years –0.099 0.011 <0.001 –0.099 0.011 <0.001 –0.099 0.011 <0.001
Aged 35–44 years 0.030 0.007 <0.001 0.030 0.007 <0.001 0.030 0.007 <0.001
Aged 45–54 years 0.038 0.009 <0.001 0.038 0.009 <0.001 0.038 0.009 <0.001
Aged 55–64 years 0.015 0.012 –0.015 0.012 –0.015 0.012 –
Ethnicity: mixed 0.248 0.021 <0.001 0.248 0.021 <0.001 0.248 0.021 <0.001
Ethnicity: Asian –0.004 0.018 ––0.004 0.018 ––0.004 0.018 –
Ethnicity: black 0.160 0.019 <0.001 0.160 0.019 <0.001 0.160 0.019 <0.001
Ethnicity: other or undefined –0.715 0.026 <0.001 –0.715 0.026 <0.001 –0.715 0.026 <0.001
Indicator for learning disability
speciality (any time)
0.233 0.202 –0.233 0.202 –0.233 0.202 –
Indicator for patient being
detained (year)
1.909 0.051 <0.001 1.909 0.051 <0.001 1.909 0.051 <0.001
Financial year 2007/8 0.050 0.020 <0.05 0.050 0.020 <0.05 0.052 0.020 <0.01
Proportion of practice patients able
to access care within 48 hours
–0.035 0.031 ––0.035 0.031 ––0.034 0.031 –
Indicator for whether GP practice is
reimbursed under PMS
0.004 0.006 –0.004 0.006 –0.004 0.006 –
Proportion of male GPs in
GP practice
0.009 0.013 –0.009 0.013 –0.009 0.013 –
Proportion of foreign GPs in
GP practice
–0.014 0.012 ––0.014 0.012 ––0.013 0.012 –
Mean age of GPs in GP practice 0.000 0.001 –0.000 0.001 –0.000 0.001 –
MH claimants –second quintile 0.029 0.015 <0.05 0.029 0.014 <0.05 0.029 0.014 <0.05
MH claimants –third quintile 0.071 0.015 <0.001 0.071 0.015 <0.001 0.071 0.015 <0.001
MH claimants –fourth quintile 0.079 0.015 <0.001 0.079 0.015 <0.001 0.079 0.015 <0.001
MH claimants –fifth quintile 0.076 0.020 <0.001 0.076 0.020 <0.001 0.076 0.020 <0.001
Proportion providing informal care,
practice catchment area
–1.771 0.610 <0.01 –1.771 0.610 <0.01 –1.761 0.610 <0.01
NHS psychiatric residents per
1000 population, practice
catchment area
0.001 0.003 –0.001 0.003 –0.001 0.003 –
Proportion living in urban setting,
practice catchment area
0.056 0.018 <0.01 0.056 0.018 <0.01 0.056 0.018 <0.01
GP practice list size (second tertile –
medium)
0.012 0.008 –0.012 0.008 –0.012 0.008 –
GP practice list size (third tertile –
large)
0.016 0.010 –0.016 0.010 –0.016 0.010 –
EMPIRICAL ANALYSIS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
48

Percentage of valid exception reporting
In previous analyses, we sought to identify the ‘switching point’at which the relationship between QOF
achievement and admissions became statistically insignificant. In the cost analysis, we found that the way
in which achievement was measured, that is, whether or not exception reported cases were included in the
measures, had no influence on the statistical significance of the relationship between achievement and cost.
Therefore, there is no ‘switching point’to identify in this analysis.
TABLE 26 Mental Health Minimum Data Set analysis: regression results from the sensitivity analyses,
working age individuals (continued )
Variable description
Joint modelling MH6 only MH9 only
Beta SE p-value Beta SE p-value Beta SE p-value
Patient population: average age 0.001 0.001 –0.001 0.001 –0.001 0.001 –
Patient population: proportion of
male patients
0.320 0.212 –0.320 0.212 –0.319 0.212 –
Constant term 6.684 0.114 <0.001 6.684 0.114 <0.001 6.685 0.114 <0.001
PCT-level variance 0.349 0.020 <0.001 0.349 0.020 <0.001 0.350 0.020 <0.001
GP-level variance 0.173 0.006 <0.001 0.173 0.006 <0.001 0.173 0.006 <0.001
Patient-level variance 1.329 0.008 <0.001 1.329 0.008 <0.001 1.329 0.008 <0.001
Number of observations 685,154 –– 685,154 –– 685,154 ––
SE, standard error.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
49

Chapter 4 Discussion
Our research investigated the relationship between quality of primary care for people with SMI, as
incentivised by the UK QOF, and hospital admissions for physical and mental health conditions.
Good-quality primary care management of patients with a SMI should reduce complications of SMI and
comorbidities and should, therefore, be associated with lower unplanned (emergency) admission rates.
Conversely, better quality of care may result in more health problems being identified as part of regular
screening activities and more frequent GP–patient contacts, thereby leading to more planned (elective)
admissions for hospital care. If better-quality primary care leads to reduced emergency admissions, it may
also be associated with lower NHS expenditure. LOS for patients with a SMI is typically much longer than
for other patients and better management in primary care could shorten their lengths of stay in hospital.
Our research questions were as follows:
1. Is better general practice performance on SMI QOF indicators associated with:
i. lower rates of emergency hospital admissions for SMIs for practice patients with a diagnosis of
aSMI?
ii. lower rates of emergency admissions for a SMI for practice patients with a diagnosis of
bipolar disorder?
iii. lower rates of emergency admissions for physical conditions for practice patients with a current or
previous diagnosis of a SMI?
iv. higher rates of elective admissions for physical conditions in patients with a current or previous
diagnosis of a SMI?
2. Is better general practice performance on SMI QOF indicators associated, with shorter LOS for practice
patients with a SMI following admission for a SMI?
3. Is better performance on SMI QOF indicators associated with lower secondary care expenditure for
mental health services for practice patients with a SMI?
Our null hypotheses were that there is no association between primary care quality and either admissions,
LOS or costs. Our alternative hypotheses were that preventative care could lower emergency hospital
admissions, reduce LOS and reduce mental health expenditure, and that regular screening could increase
elective admissions.
Our results showed, contrary to expectation, a positive and statistically significant association between QOF
achievement (particularly for MH9 –annual health checks) and emergency admissions, for both mental and
physical admissions. This implies that better quality of primary care is associated with more admissions, not
fewer. The results showed a positive association between QOF achievement and elective admissions, as
expected, although results were not statistically significant. The results for QOF achievement on MH4
(lithium therapy recorded) for patients with bipolar disorder also showed a statistically significant and
positive association with admissions. Results showed no statistically significant effect of QOF achievement
on either LOS or cost. All results were robust to sensitivity analyses.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
51

There are a number of potential explanations for our findings and we present them in order of their
perceived likelihood.
1. Higher quality of primary care, as measured by QOF, may not effectively prevent the need for secondary
care. Current policy places an emphasis on ‘upstream’prevention and ‘early intervention’to avoid the
need for more intensive and expensive specialist care. Our study findings do raise some doubt about
whether or not higher-quality primary care, as measured by the SMI QOF indicators, is delivering on this
policy aim. The QOF may be failing as an effective means of supporting early intervention and
avoidance of crisis and hence emergency admissions. Indeed this chimes more broadly with questions
about the effectiveness of current policy measures to prevent secondary care use through improved
preventative management of certain clinical conditions given that these efforts have hitherto failed to
reduce the demand for emergency care.61
2. Better-quality primary care may be picking up unmet need for secondary care. Practices that attract
people with a SMI who may be less likely to engage with services and those with relatively high levels
of unmet need may uncover and treat this unmet need which may then entail admission.
3. We analysed QOF data at the level of practices and cannot be sure whether or not the specific
individuals who were admitted had received QOF checks. It is possible that those who were admitted as
emergencies had not received an annual QOF review. It is also possible that individuals admitted as
emergencies did receive QOF reviews, but only after discharge from hospital, rather than prior to
admission, which could also explain the positive association.
4. If people with a SMI know from their experiences with the health and social care system that some
practices are more receptive to them, recognise their needs or are better organised to provide their
care, then this may systematically influence their choice of practice. These practices may therefore
achieve high scores for the QOF reviews but also may have a higher rate of emergency admissions for
these individuals if they attract and treat people whose disease is complex or at the higher end of the
severity spectrum.
5. The QOF indicators may not accurately measure the quality of primary care for a SMI. The QOF, like any
other P4P scheme, may result in tunnel vision71 or a focus on areas of activities within the scheme
which are incentivised, sometimes at the expense of other activities which are not incentivised.72
Thus, high QOF attainment may not necessarily reflect high-quality care.
6. As hypothesised, higher rates of QOF checks were associated with higher rates of elective admissions
for physical health problems for people with SMI, although these associations were not statistically
significant. However, this trend is consistent with trial evidence of increased referrals following the
introduction of regular health checks for people with long-term mental illness.73
This study makes a unique contribution in that it is the first analysis to examine the relationship between
primary care quality, as measured by SMI QOF indicators, and admission rates for patients with a SMI
for both mental and physical health. It also makes a unique contribution in examining the resource
implications of better-quality primary care for SMI patients. The strengths of the study are that it covers all
practices in England and results are representative. It uses a consistent set of primary care quality indicators
over the entire study period and employs robust panel data estimation as well as cross-sectional analysis.
A comprehensive set of GP practice and patient population characteristics are included in the models.
An array of sensitivity analyses was undertaken and results were found to be robust.
The limitations of the study include the fact that aggregate data were used to examine the association
between QOF quality and admissions. This does not enable one to specify the nature of individual care
pathways or determine causality. Although we attempted to model all known confounding factors, we
were unable to capture time-variant aspects of the influence of CMHTs and could not account for disease
severity (case mix). In addition, we cannot rule out the possibility that there are other, unknown, biases
that impact our findings. Another limitation was the incomplete data available in the cost analysis. To help
address this limitation, the resource use implications of primary care quality on LOS were also investigated,
as the LOS data were drawn from a more comprehensive data set.
DISCUSSION
NIHR Journals Library www.journalslibrary.nihr.ac.uk
52

Chapter 5 Conclusions
Implications for research
The positive association we found between higher rates of QOF checks and higher rates of emergency
admissions for both mental and physical health problems was unexpected and needs further exploration.
We recommend a number of avenues for further research in order to better understand what is happening
within the patient pathway and better understand the link between primary and secondary care for
patients with a SMI. We have added to the sparse literature on the quality of mental health services for
those with a SMI in primary care, but future research would require access to patient-level, as well as
practice-level, primary care data to build on the findings from this study and provide further insights to
improve patient care.
We identify a number of research priorities to examine (ordered by their perceived priority):
1. The patient pathway and the timing of events within that pathway.
In order to examine whether or not the specific individuals who were admitted had received QOF
checks, and whether or not these checks were done post discharge, we require patient-level data. This
will help determine causality.
2. Which QOF measures might reduce admissions?
Emergency admissions are costly for the health-care system and generally undesirable for service users.
Hence, considering ways to reduce avoidable unplanned admissions remains a policy priority. The
effectiveness of the current SMI QOF indicators in this regard needs to be confirmed or rejected by
further work using patient-level primary care data. The new SMI QOF indicators introduced in 2011/12
which cover physical health monitoring should also be assessed.
3. What other (non-QOF) measures of primary care quality might reduce admissions more effectively and
could potentially be incentivised?
In order to explore alternative measures of primary care quality, general practice data at the level of the
individual patient is required.
4. The specific physical conditions and indications for admission among people with a SMI.
This will help to determine whether or not admissions could be prevented by particular types of
intervention (e.g. detecting diabetes in patients with a SMI through screening before it becomes a
physical emergency). The role of comorbidity as a factor in higher admissions would be important to
consider. Once again, patient-level data would be needed for such research.
5. Which types of admissions are potentially avoidable for SMI care?
Currently all SMI admissions are classed as emergency admissions, and all are implicitly considered
avoidable, but more research is needed to try to understand which types of admissions are necessary
and which represent appropriate use of secondary care and which are potentially avoidable.31
6. How much unmet need there is for people with SMI?74
Estimates of unmet need should be updated from those generated by the Mental Illness Needs Index
2000 and the Community Mental Health Profile 201375 which focus more broadly on all mental health
problems and prevalence by local authority area, to focusing on unmet need for a SMI, specifically at
the GP practice level. Prevalence of unmet needs are related to the system of mental health-care
provision and to socioeconomic circumstances –the less integrated and continuous is care and the
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
53

poorer the life situation, the higher is unmet need.76 Once more precise quantitative statements can be
made about unmet need, policy initiatives can be put in place to ensure primary care is appropriately
equipped and incentivised to address these.
7. Whether or not some GPs find it harder to get patients admitted than others, given the level of
capacity constraints in secondary care to appropriately deal with unmet need in people with a SMI.
There has been high-profile media coverage of the large numbers of mental health bed closures over
the past few years and the fact that average bed occupancy rates for psychiatric beds are running at or
above 100% for around half of all mental health trusts, which is above the 85% recommended by the
Royal College of Psychiatrists.77 In light of these supply side constraints, more research is needed on
whether or not some GP practices are more successful than others in getting their patients admitted to
hospital and whether or not this is related to their achievement on the QOF.
8. How the comprehensive care plan is developed and documented for individuals with SMI.
At present, if a care plan is developed in secondary care and passed on to the GP practice, this can
be used as a measure of achievement on the QOF even though it has not been developed in primary
care. Information on the origin of the care plan could easily be captured by GP practices and research
could support a better understanding of the processes by which care plans are developed, used
and updated.
Implications for practice
There are a number of possible implications for practice based on the conclusions from this study (ordered
by their perceived priority).
1. Assess value for money of QOF health checks for people with a SMI.
An over-simplistic interpretation of our findings could lead practitioners and commissioners of services
to conclude that funding annual reviews of people with a SMI through the QOF is unlikely to be
cost-effective because it does not appear to prevent costly emergency admissions. One interpretation is
that QOF is not working and should be abandoned. However, the QOF was not specifically designed to
reduce unplanned admissions and other outcome measures, especially the health and well-being of
patients and their carers is required for a full evaluation. It is not possible from our practice-level analysis
to be sure whether or not the QOF funded reviews prevent emergency admissions of people with a SMI
and many of these emergency admissions may be appropriate and represent good-quality care by GPs.
It would therefore be inappropriate to recommend the abolition of QOF checks for a SMI on the basis
of these study findings. QOF checks, specifically those that focus on physical care, may still be important
activities that are valued by service users and may offer health benefits that this study did not consider.
2. Factor in resource requirements for likely increase in referrals following QOF checks for a SMI.
Practitioners and commissioners should be aware that carrying out regular checks on people with a SMI
is likely to lead to increased referrals for physical health problems, and ensure that funding is in place to
support those referrals. Given the particular problems that some people with a SMI might face in being
able to attend outpatient appointments for physical health problems because of an impaired ability to
organise their daily schedule or making travel arrangements, specific arrangements for care pathways
might be considered between primary and secondary care providers in order to accommodate such
referrals, such as specialists agreeing to see patients on practice premises, or on domiciliary visits,
for example.
CONCLUSIONS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
54

3. Improve diagnostic coding quality in secondary care.
As part of this study, we identified poor-quality diagnostic coding in a number of mental health
provider trusts. These organisations had a very high proportion of people given a diagnosis code of R69
(denoting ‘not known’). This prompted the strategy of running a sensitivity analysis to investigate the
impact on our results of inclusion of admissions with this broader definition. We assume that these
people are admitted for a SMI because they have been admitted to a psychiatric specialty, in many
cases to a specialist mental health hospital. The variation in coding practice of R69 admissions suggests
a need to improve diagnostic coding, particularly from certain providers who appear over-reliant on the
R69 code and improved diagnostic coding could be supported with appropriate incentives.
4. Improve data coverage and quality of the MHMDS.
The shortfalls in quality of the MHMDS data meant that we were unable to address comprehensively
the research question relating to the impact of primary care quality on secondary mental health
expenditure. Data items which were poorly coded or non-existent in the 2 years of MHMDS data
used in this study included diagnoses, attendance at NHS day-care facilities, home-based care and
community team activity, social care, CPA assessments, and HoNOS scores that assess health and social
functioning. These data items are fundamental to examining activity, resource use, severity and
outcomes for mental health patients and incentives could be considered for improving data quality,
which in turn would allow better monitoring and improvement of mental health services. LSOA codes
are also currently not provided as part of the MHMDS, but if included, would enable research to
appropriately adjust for area deprivation characteristics which are crucial as potential confounders in
mental health services research.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
55

Acknowledgements
We would like to thank the NHS National Clinical Classifications Service at NHS Connecting for Health
for providing us with the cross-mapping results for Read codes version 2 and version 3 (which are a
coded thesaurus of clinical terms used by GP practices to record patient diagnoses) to ICD-10 diagnosis
codes used in HES admissions data.
We would like to thank the following steering group members for their invaluable contributions to this
project: June Wainwright, Lauren Aylott, Suzanne McBain, Peter Bower, Paul Blenkiron, Liz England and
David Daniel. We received sound and timely advice from our medical, policy and patient and public
involvement (PPI) colleagues on a wide range of issues, not only at steering group meetings but also on
many other occasions when we sought their expertise. They helped clarify our thinking on various issues
and also provided valuable insights with the interpretations of our results. They read and commented on
the interim reports produced for the National Institute for Health Research (NIHR) and also provided
comments on this draft final report. Their input made a huge difference to the project. We are delighted
that we have their support and continued commitment to our future research plans and we look forward
to a long and fruitful collaboration with them.
We would like to thank Rachel Richardson for contributions to the project in producing the dissemination
strategy and in writing the advertising materials in lay language to invite service users and carers
(e.g. RETHINK) to a workshop we ran in March 2013 presenting interim results from the project.
We would also like to thank conference participants from the various conferences where we presented our
work for valuable comments and feedback: Health Economists Study Group (HESG), Exeter, January 2013;
Primary Care Mental Health Conference, University of Manchester, March 2013; Eleventh Workshop on
Costs and Assessment in Psychiatry (ICMPE), Venice, March 2013; HESG, Warwick, June 2013; Society for
Academic Primary Care 42nd Annual Conference, University of Nottingham, July 2013; Royal College of
General Practitioners’Annual Primary Care Conference, Harrogate, UK, October 2013; and the North
American Primary Care Research Group Annual Scientific Meeting, Ottawa, November 2013.
Publications
Gutacker N, Mason AR, Kendrick T, Goddard M, Gravelle H, Gilbody S, et al. Does the quality and
outcomes framework reduce psychiatric admissions in people with serious mental illness? A regression
analysis. BMJ Open 2015;5:e007342.
Contributions of authors
Dr Rowena Jacobs (Senior Research Fellow, health economics) led the study and contributed to study
design, interpretation of results and led the writing of the final report.
Nils Gutacker (Research Fellow, health economics) provided input to all aspects of the study, including
data analysis of the HES and QOF data, interpretation of results and leading the writing of the final report.
Anne Mason (Senior Research Fellow, health economics) provided input to all aspects of the study,
including data analysis of the MHMDS, interpretation of results and leading the writing of the final report.
Maria Goddard (Professor, health economics) contributed to study design, interpretation of results and to
the writing of the final report.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
57

Hugh Gravelle (Professor, health economics) contributed to study design, interpretation of results and to
the writing of the final report.
Tony Kendrick (Professor, GP) contributed to study design, interpretation of results, providing clinical
input and writing of the final report.
Simon Gilbody (Professor, psychiatrist) contributed to study design, provided clinical input and helped
with interpretation of results.
Lauren Aylott (service user) contributed as a service user to the interpretation of results and implications
for practice and commented on the final report.
June Wainwright (service user) contributed as a service user to the interpretation of results and
implications for practice and commented on the final report.
ACKNOWLEDGEMENTS
NIHR Journals Library www.journalslibrary.nihr.ac.uk
58

References
1. Harrison G, Hopper K, Craig T, Laska E, Siegel C, Wanderling J, et al. Recovery from psychotic
illness: a 15- and 25-year international follow-up study. Br J Psychiatry 2001;178:506–17.
http://dx.doi.org/10.1192/bjp.178.6.506
2. Audit Commission. Maximising Resources in Adult Mental Health. London: Audit Commission;
2010. URL: http://archive.audit-commission.gov.uk/auditcommission/sitecollectiondocuments/
Downloads/20100623mentalhealthbriefing.pdf (accessed 31 March 2014).
3. Saunders KEA, Goodwin GM. The course of bipolar disorder. Adv Psychiatr Treat 2010;16:318–28.
http://dx.doi.org/10.1192/apt.bp.107.004903
4. Saha S, Chant D, Welham J, McGrath J. A systematic review of the prevalence of schizophrenia.
PLOS Med 2005;2:e141. http://dx.doi.org/10.1371/journal.pmed.0020141
5. Osborn DPJ. Physical activity, dietary habits and coronary heart disease risk factor knowledge
amongst people with severe mental illness: a cross sectional comparative study in primary care.
Soc Psychiatry Psychiatr Epidemiol 2007;42:787–93. http://dx.doi.org/10.1007/s00127-007-0247-3
6. Rethink Mental Illness. Lethal Discrimination: Why People with Mental Illness are Dying Needlessly
and What Needs to Change. London: Rethink Mental Illness; 2013. URL: www.rethink.org/media/
810988/Rethink%20Mental%20Illness%20-%20Lethal%20Discrimination.pdf (accessed
31 March 2014).
7. Jaques H. NICE recommends dropping all QOF indicators on depression. BMJ Careers 2011.
URL: http://careers.bmj.com/careers/advice/view-article.html?id=20004103 (accessed 31 March 2014).
8. Miller B, Paschall C, Svendsen D. Mortality and medical comorbidity among patients with SMI.
Psychiatr Serv 2006;57:1482–7. http://dx.doi.org/10.1176/appi.ps.57.10.1482
9. Wahlbeck K, Westman J, Nordentoft M, Gissler M, Laursen T. Outcomes of Nordic mental health
systems: life expectancy of patients with mental disorders. Br J Psychiatry 2011;199:453–8.
http://dx.doi.org/10.1192/bjp.bp.110.085100
10. Brown S, Inskip H, Barraclough B. Causes of the excess mortality of schizophrenia. Br J Psychiatry
2000;177:212–17. http://dx.doi.org/10.1192/bjp.177.3.212
11. Osborn DPJ, Levy G, Nazareth I, Petersen I, Islam A, King MB. Relative risk of cardiovascular and
cancer mortality in people with severe mental illness from the United Kingdom’s General
Practice Research Database. Arch Gen Psychiatry 2007;64:242–9. http://dx.doi.org/10.1001/
archpsyc.64.2.242
12. Harris EC, Barraclough B. Excess mortality of mental disorder. Br J Psychiatry 1998;173:11–53.
http://dx.doi.org/10.1192/bjp.173.1.11
13. Marder SR, Essock SM, Miller AL, Buchanan RW, Casey DE, Davis JM, et al. Physical health
monitoring of patients with schizophrenia. Am J Psychiatry 2004;161:1334–49. http://dx.doi.org/
10.1176/appi.ajp.161.8.1334
14. Murray CJL, Lopez A. The Global Burden of Disease: A Comprehensive Assessment of Mortality
and Disability from Diseases, Injuries, and Risk Factors in 1990 and Projected to 2020. Boston:
Harvard School of Public Health on behalf of the World Health Organization and the World
Bank; 1996.
15. Bouza C, López-Cuadrado T, Amate JM. Hospital admissions due to physical disease in people with
schizophrenia: a national population-based study. Gen Hosp Psychiatry 2010;32:156–63.
http://dx.doi.org/10.1016/j.genhosppsych.2009.11.014
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
59

16. Li Y, Glance LG, Cai X, Mukamel DB. Mental illness and hospitalization for ambulatory care
sensitive medical conditions. Med Care 2008;46:1249–56. http://dx.doi.org/10.1097/
MLR.0b013e31817e188c
17. Osby U, Brandt L, Correia N, Ekbom A, Sparen P. Excess mortality in bipolar and unipolar disorder
in Sweden. Arch Gen Psychiatry 2001;58:844–50. http://dx.doi.org/10.1001/archpsyc.58.9.844
18. Brown S, Kim M, Mitchell C, Inskip H. Twenty-five year mortality of a community cohort with
schizophrenia. Br J Psychiatry 2010;196:116–21. http://dx.doi.org/10.1192/bjp.bp.109.067512
19. Connolly M, Kelly C. Lifestyle and physical health in schizophrenia. Adv Psychiatr Treat
2005;11:125–32. http://dx.doi.org/10.1192/apt.11.2.125
20. British Medical Association, NHS Employers. Revisions to the GMS Contract 2006/07. Delivering
Investment in General Practice. London: NHS Confederation (Employers) Company Ltd; 2006.
21. McElroy SL. Obesity in patients with severe mental illness: overview and management.
J Clin Psychiatry 2009;70:12–21. http://dx.doi.org/10.4088/JCP.7075su1c.03
22. Kendrick T. Cardiovascular and respiratory risk factors and symptoms among general practice
patients with long-term mental illness. Br J Psychiatry 1996;169:733–9. http://dx.doi.org/10.1192/
bjp.169.6.733
23. McManus S, Meltzer H, Campion J. Cigarette Smoking and Mental Health in England: Data
from the Adult Psychiatric Morbidity Survey. In: National Centre for Social Research; 2010.
URL: www.natcen.ac.uk/media/21994/smoking-mental-health.pdf (accessed 4 April 2014).
24. Sernyak MJ, Leslie DL, Alarcon RD, Losonczy MF, Rosenheck R. Association of diabetes mellitus
with use of atypical neuroleptics in the treatment of schizophrenia. Am J Psychiatry
2002;159:561–6. http://dx.doi.org/10.1176/appi.ajp.159.4.561
25. Jacobs R, Barrenho E. The impact of crisis resolution and home treatment teams on psychiatric
admission rates in England. Br J Psychiatry 2011;99:71–6. http://dx.doi.org/10.1192/
bjp.bp.110.079830
26. Nazareth I, King M, Haines A. Care of schizophrenia in general practice. BMJ 1993;307:910.
http://dx.doi.org/10.1136/bmj.307.6909.910
27. Kai J, Crosland A, Drinkwater C. Prevalence of enduring and disabling mental illness in the inner
city. Br J Gen Pract 2000;50:992–4.
28. Lang F, Johnstone E, Murray G. Service provision for people with schizophrenia. Role of the general
practitioner. Br J Psychiatry 1997;171:165–8. http://dx.doi.org/10.1192/bjp.171.2.165
29. Reilly S, Planner C, Hann M, Reeves D, Nazareth I, Lester H. The role of primary care in service
provision for people with severe mental illness in the United kingdom. PLOS ONE 2012;7:e36468.
http://dx.doi.org/10.1371/journal.pone.0036468
30. Purdy S, Paranjothy S, Huntley A, Thomas R, Mann M, Huws D, et al. Interventions to Reduce
Unplanned Hospital Admission: A Series of Systematic Reviews. Final Report. Bristol: University of
Bristol; 2012. URL: www.bristol.ac.uk/primaryhealthcare/docs/projects/unplannedadmissions.pdf
(accessed 31 March 2014).
31. Purdy S. Avoiding Hospital Admissions: What Does the Research Evidence Say? London:
The King’s Fund; 2010.
32. Giuffrida A, Gravelle H, Roland M. Measuring quality of care with routine data: avoiding confusion
between performance indicators and health outcomes BMJ 1999;319:94–7. http://dx.doi.org/
10.1136/bmj.319.7202.94
REFERENCES
NIHR Journals Library www.journalslibrary.nihr.ac.uk
60

33. Saxena S, George J, Barber J, Fitzpatrick J, Majeed A. Association of population and practice
factors with potentially avoidable admission rates for chronic diseases in London: cross-sectional
analysis. J R Soc Med 2006;99:81–8. http://dx.doi.org/10.1258/jrsm.99.2.81
34. Griffin S, Kinmonth A. Systems for routine surveillance for people with diabetes mellitus.
Cochrane Database Syst Rev 2006;4:CD000541.
35. McEvoy JP, Meyer JM, Goff DC, Nasrallah HA, Davis SA, Sullivan L, et al. Prevalence of the
metabolic syndrome in patients with schizophrenia: baseline results from the Clinical Antipsychotic
Trials of Intervention Effectiveness (CATIE) Schizophrenia Trial and comparison with national
estimates from NHANES III. Schizophr Res 2005;80:19–32. http://dx.doi.org/10.1016/
j.schres.2005.07.014
36. National Institute for Health and Care Excellence (NICE). About the Quality and Outcomes
Framework (QOF). NICE; 2012. URL: www.nice.org.uk/aboutnice/qof/qof.jsp (accessed
31 March 2014).
37. Downing A, Rudge G, Cheng Y, Tu YK, Keen J, Gilthorpe MS. Do the UK government’s new
Quality and Outcomes Framework (QOF) scores adequately measure primary care performance?
A cross-sectional survey of routine healthcare data. BMC Health Serv Res 2007;7:166.
http://dx.doi.org/10.1186/1472-6963-7-166
38. Bottle A, Gnani S, Saxena S, Aylin P, Mainous AG, Majeed A. Association between quality of
primary care and hospitalization for coronary heart disease in England: a national cross-sectional
study. J Gen Intern Med 2008;23:135–41. http://dx.doi.org/10.1007/s11606-007-0390-2
39. Bottle A, Millett C, Xie Y, Saxena S, Wachter RM, Majeed A. Quality of primary care and hospital
admissions for diabetes mellitus in England. J Ambul Care Manage 2008;31:226–38.
http://dx.doi.org/10.1097/01.JAC.0000324668.83530.6d
40. Purdy S, Griffin T, Salisbury C, Sharp D. Emergency admissions for coronary heart disease:
a cross-sectional study of general practice, population and hospital factors in England.
Public Health 2011;125:46–54. http://dx.doi.org/10.1016/j.puhe.2010.07.006
41. Dusheiko M, Doran T, Gravelle H, Fullwood C, Roland M. Does higher quality of diabetes
management in family practice reduce unplanned hospital admissions? Health Serv Res
2011;46:27–46. http://dx.doi.org/10.1111/j.1475-6773.2010.01184.x
42. Soljak M, Calderon-Larranaga A, Sharma P, Cecil E, Bell D, Abi-Aad G, et al. Does higher quality
primary health care reduce stroke admissions? A national cross-sectional study. Br J Gen Pract
2011;61:e801–7. http://dx.doi.org/10.3399/bjgp11X613142
43. Doran T, Fullwood C, Reeves D, Gravelle H, Roland M. Exclusion of patients from pay-for-
performance targets by English physicians. N Engl J Med 2008;359:274–84. [Erratum Published in
N Engl J Med 2008;359:546.] http://dx.doi.org/10.1056/NEJMsa0800310
44. Petersen LA, Woodard LD, Urech T, Daw C, Sookanan S. Does pay-for-performance improve
the quality of health care? Ann Intern Med 2006;145:265–72. http://dx.doi.org/10.7326/
0003-4819-145-4-200608150-00006
45. Roland M. Linking physician pay to quality of care: a major experiment in the UK. N Engl J Med
2004;35:1448–54. http://dx.doi.org/10.1056/NEJMhpr041294
46. Mason A, Walker S, Claxton K, Cookson R, Fenwick E, Sculpher M. The GMS Quality and
Outcomes Framework: Are the Quality and Outcomes Framework (QOF) Indicators a Cost-effective
Use of NHS Resources? Quality and Outcomes Framework. Joint Executive Summary: Reports to the
Department of Health from the University of East Anglia and the University of York. York:
University of York; 2008.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
61

47. Walker S, Mason AR, Claxton K, Cookson R, Fenwick E, Fleetcroft R, et al. Value for money and
the Quality and Outcomes Framework in primary care in the UK NHS. Br J Gen Pract
2010;60:213–20. http://dx.doi.org/10.3399/bjgp10X501859
48. British Medical Association. Focus on QOF Payments. London: British Medical Association; 2013.
URL: http://bma.org.uk/practical-support-at-work/contracts/independent-contractors/qof-guidance/
focus-qof-payments (accessed 31 March 2014).
49. Great Britain. Mental Health Act 1983. London: The Stationery Office; 1983.
50. Department of Health (DH). QOF Guidance. London: DH; 2004. URL: www.dh.gov.uk/en/
Policyandguidance/Organisationpolicy/Primarycare/Primarycarecontracting/QOF/DH_4125653
(accessed 31 March 2014).
51. Doran T, Kontopantelis E, Fullwood C, Lester H, Valderas JM, Campbell S. Exempting dissenting
patients from pay for performance schemes: retrospective analysis of exception reporting in the UK
Quality and Outcomes Framework. BMJ 2012;344:e2405. http://dx.doi.org/10.1136/bmj.e2405
52. Gravelle H, Sutton M, Ma A. Doctor behaviour under a pay for performance contract: treating,
cheating and case finding? Economic Journal 2010;120:F129–F56. http://dx.doi.org/10.1111/
j.1468-0297.2009.02340.x
53. Office for National Statistics. Neighbourhood Statistics. URL: www.neighbourhood.statistics.gov.uk/
dissemination/ (accessed 23 September 2014).
54. Glover G, Arts G, Babu KS. Crisis resolution/home treatment teams and psychiatric admission rates
in England. Br J Psychiatry 2006;189:441–5. http://dx.doi.org/10.1192/bjp.bp.105.020362
55. Onyett S, Linde K, Glover G, Floyd S, Bradley S, Middleton H. Implementation of crisis resolution/
home treatment teams in England: national survey 2005–2006. Psychiatric Bull 2008;32:374–7.
http://dx.doi.org/10.1192/pb.bp.107.018366
56. Glover G, Barnes D. Mental Health Service Provision for Working Age Adults in England 2003.
Durham: Centre for Public Mental Health, University of Durham; 2005.
57. Holmes AM, Deb P. Factors influencing informal care-giving. J Ment Health Policy Econ
1998;1:77–87. http://dx.doi.org/10.1002/(SICI)1099-176X(199807)1:2<77::
AID-MHP10>3.0.CO;2-5
58. Cameron AC, Trivedi PK. Regression Analysis of Count Data. Cambridge: Cambridge University
Press; 1998. http://dx.doi.org/10.1017/CBO9780511814365
59. Hausman J, Hall BH, Griliches Z. Econometric models for count data with an application to the
patents-R&D relationship. Econometrica 1984;52:909–38. http://dx.doi.org/10.2307/1911191
60. Blundell R, Griffith R, Windmeijer F. Individual effects and dynamics in count data models.
J Econom 2002;108:113–31. http://dx.doi.org/10.1016/S0304-4076(01)00108-7
61. Bardsley M, Blunt I, Davies S, Dixon J. Is secondary preventive care improving? Observational study
of 10-year trends in emergency admissions for conditions amenable to ambulatory care. BMJ Open
2013;3:e002007. http://dx.doi.org/10.1136/bmjopen-2012-002007
62. Johnstone P, Zolese G. Systematic review of the effectiveness of planned short hospital stays for
mental health care. BMJ 1999;318:1387–90. http://dx.doi.org/10.1136/bmj.318.7195.1387
63. World Health Organization (WHO). International Statistical Classification of Diseases and Related
Health Problems 10th Revision. Geneva: WHO; 2014. URL: http://apps.who.int/classifications/
icd10/browse/2010/en#/R69 (accessed 31 March 2014).
REFERENCES
NIHR Journals Library www.journalslibrary.nihr.ac.uk
62

64. Street A, Kobel C, Renaud T, Thuilliez J, on behalf of the EuroDRG group. How well do Diagnosis
Related Groups explain variations in costs or LOS among patients and across hospitals? Methods
for analysing routine patient data. Health Economics 2012; 21(Suppl. 2):6–18. http://dx.doi.org/
10.1002/hec.2837
65. mentalhealthcare. Forensic Mental Health Services. 2014. URL: www.mentalhealthcare.org.uk/
forensic_mental_health_services (accessed 31 March 2014).
66. Department of Health (DH). National Schedule of Reference Costs 2007–08 for NHS Trusts.
London: DH; 2009.
67. Sutton M, Whittaker W, Morris S, Glover G, Dusheiko M, Wildman J, et al. RARP35 Report
of the Resource Allocation for Mental health and Prescribing (RAMP) Project. Report to the
Department of Health. 2010. URL: http://webarchive.nationalarchives.gov.uk/+/www.dh.gov.uk/en/
Managingyourorganisation/Financeandplanning/Allocations/DH_4108515 (accessed 31 March 2014).
68. Curtis L, editor. Unit Costs of Health and Social Care. Canterbury: Personal Social Services Research
Unit, University of Kent, 2009.
69. Matthews F, Arthur A, Barnes LE, Bond J, Jagger C, Robinson L, et al. A two-decade comparison of
prevalence of dementia in individuals aged 65 years and older from three geographical areas of
England: results of the Cognitive Function and Ageing Study I and II. Lancet 2013;382:1405–12.
http://dx.doi.org/10.1016/S0140-6736(13)61570-6
70. Halvorsen R, Palmquist R. The interpretation of dummy variables in semilogarithmic equations.
Am Econ Rev 1980;70:474–5.
71. Smith P. On the unintended consequences of publishing performance data in the public sector.
Int J Public Admin 1995;18:277–310. http://dx.doi.org/10.1080/01900699508525011
72. Eggleston K. Multitasking and mixed systems for provider payment. J Health Econ
2005;24:211–23. http://dx.doi.org/10.1016/j.jhealeco.2004.09.001
73. Kendrick T, Burns T, Freeling P. Randomised controlled trial of teaching general practitioners to
carry out structured assessments of their long term mentally ill patients. BMJ 1995;311:93–8.
http://dx.doi.org/10.1136/bmj.311.6997.93
74. Desai MM, Rosenheck RA. Unmet need for medical care among homeless adults with
serious mental illness. Gen Hosp Psychiatry 2005;27:418–25. http://dx.doi.org/10.1016/
j.genhosppsych.2005.06.003
75. Public Health England. Community Mental Health Profiles 2013. Durham: Public Health England;
2013. URL: www.mentalhealthobservatory.org.uk/cmhp/ (accessed 31 March 2014).
76. Wiersma D. Needs of people with severe mental illness. Acta Psychiatr Scand Suppl
2006;429:115–19. http://dx.doi.org/10.1111/j.1600-0447.2005.00728.x
77. Buchanan M. BBC News Health. England’sMental Health Services ‘In Crisis’.2013.
URL: www.bbc.co.uk/news/health-24546656 (accessed 31 March 2014).
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
63

Appendix 1 Patient and public involvement
Patient involvement was an important element in our research project from the outset. Ms June
Wainwright (JW) who is a mental health service user and a carer and also a service user representative
for the NIHR Mental Health Research Network (MHRN), was a co-applicant on our proposal and has
therefore been a member of our project team throughout. June played a key role at the early stage in
helping us formulate the research question and design and write the proposal. She also helped facilitate
our overall PPI strategy for the project by advising us on the composition of our steering group, of which
she is a member.
In addition to JW, our steering group includes two other PPI representatives [Ms Lauren Aylott (a mental
health service user) and Ms Suzanne McBain (a carer)] both of whom have brought valuable perspectives
to our project throughout. The steering group has met three times during the course of the project,
in accordance with the schedule we set out in the proposal and our PPI members have contributed
significantly to these discussions. At each meeting of the steering group, the principal investigator and a
member of the project team outlined progress to date and the issues that have arisen for discussion in
the group, in order to receive feedback and guidance on the next steps. The project team gave short
presentations, which aimed to convey the main messages in an accessible way so that all members of the
group were able to take part fully in the discussion.
The first meeting revolved around discussion of the QOF measures of quality of care that were available
to the project team for analysis and the input of our PPI members was particularly valuable in terms of
reflecting on the usefulness of measures in terms of what they mean in practice to patients. Their views
on the nature of hospital admissions for people with a SMI were also important, that is, to what degree all
admissions for this patient group can be seen as ‘emergency’admissions, and also the potential impact of
the CRHT teams. At this initial meeting we also discussed the other variables we were planning to include
in our analysis and, again, our PPI members were able to give us an insight into issues related to home
care and informal care, including possible sources of data. Planning for the workshop in which we
disseminated our initial results also took place at the steering group meeting and we received advice from
the PPI members on who we might invite to the workshop, including patient and carer mental health
organisations and the PPI members were also invited to attend the workshop. JW also used her links with
the MHRN to contact potential participants for the workshop.
The second meeting focused mainly on the initial results emerging from the analyses and specific points on
which we sought and received advice from our PPI members, included the interpretation of the positive
association between QOF and hospital admissions. Their views on this (as well as those of others on the
group and project team) are reflected in this report in terms of possible explanations for the results.
Another key question for our PPI members was around the annual review for patients with a SMI in terms
of how invitations to attend are received and whether those being cared for mainly in the secondary care
sector would view the annual review with the GP as an important or useful element of their care.
Our third, and final, steering group meeting reflected in detail on the final results and, again, the views of
our PPI members on how to interpret the findings of a positive association between QOF and hospital
admissions were invaluable. In particular, they helped shape our conclusions and our suggestions for
further research to unpick and understand better our findings and the insights that the PPI members
brought to this final interpretation of the results made a significant difference to our emphasis in this
report. At this final meeting we also discussed the further involvement of two of our PPI members on a
new proposal and subsequently they have attended meetings to discuss further work and have joined us
as co-investigators on a proposal that has now been submitted to the NIHR. We are particularly pleased
that we were able to engage the continued input from the PPI members in this way and this reflects the
good relationships built up over the duration of the project.
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
65

In addition to the input at steering group meetings, we also were able to e-mail some of the PPI members
to ask their views on specific questions as they arose in the research. We also asked for, and incorporated,
their comments on the draft interim reports and the draft final report. Their involvement will continue
beyond the production of the report because we will seek their input into the dissemination activities still
to be undertaken, that is, the production of summaries targeted at decision-makers and service users and
also anticipate their help in accessing channels through which the summaries can be distributed.
One of the challenges we anticipated for involvement of our PPI members was that they may not feel able
to readily give their views at the steering group, given the range and background of other participants. We
tried to address this by (1) ensuring that we did not just have one PPI member but three; (2) by thinking
in advance of the meetings which were the key questions that would benefit most from their input, as
opposed to other areas on which we were seeking mainly clinical or academic input; and (3) chairing our
meetings in a way that was as relaxed and inclusive as possible.
Our payments to PPI members were made in accordance with the INVOLVE guidance. INVOLVE is a
national advisory group, funded by the NIHR, to support active public involvement in NHS research.
APPENDIX 1
NIHR Journals Library www.journalslibrary.nihr.ac.uk
66

Appendix 2 Further results
TABLE 27 Cross-sectional models: admissions for patients with a SMI
Admission
type and year
Joint modelling Independent modelling
MH6 MH9 MH6 MH9
IRR 95% CI IRR 95% CI IRR 95% CI IRR 95% CI
Admissions for a SMI
2006/7 1.096 0.969 to 1.240 1.343 1.124 to 1.604 1.269 1.151 to 1.399 1.453 1.263 to 1.671
2007/8 0.988 0.860 to 1.136 1.340 1.122 to 1.600 1.141 1.023 to 1.273 1.327 1.155 to 1.526
2008/9 0.988 0.857 to 1.140 1.229 1.030 to 1.467 1.092 0.970 to 1.228 1.219 1.053 to 1.411
2009/10 1.095 0.935 to 1.281 1.024 0.855 to 1.228 1.108 0.973 to 1.262 1.081 0.930 to 1.257
2010/11 0.957 0.815 to 1.124 1.192 0.995 to 1.428 1.040 0.905 to 1.195 1.164 0.996 to 1.359
Admissions for physical (elective) care
2006/7 1.312 0.887 to 1.942 1.234 0.741 to 2.053 1.458 1.112 to 1.912 1.559 1.095 to 2.221
2007/8 1.101 0.784 to 1.548 1.399 0.953 to 2.053 1.301 1.039 to 1.629 1.510 1.191 to 1.916
2008/9 1.011 0.834 to 1.226 1.180 0.930 to 1.498 1.096 0.944 to 1.272 1.189 0.988 to 1.431
2009/10 0.987 0.817 to 1.193 1.367 1.111 to 1.683 1.157 0.995 to 1.345 1.357 1.150 to 1.600
2010/11 0.957 0.814 to 1.124 1.211 1.020 to 1.439 1.049 0.913 to 1.207 1.182 1.018 to 1.373
Admissions for physical (emergency) care
2006/7 1.106 0.979 to 1.250 1.412 1.198 to 1.665 1.310 1.191 to 1.440 1.540 1.355 to 1.749
2007/8 1.029 0.899 to 1.178 1.485 1.246 to 1.770 1.250 1.125 to 1.389 1.520 1.326 to 1.742
2008/9 1.035 0.905 to 1.185 1.231 1.039 to 1.459 1.146 1.028 to 1.277 1.261 1.099 to 1.447
2009/10 1.073 0.928 to 1.241 1.249 1.062 to 1.469 1.201 1.063 to 1.357 1.303 1.137 to 1.494
2010/11 1.025 0.895 to 1.175 1.284 1.115 to 1.477 1.156 1.025 to 1.303 1.301 1.149 to 1.474
TABLE 28 Cross-sectional models: admissions for patients with bipolar disorder
Admission
type and year
Joint modelling Independent modelling
MH4 MH5 MH4 MH5
IRR 95% CI IRR 95% CI IRR 95% CI IRR 95% CI
Admissions for bipolar disorder
2006/7 1.115 0.821 to 1.513 1.054 0.855 to 1.298 1.151 0.877 to 1.511 1.083 0.898 to 1.305
2007/8 1.028 0.759 to 1.392 1.098 0.912 to 1.322 1.089 0.822 to 1.442 1.105 0.930 to 1.312
2008/9 0.999 0.720 to 1.387 0.934 0.767 to 1.138 0.955 0.715 to 1.275 0.934 0.784 to 1.113
2009/10 1.023 0.694 to 1.509 0.987 0.802 to 1.214 1.014 0.716 to 1.435 0.992 0.824 to 1.194
2010/11 1.176 0.801 to 1.727 1.008 0.818 to 1.243 1.183 0.827 to 1.692 1.039 0.854 to 1.264
DOI: 10.3310/hsdr03160 HEALTH SERVICES AND DELIVERY RESEARCH 2015 VOL. 3 NO. 16
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Jacobs et al. under the terms of a commissioning contract issued by the Secretary of State for
Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals
provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be
addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science
Park, Southampton SO16 7NS, UK.
67

TABLE 29 Results for covariates on SMI admissions model
Variable description IRR 95% CI
Achievement on MH6 QOF indicator 1.020 0.944 to 1.102
Achievement on MH9 QOF indicator 1.210 1.104 to 1.327
Indicator for whether GP practice is reimbursed under PMS, otherwise reimbursed under GMS 0.970 0.948 to 0.992
Proportion of male GPs in GP practice 1.031 0.986 to 1.078
Proportion of foreign GPs in GP practice 0.991 0.953 to 1.029
Mean age of GPs in GP practice 1.003 1.001 to 1.004
GP practice list size (second tertile –medium) 0.986 0.960 to 1.014
GP practice list size (third tertile –large) 0.936 0.904 to 0.970
Patient population: average age 0.990 0.985 to 0.994
Patient population: proportion of male patients 2.870 1.653 to 4.982
MH claimants –second quintile 1.056 1.018 to 1.095
MH claimants –third quintile 1.078 1.033 to 1.126
MH claimants –fourth quintile 1.108 1.059 to 1.159
MH claimants –fifth quintile 1.120 1.058 to 1.185
Proportion providing informal care, practice catchment area 0.150 0.030 to 0.758
NHS psychiatric residents per 1000 population, practice catchment area 1.000 0.989 to 1.011
Proportion of non-white ethnicity, practice catchment area 1.182 1.027 to 1.359
Proportion living in urban setting, practice catchment area 1.104 1.048 to 1.163
Proportion of practice patients able to access care within 48 hours 0.979 0.881 to 1.088
Distance (in miles) to closest acute hospital 0.995 0.992 to 0.998
Distance (in miles) to closest mental health hospital 1.003 1.000 to 1.005
Mean number of admissions between April 2004 and March 2006 1.021 1.018 to 1.024
Financial year 2007/8 1.003 0.979 to 1.028
Financial year 2008/9 0.928 0.905 to 0.951
Financial year 2009/10 0.931 0.906 to 0.958
Financial year 2010/11 0.786 0.760 to 0.812
GP-level variance 0.111 0.105 to 0.119
Number of observations 38,774
Note
Model includes hospital fixed effects but these are not reported.
APPENDIX 2
NIHR Journals Library www.journalslibrary.nihr.ac.uk
68

Part of the NIHR Journals Library
www.journalslibrary.nihr.ac.uk
Published by the NIHR Journals Library
This report presents independent research funded by the
National Institute for Health Research (NIHR). The views
expressed are those of the author(s) and not necessarily
those of the NHS, the NIHR or the Department of Health
EME
HS&DR
HTA
PGfAR
PHR