Content uploaded by Ivan Nyklíček
Author content
All content in this area was uploaded by Ivan Nyklíček on Sep 24, 2014
Content may be subject to copyright.
Open University Press
Buckingham • Philadelphia
RESEARCH METHODS
IN HEALTH
Investigating health and
health services
Second edition
Ann Bowling
Open University Press
Celtic Court
22 Ballmoor
Buckingham
MK18 1XW
email: enquiries@openup.co.uk
world wide web: www.openup.co.uk
and
325 Chestnut Street
Philadelphia, PA 19106, USA
First edition published 1997
Reprinted 1998, 1999, 2000
First published in this second edition 2002
Copyright © A. Bowling 2002
All rights reserved. Except for the quotation of short passages for the purpose
of criticism and review, no part of this publication may be reproduced, stored
in a retrieval system, or transmitted, in any form or by any means, electronic,
mechanical, photocopying, recording or otherwise, without the prior written
permission of the publisher or a licence from the Copyright Licensing Agency
Limited. Details of such licences (for reprographic reproduction) may be
obtained from the Copyright Licensing Agency Ltd of 90 Tottenham Court
Road, London, W1P 0LP.
A catalogue record of this book is available from the British Library
ISBN 0 335 20643 3 (pb) 0 335 20644 1 (hb)
Library of Congress Cataloging-in-Publication Data
Bowling, Ann.
Research methods in health: investigating health and health services / Ann
Bowling.– 2nd ed.
p. cm.
Includes bibliographical references and index.
ISBN 0-335-20644-1 (hb) – ISBN 0-335-20643-3 (pbk.)
1. Public health–Research–Methodology. 2. Community health services–
Research–Methodology. I. Title.
RA440.85 .B69 2001
362.1′07′2–dc21 2001045769
Typeset by Graphicraft Limited, Hong Kong
Printed in Great Britain by Biddles Limited, Guildford and King’s Lynn
Contents
Preface xv
Acknowledgements xvi
Section 1 Investigating health services and health: the scope of research 1
1 Evaluating health services: multidisciplinary
collaboration 5
Introduction 6
Health services research 6
The assessment of quality 7
Audit 8
Medical audit, clinical audit and quality assurance 8
Evaluation 9
Structure, process and outcome 10
Appropriateness and inappropriateness 11
Outcome 12
Summary of main points 15
Key questions 15
Key terms 16
Recommended reading 16
2 Social research on health: sociological and
psychological concepts and approaches 17
Introduction 18
Sociological and psychological research on health 18
1 Health and illness 19
The bio-medical model 19
The social model of health 20
Lay definitions of health 20
Lay theories of illness 21
Variations in medical and lay perspectives 22
2 Social factors in illness and responses to illness 23
Social variations in health: structural inequalities 23
Psycho-social stress and responses to stress 24
Stigma, normalisation and adjustment 27
The Sick Role and illness behaviour 29
3 Models of health behaviour 33
Health lifestyles 33
Health behaviour 34
Models of health-related actions 35
4 Health-related quality of life 38
Theoretical influences on measurement 39
5 Interactions between health professionals and
patients 41
Communication 41
Patients’ evaluations of health care 42
Summary of main points 51
Key questions 53
Key terms 53
Recommended reading 53
3 Health needs and their assessment: demography and
epidemiology 54
with Ian Rees Jones
Introduction 55
1 The assessment of health needs 55
Health needs 55
The need for health and the need for health care 56
Methods of assessing health needs 59
The role of epidemiological and demographic research 62
2 Epidemiology 63
The role of epidemiology 63
Epidemiological research 64
Methods of epidemiology 66
Assessing morbidity, mortality, incidence and prevalence 73
3 The role of demography 77
Demographical methods in relation to assessing need 78
Rates: births and deaths 79
The need to standardise 80
Analyses of survival 83
Summary of main points 86
Key questions 87
vi Contents
Key terms 87
Recommended reading 88
4 Costing health services: health economics
with Ian Rees Jones 89
Introduction 90
Health economics 90
Demand, utility and supply 92
Economic appraisal 93
Cost minimisation 94
Cost-effectiveness 94
Cost–benefit analysis 95
Marginal cost 96
Complete costs 96
Event pathways 97
Opportunity cost 98
Discounting 100
Cost–utility analysis 100
Cost–utility analysis and economic valuations of health 101
Costing health services 107
Study methods used for costings 111
Summary of main points 112
Key questions 113
Key terms 113
Recommended reading 113
Section II The philosophy, theory and practice of research 115
5 The philosophical framework of measurement 117
Introduction 118
The philosophy of science 118
Paradigms 119
Objectivity and value freedom 119
Deductive and inductive approaches 120
The survival of hypotheses and paradigm shifts 124
Theoretical influences on social research methods 125
Social science and grounded theory 125
Positivism 126
Phenomenology 128
Choice of methods 130
Summary of main points 131
Key questions 132
Key terms 132
Recommended reading 132
Contents vii
6 The principles of research 133
Introduction 134
Searching the literature 134
Literature reviews 136
Meta-analyses 136
Rigour 138
Aims, objectives and hypotheses 138
Concepts and theories 139
Research proposals 141
Research design and research methods 143
Selection of measurement instruments 144
Level of data and statistical techniques 144
Reliability and validity 147
Threats to reliability and validity 153
Ethics and ethical committees 156
Dissemination 159
Summary of main points 160
Key questions 161
Key terms 162
Recommended reading 162
Section III Quantitative research: sampling and research methods 163
7 Sample size and sampling for quantitative research 165
Introduction 166
1 Calculation of sample size, statistical significance
and sampling 166
The sampling unit 166
Calculation of sample size and statistical power 167
Testing hypotheses, statistical significance, the null
hypothesis 169
Type I and type II errors 170
One- or two-sided hypothesis testing 172
Statistical, social and clinical significance 173
Sampling frames 174
Sampling 176
Confidence intervals and the normal distribution 177
External validity of the sample results 182
2 Methods of sampling 183
Random sampling 183
Non-random sampling: quota sampling 186
Sampling for qualitative research 187
Sampling for telephone interviews 188
viii Contents
Summary of main points 189
Key questions 191
Key terms 191
Notes 191
Recommended reading 192
8 Quantitative research: surveys 193
Introduction 194
1 Survey methods 194
The survey 194
Descriptive and analytic surveys 195
Retrospective (ex post facto), cross-sectional surveys 197
Prospective, longitudinal surveys 197
Cross-sectional and longitudinal cohort studies 200
Triangulated research methods and surveys 201
2 Methods of analysing change in longitudinal
surveys 202
Analysing change 202
Sample attrition and analysing change 209
Stopping rules and analysis of interim results 212
Summary of main points 212
Key questions 213
Key terms 213
Recommended reading 214
9 Quantitative research: experiments and other
analytic methods of investigation 215
Introduction 216
The experimental method 216
Internal and external validity 219
Reducing bias in participants and the investigating team 223
Blind experiments 224
The randomised controlled trial in health care evaluation 225
Other analytic methods of investigation 228
Before–after study with non-randomised control group 231
After-only study with non-randomised control group 231
Time series studies using different samples (historical
controls) 232
Geographical comparisons 232
People acting as own controls 233
Within-person, controlled site study 233
Threats to the validity of causal inferences in other
analytic studies 233
Summary of main points 234
Key questions 235
Contents ix
Key terms 235
Recommended reading 236
10 Sample selection and group assignment methods in
experiments and other analytic methods 237
Introduction 238
Random sampling 238
Convenience and purposive sampling 238
Volunteers 239
Type of investigation and type of sampling frame 239
Response rates: experiments and other analytical studies 240
Ensuring similarity in group characteristics: random
allocation 241
Other allocation methods: cross-over methods 247
Methods of group design for improving the basic RCT 248
Common methods of controlling to obtain equivalence in
non-randomised studies 250
Summary of main points 252
Key questions 253
Key terms 253
Recommended reading 253
Section IV The tools of quantitative research 255
11 Data collection methods in quantitative research:
questionnaires, interviews and their response rates 257
Introduction 258
Structured and semi-structured questionnaires 258
Postal questionnaires and self-administration 259
Structured and semi-structured interviews 260
Non-response 263
Item non-response 269
Summary of main points 271
Key questions 271
Key terms 272
Recommended reading 272
12 Questionnaire design 273
Introduction 274
Planning 274
Piloting 275
Questionnaire layout 275
The covering letter 277
Question form, order and wording 278
xContents
Rules for form 278
Scores 286
Constructing additional items and scales 287
Attitude measurement scales 288
Rules for order and wording 294
Rules for questions by type of topic 300
Checking the accuracy of responses 306
Translating an instrument and cultural equivalence 307
Summary of main points 308
Key questions 309
Key terms 309
Recommended reading 309
13 Techniques of survey interviewing 310
Introduction 311
Types of interview 311
The interviewer 311
Interviewer handbooks 313
Sampling by interviewers 315
Interviewer training 316
Interviewer bias 317
Persistence in contacting respondents 318
Approaching respondents 319
Motivating people to respond 320
Third parties and distractions 322
Beginning the interview 323
Rules for structured interviewing 324
Interviewing techniques 326
The end of the interview 330
Recording responses 331
Debriefing 332
Quality control 332
Summary of main points 333
Key questions 333
Key terms 334
Recommended reading 334
14 Preparation of quantitative data for coding and
analysis 335
Introduction 336
Coding 336
Coding transfer sheets 338
The code book 339
Numerical values for codes 340
Coding open questions 341
Coding closed questions 343
Contents xi
Checking returned questionnaires 343
Data entry on to computer 344
Cleaning the data 345
Checking for bias in the analyses 346
Missing values and data checks 346
Computer packages for the analysis of quantitative data 347
The analysis 348
Summary of main points 349
Key questions 350
Key terms 350
Recommended reading 350
Section V Qualitative and combined research methods, and
their analysis 351
15 Unstructured and structured observational studies 355
Introduction 356
Observation 358
Participant observation 359
Gaining access 360
Hardware: video- and audiotapes 361
Establishing validity and reliability 362
Observation and triangulated methods 364
Structured observations: what to record 364
Time sampling 366
Recording observed (non-verbal) body language 366
Unstructured observations 367
Combining structured and unstructured recordings 368
Theoretical analysis of observational data 369
Categorisation of observational data 371
Narratives 373
Audio-observation: conversation sampling 373
Recording and analysing verbal communication 374
Summary of main points 375
Key questions 376
Key terms 376
Recommended reading 376
16 Unstructured interviewing and focus groups 377
Introduction 378
1 Unstructured interviews 378
Types of unstructured, in-depth interview 378
In-depth interviewing: sample selection and size 379
The process of the interview 381
xii Contents
Techniques of in-depth interviewing 382
Analysis and presentation of in-depth interview data 385
Categorising qualitative data: content analysis 387
Rules for coding 389
Computer programs for analysing qualitative data 391
Narrative format 392
2 Focus groups 394
Focus group interviews 394
Group composition 395
Appropriate topics for focus groups 395
Methods of analysis 398
Summary of main points 399
Key questions 400
Key terms 400
Recommended reading 401
17 Other methods using both qualitative and
quantitative approaches: case studies, consensus
methods, action research and document research 402
Introduction 403
1 Case studies 403
The study of single or small series of cases 403
Examples of case studies 404
The analysis 406
2 Consensus methods 406
Methods for establishing and developing consensus 406
Delphi technique 407
Consensus development panels 407
Nominal group process 407
The analysis 409
3 Action research and rapid appraisal techniques 410
Action research 410
Stages of action research 412
Rapid appraisal 414
4 Document research 415
Documents as sources of, or for, research 415
Types of documents 416
Authenticity, bias, error and interpretation 418
Types of document research 419
Analysis of documents 421
Diary methods 424
Analysis of diaries 427
Summary of main points 427
Key questions 428
Contents xiii
Key terms 429
Recommended reading 429
Glossary 430
References 439
Index 479
xiv Contents
Preface to the second edition
This book is more than a text on research methods. It is an introduction
to the theoretical concepts, as well as the descriptive and analytic research
methods, that are used by the main disciplines engaged in research on
health and health services. In order to understand why the various re-
search methods are used, it is important to be aware of the conceptual
backgrounds and scientific philosophies of those involved in research
and evaluation, in particular in demography, epidemiology, health eco-
nomics, psychology and sociology.
The second edition, while essentially similar to the first edition, in-
cludes updated references and reference to key methodological develop-
ments (for example, cluster randomisation, utility analyses, patients’
preferences and perception of risk). The book is aimed at students and
researchers of health and health services, health professionals and the
policy-makers who have the responsibility for applying research findings,
and who need to know how to judge the soundness of that research.
The idea for the book, and its structure, is grounded in my career as a
researcher on health and health service issues, and the valuable experi-
ence this has provided in meeting the challenges of research on people
and organisations in real life settings.
The varying terminology used by members of different disciplines in
relation to the same research methods is often confusing. This variation
simply reflects the multidisciplinary nature of this whole area, and the
specialised languages of each discipline. While no descriptor can be
labelled as incorrect, the multitude of them, especially when not clearly
defined, can easily lead to confusion. Therefore, I have tried to justify
the terminology used where it differs from that in other disciplines.
Towards the end of the book I have included a Glossary. The first
time each entry appears in the text it is highlighted in bold.
Section 1
Investigating health services
and health: the scope of
research
‘Would you tell me, please, which way I ought to go from
here?’, asked Alice.
‘That depends a good deal on where you want to get to,’ said
the cat.
Lewis Carroll (1865) Alice’s Adventures in Worderland
INTRODUCTION
Research is the systematic and rigorous process of enquiry which aims
to describe phenonema and to develop and test explanatory concepts and
theories. Ultimately it aims to contribute to a scientific body of know-
ledge. More specifically, in relation to the focus of this book, it aims to
improve health, health outcomes and health services.
This book aims to provide an overview of the range of research
methods that are used in investigations of health and health services.
Ultimately it is to guide the reader into choosing an appropriate research
method and design in order to address a particular research question.
However, it is not possible to place research methods in a hierarchy of
excellence, as different research methods are appropriate for addressing
different research questions.
If the research question is descriptive, for example ‘what is the health
status of population X?’, then a cross-sectional survey of a sample of that
population is required to provide population estimates. The survey
method will also enable the answers to secondary questions to be esti-
mated for that population (e.g. ‘Are men more likely than women to
report poor health status?’) and certain (non-causal) types of hypotheses
to be tested (e.g. ‘Men will be X times more likely than women to
report good health status’). If the research question is ‘Do women have
worse health outcomes than men following acute myocardial infarction
(AMI)?’ then a prospective, longitudinal survey of identified men and
women who had suffered an AMI would be undertaken in order to be
able to compare their health outcomes over time in the future.
2Investigating health services and health: the scope of research
If the research aims to find out information on a topic about which
little is known, or is too complex or sensitive for the development of
standardised instruments, then qualitative methods (e.g. observational
methods, in-depth interviews and/or focus groups) may be more appro-
priate (e.g. ‘Is there quality of life on long-stay psycho-geriatric wards?’
‘Are there dehumanising care practices in long stay institutions?’ ‘How
do doctors prioritise their patient caseload?’).
And if the research aims to investigate cause-and-effect issues then an
experimental design is, in theory, required (e.g. ‘Do women aged 75+
have worse health outcomes than men aged 75+ following thrombolysis
therapy for acute myocardial infarction?’ ‘Do patients with osteoarthritis
of the knee benefit from physiotherapy?’ ‘Are specialists’ outreach clinics
held in general practitioners’ surgeries as cost-effective as specialists’
out-patient clinics in hospitals?’). While the double-blind, randomised
controlled trial is the true experimental design, and most appropriate
for addressing these types of questions, there are also situations in
which this method is unrealistic, impractical or inappropriate and other
well-designed analytic (as opposed to descriptive) methods have to be
employed instead (see Chapter 9). For some cause-and-effect questions,
the RCT may be the most appropriate research design but it would be
unethical to randomise people to interventions that are unacceptable,
and the issue must therefore be addressed using other methods, such as
a prospective, longitudinal survey of a population (e.g. ‘Does drinking
spirits increase the risk of heart disease?’).
Finally, research methods should not be seen in isolation from each
other. A triangulated or combined methodological approach to address-
ing different facets of a research issue, using different methods which
complement each other, is increasingly recommended as a means of
establishing the external validity of the research. In the same way in
which prospective, longitudinal surveys can inform the results from
randomised controlled trials, so qualitative research findings can enhance
quantitative survey data by placing the latter into real social contexts and
enhancing understanding of relevant social processes.
The importance of using triangulated research methods is enhanced
by the multifaceted nature of health, and the multidisciplinary character
of research on health and health services. This includes investigations by
anthropologists, demographers, epidemiologists, health economists,
health geographers, health policy analysts, health psychologists, histor-
ians, medical sociologists, statisticians and health professionals (clinicians,
nurses, physiotherapists and so on). Specialists in public health medicine
play a key role in health services research, as they are equipped with a
range of research skills, including epidemiology. In Britain and in some
other countries, they also have responsibility for assessing needs for
health services in specific geographical areas, and advising purchasers
on effective health care. There is a close working relationship between
researchers investigating health and health services and health profes-
sionals, particularly in relation to the development of measures of clinical
outcomes and the appropriateness of health care interventions.
Evaluating health services: multidisciplinary collaboration 3
One consequence of this multidisciplinary activity is that a wide range of
qualitative and quantitative, descriptive and analytical research methods
is available. This diversity should enrich the approach to research design,
although there has been a tendency in research on health services to
focus mainly on the experimental method. All methods have their prob-
lems and limitations, and the over-reliance on any one method, at the
expense of using multiple research methods, to investigate the phenom-
enon of interest can lead to ‘a very limited tool box’ (Pope and Mays
1993), sometimes with questionable validity (Webb et al. 1966), and
consequently to a limited understanding of the phenomena of interest.
It is necessary at this point to distinguish between the terms health
research and health services research.
Health research
Health research has been defined in relation to health generally. As well
as having an emphasis on health services, it has an important role in
informing the planning and operation of services aiming to achieve health
(Hunter and Long 1993):
the process for obtaining systematic knowledge and technology
which can be used for the improvement of the health of individual
groups. It provides the basic information on the state of health and
disease of the population; it aims to develop tools to prevent and
cure illness and mitigate its effects and it attempts to devise better
approaches to health care for the individual and the community.
(Davies 1991)
The broader aspects of health research are described in Chapters 2 and
3 (e.g. in relation to health needs and sociological and psychological
aspects of health).
Health systems and health services research
Health systems research has been defined fairly broadly as: ‘ultimately
concerned with improving the health of a community, by enhancing the
efficiency and effectiveness of the health system as an integrated part of the
overall process of socio-economic development’ (Varkevisser et al. 1991).
In Britain and the USA the general focus is on health services re-
search, rather than on health systems research. Health services research
is defined more narrowly in relation to the relationship between health
service delivery and the health needs of the population: for example, as
‘the identification of the health care needs of communities and the study
of the provision, effectiveness and use of health services’ (Medical
Research Council, see Clarke and Kurinczuk 1992). While there is an
overlap with health research, health services research needs to be trans-
lated into action to be of value and should ‘transcend the R (acquiring
knowledge) and the D (translating that knowledge into action) divide’
(Hunter and Long 1993).
Investigating health services and health: the scope of research 3
4Investigating health services and health: the scope of research
Each of these definitions emphasises the multidisciplinary nature of
health research, health systems research and health services research.
Health services research, for example, has been described as ‘a space
within which disciplines can meet’ (Pope 1992), and as an area of applied
research, rather than a discipline (Hunter and Long 1993).
Within these definitions, the topics covered in Chapters 1, 3 and 4, on
evaluating health services, health needs and their assessment (the latter
also comes within the definition of broader health research) and the
costing of health services, are encompassed by health services research.
Chapter 2, on social research on health, also falls within both health
research and health services research. Not everyone would agree with
these definitions and distinctions. For example, some might categorise
the assessment of needs as health research rather than health services
research. What is important is not the distinctions and overlaps between
these branches of research, but a respect for each discipline in relation to
its contribution to a multidisciplinary body of knowledge about health
and disease, health systems as a whole and health services.
Finally, it should be pointed out that research on health services is not
insulated from the society within which it is placed. It is often respons-
ive to current policy and political issues (see Cartwright 1992), and is
thus dependent upon decisions taken by others in relation to research
topics and research funding. While it is common for researchers to initi-
ate new research ideas, much of the funding for this research comes
from government bodies, who tend to prioritise research and develop-
ment on a local or national basis. The research topics are rarely value
free. The research findings are also disseminated to members of a wide
range of professional and management groups. In relation to this
multidisciplinary nature, the agenda for research and the consumers of
the research findings, it contrasts starkly with the traditional biomedical
model of research.
Evaluating health services: multidisciplinary collaboration 5
1 Evaluating health services:
multidisciplinary collaboration
Introduction 6
Health services research 6
The assessment of quality 7
Audit 8
Medical audit, clinical audit and quality assurance 8
Evaluation 9
Structure, process and outcome 10
Appropriateness and inappropriateness 11
Outcome 12
Summary of main points 15
Key questions 15
Key terms 16
Recommended reading 16
6Investigating health services and health: the scope of research
Introduction
Research on health and health services ranges from descriptive investiga-
tions of the experience of illness and people’s perceptions of health and
ill health (known as research on health, or health research) to evaluations
of health services in relation to their appropriateness, effectiveness and
costs (health services research). However, these two areas overlap and
should not be rigidly divided, as it is essential to include the perspective
of the lay person in health service evaluation and decision-making. Other
related fields of investigation include audit, quality assurance, and the
assessment of needs for health services (usually defined in terms of the
need for effective services), which comes within the umbrella of health
research but also has a crucial link with health services research. Audit
and quality assurance are not strictly research in the sense of contributing
to a body of scientific knowledge and adherence to rigorous methods
of conducting research (quantitative or qualitative). Instead they are con-
cerned with monitoring in order to ensure that predefined standards of
care are met. They are increasingly important activities with the emphasis
on clinical governance in health care (Lugon and Secker-Walker 1999).
They are described briefly below with the other main areas of research
activity.
Health services research
It was explained in the introduction to Section I that health services
research is concerned with the relationship between the provision,
effectiveness and efficient use of health services and the health needs of
the population. It is narrower than health research. More specifically,
health services research aims to produce reliable and valid research data
on which to base appropriate, effective, cost-effective, efficient and
acceptable health services at the primary and secondary care levels. Thus,
the research knowledge acquired needs to be developed into action if the
discipline is to be of value; hence the emphasis throughout industry and
service organisations on ‘research and development’. The focus is gener-
ally on:
• the relationships between the population’s need and demand for
health services, and the supply, use and acceptability of health
services;
• the processes and structures, including the quality and efficiency, of
health services;
• the appropriateness and effectiveness of health service interventions, in
relation to effectiveness and cost-effectiveness, including patients’ per-
ceptions of outcome in relation to the effects on their health, health-
related quality of life and their satisfaction with the outcome.
Evaluating health services: multidisciplinary collaboration 7
These areas of research are addressed in more detail in this chapter and in
the other chapters included in Section I.
Health services research is distinct from audit and quality assurance,
although they share the same concepts in relation to the evaluation of
structure, process and outcome. Audit and quality assessment aim to
monitor whether predefined and agreed standards have been met. Health
services research has evaluation – rather than monitoring – as its aim.
Health services research is also broader than traditional clinical research,
which directly focuses on patients in relation to their treatment and care.
Clinical research has traditionally focused on biochemical indicators, and
more recently, and in selected specialties only, on the measurement of
the broader quality of life of the patients. Health services research invest-
igates the outcome of medical interventions from social, psychological,
physical and economic perspectives. It has also been cogently argued
that health services research should be concerned with the evaluation of
the health sector in the broadest sense, and not limited to health services
alone (Hunter and Long 1993).
Quality assessment and audit will be described next, followed by the
concepts central to the latter and to health services research: the evalu-
ation of the structure, process and outcome, including appropriateness, of
health services.
The assessment of quality
The quality of care can be defined in relation to its effectiveness with
regard to improving the patient’s health status, and how well it meets
professionals’ and the public’s standards about how the care should be
provided (Donabedian 1980). Higginson (1994) stated that quality of
care needs to include effectiveness, acceptability and humanity, equity,
accessibility and efficiency. Building on work by Shaw (1989) and Black
(1990), she defined quality of health care in broad terms:
• effectiveness (achieving the intended benefits in the population, under
usual conditions of care);
• acceptability and humanity (to the consumer and provider);
• equity and accessibility (the provision and availability of services to
everyone likely to benefit (in ‘need’));
• efficiency (greatest benefit for least cost).
Higginson adds that patient empowerment might also be included, in
order that they may increase their control over the services received, and
each patient should be offered care that is appropriate.
Quality is clearly relevant to health services research. Quality assur-
ance and medical and clinical audit are all initiatives to establish and
maintain quality in health care, and also involve the evaluation of struc-
ture, process and outcome in relation to quality.
8Investigating health services and health: the scope of research
Audit
Audit is directed at the maintenance and achievement of quality in health
care. Audit aims to improve patient outcome, to develop a more cost-
effective use of resources and to have an educational function for health
professionals. In theory, it should lead to change in clinical practice by
encouraging a reflective culture of reviewing current practice, and by
inducing changes which lead to better patient outcomes and satisfaction.
Suggested criteria for undertaking an audit include: the issue addressed
should be a common, significant or serious problem; any changes fol-
lowing audit should be likely to benefit patients and to lead to greater
effectiveness; the issue is relevant to professional practice or develop-
ment; there is realistic potential for improvement; and the end result is
likely to justify the investment of the time and effort involved (Clinical
Resource and Audit Group 1994). Investigators of audit have reported
that most audit has focused on process, rather than structure or out-
comes (e.g. Packwood 1995).
Medical audit, clinical audit and quality assurance
Audit consists of reviewing and monitoring current practice, and evalu-
ation (comparison of performance) against agreed predefined standards
(Standing Committee on Postgraduate Medical Education 1989). It is
divided into medical and clinical audit, and is related to quality assur-
ance. These have become commonplace in the British National Health
Services (NHS) and are now built into the structure of provider units
(e.g. hospitals and, increasingly, general practice). These three concepts
have been clarified by Higginson (1994) as follows.
• Medical audit is the systematic critical analysis of the quality of medical
care, including a review of diagnosis, and the procedures used for
diagnosis, clinical decisions about the treatment, use of resources and
patient outcome (Secretaries of State for Health, Wales, Northern Ire-
land and Scotland 1989a). Examples of medical audit include analyses
of avoidable deaths, and the assessment of medical decision-making,
resources and procedures used in relation to patient outcome.
• Clinical audit is conducted by doctors (medical audit) and other health
care professionals (e.g. nurses, physiotherapists, occupational and speech
therapists), and is the systematic critical analysis of the quality of clin-
ical care. It includes collecting information to review diagnosis and the
procedures used for diagnosis, clinical decisions about the treatment,
use of resources and patient outcome (Secretaries of State for Health,
Wales, Northern Ireland and Scotland 1989a).
• Quality assurance is a clinical and management approach which in-
volves the systematic monitoring and evaluation of predefined and
agreed levels of service provision. Quality assurance is the definition
Evaluating health services: multidisciplinary collaboration 9
of standards, the measurement of their achievement and the mechan-
isms employed to improve performance (Shaw 1989). Medical and
clinical audit is usually one part of a quality assurance programme.
Quality assurance usually implies a planned programme involving the
whole of a particular health service.
Audit can be carried out internally by organisations, members of a discip-
line (peer review), individuals who systematically review their work or
that of their teams, or external bodies (e.g. purchasers for contract mon-
itoring, or professional bodies). Certain criteria need to be met for con-
ducting successful audit: for example, effective clinical leadership; strategic
direction (vision, strategy, objectives and planning); audit staff and sup-
port (e.g. high calibre, right skill mix, reward, staff development); basic
structures and systems (e.g. business planning); training and education;
understanding and involvement (e.g. communication, leadership and so on);
organisational environment (e.g. structure, relationships) (Walshe 1995).
The process of audit involves multiple methods, such as document
searching and analysis (e.g. analysis of complaints files, random or sys-
tematic selection of nursing and medical records for routine reviews),
analysis of routine data, clinical case reviews and presentations in team
meetings (see Hopkins 1990 for review). It can also include the collec-
tion of information by focus groups of patients or by questionnaire,
e.g. patient satisfaction, patient assessed outcome (see Riordan and
Mockler 1996 for an example of this in an audit of a psychogeriatric
assessment unit). While quantitative research methodology is most
appropriate for audit, much can also be gained by supplementing this
with qualitative methods such as observation
(e.g. visits to wards and
clinics to assess quality by observation). The design of audits should also
aim to be scientifically and methodologically rigorous (Russell and Wilson
1992; Department of Health 1993b).
Evaluation
Evaluation is the use of the scientific method, and the rigorous and
systematic collection of research data to assess the effectiveness of organ-
isations, services and programmes (e.g. health service interventions)
in achieving predefined objectives (Shaw 1980). Evaluation is central to
health services research and audit. Evaluation is more than audit because
it aims to record not only what changes occur, but also what led to those
changes. Evaluation can be divided into two types: formative and
summative. Formative evaluation involves the collection of data while
the organisation or programme is active, with the aim of developing or
improving it. Summative evaluation involves collecting data about the
active (or terminated) organisation or programme with the aim of decid-
ing whether it should be continued or repeated (a health promotion
activity or screening programme) (Kemm and Booth 1992).
10 Investigating health services and health: the scope of research
Structure, process and outcome
The evaluation of health services is usually based on the collection of
data about the structure, inputs, process, outputs and outcomes of the
service (Donabedian 1980). Structure refers to the organisational frame-
work for the activities; process refers to the activities themselves; and
outcome refers to the impact (effectiveness) of the activities of interest
(e.g. health services and interventions) in relation to individuals (e.g.
patients) and communities. Health outcome relates to the impact of the
service on the patient (effectiveness). The structure and process of ser-
vices can influence their effectiveness. These concepts have been clearly
described in relation to the evaluation of health services by St Leger et al.
(1992).
Thus, it is often necessary to measure structure and process in order
to interpret the outcome of the care. For example, the collection of qual-
itative and quantitative descriptive data about process and structure is
essential if the investigator wishes to address the question of whether –
and how – the outcome was caused by the activity itself, and/or by
variations in the structure, or the way it was organised or delivered
(process). These data can enhance the influence of the research results.
These concepts, and their operationalisation, are described below.
Structure and inputs
The structure of an organisation refers to the buildings, equipment,
staff, beds and so on needed to meet defined standards. The assessment
of quality will be in relation to their numbers, type and suitability. It is
represented in economic terms by its fixed costs (see Chapter 4). The
operationalisation of this concept requires measurement of the raw mater-
ials forming the inputs. These can be operationalised in relation to the
distribution of staff, their mix in relation to level of training, grade and
skill, availability, siting and type of buildings (e.g. hospitals, clinics
and types), facilities and equipment, numbers and types of services,
consumables (e.g. medication) used and other types of capital and financial
resources.
Data on structure and inputs can be obtained by questionnaire and
document analysis. The study design might be a descriptive survey or
the data might be collected within an experimental design comparing
organisations in relation to outcome.
Process and outputs
The process refers to how the service is organised, delivered and used. It is
assessed in medical audit in relation to deviation from predefined and agreed
standards. It includes accessibility (e.g. proximity to public transport,
Evaluating health services: multidisciplinary collaboration 11
waiting lists), the way in which personnel and activities interact, and
interaction between personnel and patients. In other words, it is the
documentation and analysis of dynamic events and interactions. Data on
processes are essential for the evaluation of whether scarce health service
resources are used efficiently.
The types of data to be collected include outputs (e.g. the activities
that occur through the use of the resources in the system). These can be
operationalised in relation to rates of hospital discharge, number and
type of supplies given (e.g. medication, equipment), the number of
patient–professional contacts and their type, the number of home visits,
average lengths of hospital stay, length of consultation, medical and
surgical intervention rates, waiting lists and waiting times. Donabedian
(1980) included accessibility as a process indicator (e.g. levels of use by
different population groups, adequacy and appropriateness of services
provided). The analysis of process also involves the collection of data
about the quality of the relationship, and communications, between
professional and professional, and professional and patient (e.g. timely
provision of information to general practitioners (GPs) about their
patients’ treatment/discharge, provision of information to patients), plans
or procedures followed and documentation.
Some of the information can be extracted from records and, increas-
ingly, computer databases, combined with checks with patients and pro-
fessionals in relation to its accuracy and completeness. Alternatively, it
can be collected by asking patients to provide the information. Appro-
priate methods include questionnaire surveys and document analyses.
Appropriateness and inappropriateness
Appropriateness
Appropriateness is relevant to outcome. Appropriateness of health care
interventions has been variously defined. Investigators at Rand in the
USA defined it in terms of whether the expected health benefit of the
procedure exceeds its expected negative health consequences by a suffi-
ciently wide margin to justify performing the procedure, excluding con-
siderations of financial cost (Chassin 1989). The view of the British
NHS Executive is that appropriateness of care refers to the selection, on
the basis of the evidence, of interventions of demonstrable effectiveness
that are most likely to lead to the outcome desired by the individual
patient (Hopkins 1993). The definition used in Britain often includes
consideration of resources (Chantler et al. 1989; Maxwell 1989), and of
the individuality of the patient. There is no consensus internationally on
a definition of appropriateness.
The emphasis in health services research is on the measurement of the
appropriateness of, as well as the effectiveness of, interventions in the
12 Investigating health services and health: the scope of research
broadest sense. Policy-makers, purchasers and providers of health ser-
vices aim, in theory, to identify the most appropriate treatments and
services to deliver and purchase (outcome assessment) and the level of
need in the population for the interventions, and to monitor their provi-
sion and mode of delivery (measurement of processes and structure).
Patients themselves also want to know whether the treatment will work
and whether they will recover – as well as where to go for their treat-
ment. The difficulties at policy level stem from the relative dearth of
research data on appropriateness and effectiveness. Appropriateness is
not limited to interventions, but also applies to organisational factors.
For example, there is an increasing literature on the appropriateness of
length of hospital inpatient stays (Houghton et al. 1997).
Inappropriateness
All medical treatments aim to save or prolong life, to relieve symptoms,
to provide care and/or to improve health-related quality of life. How-
ever, the assessment of health outcomes and appropriateness of treat-
ments has been given impetus by the increasing evidence about high
rates of inappropriate treatments. For example, in the USA, relatively
high levels of inappropriateness rates have been found in relation to
surgical interventions for coronary heart disease (Chassin et al. 1987;
Winslow et al. 1988; Smith 1990). High levels of inappropriate care
and wide variations in practice (e.g. intervention rates) have been docu-
mented in the UK in relation to various procedures (Brook et al. 1988;
Anderson and Mooney 1990; Coulter et al. 1993). While Brook (1994)
argued that there is too much literature on medical practice for doctors to
assimilate routinely, it is also the case that there is insufficient research
evidence on the appropriateness of many medical interventions. Methods
for developing consensus on appropriateness criteria are described in
Chapter 17.
Outcome
Health service outcomes are the effects of health services on patients’
health (e.g. their health gain) as well as patients’ evaluations of their
health care. Reliable and valid information on outcomes of health ser-
vices is essential for audit, as well as for purchasing policies. Donabedian
(1980) defined health outcome as a change as a result of antecedent health
care. This is a narrow definition, although widely used, and excludes the
maintenance of patients in a stable condition, which can also be a valid
aim of treatment. It also excludes many health promotion and preven-
tion activities. Outcome refers to the effectiveness of the activities in
relation to the achievement of the intended goal. Purchasing debates in
health care have focused on health care costs in relation to broader ‘health
Evaluating health services: multidisciplinary collaboration 13
gains’ or ‘benefits’ from the treatments and interventions that are being
contracted for.
There is similar debate about the definition and measurement of out-
come in relation to social care and input from social services. Outcome
is more complex in the context of social care, and also in the case of
long-term health care, than it is with specific, time-limited treatments
and interventions. In relation to social care, and long-term health care,
the objective is to measure what difference this made to the recipient’s
life in the broadest sense (Qureshi et al. 1994).
Health outcome measurement has traditionally focused on survival
periods, toxicity, biochemical indicators and symptom rates, relapses,
various indicators of physical and psychological morbidity, and easily
measured social variables (e.g. days off work or school, number of bed
days, hospital readmission rates, other indicators of health service use).
Lohr (1988) defined outcome in relation to death, disease, disability,
discomfort and dissatisfaction (‘the five Ds’), and argued that measure-
ment instruments should focus on each of these concepts. However, the
trend now is to incorporate positive indicators (e.g. degrees of well-
being, ability, comfort, satisfaction), rather than to focus entirely on
negative aspects of outcome.
Broader measures of outcome
In health and social services research, more positive criteria of quality of
life are increasingly being incorporated into the broader assessment of
outcome. Treatment and care need to be evaluated in terms of whether
they are more likely to lead to an outcome of a life worth living in
social, psychological and physical terms. Health and ill health is a con-
sequence of the interaction of social, psychological and biological events
(sometimes called the biopsychosocial model of ill health). Thus each of
these elements requires measurement in relation to: patients’ perceived
health status and health-related quality of life (physical, psychological
and social); reduced symptoms and toxicity; and patients’ (and carers’
where appropriate) satisfaction with the treatment and outcome (see
Chapter 2). Thus, the assessment of outcome needs to incorporate both
the medical model and the patient’s perspective.
Health and health-related quality of life
Health status and health-related quality of life are two distinct concep-
tual terms which are often used interchangeably. Health status is one
domain of health-related quality of life. The definition of health status
traditionally focused on physical morbidity and mental health, and was
negative in its operationalisation. Because the current usage of health
status implies a multifaceted concept, it overlaps with the broader concept
14 Investigating health services and health: the scope of research
of health-related quality of life. Both can encompass physical health (e.g.
fitness, symptoms, signs of disease and wellness), physical functioning
(ability to perform daily activities and physical roles), social functioning
and social health (relationships, social support and activities), psycho-
logical well-being (depression, anxiety), emotional well-being (life satis-
faction, morale, control, coping and adjustment) and perceptions. It is
increasingly accepted that an instrument which encompasses the above
domains is measuring health-related quality of life, rather than a nar-
rower aspect of physical or mental health status (see WHOQOL Group
1993; Bowling 1995b, 1997). In addition, the concepts of perceived health
status, quality of life and health-related quality of life can be complex
to analyse as they might be mediated by several interrelated variables,
including self-related constructs (e.g. self-efficacy, self-esteem, perceived
control over life) and subjective evaluations could be influenced, in theory,
by cognitive mechanisms (e.g. expectations of life, level of optimism or
pessimism, social and cultural values, aspirations, standards for social
comparisons of one’s circumstances in life). Few investigators have taken
these variables, and their interplay, into account, although associations
between expectations of treatment and patient outcome, and between
level of optimism and patients’ coping strategies, have been reported
(Higginson 2000; Koller et al. 2000).
Health-related quality of life as an outcome measure broadens out-
come towards considering the impact of the condition and its treatment
on the person’s emotional, physical and social functioning and lifestyle.
It addresses the question of whether the treatment leads to a life worth
living, and it provides a more subjective, patient-led baseline against
which the effects of interventions can be evaluated. It can only do this,
however, if the measurement scale reflecting its components is valid,
reliable, precise, specific, responsive to change and sensitive. A universal
questionnaire to elicit the relevant information for a number of condi-
tions would need to be of enormous length. Disease-specific quality of
life scales are needed, not simply for greater brevity, but to ensure sen-
sitivity to sometimes small, but clinically significant, changes in health
status and levels of disease severity. A quality of life measure used in
research on health and health care should be able to inform the investig-
ator of the effects of the condition or treatment on the patient’s daily, as
well as long-term, life. It should also be capable of providing informa-
tion on whether, and to what extent, any gains in survival time among
patients with life-threatening conditions are at the expense of reductions
in quality of life during the period of the treatment and in the long term.
A disease-specific, or condition-specific instrument will have a nar-
rower focus generally, but contain more details of relevance to the area
of interest. If the investigator is interested in a single disease or condi-
tion, then a disease-specific indicator is appropriate, although if the re-
spondent has multiple health problems it may be worth combining it
with a generic measure. If the research topic covers more than one con-
dition, or general health, then generic measures might be more appro-
priate. It is not possible in this short space to recommend specific
Evaluating health services: multidisciplinary collaboration 15
measures; generic and disease-specific measures have been reviewed by
the author elsewhere (Bowling 1995b, 1997). The theoretical influences
which shaped the development of health status and health-related quality
of life scales are described briefly in Chapter 2.
• Research is the systematic and rigorous process of enquiry that aims to
describe processes and develop explanatory concepts and theories, in
order to contribute to a scientific body of knowledge.
• Health services research aims to produce reliable and valid research data
on which to base appropriate, effective, cost-effective, efficient and
acceptable health services in the broadest sense.
• The quality of care relates to its effectiveness at improving patients’
health status and how well it meets predefined and agreed standards
about how the care should be provided.
• Audit is directed at the maintenance and achievement of quality in
health care. It consists of review and monitoring of current practice,
and evaluation against standards.
• Medical audit is the systematic critical analysis of the quality of med-
ical care; clinical audit is the systematic critical analysis of the quality of
clinical care by all health care professionals.
• Quality assurance is a clinical and management approach which is the
systematic monitoring and evaluation of predefined and agreed levels
of service provision.
• Evaluation is the use of the scientific method, and the rigorous and
systematic collection of research data to assess the effectiveness of
organisations, services and programmes (e.g. health service interven-
tions) in achieving predefined objectives.
• Evaluation is more than audit because it aims to record not only what
changes occur, but also what led to those changes.
• The evaluation of health services is usually based on collecting data
about the structure, process and outcomes of services, as well as the
appropriateness of the services.
• Outcome should usually include measurement of the impact of the
condition and the service (i.e. health care intervention) on the broader
health-related quality of life of the patient.
Define research.
Distinguish between health research, health systems research and health
services research.
Summary of
main points
Key
questions
16 Investigating health services and health: the scope of research
What are the key components of health services research?
Distinguish between evaluation and audit.
What is the difference between audit and quality assurance?
Distinguish between the structure, process and outcome of health services.
What are health service inputs and outputs?
What are the main domains of health-related quality of life which should
be included in the measurement of health outcomes?
appropriateness
audit
clinical audit
disease-specific quality of life
evaluation
health-related quality of life
health research
health services research
health status
Recommended reading
Black, N., Brazier, J., Fitzpatrick, R. and Reeve, B. (eds) (1998) Health Services
Research Methods. A Guide to Best Practice. London: BMJ Publishing.
Bowling, A. (1995) Measuring Disease. A Review of Disease-specific Quality of Life
Measurement Scales. Buckingham: Open University Press.
Donabedian, A. (1980) Explorations in Quality Assessment and Monitoring. Vol. 1.
The Definition of Quality and Approaches to its Assessment. Ann Arbor, MI:
Health Administration Press.
Higginson, I. (1994) Quality of care and evaluating services. International Review
of Psychiatry, 6, 5–14.
Hunter, D.J. and Long, A.F. (1993) Health research. In W. Sykes, M. Bulmer
and M. Schwerzel (eds) Directory of Social Research Organizations in the UK.
London: Mansell.
Jenkinson, C. (ed.) (1997) Assessment and Evaluation of Health and Medical Care.
Buckingham: Open University Press.
Long, A. (1994) Assessing health and social outcomes. In J. Popay and G. Williams
(eds) Researching the People’s Health. London: Routledge.
St Leger, A.S., Schnieden, H. and Wadsworth-Bell, J.P. (1992) Evaluating Health
Services’ Effectiveness. Buckingham: Open University Press.
Key
terms inputs
medical audit
outcome
outputs
process
quality of life
quality assurance
structure
A preview of this full-text is provided by Hogrefe Publishing.
Content available from Journal of Psychophysiology
This content is subject to copyright. Terms and conditions apply.
