The causes and effects of
from clinical trials
C Bartlett,1L Doyal,2S Ebrahim,1P Davey,3
M Bachmann,4M Egger5and P Dieppe1*
1Medical Research Council Health Services Research Collaboration,
Department of Social Medicine, University of Bristol, UK
2School for Policy Studies, University of Bristol, UK
3MEMO, Department of Clinical Pharmacology, University of Dundee,
Ninewells Hospital, Dundee, UK
4School of Medicine, Health Policy and Practice, University of East Anglia,
5Department of Social and Preventative Medicine, University of Berne,
* Corresponding author
Health Technology Assessment
NHS R&D HTA Programme
Health Technology Assessment 2005; Vol. 9: No. 38
The causes and effects of socio-demographic exclusions
from clinical trials
The exclusion from trials of people likely to be in
need of or to benefit from an intervention could
compromise the trials’ generalisability. We
investigated the exclusion of women, older people
and minority ethnic groups, focusing on two drug
exemplars, statins and non-steroidal anti-
inflammatory drugs (NSAIDs).
G Scope the social, legal and ethical contexts of
trial exclusion, comparing the UK with the USA.
G Document disparities between people included
in trials, those using the drugs and those in
need of the treatment.
G Project the effects of exclusion on the
generalisability of trials, referring to
effectiveness (statins) and adverse effects
G Develop a theoretical model for the causes and
effects of exclusions.
We reviewed literature on the exclusion of women,
older people and ethnic minorities in healthcare
research and held three workshops with
We analysed 27 randomised controlled trials (RCTs)
of statins use for secondary prevention of coronary
heart disease (CHD), lasting at least 6 months (up
to August 2001). We analysed a stratified sample of
25 NSAIDs trials for pain in osteoarthritis (OA) (up
to 1998, prior to the introduction of coxibs).
Using a Scottish cohort with record-linkage
[Medicines Monitoring Unit (Dundee) (MEMO)],
we profiled 3188 people needing secondary
prevention for CHD (1993–1996), ascertaining the
independent effects of statins, and 131,410 people
dispensed NSAIDs (1989–1996), examining
Use and need
To profile the need for secondary prevention of
CHD in England we accessed routine data sources
including Hospital Episode Statistics (HES). To
estimate usage we consulted published surveys. For
potential need and usage of NSAIDs in OA we
accessed the Somerset and Avon Survey of Health
(SASH) 1996–97 and published data.
For both drugs, we compared the socio-
demographic profiles of trial samples, the
population in potential need and those on
We produced an evidence synthesis to clarify the
effects of statins on women and older people. We
modelled the relationship of absolute effectiveness
outcomes (e.g. numbers needed to treat) with
underlying risk levels of disease events, examining
the likely effects of trial exclusions.
In the USA, the discourse has expanded from
protecting the vulnerable to include justice and
the equitable access of different groups to trials.
Appropriate representation of women and ethnic
minorities in publicly funded trials is required by
legislation. Guidelines recommend appropriate
inclusion by age. In the UK, the debate is more
limited, and equity and inclusivity in research are
not formally promoted.
The average age of trial participants was
58.5 years; only 16.3% were women. Statins
reduced cardiovascular disease (CVD) incidence
by about 25% in both men and women. Older
people up to 75 years of age also benefited.
Meta-analysis and two landmark trials, containing
large proportions of women and older people
(published after 2001), confirmed these results.
The average age of trial participants was
61.9 years and women were well represented
Executive summary: The causes and effects of socio-demographic exclusions from clinical trials
(68.5%). Gastrointestinal (GI) adverse events were
commonly reported, but renal side-effects were
not. Outcomes were seldom reported according to
For both drugs, USA trials were more inclusive
than UK/European trials. Ethnicity was not well
reported for either drug.
Some 23% of the cohort were treated with statins.
Statins users were younger than non-statins users
(but no more likely to be male) and had superior
High current exposure to NSAIDs elevated the
risk of GI side-effects by about 50% versus no
current exposure and renal impairment risk by
nearly 140%. Side-effect risk increased with age;
being female diminished risk.
Use and need
Approximately 537,000 incident cases of CVD
would qualify for statins use in England each year.
Women constitute 45% of this population with
need, two-thirds of whom are aged 65 years or
over. Need varies by ethnic group. No sex bias in
prescribing was detected, but use was commoner
in younger people.
6.3% of adults aged 35+ years reported hip and/or
knee pain associated with OA; 3.9% of adults used
prescribed analgesics for this and they were more
likely to be women and to be >65 years old.
Women formed almost half of the ‘with need’ and
‘on treatment’ populations, but were markedly
under-represented in trials. Those aged 65+ years
formed nearly two-thirds of the ‘with need’
population, but only one-fifth of trial samples, and
were less likely to be treated than younger
Women formed similar proportions (two-thirds) of
trial samples, and of the ‘with need’ and ‘on
treatment’ populations. People aged 65+ years
formed about three-fifths of the ‘on treatment’
population, but were under-represented in trials.
Association of side-effects with socio-demographic
factors was revealed in cohort data but not in trials.
Meta-analysis might overcome problems of low
inclusion for the assessment of relative
effectiveness, but the assessment of side-effects in
different groups would require massive trials.
Measures of absolute effectiveness are vital for the
analyses of benefit and harm and cost-effectiveness.
Such measurements, involving underlying risk
levels, will be severely biased if different population
groups are not adequately represented.
The issue of exclusion from trials of women, older
people and ethnic minorities has been relatively
neglected in the UK research community, and
there is confusion about diversity issues. Under-
representation occurs, but in drug trials at least this
may not always affect the external validity of
relative effect estimates. However, measures of
absolute effectiveness, absolute harm and cost-
effectiveness are associated with underlying risk
levels in different socio-demographic groups.
Under-representation will therefore bias absolute
effect estimates. The complexity of the issues made
development of a theoretical model impossible.
Recommendations for future
The following areas are suggested for future
G Multi-disciplinary assessment of realistic options
for trialists to address the issue of exclusions.
G Clarification of the use of ethnic categories in
health research and of the implications of the
different dimensions of ageing and sex/gender.
G Identification of barriers and facilitators to the
involvement of different population groups in
G Further investigation of the susceptibility of
older men to NSAID adverse events.
G Development of a ‘register of registries and
databases’ and exploration of how linked health
information systems in the UK could be
Bartlett C, Doyal L, Ebrahim S, Davey P,
Bachmann M, Egger M, et al. The causes and
effects of socio-demographic exclusions from
clinical trials. Health Technol Assess 2005;9(38).
Health Technology Assessment 2005; Vol. 9: No. 38 (Executive summary)
NHS R&D HTA Programme Download full-text
Excellence (NICE) and the National Screening Committee (NSC) who rely on HTA outputs to help raise
standards of care. HTA findings also help to improve the quality of the service in the NHS indirectly in
that they form a key component of the ‘National Knowledge Service’ that is being developed to improve
the evidence of clinical practice throughout the NHS.
The HTA Programme was set up in 1993. Its role is to ensure that high-quality research information on
the costs, effectiveness and broader impact of health technologies is produced in the most efficient way
for those who use, manage and provide care in the NHS. ‘Health technologies’ are broadly defined to
include all interventions used to promote health, prevent and treat disease, and improve rehabilitation
and long-term care, rather than settings of care.
The HTA Programme commissions research only on topics where it has identified key gaps in the
evidence needed by the NHS. Suggestions for topics are actively sought from people working in the
NHS, the public, service-users groups and professional bodies such as Royal Colleges and NHS Trusts.
Research suggestions are carefully considered by panels of independent experts (including service users)
whose advice results in a ranked list of recommended research priorities. The HTA Programme then
commissions the research team best suited to undertake the work, in the manner most appropriate to find
the relevant answers. Some projects may take only months, others need several years to answer the
research questions adequately. They may involve synthesising existing evidence or conducting a trial to
produce new evidence where none currently exists.
Additionally, through its Technology Assessment Report (TAR) call-off contract, the HTA Programme is
able to commission bespoke reports, principally for NICE, but also for other policy customers, such as a
National Clinical Director. TARs bring together evidence on key aspects of the use of specific
technologies and usually have to be completed within a short time period.
he research findings from the NHS R&D Health Technology Assessment (HTA) Programme directly
influence key decision-making bodies such as the National Institute for Health and Clinical
Criteria for inclusion in the HTA monograph series
Reports are published in the HTA monograph series if (1) they have resulted from work commissioned
for the HTA Programme, and (2) they are of a sufficiently high scientific quality as assessed by the referees
Reviews in Health Technology Assessment are termed ‘systematic’ when the account of the search,
appraisal and synthesis methods (to minimise biases and random errors) would, in theory, permit the
replication of the review by others.
The research reported in this monograph was commissioned by the HTA Programme as project number
98/24/02. The contractual start date was in June 2001. The draft report began editorial review in August
2003 and was accepted for publication in March 2005. As the funder, by devising a commissioning brief,
the HTA Programme specified the research question and study design. The authors have been wholly
responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA
editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the
referees for their constructive comments on the draft document. However, they do not accept liability for
damages or losses arising from material published in this report.
The views expressed in this publication are those of the authors and not necessarily those of the
HTA Programme or the Department of Health.
Editor-in-Chief: Professor Tom Walley
Series Editors: Dr Peter Davidson, Dr Chris Hyde, Dr Ruairidh Milne,
Dr Rob Riemsma and Dr Ken Stein
Managing Editors: Sally Bailey and Sarah Llewellyn Lloyd
© Queen’s Printer and Controller of HMSO 2005
This monograph may be freely reproduced for the purposes of private research and study and may be included in professional journals provided
that suitable acknowledgement is made and the reproduction is not associated with any form of advertising.
Applications for commercial reproduction should be addressed to NCCHTA, Mailpoint 728, Boldrewood, University of Southampton,
Southampton, SO16 7PX, UK.
Published by Gray Publishing, Tunbridge Wells, Kent, on behalf of NCCHTA.
Printed on acid-free paper in the UK by St Edmundsbury Press Ltd, Bury St Edmunds, Suffolk.