The causes and effects of socio-demographic exclusions from clinical trials

ArticleinHealth technology assessment (Winchester, England) 9(38):iii-iv, ix-x, 1-152 · October 2005with27 Reads
Impact Factor: 5.03 · DOI: 10.3310/hta9380 · Source: PubMed
Abstract

To investigate the exclusion from trials of women, older people and minority ethnic groups, focusing on two drug exemplars, statins and non-steroidal anti-inflammatory drugs (NSAIDs). Medical and ethical databases. Workshops with stakeholders. Literature was reviewed on exclusions in healthcare research and three workshops were held with stakeholders. Twenty-seven randomised controlled trials (RCTs) of statins use for secondary prevention of coronary heart disease (CHD) and 25 NSAIDs trials for pain in osteoarthritis (OA) were analysed. Using a Scottish cohort with record-linkage, profiling was carried out for 3188 people needing secondary prevention for CHD (1993-1996), ascertaining the independent effects of statins, and 131,410 people dispensed NSAIDs (1989-1996), examining adverse effects. Routine data sources were accessed to profile the need for secondary prevention of CHD in England and usage was estimated by consulting published surveys. The Somerset and Avon Survey of Health (SASH) 1996-97 and published data were accessed for information on potential need and usage of NSAIDs in OA. For both drugs, the socio-demographic profiles of trial samples, the population in potential need and those on treatment were compared. An evidence synthesis was produced to clarify the effects of statins on women and older people and the relationship of absolute effectiveness outcomes with underlying risk levels of disease events was modelled, examining the likely effects of trial exclusions. The average age of statins trial participants was 58.5 years; only 16.3% were women. Statins reduced cardiovascular disease (CVD) incidence by about 25% in both men and women. Older people up to 75 years of age also benefited. Meta-analysis and two landmark trials confirmed these results. The average age of NSAIDs trial participants was 61.9 years and women were well represented (68.5%). Gastrointestinal (GI) adverse events were commonly reported, but renal side-effects were not. Outcomes were seldom reported according to socio-demographic group. For both drugs, USA trials were more inclusive than UK/European trials. Ethnicity was not well reported for either drug. Some 23% of the cohort were treated with statins. Users were younger than non-statins users (but no more likely to be male) and had superior outcomes. High current exposure to NSAIDs elevated the risk of GI side-effects by about 50% versus no current exposure and renal impairment risk by nearly 140%. Side-effect risk increased with age; being female diminished risk. Approximately 537,000 incident cases of CVD would qualify for statins use in England each year. Women constitute 45% of this population with need, two-thirds of whom are aged 65 years or over. Need varies by ethnic group. No sex bias in prescribing statins was detected, but use was commoner in younger people. For NSAIDs, 6.3% of adults aged 35+ years reported hip and/or knee pain associated with OA; 3.9% of adults used prescribed analgesics for this and they were more likely to be women and to be >65 years old. For statins, women formed almost half of the 'with need' and 'on treatment' populations, but were markedly under-represented in trials. Those aged 65+ years formed nearly two-thirds of the 'with need' population, but only one-fifth of trial samples, and were less likely to be treated than younger subjects. For NSAIDs, women formed similar proportions. Associations of side-effects with socio-demographic factors was revealed in cohort data but not in trials. The issue of exclusion from trials of women, older people and ethnic minorities has been relatively neglected in the UK research community, and there is confusion about diversity issues. Under-representation occurs, but in drug trials at least this may not always affect the external validity of relative effect estimates. However, measures of absolute effectiveness, absolute harm and cost-effectiveness are associated with underlying risk levels in different socio-demographic groups. Under-representation will therefore bias absolute effect estimates. The following areas are suggested for future research: multi-disciplinary assessment of realistic options for trialists to address the issue of exclusions; clarification of the use of ethnic categories in health research and of the implications of the different dimensions of ageing and sex/gender; identification of barriers and facilitators to the involvement of different population groups in research, further investigation of the susceptibility of older men to NSAID adverse events, and the development of a 'register of registries and databases' and exploration of how linked health information systems in the UK could be improved.

Full-text

Available from: Paul Dieppe, May 04, 2015
The causes and effects of
socio-demographic exclusions
from clinical trials
C Bartlett,
1
L Doyal,
2
S Ebrahim,
1
P Davey,
3
M Bachmann,
4
M Egger
5
and P Dieppe
1
*
1
Medical Research Council Health Services Research Collaboration,
Department of Social Medicine, University of Bristol, UK
2
School for Policy Studies, University of Bristol, UK
3
MEMO, Department of Clinical Pharmacology, University of Dundee,
Ninewells Hospital, Dundee, UK
4
School of Medicine, Health Policy and Practice, University of East Anglia,
Norwich, UK
5
Department of Social and Preventative Medicine, University of Berne,
Switzerland
* Corresponding author
HTA
Health Technology Assessment
NHS R&D HTA Programme
Health Technology Assessment 2005; Vol. 9: No. 38
Executive summary
The causes and effects of socio-demographic exclusions
from clinical trials
Page 1
Background
The exclusion from trials of people likely to be in
need of or to benefit from an intervention could
compromise the trials’ generalisability. We
investigated the exclusion of women, older people
and minority ethnic groups, focusing on two drug
exemplars, statins and non-steroidal anti-
inflammatory drugs (NSAIDs).
Objectives
Scope the social, legal and ethical contexts of
trial exclusion, comparing the UK with the USA.
Document disparities between people included
in trials, those using the drugs and those in
need of the treatment.
Project the effects of exclusion on the
generalisability of trials, referring to
effectiveness (statins) and adverse effects
(NSAIDs).
Develop a theoretical model for the causes and
effects of exclusions.
Methods
Scoping
We reviewed literature on the exclusion of women,
older people and ethnic minorities in healthcare
research and held three workshops with
stakeholders.
Trials
We analysed 27 randomised controlled trials (RCTs)
of statins use for secondary prevention of coronary
heart disease (CHD), lasting at least 6 months (up
to August 2001). We analysed a stratified sample of
25 NSAIDs trials for pain in osteoarthritis (OA) (up
to 1998, prior to the introduction of coxibs).
Cohorts
Using a Scottish cohort with record-linkage
[Medicines Monitoring Unit (Dundee) (MEMO)],
we profiled 3188 people needing secondary
prevention for CHD (1993–1996), ascertaining the
independent effects of statins, and 131,410 people
dispensed NSAIDs (1989–1996), examining
adverse effects.
Use and need
To profile the need for secondary prevention of
CHD in England we accessed routine data sources
including Hospital Episode Statistics (HES). To
estimate usage we consulted published surveys. For
potential need and usage of NSAIDs in OA we
accessed the Somerset and Avon Survey of Health
(SASH) 1996–97 and published data.
Disparities
For both drugs, we compared the socio-
demographic profiles of trial samples, the
population in potential need and those on
treatment.
Epidemiological/statistical assumptions
We produced an evidence synthesis to clarify the
effects of statins on women and older people. We
modelled the relationship of absolute effectiveness
outcomes (e.g. numbers needed to treat) with
underlying risk levels of disease events, examining
the likely effects of trial exclusions.
Results
Scoping
In the USA, the discourse has expanded from
protecting the vulnerable to include justice and
the equitable access of different groups to trials.
Appropriate representation of women and ethnic
minorities in publicly funded trials is required by
legislation. Guidelines recommend appropriate
inclusion by age. In the UK, the debate is more
limited, and equity and inclusivity in research are
not formally promoted.
Trials
Statins
The average age of trial participants was
58.5 years; only 16.3% were women. Statins
reduced cardiovascular disease (CVD) incidence
by about 25% in both men and women. Older
people up to 75 years of age also benefited.
Meta-analysis and two landmark trials, containing
large proportions of women and older people
(published after 2001), confirmed these results.
NSAIDs
The average age of trial participants was
61.9 years and women were well represented
Executive summary: The causes and effects of socio-demographic exclusions from clinical trials
Executive summary
Page 2
(68.5%). Gastrointestinal (GI) adverse events were
commonly reported, but renal side-effects were
not. Outcomes were seldom reported according to
socio-demographic group.
For both drugs, USA trials were more inclusive
than UK/European trials. Ethnicity was not well
reported for either drug.
Cohorts
Statins
Some 23% of the cohort were treated with statins.
Statins users were younger than non-statins users
(but no more likely to be male) and had superior
outcomes.
NSAIDs
High current exposure to NSAIDs elevated the
risk of GI side-effects by about 50% versus no
current exposure and renal impairment risk by
nearly 140%. Side-effect risk increased with age;
being female diminished risk.
Use and need
Statins
Approximately 537,000 incident cases of CVD
would qualify for statins use in England each year.
Women constitute 45% of this population with
need, two-thirds of whom are aged 65 years or
over. Need varies by ethnic group. No sex bias in
prescribing was detected, but use was commoner
in younger people.
NSAIDs
6.3% of adults aged 35+ years reported hip and/or
knee pain associated with OA; 3.9% of adults used
prescribed analgesics for this and they were more
likely to be women and to be >65 years old.
Disparities
Statins
Women formed almost half of the ‘with need’ and
‘on treatment’ populations, but were markedly
under-represented in trials. Those aged 65+ years
formed nearly two-thirds of the ‘with need’
population, but only one-fifth of trial samples, and
were less likely to be treated than younger
subjects.
NSAIDs
Women formed similar proportions (two-thirds) of
trial samples, and of the ‘with need’ and ‘on
treatment’ populations. People aged 65+ years
formed about three-fifths of the ‘on treatment’
population, but were under-represented in trials.
Association of side-effects with socio-demographic
factors was revealed in cohort data but not in trials.
Epidemiological/statistical assumptions
Meta-analysis might overcome problems of low
inclusion for the assessment of relative
effectiveness, but the assessment of side-effects in
different groups would require massive trials.
Measures of absolute effectiveness are vital for the
analyses of benefit and harm and cost-effectiveness.
Such measurements, involving underlying risk
levels, will be severely biased if different population
groups are not adequately represented.
Main conclusions
The issue of exclusion from trials of women, older
people and ethnic minorities has been relatively
neglected in the UK research community, and
there is confusion about diversity issues. Under-
representation occurs, but in drug trials at least this
may not always affect the external validity of
relative effect estimates. However, measures of
absolute effectiveness, absolute harm and cost-
effectiveness are associated with underlying risk
levels in different socio-demographic groups.
Under-representation will therefore bias absolute
effect estimates. The complexity of the issues made
development of a theoretical model impossible.
Recommendations for future
research
The following areas are suggested for future
research:
Multi-disciplinary assessment of realistic options
for trialists to address the issue of exclusions.
Clarification of the use of ethnic categories in
health research and of the implications of the
different dimensions of ageing and sex/gender.
Identification of barriers and facilitators to the
involvement of different population groups in
research.
Further investigation of the susceptibility of
older men to NSAID adverse events.
Development of a ‘register of registries and
databases’ and exploration of how linked health
information systems in the UK could be
improved.
Publication
Bartlett C, Doyal L, Ebrahim S, Davey P,
Bachmann M, Egger M, et al. The causes and
effects of socio-demographic exclusions from
clinical trials. Health Technol Assess 2005;9(38).
Health Technology Assessment 2005; Vol. 9: No. 38 (Executive summary)
Page 3
NHS R&D HTA Programme
T
he research findings from the NHS R&D Health Technology Assessment (HTA) Programme directly
influence key decision-making bodies such as the National Institute for Health and Clinical
Excellence (NICE) and the National Screening Committee (NSC) who rely on HTA outputs to help raise
standards of care. HTA findings also help to improve the quality of the service in the NHS indirectly in
that they form a key component of the ‘National Knowledge Service’ that is being developed to improve
the evidence of clinical practice throughout the NHS.
The HTA Programme was set up in 1993. Its role is to ensure that high-quality research information on
the costs, effectiveness and broader impact of health technologies is produced in the most efficient way
for those who use, manage and provide care in the NHS. ‘Health technologies’ are broadly defined to
include all interventions used to promote health, prevent and treat disease, and improve rehabilitation
and long-term care, rather than settings of care.
The HTA Programme commissions research only on topics where it has identified key gaps in the
evidence needed by the NHS. Suggestions for topics are actively sought from people working in the
NHS, the public, service-users groups and professional bodies such as Royal Colleges and NHS Trusts.
Research suggestions are carefully considered by panels of independent experts (including service users)
whose advice results in a ranked list of recommended research priorities. The HTA Programme then
commissions the research team best suited to undertake the work, in the manner most appropriate to find
the relevant answers. Some projects may take only months, others need several years to answer the
research questions adequately. They may involve synthesising existing evidence or conducting a trial to
produce new evidence where none currently exists.
Additionally, through its Technology Assessment Report (TAR) call-off contract, the HTA Programme is
able to commission bespoke reports, principally for NICE, but also for other policy customers, such as a
National Clinical Director. TARs bring together evidence on key aspects of the use of specific
technologies and usually have to be completed within a short time period.
Criteria for inclusion in the HTA monograph series
Reports are published in the HTA monograph series if (1) they have resulted from work commissioned
for the HTA Programme, and (2) they are of a sufficiently high scientific quality as assessed by the referees
and editors.
Reviews in Health Technology Assessment are termed ‘systematic’ when the account of the search,
appraisal and synthesis methods (to minimise biases and random errors) would, in theory, permit the
replication of the review by others.
The research reported in this monograph was commissioned by the HTA Programme as project number
98/24/02. The contractual start date was in June 2001. The draft report began editorial review in August
2003 and was accepted for publication in March 2005. As the funder, by devising a commissioning brief,
the HTA Programme specified the research question and study design. The authors have been wholly
responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA
editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the
referees for their constructive comments on the draft document. However, they do not accept liability for
damages or losses arising from material published in this report.
The views expressed in this publication are those of the authors and not necessarily those of the
HTA Programme or the Department of Health.
Editor-in-Chief: Professor Tom Walley
Series Editors: Dr Peter Davidson, Dr Chris Hyde, Dr Ruairidh Milne,
Dr Rob Riemsma and Dr Ken Stein
Managing Editors: Sally Bailey and Sarah Llewellyn Lloyd
ISSN 1366-5278
© Queen’s Printer and Controller of HMSO 2005
This monograph may be freely reproduced for the purposes of private research and study and may be included in professional journals provided
that suitable acknowledgement is made and the reproduction is not associated with any form of advertising.
Applications for commercial reproduction should be addressed to NCCHTA, Mailpoint 728, Boldrewood, University of Southampton,
Southampton, SO16 7PX, UK.
Published by Gray Publishing, Tunbridge Wells, Kent, on behalf of NCCHTA.
Printed on acid-free paper in the UK by St Edmundsbury Press Ltd, Bury St Edmunds, Suffolk.
Page 4
    • "One frequently cited challenge in reference to minority group participation in RCTs relates to patients' language proficiency and literacy [19,20]. Among those living in the United States, around 25 million people are unable to speak English fluently [21], while it is estimated that almost 300,000 adults in England and Wales from the 4 most common ethnic minority groups speak little or no English [22]. "
    [Show abstract] [Hide abstract] ABSTRACT: Background: Type 2 diabetes is common, on the rise, and disproportionately affects ethnic minority groups. Telehealth interventions may mitigate diabetes-related complications, but might under-recruit or even exclude ethnic minorities, in part because of English language requirements. The under-representation of minority patients in trials could threaten the generalizability of the findings, whereby the patients who might stand to benefit most from such interventions are not being included in their evaluation. Objective: The aims of this systematic review are twofold: (1) to assess the reporting and prevalence of ethnic minorities in published telehealth trials for type 2 diabetes, including identifying trial features associated with successful patient recruitment; and (2) to determine the proportion of such trials that report English language proficiency as an inclusion/exclusion criterion, including how and why they do so. Methods: Randomized controlled trials (RCTs) of adults with type 2 diabetes in Western, English-speaking countries that included telehealth interventions targeting diabetes as a primary condition, and those that did not specifically recruit minority groups will be included. Search strategies were devised for indexed and keyword terms capturing type 2 diabetes, telehealth/health technology, and RCTs in English language publications from 2000 to July 2015 in MEDLINE, PsycINFO, EMBASE, CINAHL, and CENTRAL. Reference lists of included studies will also be searched. Two reviewers will independently screen abstracts and full-text articles against inclusion criteria, mediated by a third reviewer if consensus cannot be reached. Data extracted from included studies will be checked by a second reviewer and will be summarized using narrative synthesis. Results: This research is in progress, with findings expected by Spring 2016. Conclusions: This review will address research reporting and recruitment practices of ethnic minorities in telehealth RCTs for type 2 diabetes. Prevalence estimates will elucidate generalizability of existing research, with implications for researchers, health professionals, and policy makers. Identifying trial or intervention features that appear to facilitate ethnic minority recruitment, as well as language barriers that impede it might suggest ways to improve recruitment in future trials. Trial registration: PROSPERO International Prospective Register of Systematic Reviews: CRD42015024899; http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42015024899 (Archived by WebCite at http://www.webcitation.org/6fUMqbJ0f).
    Full-text · Article · Mar 2016
    • "Lack of external validity is a concern when the RCT participants are different from the target population. Findings from recent studies have heightened these concerns by showing that RCT samples might not represent the types of patients encountered in usual clinical practice settings [2,3]. In the context of substance use disorders (SUD), recent studies have shown that tight exclusion criteria commonly employed in RCTs might have resulted in RCT samples that are different with regard to sex and race distribution from the treatmentseeking populations in usual care settings4567 . "
    [Show abstract] [Hide abstract] ABSTRACT: To compare characteristics of individuals participating in randomized control trials (RCTs) of treatments of substance use disorder (SUD) with individuals receiving treatment in usual care settings, and to provide a summary quantitative measure of differences between characteristics of these two groups of individuals using propensity score methods. Analyses using data from RCT samples from the National Institute of Drug Abuse Clinical Trials Network (CTN) and target populations of patients drawn from the Treatment Episodes Data Set-Admissions (TEDS-A). Multiple clinical trial sites and nationwide usual SUD treatment settings in the United States. A total of 3,592 individuals from 10 CTN samples and 1,602,226 individuals selected from TEDS-A between 2001 and 2009. The propensity scores for enrolling in the RCTs were computed based on the following nine observable characteristics: sex, race/ethnicity, age, education, employment status, marital status, admission to treatment through criminal justice, intravenous drug use, and the number of prior treatments. The proportion of those with ≥12 years of education and the proportion of those who had full-time jobs were significantly higher among RCT samples than among target populations (in seven and nine trials, respectively, at p<.001). The pooled difference in the mean propensity scores between the RCTs and the target population was 1.54 standard deviations and was statistically significant at p<.001. In the USA, individuals recruited into randomized control trials (RCT) of substance use disorder treatments appear to be very different from individuals receiving treatment in usual care settings. Notably, RCT participants tend to have more years of education and a greater likelihood of full-time work compared with people receiving care in usual care settings.
    Full-text · Article · Jan 2016 · Addiction
    • "As compared to screening studies among European populations, the uptake is only slightly lower in our study [4,6789101112. This lower uptake was expected as a lower participation is often observed in studies among migrant populations in industrialised countries [4,282930. Differences in uptake in HbA1c vs. OGTT group. The higher uptake for HbA1c is in line with the assumption that a more burdensome test is associated with a lower response [4, 20]. "
    [Show abstract] [Hide abstract] ABSTRACT: Direct comparisons of the effect of a glycated haemoglobin measurement or an oral glucose tolerance test on the uptake and yield of screening in people of South Asian origin have not been made. We evaluated this in 18 to 60-year-old South Asian Surinamese. We invited 3173 South Asian Surinamese for an oral glucose tolerance test between June 18th 2009- December 31st 2009 and 2012 for a glycated hemoglobin measurement between April 19th 2010-November 11th, 2010. Participants were selected from 48 general practices in The Hague, The Netherlands. We used mixed models regression to analyse differences in response and participation between the groups. We described differences in characteristics of participants and calculated the yield as the percentage of all cases identified, if all invitees had been offered screening with the specified method. The response and participation in the glycated hemoglobin group was higher than in the group offered an oral glucose tolerance test (participation 23.9 vs. 19.3; OR: 1.30, 95%-confidence interval1.01-1.69). After adjustment for age and sex, characteristics of participants were similar for both groups. Overall, glycated hemoglobin identified a similar percentage of type 2 diabetes cases but a higher percentage of prediabetes cases, in the population than the oral glucose tolerance test. We found that glycated hemoglobin and the oral glucose tolerance test may be equally efficient for identification of type 2 diabetes in populations of South Asian origin. However, for programs aimed at identifying people at high risk of type 2 diabetes (i.e. with prediabetes), the oral glucose tolerance test may be a less efficient choice than glycated hemoglobin.
    Full-text · Article · Aug 2015 · PLoS ONE
Show more

Similar publications

Discover more