Adherence to Federal Guidelines for Reporting of Sex and Race/Ethnicity in Clinical Trials

Article (PDF Available)inJournal of Women's Health 15(10):1123-31 · January 2007with19 Reads
DOI: 10.1089/jwh.2006.15.1123 · Source: PubMed
The National Institutes of Health Revitalization Act of 1993 requires that NIH-funded clinical trials include women and minorities as subjects; other federal agencies have adopted similar guidelines. The objective of this study was to determine the current level of compliance with these guidelines in federally funded randomized controlled trials. Randomized controlled trials published in nine influential medical journals in 2004 were identified by PubMed search. Studies where individuals were not the unit of analysis, those begun before 1994, and those not receiving federal funding were excluded. Included studies were examined to determine sample characteristics and presence of subgroup reporting. PubMed located 589 published papers. After exclusion of ineligible papers, 69 remained for analysis. Among 46 clinical studies enrolling both men and women, women were generally underrepresented, comprising on average 37% of the sample and only 24% of the sample when analysis was restricted to drug trials. Eighty-seven percent of the studies did not report any outcomes by sex or include sex as a covariate in modeling. Among all 69 studies, 18% did not break down sample sizes by racial and ethnic groups, and 87% did not provide any analysis by racial or ethnic groups. Only 5 studies indicated that the generalizability of their results may be limited by lack of diversity among those studied. These findings illustrate inadequate compliance with the NIH guidelines. Researchers, editors, and journal audiences share the responsibility of ensuring compliance with our country's policies regarding federally funded research to effect healthcare improvements for all.
Volume 15, Number 10, 2006
© Mary Ann Liebert, Inc.
Adherence to Federal Guidelines for Reporting of Sex
and Race/Ethnicity in Clinical Trials
Background: The National Institutes of Health Revitalization Act of 1993 requires that NIH-
funded clinical trials include women and minorities as subjects; other federal agencies have
adopted similar guidelines. The objective of this study was to determine the current level of
compliance with these guidelines in federally funded randomized controlled trials.
Methods: Randomized controlled trials published in nine influential medical journals in 2004
were identified by PubMed search. Studies where individuals were not the unit of analysis,
those begun before 1994, and those not receiving federal funding were excluded. Included stud-
ies were examined to determine sample characteristics and presence of subgroup reporting.
Results: PubMed located 589 published papers. After exclusion of ineligible papers, 69 re-
mained for analysis. Among 46 clinical studies enrolling both men and women, women were
generally underrepresented, comprising on average 37% of the sample and only 24% of the
sample when analysis was restricted to drug trials. Eighty-seven percent of the studies did
not report any outcomes by sex or include sex as a covariate in modeling. Among all 69 stud-
ies, 18% did not break down sample sizes by racial and ethnic groups, and 87% did not pro-
vide any analysis by racial or ethnic groups. Only 5 studies indicated that the generalizabil-
ity of their results may be limited by lack of diversity among those studied.
Conclusions: These findings illustrate inadequate compliance with the NIH guidelines. Re-
searchers, editors, and journal audiences share the responsibility of ensuring compliance with
our country’s policies regarding federally funded research to effect healthcare improvements
for all.
(NIH) Re-
vitalization Act of 1993 (effective March 9,
1994) requires that NIH-supported clinical re-
search include women and minorities as subjects
“in approximately equal numbers of both sexes.
. . . unless different proportions are appropriate
because of the known prevalence, incidence, mor-
bidity, mortality rates, or expected intervention
Department of Obstetrics and Gynecology, College of Medicine, and
Center for Research on Women and Gen-
der, University of Illinois at Chicago, Chicago, Illinois.
Departments of Medicine, Psychiatry, and Industrial & Systems Engineering, Center for Women’s Health Research,
University of Wisconsin-Madison, Madison, Wisconsin.
The guidelines for implementation fur-
ther state that phase III clinical trials must be de-
signed to allow separate planning, conducting,
and reporting of analyses for these groups when
prior research has indicated that it may be im-
portant and that preliminary trials must provide
enough information to inform the design of sub-
sequent phase III trials. Women of childbearing
potential cannot be routinely excluded, and cost
may not be used as an excuse to avoid adherence
to these guidelines. The Agency for Health Re-
search and Quality (AHRQ) adopted the NIH
guidelines, and the Centers for Disease Control
and Prevention (CDC) developed a similar set of
guidelines that became effective October 1, 1995.
Previous analyses indicate that federally
funded clinical trials have not consistently ad-
hered to these standards. Female subjects gener-
ally were underrepresented and sometimes ab-
sent, and sex-specific and race/ethnicity-specific
analyses were frequently not performed or not re-
Given the multiple years involved in
planning and executing clinical trials, it is possi-
ble that previous evaluations failed to find com-
pliance with federal policies because the clinical
trials were designed or initiated prior to the 1993
Act. Therefore, the current analysis evaluates ad-
herence to these guidelines for reporting of sex
and race/ethnicity in clinical trials by evaluating
federally funded studies initiated after 1993 in the
areas of general internal medicine, oncology, car-
diology, infectious disease, and obstetrics and gy-
necology (race/ethnicity only). This analysis also
examines whether obstetrics and gynecology
journals, given that they are sex specific, are more
likely to publish results by race and ethnicity than
other journals, a topic that has not previously
been explored. To the extent that the course of
disease and effectiveness of treatment differ by
sex and by race/ethnicity, compliance with these
guidelines is critically important to ensure ap-
propriate, evidence-based medicine.
Randomized controlled trials (RCTs) pub-
lished in 2004 were located by computerized
search of PubMed, focusing on the areas of gen-
eral internal medicine, oncology, cardiology, in-
fectious disease, and obstetrics and gynecology.
Journals were chosen based on both their impact
factor in 2003, as determined by Journal Citation
Reports retrieved from the ISI Web of Knowl-
edge, and by the number of RCTs published in
2004. The impact factor reflects the frequency
with which papers in a specific journal are cited
in a given year. For general medicine, the four
journals with the highest impact scores published
in the United States were selected; each had at
least 20 RCTs. In the subspecialty areas of oncol-
ogy, cardiology, and infectious disease, journals
were selected that were not limited to a single dis-
ease, had the highest impact score, and contained
at least 50 RCTs. For obstetrics and gynecology,
two journals were chosen using the same criteria.
The following nine journals met our criteria:
New England Journal of Medicine, Journal of the
American Medical Association, Annals of Internal
Medicine, American Journal of Medicine, Journal of
Clinical Oncology, Circulation, the Journal of In-
fectious Diseases, Obstetrics and Gynecology, and
the American Journal of Obstetrics and Gynecology.
A PubMed search in each journal used limits as
indexed by the National Library of Medicine to
select all papers described as “Randomized
Controlled Trial” that were in English, based on
data from humans, and published during 2004.
In cases where a paper was published online
and also in print, the date of publication used
for selection refers to the earlier of the two
Each paper was examined by a single reviewer,
in collaboration with the coauthors, to determine
the source of funding and the date when study
recruitment began. Studies identifying no federal
support were excluded. Letters and Brief Com-
munications and clinical trials begun before 1994
were also excluded. Studies were excluded where
an individual was not the unit of randomization
or analysis, where only a portion of a trial’s sub-
jects (such as only enrolled subjects with severe
disease) was analyzed, where data were com-
bined from several trials, or where no subjects
resided in the United States. All information was
captured using a data collection form and entered
into Microsoft Excel for analysis.
For the analysis of sex-based reporting, stud-
ies that were specific to only males or females
were excluded. Conditions that are not exclusive
to one sex but may disproportionately affect
members of one sex (e.g., autoimmune diseases)
were not excluded. Studies based in veteran’s
hospitals or treatment facilities were also not ex-
cluded unless they addressed a condition found
only in men (e.g., prostate cancer).
Papers were evaluated to determine if sex-spe-
cific and race/ethnicity-specific results were re-
ported. Obstetrics and gynecology papers were
evaluated for the reporting of race/ethnicity-
specific results only. It was also noted whether
race/ethnicity and sex were taken into account
during the analysis of outcomes and if the au-
thors acknowledged the impact that sex, race, or
ethnicity might have on either the results or their
generalizability to broader populations. The sam-
ple distribution across sex and race/ethnicity was
recorded in terms of both percent distribution
and absolute numbers (because sample size dri-
ves the ability to find statistical significance).
Comparisons between subsets of the papers were
made using Fisher exact test. Each paper was ex-
amined in its entirety, including abstract, text,
and tables. In addition to the papers themselves,
any published follow-up papers or comments by
either the author(s) or another researcher were
examined for information relating to sex, race,
and ethnicity. The race/ethnicity portion of the
analysis is limited to black and Hispanic, as other
racial/ethnic minorities were rarely reported.
The search resulted in 484 publications in the
areas of general and internal medicine, oncology,
cardiac and cardiology, and infectious disease, all
meeting the search criteria as of June 15, 2005. Of
these, 423 were eliminated for one or more of the
following reasons: no federal funding, no support
described, funding prior to 1994 or unknown date
of funding, no subjects residing in the United
States, full sample of subjects not described, or
subjects pooled across several trials reported else-
where. The remaining 61 studies were included
in the gender analyses. Among these, 29 were
funded solely by federal agencies, and 32 were
supported using federal support in combination
with other funds (private, state, university en-
dowment, or other sources). Fifty-one studies
received some or all funding from the NIH. For
use in analysis of race/ethnicity only, an addi-
tional 105 obstetrics and gynecology papers were
found; after eliminating papers for the above-
mentioned reasons, 8 remained for analysis.
Thus, 69 papers were included for analysis of
Of the 61 studies, 15 were sex specific and 46
were not. Table 1 provides the distribution of
these studies by journal type. Of the 15 sex-spe-
cific studies, 7 studies focused on diseases that
are specific to women (e.g., menopause, breast
cancer, cancers of the female reproductive or-
gans), 5 were specific to men (e.g., prostate can-
cer, hernia repair), 3 studies chose to include only
one sex, even though the condition may affect
both men and women (e.g., treatment of lipody-
strophy in HIV-infected men, smoking cessation
among women with cardiovascular disease, and
HIV prevention in African American girls). These
15 sex-specific studies are excluded from the fol-
lowing analysis.
Among the 46 studies that were not sex spe-
cific, the majority of studies (n 32, 70%) en-
1. F
, n (%)
General Infectious
Sex-specific 9 (20) 4 (36) 2 (67) 0 15 (25)
Male only 4 (9) 2 (18) 0 0 6 (10)
Female only 5 (11) 2 (18) 2 (67) 0 9 (15)
Studies of 36 (80) 7 (64) 1 (33) 2 (100) 46 (75)
conditions in
both sexes
Total 45 (100) 11 (100) 3 (100) 2 (100) 61 (100)
New England Journal of Medicine (impact factor [IF] 34.8), JAMA (IF 21.5), Annals of Internal Medicine (IF 12.4),
American Journal of Medicine (IF 4.4).
Journal of Clinical Oncology (IF 10.9).
Circulation (IF 11.2).
Journal of Infectious Diseases (IF 4.5).
rolled 30% or more women. The percentage of
women in each sample ranged from 1% to 79%
(median 43%). The number of women enrolled
ranged from 2 to 1964 (median 91). Twelve stud-
ies had more than 300 female subjects. Of the
studies that reported on a drug as an interven-
tion (as opposed to a device or other nonmedical
intervention), 13 of 23 (56%) had 30% or more wo-
men enrolled. In contrast, among the nondrug
studies, 20 of 23 (87%) had 30% or more women
(p 0.02).
NIH-funded studies did not fare any better
than the group as a whole. Among the 38 non-
sex-specific studies that were supported by NIH,
74% had more than 30% women. Among NIH-
funded drug studies, 57% had 30% or more wo-
men, whereas among nondrug studies, 94% had
more than 30% women.
Among the 46 studies that were not sex spe-
cific, 40 (87%) did not report any outcomes by sex
or include sex as a covariate in modeling. Only 9
(20%) discussed reasons why sex-specific data
were not provided, the most common reason be-
ing that differences by sex were found not to be
statistically significant. These 9 studies all re-
ceived NIH funding. Table 2 provides a break-
down by journal type. The 31 studies that did not
report by sex, did not include it in modeling, and
did not provide a rationale for disregarding it
were predominantly focused on the topics of can-
cer (n 7), cardiovascular disease (n 5), HIV
(n 3), and psychological disorders (n 4). Thir-
teen of these 31 studies had more than 200 wo-
men enrolled.
No study acknowledged the limits of general-
izability related to the study’s findings to a par-
ticular sex. This includes the 7 studies with less
than 20% women subjects. In fact, 1 study in
which only 19% of the subjects were women
stated that results were generalizable to the gen-
eral population because of “diverse representa-
tion from women.” Twenty-four of the 31 stud-
ies that did not report results by sex or provide
a rationale for this exclusion had a follow-up let-
ter or comment referenced in PubMed. Of these,
1 questioned the lack of reporting related to sex,
stating that analyses should have taken women’s
menopausal status into account. The author
replied that sample size was insufficient to per-
form such an analysis.
Of the 4 studies reporting on a Veterans Ad-
ministration study population, the representation
by women subjects ranged from 2% to 20%, with
fewer than 72 female subjects in each. One study
included sex in a model; the other 3 did not ac-
knowledge how the sample distribution might in-
fluence generalizability.
Examining the 69 studies in all nine journals
with respect to race/ethnicity, none addressed a
disease or condition that might be considered
race specific (e.g., sickle cell disease in African
Americans). About half of all studies reported the
percent of subjects that were black (n 46, 67%)
or Hispanic (n 33, 48%). Twelve (18%) studies
did not report the number of subjects in any
racial/ethnic categories; none of these gave any
indication that recruitment was limited to a sin-
gle race.
Just over three quarters of studies (n 54, 78%)
reported including both white and nonwhite sub-
jects, with the proportion of white subjects rang-
ing from 30% to 94%. In 8 (12%) of the studies, at
least 90% of subjects were white, but only 1 study
had only white subjects. Similarly, only 1 study
recruited only subjects who were black, and 1 in-
2. A
, n (%)
General Infectious
medicine Oncology Cardiovascular disease Total
Analysis by sex provided 5 (14) 0 0 1 (50) 6 (13)
or sex included in model
Did not analyze by sex but 8 (22) 1 (14) 0 0 9 (20)
provided an explanation
Did not include sex in 23 (64) 6 (86) 1 (100) 1 (50) 31 (67)
analysis or did not
provide an explanation
or both
Total 36 (100) 7 (100) 1 (100) 2 (100) 46 (100)
cluded only black or Hispanic subjects. Among
56 studies funded by NIH, 79% reported en-
rolling both white and nonwhite subjects com-
pared with 78% overall. Table 3 provides infor-
mation on the extent to which sample sizes for
racial and ethnic groups are reported.
Two studies that provided sample counts by
race subgroups treated Hispanic as an ethnic cat-
egory, separate from racial description, in accor-
dance with the Office of Management and Bud-
get census categories. The remaining studies
considered Hispanic to be a mutually exclusive
race category (i.e., no subject could be classified
as both black and Hispanic).
Looking at the types of analyses performed in
these studies, of the 61 studies that did not report
by race or include it in a model, 6 (9%) had more
than 200 black subjects, 3 (4%) had greater than
200 Hispanic participants, and 1 had more than
200 subjects classified as “other.” Examining rea-
sons the authors gave for not including racial/
ethnic results, 3 stated that inclusion of the vari-
able in their model was not significant, and 3
stated that differences in bivariate comparisons
were found to be not significant. No study men-
tioned results specific to any racial/ethnic group
other than black or Hispanic. Table 4 provides the
distribution by journal type for those that pro-
vided analysis by racial/ethnic group.
The discussion and conclusion sections of the
papers were examined for statements limiting the
generalizability of results based on the race/eth-
nicity of subjects involved in the study. Five stud-
ies noted that the racial/ethnic distribution in
their samples differed from that of the general
population and that interpretation of results
should take this into account. In 3 of these, the
samples predominantly comprised minority sub-
jects, and the other 2 studies had a preponder-
ance of white subjects. In the other 64 papers, the
findings were generalized without regard to race
or ethnicity.
Examining the 8 eligible studies published in
two obstetrics and gynecology journals sepa-
rately, 7 reported the number of black women
enrolled, which ranged from 14% to 81%, and 3
reported the number of Hispanic women, which
ranged from 2% to 40%. There were no differ-
ences in reporting of sample size for race/eth-
nicity between these journals and the 7 medicine
journals (p 0.15 for blacks and p 0.24 for
Hispanics). None of these 8 obstetrics and gy-
necology studies reported results by race or eth-
nicity. Two studies reported testing race for pos-
sible inclusion in a multivariate model; 1 of these
found the variable to be not significant and re-
moved it from the final model reported. Simi-
larly, the proportion of obstetrics and gynecol-
ogy studies that did not report results by
race/ethnicity, include it in a model, or provide
an explanation did not differ from the other
seven journals (p 0.19).
3. R
White Black Hispanic
Studies with unknown 17 (25%) 23 (33%) 36 (52%)
number of subjects in
given racial/ethnic
Studies with no 1 (1%) 1 (1%) 2 (3%)
subjects enrolled in
given racial/ethnic
Studies with less than 0 ( 9 (14%) 19 (28%)
10% of subjects in
given racial/ethnic
Studies with all 1 (1%) 1 (1%) 0
subjects in given
racial/ethnic group
Percent of sample 30%–94% 4%–81% 1%–57%
among studies with
more than one race
Percents based on 69 studies.
Phase III studies
Of the 69 studies described here, only 4 were
described as phase III studies (none were in the
obstetrics and gynecology journals). For these, we
would expect compliance with the NIH require-
ments that the sample sizes be adequate to allow
subgroup comparisons. No study provided sex-
specific results or stated that there were no sta-
tistically significant sex differences, nor did any
of these phase III studies provide race/ethnicity-
specific results or state that differences were not
significant by race. One phase III trial was female
specific, and one was male specific. The other 2
had 58 (14%) and 305 (38%) female subjects, re-
spectively. The percent of black subjects ranged
from 7% to 29% in the 4 studies for which race
was reported, and the percent Hispanic was 4%
to 20% in the 3 for which it was reported.
The Institute of Medicine (IOM) has stressed
the importance of research that acknowledges sex
differences, describing sex categorization as a
“basic human variable.” It states that sex influ-
ences human health not only through biology but
through gender-related differences in behaviors,
perceptions, environmental exposures, socioeco-
nomic status, and public policy.
The IOM ac-
knowledges that racial/ethnic disparities in
health status are similarly a complex interaction
of socioeconomic, behavioral, and genetic fac-
It is only through acknowledgment and
critical examination of these multiple factors that
the true picture of health differences and dispar-
ities between men and women and among dif-
ferent racial/ethnic populations in the United
States will become clear.
asserts that ev-
idence-based medicine in general is only as good
as the studies on which it is based and bears the
risk of reinforcing the existing biases in the re-
search supporting it. Research that excludes cer-
tain groups should not be blindly generalized to
members of these groups; evidence-based medi-
cine depends on appropriate representation of all
race/ethnicity and sex groups in the research that
is its foundation.
The importance of examining sex and gender
differences in clinical research is underscored by
recent findings that aspirin is effective in pre-
venting stroke but not myocardial infarction (MI)
or death from other cardiovascular causes in wo-
men, despite several studies showing it to be ef-
fective at preventing MI but not stroke in men.
There are known differences in the way men and
women experience symptoms, react to certain
drugs, and respond to doses for maximal benefit
with lowest risk of side effects.
Failure to do
sex-specific analyses in the original trial of
digoxin in chronic congestive heart failure missed
the significant harm of this treatment to women,
4. A
, n (%)
Internal Infectious and
medicine Oncology Cardiovascular disease gynecology Total
Analysis by 4 (9) 1 (9) 0 0 1 (13) 6 (9)
provided or variable
included in model
Did not analyze by 5 (11) 0 0 0 1 (13) 6 (9)
race/ethnicity but
Did not include 34 (76) 10 (91) 3 (100) 2 (100) 6 (75) 55 (80)
race/ethnicity in
analysis or did
not provide an
explanation or both
Sample not racially 2 (4) 0 0 0 0 2 (3)
Total 45 (100) 11 (100) 3 (100) 2 (100) 8 (100) 69 (100)
Percentages may not total 100% because of rounding.
an important finding later discovered by re-
searchers not involved in the original study.
Although race may be primarily a socially
rather than biologically constructed category, the
recent FDA approval of BiDil (NitroMed, Inc.,
Lexington, MA) for treatment of heart failure
among black patients resulted from a clinical trial
among African American subjects after an initial
study revealed potential differences by race in
subgroup analysis.
An example such as this jus-
tifies the need to analyze study results among
subgroups and underscores the responsibility for
authors and journal editors of studies unable to
perform such analyses to acknowledge the lim-
ited generalizability of their results.
The goal of this study was to examine only fed-
erally funded clinical trials initiated after passage
of the NIH Revitalization Act, yet our results are
similar to those found in earlier studies.
sults suggest that although most studies enrolled
women, more than half of the studies enrolled
more men than women. In an analysis of studies
published from 1994 to 1999 where mortality was
the end point, Ramasubbu et al.
found that wo-
men comprised 26% of subjects in studies that be-
gan enrolling after 1993, with half of the studies
in the area of cardiology. Congruent with our
findings, they did not find a significant difference
in the percent of women enrolled in trials that be-
gan enrolling before 1993 compared with those
enrolling after 1993.
Our analysis found that two thirds of studies
that included men and women did not acknowl-
edge the consideration of differences by sex, and
only 13% reported sex-specific results or included
sex in analytical models. Among 5-year R01
grants, publications generally peak in year 5 and
decrease steadily thereafter.
Therefore, the
studies we examined published in 2004 are likely
to have first received funding in the late 1990s,
well after implementation of the NIH Revitaliza-
tion Act.
NIH reports that 50% of people enrolled during
fiscal year 2004 in trials that were not sex-specific
were women.
This is in contrast to 37% that we
found in the 46 such studies we reviewed and
36% in 38 NIH studies. Time period differences
may account for some of the inconsistency. It is
possible that studies published in less visible
journals tend to enroll a greater proportion of wo-
men than those found in the high-impact journals
included for this study. Despite the reasons for
this discrepancy, the fact remains that published
research in the highest-impact journals continues
to reflect inadequate attention to equitable enroll-
ment. A recent report from the Office of Research
of Women’s Health
indicates that in 2000, more
than 94% of grant applications were compliant
with the enrollment guidelines. This may be in-
dicative of the discrepancies between investiga-
tor intention at the time a grant application is
made and the realities of subject enrollment.
Reporting by race/ethnicity has also not shown
much improvement. Corbie-Smith et al.
that 41% of RCTs in three general medical jour-
nals reported no sample sizes by race categories,
compared with the 18% in nine journals in our
analysis. However, they did not restrict studies
in their analysis to those supported by federal
funds, and 82% of the studies they examined be-
gan enrolling subjects before 1994.
There are several potential limitations to this
study. Differences by race or gender may be
reported in smaller or specialty journals not
analyzed here. This may be the case, given that
each R01 grant produces approximately 7.6 Med-
line-listed papers, with 1.61 in a core journal.
However, the journals we selected have wider
readership and impact than more targeted sub-
specialty journals and, thus, are more likely to in-
fluence future research and practice guidelines.
Second, because we excluded studies where the
funding source could not be identified, it is pos-
sible that some of these were in fact supported by
federal dollars. However, citation of federal fund-
ing is required in resulting publications so incor-
rect exclusions in our analysis should be few.
Third, we do not have a pre-1994 comparison
group, but this was deliberate. Our interest was
to describe the current state of clinical trials re-
porting with respect to the NIH Revitalization
Act in a broad set of journals and subjects, re-
gardless of what the situation was in past years.
Fourth, although we initially reviewed almost 500
papers, only 69 met our inclusion criteria, and this
limited our ability to compare journals. Last, be-
cause we were not always able to determine if a
particular study was designed in accordance with
the guidelines regarding subject enrollment, our
analysis was limited to that which was reported;
some studies may have been designed in accor-
dance with the guidelines without giving enough
information in the paper to inform the reader. We
were also not able to take into consideration, per
the guidelines, how the prevalence of a disease
may vary by race/ethnicity or sex subgroups and
how this may affect sample characteristics. The
guidelines for phase III trials do include report-
ing of subgroup analyses, and so our analysis of
what was reported for these trials is appropriate.
It is important to consider the statistical ap-
propriateness of subgroup reporting. It is true
that, by definition, a study is designed to look at
an outcome within the entire study population.
Examination of particular subgroups may inad-
vertently introduce confounding factors that can
bias the result. However, subgroup reporting
should still be performed in order to inform fu-
ture research. Recognition of significant findings
within a particular subgroup is the precursor to
designing subsequent studies to look specifically
at that group and to make valid comparisons
across subgroups. Freedman et al.
describe an
ideal progression in research, where potential dif-
ferences among sex or race groups found as a re-
sult of subgroup analysis lead to studies where
the primary question is asked separately for each
group, and the resulting study is designed with
separate groups in mind. It may be the case that
many of the studies included in our analysis were
not adequately powered to detect statistical sig-
nificance, but if the data could be combined with
results of other studies, the question could more
definitively be answered.
Of the seven journals included here, only one
(Circulation) requests subanalyses, asking in its
Instructions to Authors to “please provide sex-
specific and/or racial/ethnic-specific data when
appropriate, or specifically state that no sex-
based or racial/ethnic-based differences were
present.” However, this request was not heeded
by the authors of the three papers reviewed here
or reinforced by the editors. One journal (JAMA)
specifically indicates that if race/ethnicity is dis-
cussed, justification for doing so must be pro-
vided. This is in accordance with the International
Committee of Medical Journal Editors’ Uniform
Requirements for Manuscripts Submitted to Bio-
medical Journals statement: “When authors use
variables such as race or ethnicity, they should
define how they measured the variables and jus-
tify their relevance” and “Where scientifically ap-
propriate, analyses of the data by variables such
as age and sex should be included.”
There is
much debate in the literature, however, about the
value of race/ethnicity data. To the extent that
race/ethnicity is a proxy for socioeconomic and
other measures, differences found by race/eth-
nicity may reflect the effects of racism and clas-
sism as opposed to true differences.
We examined nine high-impact journals in the
fields of general medicine, cardiology, oncology,
infectious disease, and obstetrics and gynecology
for publication of federally funded clinical trials
begun after the congressionally mandated inclu-
sion of women and minorities. We found contin-
ued evidence of underrepresentation of women
based on published reports of federally funded
studies. In studies of interventions for diseases
that affect both men and women, nearly one third
enrolled fewer than 30% women as subjects. We
also found widespread inattention to federal
mandates regarding inclusion of racial/ethnic
minority subjects in clinical trials. Thirteen per-
cent of clinical trials included 10% or fewer non-
white subjects. Results in all but 5 studies were
generalized to all populations despite the lack of
diversity among those studied.
In some cases, funding agencies have a clear-
ance process for manuscripts prior to submission.
This could provide an opportunity for agencies
to review compliance with their policies. Fur-
thermore, the NIH Public Access Policy, effective
May 2, 2005, requests the submission of papers
accepted for publication in peer-reviewed jour-
nals to PubMed Central when the research is sup-
ported by NIH.
The intention is to make these
taxpayer-supported results widely available. The
notice indicates that the submitted publications
will be monitored so that NIH can manage its re-
search portfolio and ultimately set research pri-
Our research suggests a loophole in the NIH
Revitalization Act: although the Act mandates re-
searchers to report results for phase III trials by
sex and race, it has no power to require that it be
done in any particular forum other than reports
made to the funding agency. The responsibility,
in part, falls to the journals, for they are a source
of information on which healthcare providers
base their treatment plans. We encourage jour-
nals to consider the importance of equitable en-
rollment and reporting and to revise their edito-
rial requirements to reflect these goals. The
continued lack of compliance with federal guide-
lines years after passage of the Act indicates the
need for greater scrutiny and accountability
among investigators. We also encourage NIH and
other funding agencies to use this as an oppor-
tunity to monitor the representation and analysis
and publication of sex and race/ethnicity in all
federally funded clinical research, for research
that does not equally reflect men, women, and
people of all races cannot adequately contribute
to evidence-based models for care.
1. National Institutes of Health. NIH guidelines on the
inclusion of women and minorities as subjects in clin-
ical research. Fed Regist 1994;59:14508.
2. Vidaver RM, Lafleur B, Tong C, Bradshaw R, Marts
SA. Women subjects in NIH-funded clinical research
literature: Lack of progress in both representation and
analysis by sex. J Womens Health Gend Based Med
3. Corbie-Smith G, St George DM, Moody-Ayers S, Ran-
sohoff DF. Adequacy of reporting race/ethnicity in
clinical trials in areas of health disparities. J Clin Epi-
demiol 2003;56:416.
4. Ramasubbu K, Gurm H, Litaker D. Gender bias in
clinical trials: Do double standards still apply? J
Womens Health Gend Based Med 2001;10:757.
5. Exploring the biological contributions to human
health: Does sex matter? Washington, DC: Institute of
Medicine; 2001.
6. Unequal treatment: Confronting racial and ethnic dis-
parities in health care. Washington, DC: Institute of
Medicine, 2002.
7. Olden K, White SL. Health-related disparities: Influ-
ence of environmental factors. Med Clin North Am
8. Rogers W. Evidence-based medicine and women: Do
the principles and practice of EBM further women’s
health? Bioethics 2004;18:50.
9. Ridker PM, Cook NR, Lee IM, et al. A randomized
trial of low-dose aspirin in the primary prevention of
cardiovascular disease in women. N Engl J Med
10. Chen W, Woods SL, Puntillo KA. Gender differences
in symptoms associated with acute myocardial in-
farction: A review of the research. Heart Lung
11. Kaiser J. Gender in the pharmacy: Does it matter? Sci-
ence 2005;308:1572.
12. Khan A, Brodhead AE, Schwartz KA, Kolts RL, Brown
WA. Sex differences in antidepressant response in re-
cent antidepressant clinical trials. J Clin Psychophar-
macol 2005;25:318.
13. Rathore SS, Wang Y, Krumholz HM. Sex-based dif-
ferences in the effect of digoxin for the treatment of
heart failure. N Engl J Med 2002;347:1403.
14. Taylor AL, Ziesche S, Yancy C, et al. Combination of
isosorbide dinitrate and hydralazine in blacks with
heart failure. N Engl J Med 2004;351:2049.
15. Murthy VH, Krumholz HM, Gross CP. Participation
in cancer clinical trials: Race-, sex-, and age-based dis-
parities. JAMA 2004;291:2720.
16. Druss BG, Marcus SC. Tracking publication outcomes
of National Institutes of Health grants. Am J Med
17. NIH Tracking/Inclusion Committee. Comprehensive
report: Tracking of human subjects research as re-
ported in fiscal year 2003 and fiscal year 2004. De-
partment of Health and Human Services, National In-
stitutes of Health, 2005.
18. Stone J, Pinn VW, Rudick J, Lawrence M, Carlyn M.
Evaluation of the first 10 years of the Office of Re-
search on Women’s Health at the National Institutes
of Health: Selected findings. J Womens Health 2006;
19. Freedman LS, Simon R, Foulkes MA, et al. Inclusion
of women and minorities in clinical trials and the
NIH Revitalization Act of 1993—The perspective of
NIH clinical trialists. Control Clin Trials 1995;16:
20. Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR.
Publication bias in clinical research. Lancet 1991;337:
21. International Committee of Medical Journal Editors.
Uniform requirements for manuscripts submitted to
biomedical journals: Writing and editing for biomed-
ical publication, 2005. Available at Ac-
cessed January 4, 2006.
22. Jones CP. Invited commentary: “Race,” racism, and
the practice of epidemiology. Am J Epidemiol 2001;
23. National Institutes of Health. Policy on enhancing
public access to archived publications resulting from
NIH-funded research. Fed Regist 2005;70:689.
Address reprint requests to:
Stacie E. Geller, Ph.D.
College of Medicine
820 S. Wood Street (MC 808)
University of Illinois at Chicago
Chicago, IL 60612
    • "Although numerous recommendations, guidelines, and legislation aimed at addressing gaps in scientific knowledge and reducing sex-and gender-based health disparities in CVD have been crafted over the last 20 years (Hamburg, 2014; Institute of Medicine, 2001 Mastroianni, Faden, & Federman, 1994; National Institutes of Health, 2001) most have not been fully implemented, nor has the benefit been fully realized (Blauwet, Hayes, McManus, Redberg, & Walsh, 2007; Dhruva, Bero, & Redberg, 2011; Dolor et al., 2012 ; US Food and Drug Adminis- tration [FDA] Office of Women's Health, 2015; Geller, Adams, & Carnes, 2006; Mazure & Jones, 2015). The U.S. General Accountability Office released its report, " National Institutes of Health: Better Oversight Needed to Help Ensure Continued Progress Including Women in Health Research " (US Government Accountability Office, 2015) just days before the summit was convened. "
    Full-text · Article · Apr 2016
    • "A survey of 1,800 articles in American medical journals from 1985–1996 found that when only one sex was used to study conditions affecting both sexes, males were used more frequently than females [2]. Another survey of 69 articles in 2004 American medical journals found inadequate compliance with the NIH guidelines [3] . The topic of sex balance in biomedical research designs has gained renewed interest in 2014 when NIH leadership decided to provide additional focus on sex balance in preclinical research involving animals and cells [4] . "
    [Show abstract] [Hide abstract] ABSTRACT: Study samples should be appropriately selected to maximize generalizability of results. Excluding one sex from studies of conditions that affect both sexes is problematic and has received attention as a public policy issue in the United States, resulting in legislation and recommendations made by the National Institutes of Health to address this deficiency of study designs. It is unknown to what extent biomechanical studies have inappropriately excluded one sex. The objective of this study was to provide objective data on this question. A retrospective review of random samples of abstracts presented at American Society of Biomechanics annual meetings from 1983 to 2013 was conducted to assess reporting of sex of study samples and whether the study samples were approximately balanced with respect to sex. We did not find a statistically significant increasing trend in the percentage of abstracts reporting sex over time. However, increasing trends were noted in the percentage of abstracts including both sexes (p < 0.05) and percentage of abstracts having an "approximately balanced" study sample containing 50 ± 20% females (p > 0.05). In 2013 the percentage of abstracts reporting studies having approximately balanced study samples was only 28%, far from the ideal level of 100%. While there has been modest change since 1983, there remains significant room for improvement in the reporting and composition of experimental studies reported at American Society of Biomechanics annual meetings.
    Full-text · Article · Mar 2015
    • "The Ingalhalikar et al. [1] study is emblematic of this tradition, seeking to identify neurobiological variation between males and females in the hope of explaining differences in their psychological and behavioural characteristics. There are several sound reasons for screening neuroscientific data for sexual differentiation, chief among them remediating the historical underrepresentation of females in biomedical research, which has disadvantaged women in respect to disease understanding and treatment9101112. However, neuroscientific research on sexual dimorphism has recently elicited intense criticism from scholars in both natural and social sciences. "
    [Show abstract] [Hide abstract] ABSTRACT: Neuroscience research on sex difference is currently a controversial field, frequently accused of purveying a 'neurosexism' that functions to naturalise gender inequalities. However, there has been little empirical investigation of how information about neurobiological sex difference is interpreted within wider society. This paper presents a case study that tracks the journey of one high-profile study of neurobiological sex differences from its scientific publication through various layers of the public domain. A content analysis was performed to ascertain how the study was represented in five domains of communication: the original scientific article, a press release, the traditional news media, online reader comments and blog entries. Analysis suggested that scientific research on sex difference offers an opportunity to rehearse abiding cultural understandings of gender. In both scientific and popular contexts, traditional gender stereotypes were projected onto the novel scientific information, which was harnessed to demonstrate the factual truth and normative legitimacy of these beliefs. Though strains of misogyny were evident within the readers' comments, most discussion of the study took pains to portray the sexes' unique abilities as equal and 'complementary'. However, this content often resembled a form of benevolent sexism, in which praise of women's social-emotional skills compensated for their relegation from more esteemed trait-domains, such as rationality and productivity. The paper suggests that embedding these stereotype patterns in neuroscience may intensify their rhetorical potency by lending them the epistemic authority of science. It argues that the neuroscience of sex difference does not merely reflect, but can actively shape the gender norms of contemporary society.
    Full-text · Article · Oct 2014
Show more