COMMENTS AND RESPONSES
The Effect of Financial Incentives on Hospitals That Serve
TO THE EDITOR: Jha and colleagues (1) made excellent efforts to
define the effects of the pay-for-performance Premier Hospital Qual-
ity Incentive Demonstration program (referred to hereafter as the
“Premier program”) on hospitals that serve poor patients. However,
they missed the essence of the question in their methodology, inter-
pretation, conclusion, and discussion. The disproportionate-share in-
dex, which these authors used to identify hospitals caring for poorer
populations, is also used by the Centers for Medicare & Medicaid
Services to compensate hospitals for caring for poorer Medicare pa-
tients and has never been validated as a marker for hospitals that care
for “poor” patients in general.
Hospitals that care for poor patients have a significant propor-
tion of uninsured patients, and their Medicare patients are the
“wealthiest” ones. The association between the disproportionate-
share index and the proportion of uninsured patients is unknown.
Medicare recipients comprised less than 30% of patients with heart
failure who were admitted to an inner-city teaching hospital that
cared for an indigent population (2).
In addition, the measurement tool that Jha and colleagues used
to assess the effects of the Premier program was invalid. Neither a
higher percentage of hospitals that conform to the Premier program’s
measures nor better conformity to quality care measures of the Pre-
mier program equates to higher health care quality and should not be
confused with actual improvement in health care quality and patient
outcome (3); the relation between the 2 entities is controversial at
best (3, 4).
Confusion between reporting higher statistics (which would
qualify hospitals for more payments) and actual improvement in
health care and patient outcome appears throughout Jha and col-
leagues’ article. In my experience working at a hospital that cared for
poor patients and received bonus payments from the Premier pro-
gram, conforming to heart failure quality measures defined by The
Joint Commission was associated with worse clinical outcomes—
specifically, a higher readmission rate for patients with heart failure
(2). Almost 15% of this patient population abused alcohol, 15%
abused illicit substances, and 22% were nonadherent to medications
and treatment regimens. The current quality measures are not ex-
pected to help any of these patients.
In hospitals caring for poor patients, scarce resources are redi-
rected away from patient care toward administrative efforts to
achieve better statistics that would qualify these institutions for a few
extra needed dollars from the Premier program. Report cards on
physician performance also greatly helped such hospitals to success-
fully achieve statistics that would please the wealthy payer. Many
practicing physicians can identify with the recently published per-
spective on these statistics (5).
The correct conclusion for Jha and colleagues’ study is that
hospitals that cared for poorer Medicare and Medicaid patients could
report higher rates of conformity to the Premier program. I do not
believe that we have achieved much.
Ishak A. Mansi, MD
Brooke Army Medical Center
Fort Sam Houston, TX 78234
Disclaimer: The views expressed herein are those of the author and do
not reflect the official policy or position of Brooke Army Medical Center,
the U.S. Army Medical Department, the U.S. Army Office of the Sur-
geon General, the Department of the Army, the Department of Defense,
or the U.S. government.
Potential Conflicts of Interest: None disclosed.
1. Jha AK, Orav EJ, Epstein AM. The effect of financial incentives on hospitals that
serve poor patients. Ann Intern Med. 2010;153:299-306. [PMID: 20820039]
2. Mansi IA, Shi R, Khan M, Huang J, Carden D. Effect of compliance with quality
performance measures for heart failure on clinical outcomes in high-risk patients. J
Natl Med Assoc. 2010;102:898-905. [PMID: 21053704]
3. Mansi IA. Public reporting and pay for performance [Letter]. N Engl J Med.
2007;356:1783. [PMID: 17465041]
4. Ko DT, Tu JV, Masoudi FA, Wang Y, Havranek EP, Rathore SS, et al. Quality of
care and outcomes of older patients with heart failure hospitalized in the United States
and Canada. Arch Intern Med. 2005;165:2486-92. [PMID: 16314545]
5. Ofri D. Quality measures and the individual physician. N Engl J Med. 2010;363:
606-7. [PMID: 20818853]
IN RESPONSE: Dr. Mansi highlights 2 important issues that we agree
warrant discussion. The first issue is the inadequacy of available data
on the proportion of poor patients cared for in any specific hospital.
We used the disproportionate-share index, which is a composite of
the proportion of elderly Medicare patients who are poor (defined as
receiving Supplemental Security Income) and the proportion of pa-
tients who are not elderly and have Medicaid insurance (1).
Dr. Mansi is correct that the disproportionate-share index ex-
cludes uninsured patients. Thus, if a hospital had a high proportion
of uninsured patients but also had very few elderly poor and noneld-
erly Medicaid patients, that hospital might be misclassified. How-
ever, we expect that misclassification occurs infrequently and that the
disproportionate-share index is therefore a reasonable proxy for these
data. Unfortunately, we are not aware of an available metric for a
national sample of hospitals that accounts for the proportion of un-
insured patients at a hospital.
Dr. Mansi’s second point on the inadequacies of the quality
measures is also worth noting. The quality measures adopted by the
Centers for Medicare & Medicaid Services capture only a small frac-
tion of care provided to patients. However, most of the processes of
care that comprise these measures are evidence-based, and many pro-
cesses are based on multiple randomized, controlled trials. Further-
more, studies (2) have demonstrated that hospitals that perform bet-
ter according to these measures are likely to have better patient
Dr. Mansi raises the concern that improvements in perfor-
mance according to these measures may be caused by better doc-
umentation or greater exclusion of marginal cases rather than
actual improvement in care. Other investigators (3) have ex-
pressed concern that greater attention to incentivized aspects of
care will cause other nonincentivized components to suffer. Al-
though these concerns seem reasonable, we are unaware of em-
pirical data that support these suggestions. Moreover, a recent
Annals of Internal Medicine
370 © 2011 American College of Physicians
analysis by Werner and Bradlow (4) offers some reassurance that
hospitals that improved their performance according to these pro-
cess measures had greater concomitant reductions in mortality
than hospitals that did not improve.
Although no single ideal measure of health care quality exists,
greater adherence to evidence-based care alone is surely worth ap-
plauding. The fact that hospitals with a high proportion of poor
elderly patients and poor patients who are not elderly were able to
improve their provision of these evidence-based services under finan-
cial incentives has important implications for ongoing efforts to en-
sure that all Americans receive high-quality health care.
Ashish K. Jha, MD, MPH
Arnold Epstein, MD, MA
Harvard School of Public Health
Boston, MA 02115
E. John Orav, PhD
Brigham and Women’s Hospital
Boston, MA 02120-1613
Potential Conflicts of Interest: Disclosures can be viewed at www.acponline
1. Centers for Medicare & Medicaid Services. Disproportionate share hospital (DSH).
Accessed at www.cms.gov/AcuteInpatientPPS/05_dsh.asp#TopOfPage on 20 October
2. Jha AK, Orav EJ, Li Z, Epstein AM. The inverse relationship between mortality
rates and performance in the Hospital Quality Alliance measures. Health Aff (Mill-
wood). 2007;26:1104-10. [PMID: 17630453]
3. Werner RM, Asch DA. The unintended consequences of publicly reporting quality
information. JAMA. 2005;293:1239-44. [PMID: 15755946]
4. Werner RM, Bradlow ET. Public reporting on hospital process improvements is
linked to better patient outcomes. Health Aff (Millwood). 2010;29:1319-24. [PMID:
Patients’ and Cardiologists’ Perceptions of the Benefits of
Percutaneous Coronary Intervention for Stable Coronary
TO THE EDITOR: The results of the study by Rothberg and col-
leagues (1) mirror the findings of previous research in different set-
tings and widespread clinical experience: Patient comprehension of
informed consent is often poor. Although such observations have led
some experts to conclude that expectations for patient understanding
and involvement in medical decision making are unreasonable, we
agree with Fernandez (2) that improving the quality of informed
consent is an attainable and important goal (3). In the discussion of
how to translate Rothberg and colleagues’ findings into practice
changes, 3 additional points warrant consideration.
First, more is not always better. Additional information on con-
sent forms does not guarantee that this information will be read or
understood and may even have the opposite effect (as anyone who
has signed a form without reading the fine print can attest). In a
similar manner, longer discussions with a knowledgeable cardiologist
do not translate into improved patient understanding of the benefits
of percutaneous coronary intervention (1). Efforts to improve in-
formed consent must focus not only on what information is given
but also on how such information is delivered and received. “Teach
back,” a technique in which patients repeat key elements of a discus-
sion to demonstrate understanding, can help to focus patients and
providers on what is important (4).
Second, timing is everything. Often in clinical practice, the con-
sent process occurs immediately before the procedure (that is, after
the decision to undergo the procedure has been made and the time
for weighing risks and benefits has passed). Additional information is
unlikely to be of value at this point because patients are psycholog-
ically committed to undergoing the procedure. If we expect patients
to engage in informed consent as a meaningful process of shared
decision making, we must give them time for contemplation before
having to decide.
Finally, we need strategies to improve informed consent that
do not involve physicians. Although the traditional model of
informed consent involves a discussion with the physician per-
forming the procedure, in reality such discussions are often ill-
timed or ineffective. Given the constraints of clinical practice,
this is not surprising. A busy gastroenterologist, for example, may
perform 15 colonoscopies (accompanied by 15 informed consent
discussions) in 1 day. Is it any wonder that informed consent
often amounts to little more than a signature on a form? Al-
though physicians must establish trust and answer questions, in-
teractive computer-based programs may be more suitable and
practical vehicles for improving patient understanding (5).
Yael Schenker, MD, MAS
Alan Meisel, JD
University of Pittsburgh
Pittsburgh, PA 15213
Potential Conflicts of Interest: None disclosed.
1. Rothberg MB, Sivalingam SK, Ashraf J, Visintainer P, Joelson J, Kleppel R, et al.
Patients’ and cardiologists’ perceptions of the benefits of percutaneous coronary inter-
vention for stable coronary disease. Ann Intern Med. 2010;153:307-13. [PMID:
2. Fernandez A. Improving the quality of informed consent: it is not all about the risks
[Editorial]. Ann Intern Med. 2010;153:342-3. [PMID: 20820045]
3. Schenker Y, Fernandez A, Sudore R, Schillinger D. Interventions to improve patient
comprehension in informed consent for medical and surgical procedures: a systematic
review. Med Decis Making. 2010. [PMID: 20357225]
4. Fink AS, Prochazka AV, Henderson WG, Bartenfeld D, Nyirenda C, Webb A, et al.
Enhancement of surgical informed consent by addition of repeat back: a multicenter,
randomized controlled clinical trial. Ann Surg. 2010;252:27-36. [PMID: 20562609]
5. Tait AR, Voepel-Lewis T, Moscucci M, Brennan-Martinez CM, Levine R. Patient
comprehension of an interactive, computer-based information program for cardiac
catheterization: a comparison with standard information. Arch Intern Med. 2009;169:
1907-14. [PMID: 19901144]
Limitations of the MEDLINE Database in Constructing
TO THE EDITOR: Focusing on the search limits “clinical trial” and
“human,” Winchester and Bavry (1) discuss the limitations of the
search limit function in MEDLINE with respect to constructing
1 March 2011 Annals of Internal Medicine Volume 154 • Number 5 371
meta-analyses. We agree with the authors that a thorough litera-
ture search is of the utmost importance in constructing meta-
analyses. This process requires a detailed description of the search
strategy and a thorough understanding of the limitations of Med-
ical Subject Headings (MeSH) and the search limit function of
The exact way in which the authors searched MEDLINE via the
PubMed interface is unclear because they provided few details about
their search strategy. The authors indicated searching for the term
epitifibatide. We assume that they used the correct term eptifibatide;
otherwise, fewer articles would have been identified.
The search strategy was suboptimal: The authors searched for
3 glycoprotein 2b/3a inhibitors by using free search terms and for
the substrate glycoprotein by using MeSH search terms only.
However, their goal might not have been to identify all data on
We were unable to reproduce the authors’ results. The com-
bined search ((eptifibatide OR tirofiban OR abciximab) OR platelet
glycoprotein GPIIb-IIIa complex[MeSH]) with a date limit of 1
February 2010 yielded 6515 articles. We found 1003 records by
using the search limit “clinical trial,” whereas the authors reported
finding 671 records.
Using search limits in PubMed can be tricky because most lim-
its are based on MeSH terms. Not all PubMed records are indexed.
New records are downloaded into PubMed as the publisher sup-
plies them and do not yet have MeSH terms. In addition,
OLDMEDLINE records are indexed with few MeSH terms, and
those that are included usually relate only to disease. When search
limits are used, researchers may miss important and relevant records
(2). The only search limits that can safely be used are dates and
languages that are not based on MeSH terms.
We recommend limiting a search by using a combination of
MeSH terms and free search terms. The Clinical Queries function in
PubMed (3) can be a good starting point. In this case, components
of the “Therapy” category and the “Broad” scope filter (4) relating to
clinical trials could be used together with free search terms, such as
“randomized, controlled trial,” to obtain more sensitive and precise
In addition, we recommend excluding all studies indexed as
“animals” but not as “humans” (NOT (Animals[MeSH] NOT
Humans[MeSH])). This step would ensure that no nonindexed
human studies are missed. Finally, in cases of uncertainty, con-
sulting a clinical or medical information specialist at a library is
Edith Leclercq, PhD
Cochrane Childhood Cancer Group
1105 AZ Amsterdam, the Netherlands
Bianca Kramer, PhD
Utrecht University Library
3508 TC Utrecht, the Netherlands
1066 EC Amsterdam, the Netherlands
Potential Conflicts of Interest: None disclosed.
1. Winchester DE, Bavry AA. Limitations of the MEDLINE database in constructing
meta-analyses [Letter]. Ann Intern Med. 2010;153:347-8. [PMID: 20820050]
2. Leclercq E, Leeflang MM, van Dalen EC, Kremer LC. Validation of a PubMed
search filter for identifying studies including children [Abstract]. Presented at the 2010
Joint Colloquium of the Cochrane & Campbell Collaborations, Keystone, Colorado,
18–22 October 2010.
3. National Center for Biotechnology Research. PubMed Clinical Queries. Accessed at
www.ncbi.nlm.nih.gov/sites/pubmedutils/clinical on 7 January 2011.
4. National Center for Biotechnology Research. PubMed Help: NCBI Help Manual.
Bethesda, MD: National Center for Biotechnology Information; 2005.
IN RESPONSE: We appreciate the comments from Dr. Leclercq and
colleagues about our letter on limitations of MEDLINE. The au-
thors elucidate the central point of our letter, which is that incom-
plete MeSH indexing can contribute to inaccurate searches when
limits are used. To clarify our search strategy, the precise search
string that we used was (“Platelet Glycoprotein GPIIb-IIIa Com-
plex”[MeSH]) OR eptifibatide OR abciximab OR tirofiban. On re-
peating this search with the publication date limit of 1 February
2010 and the search limit “clinical trial,” we found the same results
as Dr. Leclercq and colleagues: 1003 records.
After discussion with researchers at our institution and at the
National Library of Medicine, we believe that 2 factors account for
the increase in the number of studies from our initial report to the
present. First, although we used the publication date limit, additional
trials that were not in the database at the time of our search may
since have been added. Second, as each database entry is fully in-
dexed with the appropriate MeSH terms, the “clinical trial” term will
be applied and more studies will be properly identified by the search
limit. This reflects the shortcoming that we illustrated in our letter.
The suggestions to use the Clinical Queries function in
PubMed and to exclude studies indexed as “animal” are helpful. We
have found the support of medical librarians to be invaluable for
constructing database searches for our research projects and recom-
mend that other researchers seek library support as part of their
research methods. We agree that search limits must be used with
caution, and our suggested search strategy is a potential compromise
between a search that improperly eliminates important trials and one
that yields so many trials that hand-searching of abstracts is not
David E. Winchester, MD
Anthony A. Bavry, MD, MPH
University of Florida
Gainesville, FL 32610
Potential Conflicts of Interest: None disclosed.
372 1 March 2011 Annals of Internal Medicine Volume 154 • Number 5