The role of outpatient facilities in explaining variations in risk-adjusted readmission rates between hospitals.
ABSTRACT Validate risk-adjusted readmission rates as a measure of inpatient quality of care after accounting for outpatient facilities, using premature infants as a test case.
Surviving infants born between January 1, 1998 and December 12, 2001 at five Northern California Kaiser Permanente neonatal intensive care units (NICU) with 1-year follow-up at 32 outpatient facilities.
Using a retrospective cohort of premature infants (N=898), Poisson's regression models determined the risk-adjusted variation in unplanned readmissions between 0-1 month, 0-3 months, 3-6 months, and 3-12 months after discharge attributable to patient factors, NICUs, and outpatient facilities.
Prospectively collected maternal and infant hospital data were linked to inpatient, outpatient, and pharmacy databases.
Medical and sociodemographic factors explained the largest amount of variation in risk-adjusted readmission rates. NICU facilities were significantly associated with readmission rates up to 1 year after discharge, but the outpatient facility where patients received outpatient care can explain much of this variation. Characteristics of outpatient facilities, not the NICUs, were associated with variations in readmission rates.
Ignoring outpatient facilities leads to an overstatement of the effect of NICUs on readmissions and ignores a significant cause of variations in readmissions.
- [Show abstract] [Hide abstract]
ABSTRACT: Good quality indicators should have face validity, relevance to patients, and be able to be measured reliably. Beyond these general requirements, good quality indicators should also have certain statistical properties, including sufficient variability to identify poor performers, relative insensitivity to severity adjustment, and the ability to capture what providers do rather than patients' characteristics. We assessed the performance of candidate indicators of ICU quality on these criteria. Indicators included ICU readmission, mortality, several length of stay outcomes, and the processes of venous-thromboembolism and stress ulcer prophylaxis provision. Retrospective cohort study SETTING:: One hundred thirty-eight U.S. ICUs from 2001-2008 in the Project IMPACT database. Two hundred sixty-eight thousand eight hundred twenty-four patients discharged from U.S. ICUs. None. We assessed indicators' (1) variability across ICU-years; (2) degree of influence by patient vs. ICU and hospital characteristics using the Omega statistic; (3) sensitivity to severity adjustment by comparing the area under the receiver operating characteristic curve (AUC) between models including vs. excluding patient variables, and (4) correlation between risk adjusted quality indicators using a Spearman correlation. Large ranges of among-ICU variability were noted for all quality indicators, particularly for prolonged length of stay (4.7-71.3%) and the proportion of patients discharged home (30.6-82.0%), and ICU and hospital characteristics outweighed patient characteristics for stress ulcer prophylaxis (ω, 0.43; 95% CI, 0.34-0.54), venous thromboembolism prophylaxis (ω, 0.57; 95% CI, 0.53-0.61), and ICU readmissions (ω, 0.69; 95% CI, 0.52-0.90). Mortality measures were the most sensitive to severity adjustment (area under the receiver operating characteristic curve % difference, 29.6%); process measures were the least sensitive (area under the receiver operating characteristic curve % differences: venous thromboembolism prophylaxis, 3.4%; stress ulcer prophylaxis, 2.1%). None of the 10 indicators was clearly and consistently correlated with a majority of the other nine indicators. No indicator performed optimally across assessments. Future research should seek to define and operationalize quality in a way that is relevant to both patients and providers.Critical care medicine 04/2014; · 6.15 Impact Factor
- [Show abstract] [Hide abstract]
ABSTRACT: The Pediatric Quality Measures Program is developing readmission measures for pediatric use. We sought to describe the importance of readmissions in children and the challenges of developing readmission quality measures. We consider findings and perspectives from research studies and commentaries in the pediatric and adult literature, characterizing arguments for and against using readmission rates as measures of pediatric quality and discussing available evidence and current knowledge gaps. The major topic of debate regarding readmission rates as pediatric quality measures is the relative influence of hospital quality versus other factors within and outside of health systems on readmission risk. The complex causation of readmissions leads to disagreement, particularly when rates are publicly reported or tied to payment, about whether readmissions can be prevented and how to achieve fair comparisons of readmission performance. Despite these controversies, the policy focus on readmissions has motivated widespread efforts by hospitals and outpatient providers to evaluate and reengineer care processes. Many adult studies demonstrate a link between successful initiatives to improve quality and reductions in readmissions. More research is needed on methods to enhance adjustment of readmission rates and on how to prevent pediatric readmissions.Academic Pediatrics 09/2014; 14(5):S39–S46. · 2.23 Impact Factor
- [Show abstract] [Hide abstract]
ABSTRACT: Objective To examine the viability of a hospital readmission quality metric for infants requiring neonatal intensive care. Methods Two cohorts were constructed. First, a cohort was constructed from infants born in California from 1995 to 2009 at 23 to 34 weeks' gestation, using birth certificates linked to maternal and infant inpatient records (N = 343,625). Second, the Medicaid Analytic eXtract (MAX) identified Medicaid-enrolled infants admitted to the neonatal intensive care unit (NICU) during their birth hospitalization in 18 states during 2006 to 2008 (N = 254,722). Hospital and state-level unadjusted readmission rates and rates adjusted for gestational age, birth weight, insurance status, gender, and common complications of preterm birth were calculated. Results Within California, there were wide variations in hospital-level readmission rates that were not completely explained through risk adjustment. Similar unadjusted variation was seen between states using MAX data, but risk adjustment and calculation of hospital-level rates were not possible because of missing gestational age, birth weight, and birth hospital data. Conclusions The California cohort shows significant variation in hospital-level readmission rates after risk adjustment, supporting the premise that readmission rates of prematurely born infants may reflect care quality. However, state data do not include term and early term infants requiring neonatal intensive care. MAX allows for multistate comparisons of all infants requiring NICU care. However, there were extensive missing data in the few states with sufficient information on managed care patients to calculate state-level measures. Constructing a valid readmission measure for NICU care across diverse states and regions requires improved data collection, including potential linkage between MAX data and vital statistics records.Academic Pediatrics 09/2014; 14(5):S47–S53. · 2.23 Impact Factor
The Role of Outpatient Facilities in
Explaining Variations in Risk-Adjusted
Readmission Rates between Hospitals
Scott A. Lorch, Michael Baiocchi, Jeffrey H. Silber,
Orit Even-Shoshan, Gabriel J. Escobar, and Dylan S. Small
care after accounting for outpatient facilities, using premature infants as a test case.
Study Setting. Surviving infants born between January 1, 1998 and December 12,
2001 at five Northern California Kaiser Permanente neonatal intensive care units
(NICU) with 1-year follow-up at 32 outpatient facilities.
Study Design. Using a retrospective cohort of premature infants (N5898), Poisson’s
regression models determined the risk-adjusted variation in unplanned readmissions
between 0–1 month, 0–3 months, 3–6 months, and 3–12 months after discharge attrib-
utable to patient factors, NICUs, and outpatient facilities.
Data Collection. Prospectively collected maternal and infant hospital data were
linked to inpatient, outpatient, and pharmacy databases.
Principal Results. Medical and sociodemographic factors explained the largest
associated with readmission rates up to 1 year after discharge, but the outpatient facility
where patients received outpatient care can explain much of this variation. Character-
istics of outpatient facilities, not the NICUs, were associated with variations in read-
Conclusion. Ignoring outpatient facilities leads to an overstatement of the effect of
NICUs on readmissions and ignores a significant cause of variations in readmissions.
Key Words. Quality of care assessment, readmissions, premature infants
The assessment of health care quality has become more important with the
increase in information available to the general public, which includes Med-
icare’s ‘‘Hospital Compare’’ website (U.S. Department of Health & Human
Services 2008) and the publication of hospital infection rates in Pennsylvania
(Pennsylvania Health Care Cost Containment Council 2008). One proposed
measure is risk-adjusted readmission rates. Early research (Ashton et al. 1997)
rHealth Research and Educational Trust
suggested that lower-quality providers incompletely evaluate or manage a
patient in 13 studies, leading to higher readmission rates. Similar results were
noted more recently for coronary artery bypass graft surgery (Hannan et al.
2003). However, recently published work in congestive heart failure (Ko-
ssovsky et al. 2000; Luthi et al. 2004; Fonarow et al. 2007) did not find such an
association, leading to questions about the value of readmission rates as a
quality measure (Clarke 2004). There are several potential reasons for these
conflicting results. Hospitals may have different admission criteria for patients
(Goodman et al. 1994). Available risk-adjustment models may not adequately
control for these differences. The prevalence of readmissions may be too low
to detect a difference between hospitals. Finally, most studies did not account
for the outpatient facility where the patient receives care after discharge. Out-
patient care may influence a patient’s risk of readmission, as demonstrated by
research on medication discrepancies (Coleman et al. 2005) or other inter-
ventions (Rich et al. 1995; Benbassat and Taragin 2000). However, no study
has examined the combined effect of inpatient and outpatient facilities on
To properly evaluate risk-adjusted readmission rates as an inpatient
quality of care measure, we must examine a population of patients with high
readmission rates, uniform admission criteria, and valid measures of inpatient
care as a ‘‘test case.’’ Prematurely born infants are such a group. Besides their
high overall readmission rates, (Cavalier et al. 1996; Furman et al. 1996; Joffe
et al. 1999; Smith et al. 2004; Morris, Gard, and Kennedy 2005; Underwood,
Danielsen, and Gilbert 2007) all infants born under 34 weeks gestational age
Address correspondence to Scott A. Lorch, M.D., M.S.C.E., Department of Pediatrics, Center for
Outcomes Research, The Children’s Hospital of Philadelphia, The University of Pennsylvania
School of Medicine; The Leonard Davis Institute of Health Economics, The University of Penn-
sylvania; Division of Neonatology, The Children’s Hospital of Philadelphia; 3535 Market Street,
the Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, PA.
JeffreyH.Silber,M.D.,Ph.D.,iswiththeCenterfor Outcomes Research,TheChildren’sHospital
of Philadelphia; Departments of Pediatrics and Anesthesiology and Critical Care, The University
of Pennsylvania School of Medicine; Department of Health Care Management, The Wharton
School, The University of Pennsylvania; The Leonard Davis Institute of Health Economics, The
Hospital of Philadelphia, Center for Outcomes Research, The Leonard Davis Institute of Health
Economics, The University of Pennsylvania, Philadelphia, PA. Gabriel J. Escobar, M.D., is with
the Systems Research Initiative & Perinatal Research Unit, Kaiser Permanente Division of Re-
search, Oakland, CA. Dylan S. Small, Ph.D., is with the Department of Statistics, The Wharton
School, University of Pennsylvania, Philadelphia, PA.
2 HSR: Health Services Research xx:xx
(GA) are admitted to a neonatal intensive care unit (NICU), which eliminates
differences in admission criteria. Prior work has also suggested that higher
quality NICUs have higher volumes and lower complication rates than their
peers (Phibbs et al. 1996; Lorch et al. 2007; Phibbs et al. 2007). Premature
infants, who are interesting in their own right, also can serve as an analog for
patients with other chronic illnesses, such as congestive heart failure. The goal
inpatient quality of care by studying their role in NICUs. The specific aims
were to (1) determine the statistical significance and explained variation in
risk-adjusted readmission rates attributable to the NICU when outpatient fa-
cilities are omitted——the most commonly used analysis; (2) determine the
change in explained variation when outpatient facilities were added to the
analysis; and (3) measure the association between readmission rates and
Lorch et al. 2007; Phibbs et al. 2007). To answer these three aims, we will
present the results of three separate analyses using data from a retrospective,
population cohort of premature infants with complete follow-up data through
1 year after discharge. The first, ‘‘naı ¨ve’’ analysis determines the variation in
readmission rates attributable to the NICU while ignoring outpatient facilities
variation explained by NICUs and outpatient facilities (aim 2). The third aim
of this study will clarify which previously validated facility-quality metrics are
associated with lower readmission rates.
The Infant Functional Status Study (IFS) (Bakewell-Sachs et al. 2009) was a
Permanente Medical Center Program (KPMCP), which is an integrated man-
aged care organization whose perinatal outcomes have been previously de-
scribed (Escobar et al. 1995; Escobar 1999; Joffe et al. 1999; Newman et al.
1999; Smith et al. 2004). Infants eligible for the IFS study were born at five
KPMCP hospitals between 1998 and 2001 with a GA of 34 weeks or less and
received outpatient care at 1 of 32 outpatient practices between 1998 and
by the American Academy of Pediatrics and had an average daily census
over 10. None of these units trained pediatric residents (Silber et al. 2009).
Outpatient Facilities and Hospital Readmission Rates3
Exclusion criteria included major congenital anomalies; need for mechanical
ventilation at home after discharge; placement of a ventriculo-peritoneal
shunt; and loss to follow-up within 1 year of discharge because of cancellation
of32 weeks orless.Excluded infantsinthis GArange were similartoincluded
infants in terms of demographic factors, length of NICU stay, and need for
mechanical ventilation. The Institutional Review Boards of both The Chil-
dren’s Hospital of Philadelphia and KPMCP approved this project.
Theinitial sourceforourdata wastheKaiserPermanenteNeonatal Minimum
Data Set, which prospectively collects information on demographic, maternal
andbirthhistory,andthepostnatalhospital courseforallNICU admissionsin
KPMCP. These data are validated monthly through random chart review
(Escobar et al. 1997). We then linked these records to the KPMCP hospital-
ization database, the outpatient visit database, and the pharmacy database.
Data on medical care occurring outside the KPMCP system, which made up
o1 percent of all encounters, were also included using insurance claims re-
ceived by KPMCP.
Site of Care
Each infant was assigned to the NICU that provided the majority of care over
the last month of the child’s NICU course. We assigned each infant to the
outpatient facility used for a minimum of 50 percent of all well-child visits.
These visits were identified through an ICD-9CM code of V20.0 or V20.1.
Our primary outcome was any hospital admission between 0 and 1 month of
NICU discharge; 0–3 months; 3–6 months; and 3–12 months. The time
frames were chosen to encompass the wide range of times (between 30 days
after discharge and 1 year) in previous studies. Even though the validity of an
association between inpatient care and a rehospitalization 90 or more days
ignoring outpatient facilities could result in an association between late re-
hospitalizations and inpatient care. As a secondary analysis, we limited our
analysis to those conditions that may be reasonably associated with the care
delivered by a NICU. These conditions included (1) all respiratory readmis-
sions, as NICUs have different rates of bronchopulmonary dysplasia (BPD)
4 HSR: Health Services Research xx:xx
(Zeitlin et al. 2008), (2) failure-to-thrive or placement of gastrostomy tube, as
premature infants have high rates of feeding difficulties (Rommel et al. 2003),
and (3) eye surgery from retinopathy of prematurity (Zeitlin et al. 2008).
Confounding Variable Definitions
We included factors previously associated with increased readmission rates of
premature infants in the literature (Cavalier et al. 1996; Furman et al. 1996;
Joffe et al. 1999; Smith et al. 2004; Morris, Gard, and Kennedy 2005; Un-
from available clinical and ultrasound data; gender; mother’s racial/ethnic
status; and 12-hour SNAP score as a measure of initial illness severity. Four
supplemental oxygen support at 36 weeks postmenstrual age (Shennan et al.
Kliegman 1986); stage III or IV intraventricular hemorrhage (Papile et al.
1978); and stage II or higher retinopathy of prematurity (Anonymous 1984).
The number of previous children from obstetric records was used as a proxy
for the number of siblings currently in the household (Lorch et al. 2007).
Individual information on income during the year of the study was not avail-
able, so we used the median household income of the home zip code of the
family as a proxy measure (Krieger et al. 2003).
Development of Risk-Adjustment Model
We developed a severity adjustment model using all confounding factors
listedabove exceptformedicalcomplications because they maybeassociated
with the quality of NICU care received by the child (Rosenbaum 1984). Non-
reduced model was statistically different from the full model using the like-
lihood ratio test.
We defined several NICU characteristics that may be associated with poor
quality: NICU volume and average postmenstrual age at discharge. Higher
volume has been suggested as a measure of NICU quality (Phibbs et al. 1996,
2007; Lorch et al. 2007). Discharge at a younger postmenstrual age, when
infants may be less physiologically mature, may result in higher readmission
rates. We also defined several risk-adjusted characteristics of outpatient facil-
Outpatient Facilities and Hospital Readmission Rates5
efficacy (Gadomski et al. 1994; Patel, Gouin, and Platt 2003) or prior con-
sensus conference recommendations (National Heart Lung and Blood Insti-
tute National Asthma Education and Prevention Program 1997, 2003).
(2) Higher-than-expected use of antibiotics for viral conditions
(3) Lower-than-expected use of inhaled albuterol for ongoing respira-
We dichotomized each variable into higher- and lower-than-expected
categories and added singly to each readmission model.
To determine the amount of variation attributable to the NICU or to the
outpatient facility and address potential colinearity between these factors, we
constructed several multivariable Poisson’s models. We added NICU indica-
patient factors listed above in the following order:
(1) NICU indicators alone (Model N);
(2) NICU indicators, then outpatient facility indicators (Model N1O);
(3) Outpatient facility indicators, then NICU indicators (Model O1N).
We useda measureofR2(Cameron and Windmeijer1996) forPoisson’s
models that maintains the interpretation of R2as ‘‘explained variation.’’ We
report the marginal contribution of the NICU and outpatient factors to the R2
calculation. The variation attributed to the NICU should be similar in the
N1O and O1N models if the variation contributed by the NICU and out-
patient facilities was independent of each other. However, the results would
differ if some of the impact of NICUs on readmission rates is a result of the
outpatient facilities they discharge patients to. Statistical significance was de-
termined from the contribution of the NICU and outpatient facility indicators
in full model. These p-valuesdo not change whetherwe usedtheN1O model
or the O1N model. For aim 3, we added facility and NICU characteristics to
the final risk-adjustment model and report the incident rate ratio for each
readmission outcome measure. Outpatient facility characteristics were added
singly to the final model because of colinearity between these characteristics.
Standard errors were calculated with bootstrap techniques because of
potential overdispersion of the data.
6HSR: Health Services Research xx:xx
Five NICUs and 32 outpatient sites were identified as sites of care for the 892
infants in this study. Each NICU sent children to a median of 12 outpatient
centers (range 9–16) after discharge, whereas over 80 percent of the outpatient
centers received infants from two or fewer NICUs. The infants had an average
GA of 29.5 ? 2.2 weeks and an average birth weight of 1.358 ? 428g. 16.6
percent had a diagnosis of BPD and 1.8 percent had NEC. 45.5 percent were
white non-Hispanics, 20.5 percent were white Hispanics, and 11.2 percent
were black. There were 330 readmissions within 1 year of discharge
(Figure 1). The most common reasons for a readmission were respiratory vi-
ral illnesses, diarrhea, and dehydration. Over 52 percent of the readmissions
occurred within 3 months of discharge, with a steady decrease until 12 months
after discharge (po.05 by ANOVA for trend). A similar trend was noted for
Patient Factors Associated with Readmissions
Several medical and socioeconomic factors were associated with higher re-
admission rates (Table 1). Younger GA was strongly associated with higher
readmission rates, regardless of the chosen time frame after discharge. NEC
was the only complication associated with higher readmission rates 0–3
months after discharge. Among the socioeconomic factors, only additional
children at home were associated with higher readmissions. SNAP score was
not associated with any readmission measure and was dropped from the final
Peak time for readmission was within 4 months of discharge, with a consistent
but low volume of readmissions out to 1 year
The Timing of Hospital Readmission after Hospital Discharge.
Outpatient Facilities and Hospital Readmission Rates7
Explained Variation by NICUs and Outpatient Facilities
Figure 2 shows the amount of variation in each readmission measure that was
Hospital Readmissions within Certain Times after Discharge from the NICU
Adjusted Associations between Patient-Level Risk Factors and
0–1 months0–3 months3–6 months3–12 months
95% CI IRRn
Gestational age (weeks)
IVH grades III–IV
ROP stages 2 or higher
Discharge on oxygen
Each additional child
Increase in median income
(per U.S. $10,000)
Added average day
Additional preterm infant
Higher oral albuterol use
Higher viral antibiotic use 0.89
Lower inhaled albuterol
Note: All estimates control for patient-level factors included in this table. All standard errors
calculatedusingbootstrapestimates tocontrolfor potentialoverdispersionofdata,usingPoisson’s
regression models to calculate the adjusted change in incidence rate for readmissions within a
given time period conditioned on each patient-level or facility-level factor.
nIRR, incident rate ratio.
wOutpatient facility variables dichotomized into ‘‘higher’’ or ‘‘lower’’ than expected rates of each
8HSR: Health Services Research xx:xx
variation in readmission rates regardless of the time after discharge. In the
NICU care was significantly associated with readmissions 0–3, 3–6, and 3–12
months after discharge. When we added the outpatient facilities to this naı ¨ve
model (Model N1O), very little additional variation was attributed to
the outpatient facilities. However, when we included outpatient facilities in
the analysis before the NICU (Model O1N), much of the variation in read-
mission rates is now attributed to the outpatient facility. For example, these
three groups of factors explained 14.2 percent of the variation in readmissions
Level Factors (black bars), Site of NICU Care (gray bars), or Site of Outpatient
Care (white bars)
Amount of Variation in Readmission Rates Attributed to Patient-
Patient factors and NICU factors (Model N); (1) Patient factors, NICU factors, and
outpatient facility variables (Model N1O); or (2) Patient factors, outpatient facility
variables, and NICU factors (Model O1N). The marginal contribution to R2from the
addition of a variable is shown. Outcome measures where the NICU was statistically
significant in the naı ¨ve Models N are starred; outcome measures where the outpatient
Outpatient Facilities and Hospital Readmission Rates9
between 3 and 12 months after discharge. Models N and N1O attribute 29.6
2) and 6.3 percent of this explainable variation to the outpatient facility (0.9/
variation attributable to NICUs and outpatient facilities is reversed: now, 29.6
percent of the variation is attributed to the outpatient facilities and charac-
teristics, while only 6.3 percent of the variation was attributed to the NICUs.
The collinearity between NICUs and outpatient facilities makes it difficult to
confidently assign variations in readmission rates to the NICUs alone using
just this type of data.
with readmissions at 0–3, 3–6, and 3–12 months after discharge, NICUs lost
their significant association with readmission rates when both NICU and out-
patient facilities were analyzed together. Outpatient facilities were signifi-
cantly associated with readmissions between 3–6 and 3–12 months after
discharge. These results did not change when we limited the study to patients
who attended outpatient facilities that cared for 10 or more patients in our
cohort. For the predefined list of NICU-sensitive conditions, neither the site
of NICU care nor the site of outpatient care was associated with significant
variation at any time point after discharge.
Outpatient Facility Characteristics Associated with Increased Readmission Rates
The results from our first two aims suggest that, when outpatient facilities are
ignored from an analysis of NICUs, variations in readmission rates may be
erroneously attributed to the NICU when they could be attributed to atten-
dance at specific outpatient facilities or the combination of the two facility
types. To further explore these results, we then examined the association
between facility characteristics and readmission rates (Table 1). No charac-
teristic of the NICU was associated with higher readmission rates except day
of discharge and readmissions 3–6 months after discharge. Use of oral
albuterol at higher-than-expected rates was associated with more readmis-
sions within 3 months of NICU discharge [incident rate ratio (IRR) 1.74, 95
percent CI 1.18–2.56], and higher use of antibiotics for viral illness was
associated with a higher rate of readmissions between 3 and 6 months after
NICU discharge (IRR 2.09, 95 percent CI 1.08–4.07) and between 3 and 12
months after NICU discharge (IRR 1.92, 95 percent CI 1.11–3.31). Neither
measure is recommended treatment (Figure 3). Our results did not change
when we limited the analysis to patients who attended facilities that care
10HSR: Health Services Research xx:xx
for ten or more infants in our cohort. When we analyzed these factors as
continuous variables, instead of dichotomized into ‘‘higher’’ or ‘‘lower-than-
expected,’’ we found similar results. Use of oral albuterol at higher rates was
associated with more readmissions within 3 months of NICU discharge, up
to a 9 percent higher observed rate (IRR for 1 percent difference between
observed and expected rates of use 1.06 [95 percent CI 1.02–1.10)]. Above
this threshold, the risk of readmission decreased slightly, suggesting that the
association is highest around 9 percent. When observed rates of antibiotic
use increased from 0 to 5 percent above expected, each 1 percent increase
was associated with a 41 percent (3–6 monthsafter discharge) and 38 percent
increase (3–12 months after discharge) in readmissions. Changes in antibi-
otic rates above the 5 percent level were not associated with any further
increase in risk of readmission. No other facility variable when analyzed as a
continuous measure was associated with a change in readmission rates at any
Measures of Outpatient Quality of Care
Adjusted Association between Readmission Rates and Three
Note: For each measure, the incidence rate ratio is the change in readmission rates
by attending a poor-quality facility compared with a high-quality facility. Statistically
expected use of oral albuterol or antibiotics for viral conditions had higher rates of
readmissions at three of the four time periods studied.
Outpatient Facilities and Hospital Readmission Rates 11
Although insurers and public health agencies use outcome measures of in-
patient care because of their ease of use and ready availability, these measures
demonstrate this problem. Although early work found higher readmission
rates at facilities with inadequate processes of care, such as poor discharge
readiness and medication changes close to discharge (Ashton et al. 1997),
there is no conclusive evidence to use risk-adjusted readmission rates as a
measure of quality of inpatient care for all conditions. These prior studies did
not account for the quality of care provided by outpatient facilities after dis-
charge. Our study examined the effect of both inpatient and outpatient fa-
cilities on risk-adjusted readmission rates at various time periods after
discharge with premature infants treated in NICUs as a test case. We initially
found a large variation in readmission rates across NICUs up to 1 year after
discharge. These results are more complex, as much of this variation may be
with higher readmission rates were more likely to have decreased quality of
care, whereas no characteristic of poor-quality NICUs was associated with
higher readmission rates. Outpatient facilities, not NICUs, were statistically
associated with changes in readmission rates when both sites of care were
included in the analysis. Therefore, before using readmission rates to measure
care received by the patients after discharge. Alternatively, for outcomes such
as readmission rates that measure the care of more than one provider, vari-
ations may suggest something about the quality of the inpatient–outpatient
dyad, not one or the other. Because most outpatient providers admit patients
to a limited number of facilities, similar analyses should occur to determine
whether variations in readmission rates for other conditions measure the care
provided by hospitals, outpatient practices, or a combination of the two.
Our study is consistent with prior work that failed to detect an associ-
ation between lower-quality inpatient care and higher readmission rates.
There are several potential explanations. Severity of illness was consistently
as seen in other studies (Weissman et al. 1999; Kossovsky et al. 2000). When
the effect of inpatient care on readmission risk is greatest, a patient’s illness
severity may overwhelm the effect from NICUs. We saw this effect, as patient
factors explained 10 times more variation than NICU site of care within 3
months of discharge. It is possible that readmission rates immediately after
12 HSR: Health Services Research xx:xx
discharge may distinguish higher-performing hospitals from lower-quality
facilities at hospitals that see thousands of very-low-birth weight infants each
year. However, our data suggest that illness severity will continue to be a
powerful predictor of readmissions and hamper our ability to detect clinically
relevant differences between hospitals.
Further after discharge, when the association between illness severity
and readmission rates begins to lessen, other factors such as the quality of
outpatient care and sociodemographic factors become more important. The
problem with analyzing the effect of NICUs on the outcomes of premature
the care provided by the NICU; much of this care is also carried out in an
outpatient setting. The fact that hospitals tend to discharge patients to a group
of outpatient facilities must be accounted for in any analysis of readmission
rates. When outpatient facilities were added to the typical, naı ¨ve model
(Model N), little additional explained variation was added to the model.
However, when we reverse the inclusion sequence, the amount of variation
associated with outpatient facilities was reversed. This feature of the data, and
the fact that characteristics of poor outpatient facilities, not characteristics of
NICUs, were associated with higher readmission rates, support the idea
that both inpatient and outpatient sites of care must be accounted for in any
analysis of readmission rates as a quality measure.
The explained variation in readmission rates found in this study is con-
sistent with other studies of length of stay or costs, which found an R2of 10–18
percent for various risk adjustment models that predict health care payments
and an R2of 9–14 percent for predicted length of stay for patients with pneu-
monia (Schwartz and Ash 2003). In each case, there is random variation in
these outcomes. For readmissions, potential causes of random variation in-
clude accidental trauma or an outbreak of influenza. As with length of stay or
cost, though, the combination of inpatient and outpatient facilities is signifi-
cantly associated with variations in readmissions even though many readmis-
sions are unavoidable.
the time after discharge increased. Besides the quality of these outpatient fa-
cilities, the outpatient site may add information about the patients who attend
that facility (Huang et al. 2005). The choice to visit a given caregiver may
provide additional information about the patient, which is not otherwise mea-
surable. For example, a recent study of medical report cards found that ap-
proximately 40 percent of patients would choose a provider based on their
interpersonal skills rather than their measured technical quality (Fung et al.
Outpatient Facilities and Hospital Readmission Rates13
2005). Other studies have shown that factors such as gender and race may
influencethechoice of a provider (Montgomeryand Fahey 2001; Schnatz et al.
2007). Some of the variations in readmission rate explained by outpatient site
This study has several limitations. Our data included all inpatient and
outpatient encounterswiththemedical system,linkedtoclinicaldata fromthe
care experienced by the child after discharge. Also, physicians who practice
by adhering more rigidly to practice guidelines (Schur, Mueller, and Berk
1999). This self-selection could affect the quality of care provided by the staff.
Even so, we found significant differences in the care provided by individual
NICUs, who had different styles of discharging prematurely born infants, and
outpatient facilities. Finally, some inpatient or outpatient facilities may have
unstable results because of their small volume of patients. Since some patients
will continue to receive treatment at low-volume facilities, additional work
needs to develop fair methods of measuring the care provided by these fa-
cilities. Our results remained similar after excluding low-volume facilities,
suggesting that attendance at very small facilities was not the primary expla-
nation for the results in our study.
In summary, measures such as readmission rates that are influenced by
the care received by multiple different providers may not be able to assess the
care of one group independent of the other. Similar evaluations should occur
for other conditions before using risk-adjusted readmission rates to measure
inpatient or outpatient quality of care.
Joint Acknowledgment/Disclosure Statement: This project was funded by a grant
from the Maternal and Child Health Bureau, grant number 1 R40 MC05474-
01-00. The authors have no conflicts of interest to disclose. The authors would
with data collection, and John D. Greene for his assistance with data man-
agement in this project.
14HSR: Health Services Research xx:xx
Anonymous. 1984. ‘‘An International Classification of Retinopathy of Prematurity.’’
Pediatrics 74 (1): 127–33.
Ashton,C.M.,D.J.DelJunco,J.Souchek, N.P.Wray, andC. L.Mansyur. 1997. ‘‘The
Association between the Quality of Inpatient Care and Early Readmission: A
Meta-analysis of the Evidence.’’ Medical Care 35 (10): 1044–59.
Bakewell-Sachs, S., B. Medoff-Cooper, G. J. Escobar, J. H. Silber, and S. A. Lorch.
2009. ‘‘Infant Functional Status: The Timing of Physiologic Maturation of Pre-
mature Infants.’’ Pediatrics 123 (5): e878–86.
Benbassat, J., and M. Taragin. 2000. ‘‘Hospital Readmissions as a Measure of Quality
of Health Care: Advantages and Limitations.’’ Archives of Internal Medicine
160 (8): 1074–81.
Cameron, A. C.,andF.A. G.Windmeijer. 1996. ‘‘R-squared Measures for CountData
Regression Models with Applications to Health-care Utilization.’’ Journal of
Business and Economic Statistics 14 (2): 209–20.
Cavalier, S., G. J. Escobar, S. A. Fernbach, C. P. Jr. Quesenberry, and M. Chellino.
Infants: Experience in a Large Managed Care Organization.’’ Pediatrics 97 (5):
Clarke, A. 2004. ‘‘Readmission to Hospital: A Measure of Quality or Outcome?’’
Quality and Safety in Health Care 13 (1): 10–1.
Coleman, E. A., J. D. Smith, D. Raha, and S. J. Min. 2005. ‘‘Posthospital Medication
165 (16): 1842–7.
Escobar, G. J. 1999. ‘‘The Neonatal ‘Sepsis Work-up’: Personal Reflections
on the Development of an Evidence-based Approach toward Newborn
Infections in a Managed Care Organization.’’ Pediatrics 103 (1 Suppl E):
Escobar, G. J., A. Fischer, R. Kremers, M. S. Usatin, A. M. Macedo, and M. N.
Gardner. 1997. ‘‘Rapid Retrieval of Neonatal Outcomes Data: The Kaiser
Permanente Neonatal Minimum Data Set.’’ Quality Management in Health Care
5 (4): 19–33.
Neonatal Acute Physiology: Validation in Three Kaiser Permanente Neonatal
Intensive Care Units.’’ Pediatrics 96 (5, Part 1): 918–22.
Fonarow, G. C., W. T. Abraham, N. M. Albert, W. G. Stough, M. Gheorghiade, B. H.
Greenberg, C. M. O’Connor, K. Pieper, J. L. Sun, C. Yancy, and J. B. Young.
2007. ‘‘Association between Performance Measures and Clinical Outcomes for
Patients Hospitalized with Heart Failure.’’ Journal of the American Medical Asso-
ciation 297 (1): 61–70.
Fung, C. H., M. N. Elliott, R. D. Hays, K. L. Kahn, D. E. Kanouse, E. A. McGlynn,
M. D. Spranca, and P. G. Shekelle. 2005. ‘‘Patients’ Preferences for Technical
versus Interpersonal Quality When Selecting a Primary Care Physician.’’ Health
Services Research 40 (4): 957–77.
ofMedical Servicesby High-risk
Outpatient Facilities and Hospital Readmission Rates15