Effect of critical access hospital conversion on patient safety.
ABSTRACT The Medicare Rural Hospital Flexibility Program of the 1997 Balanced Budget Act allowed hospitals meeting certain criteria to convert to critical access hospitals (CAH) and changed their Medicare reimbursement mechanism from prospective payment system (PPS) to costbased.
To examine the impact of CAH conversion on hospital patient safety.
Secondary data on hospital patient safety indicators (PSIs), hospital CAH status, patient casemix, and market variables, for 89 Iowa rural hospitals during 19972004.
We employed quasiexperimental designs that use both control groups and pretests. The hospitalyear was the unit of analysis. We used generalized estimating equations logit and randomeffects Tobit models to assess the effects of CAH conversion on hospital patient safety. The models were adjusted for patient casemix and market variables. Sensitivity analyses, which varied by sample and statistical model, were used to examine the robustness of our findings.
PSIs were computed from Iowa State Inpatient Databases (SIDs) using Agency for Healthcare Research and Quality indicators software. Hospital CAH status was extracted from Iowa Hospital Association. Patient casemix variables were extracted from Iowa SIDs. Market variables came from Area Resource File (ARF).
CAH conversion in Iowa rural hospitals was associated with better performance of riskadjusted rates of iatrogenic pneumothorax, selected infections due to medical care, accidental puncture or laceration, and composite score of four PSIs, but had no significant impact on the observed rates of death in lowmortality diagnosisrelated groups (DRGs), foreign body left during procedure, riskadjusted rate of decubitus ulcer, or composite score of six PSIs. Conclusion. CAH conversion is associated with enhanced performance of certain PSIs.

Article: The AHRQ Health IT Value Grant Initiative: A Programmatic Review of the PeerReviewed Literature
 [Show abstract] [Hide abstract]
ABSTRACT: The purpose of this study was to develop a marketing plan for the Physical and Occupational Therapy (PT/OT) department at a Critical Access Hospital (CAH). We took the approach of understanding and analyzing the rural community and health care environment, problems faced by the PT/OT department, and developing a strategic marketing plan to resolve those problems. We used hospital admissions data, public and physician surveys, a SWOT analysis, and tools to evaluate alternative strategies. Lack of awareness and negative perception were key issues. Recommended strategies included building relationships with physicians, partnering with the school district, and enhancing the wellness program.Health Marketing Quarterly 07/2013; 30(3):26380.  SourceAvailable from: John E SchneiderJames Bahensky, John Ely, Paul James, Robert Ohsfeldt, John Schneider, Tanya Uden, Tom Vaughn, Bonnie Wakefield, Doug Wakefield, Veerasathpurush Allareddy, [......], Pengxiang Li, Troy Mills, Kwame Nyarko, Qian Qui, Lance Roberts, Smruti Vartak, Tracy Lewis, Bren Lowe, Belinda Udeh, Mirou Jaana
Page 1
RESEARCH ARTICLE
Hospitals
Effect of Critical Access Hospital
Conversion on Patient Safety
Pengxiang Li, John E. Schneider, and Marcia M. Ward
Background. The Medicare Rural Hospital Flexibility Program of the 1997 Balanced
Budget Act allowed hospitals meeting certain criteria to convert to critical access hos
pitals (CAH) and changed their Medicare reimbursement mechanism from prospective
payment system (PPS) to costbased.
Objective. To examine the impact of CAH conversion on hospital patient safety.
Data Source. Secondary data on hospital patient safety indicators (PSIs), hospital
CAH status, patient casemix, and market variables, for 89 Iowa rural hospitals during
1997–2004.
Study Design. Weemployedquasiexperimentaldesignsthatusebothcontrolgroups
and pretests. The hospitalyear was the unit of analysis. We used generalized estimating
equationslogitandrandomeffectsTobitmodelstoassesstheeffectsofCAHconversion
on hospital patient safety. The models were adjusted for patient casemix and market
variables. Sensitivity analyses, which varied by sample and statistical model, were used
to examine the robustness of our findings.
DataExtractionMethods. PSIswerecomputedfromIowaStateInpatientDatabases
(SIDs) using Agency for Healthcare Research and Quality indicators software. Hospital
CAH status was extracted from Iowa Hospital Association. Patient casemix variables
wereextractedfromIowaSIDs.MarketvariablescamefromAreaResourceFile(ARF).
Principal Findings. CAH conversion in Iowa rural hospitals was associated with
better performance of riskadjusted rates of iatrogenic pneumothorax, selected infec
tionsduetomedicalcare,accidentalpunctureorlaceration,andcompositescoreoffour
PSIs, but had no significant impact on the observed rates of death in lowmortality
diagnosisrelated groups (DRGs), foreign body left during procedure, riskadjusted rate
of decubitus ulcer, or composite score of six PSIs.
Conclusion. CAH conversion is associated with enhanced performance of certain
PSIs.
Key Words. AHRQ patient safety indicators, critical access hospital, policy
evaluation, observational data quasiexperiments
The Institute of Medicine (IOM) definition of patient safety is ‘‘freedom from
accidentalinjuryduetomedicalcare,ormedicalerrors’’(Kohn,Corrigan,and
r Health Research and Educational Trust
DOI: 10.1111/j.14756773.2007.00731.x
2089
Page 2
Donaldson 1999). More than 44,000 Americans die as a result of preventable
medical errors each year and national costs of preventable adverse events
were approximately $17 billion in 1996 (Kohn, Corrigan, and Donaldson
1999).
Ruralhospitalsplayakeyroleinprovidinghealthservicesinruralareas.
Quality of care is one of the main issues that rural hospitals face. In improving
quality of care, rural hospitals face several problems, including financial con
straints (Moscovice, Gregg, and Klingner 2002), limited resources and knowl
edge (Moscovice 1989), and staffing (Moscovice and Stensland 2002). It was
difficultformanyofthesmallestruralhospitalstorecovertheirMedicarecosts
under the prospective payment system (PPS) rates (Moscovice and Stensland
2002; Dalton et al. 2003; Stensland, Davidson, and Moscovice 2004). In order
to protect these small, financially vulnerable rural hospitals, the Medicare
Rural Hospital Flexibility Program (Flex Program) of 1997 allowed some
small hospitals to convert to critical access hospitals (CAH) and changed their
Medicare reimbursement mechanism from prospective to cost based (Came
ron, Zelman, and Stewart 2001). One objective of the Flex Program is to
improve quality of care for those financially vulnerable rural hospitals.
The Rural Hospital Flexibility Program Tracking Team conducted a
phone survey of 217 CAHs in 2001 and found that most CAHs were involved
in a broad range of quality assurance (QA) or quality improvement (QI)
activities. Almost all CAHs (99–100 percent) used continuing education pro
grams for staff, medical error reporting policy systems, and hospital QA or QI
training initiatives. Eightyseven percent of CAHs reported the use of data
feedback to staff and 38–52 percent of CAHs reported that they strengthened
QA/QI activities after converting to CAH status. Only 1–2 percent of CAHs
reported that these activities became weaker (Moscovice and Gregg 2001). In
their second year survey, they found that all of these activities were stronger
over time after CAH conversion (Moscovice, Gregg, and Klingner 2002).
Compared with preconversion, 52 percent of hospitals increased pooling
or coordinating resources; 31 percent of hospitals increased the number of
available QA or QI staff; 19 percent of hospitals increased appropriateness of
Address correspondence to Pengxiang Li, Ph.D., Division of General Internal Medicine, Uni
versity of Pennsylvania, 1215 Blockley Hall, 423 Guardian Drive, Philadelphia, PA 19104. John
E. Schneider, Ph.D., Assistant Professor, and Marcia M. Ward, Ph.D., Professor, are with the
Department of Health Management and Policy, College of Public Health, University of Iowa,
Iowa City, IA. Dr. Schneider is also with the Center for Research in the Implementation of
Innovative Strategies in Practice, Iowa City VA Medical Center, Iowa City, IA.
2090 HSR: Health Services Research 42:6, Part I (December 2007)
Page 3
credentials;35percentofhospitalsincreaseduseofcriticalpathwayprotocols;
38percentofhospitalsincreaseduseofadmissionprotocols;and43percentof
hospitalsincrease theuseoftransferprotocols.Lessthan5percentofhospitals
decreased QA/QIrelated characteristics (Moscovice and Gregg 2001). While
these reports document an increase in QA and QI activities after conversion,
no studies have been published to date that examine the effect of conversion
on outcomes. The aim of our study is to examine indicators of patient safety
over time in Iowa rural hospitals to explore changes that occurred after hos
pitals converted from PPS to CAH status.
METHODS
Sample and Data Sources
We employed quasiexperimental designs that use both control groups and
pretests. Iowa has 96 nonfederal hospitals, which are located in nonMSA
areas. These hospitals are classified as Rural Referral Hospitals, Rural PPS
Hospitals, or CAHs. The seven Rural Referral Hospitals are located in non
MSAareas,buthave similar operatingcharacteristicsto urban hospitals (Iowa
Hospital Association [IHA] 2005). The other 89 rural hospitals (rural PPS
hospitals and CAHs) are the sample of the study. All 89 rural hospitals were
rural PPS in 1997. By the end of 2005, 81 hospitals had converted to CAHs
and only eight hospitals remained as rural PPS hospitals. The study period
from 1997 to 2004 includes 712 observations (hospitalyears).
We developed a longitudinal database for 89 Iowa rural hospitals, span
ning the 8year period, which merged together: (1) patient safety indicators
(PSIs)computed fromIowa State Inpatient Databases (SIDs) using Agencyfor
Healthcare Research and Quality (AHRQ) PSIs software (AHRQ 2006a,c),
(2) Hospital CAH status extracted from IHA records, (3) market variables
extracted from the Area Resource File (ARF), and (4) other variables com
puted from Iowa SIDs. The definitions and sources of the variables are de
scribed in Table 1. Ordinary least squares regression techniques were used to
impute temporal gapsin theARF data (Alexander, D’Aunno, and Succi 1996;
Little and Rubin 2002). Information on observed years in a county’s time
series was used to estimate data in missing years.
Patient Safety Measurement
AHRQ provides software to compute quality indicators for hospitals using
inpatient discharge data. The quality indicators include three modules:
Effect of CAH Conversion on Patient Safety2091
Page 4
Table1:
Sources of Variables and Means for 1997 and 2004
Variable
Definition
Data Sources
1997
2004
PSI2
Observed rate of inhospital deaths per 1,000 patients in DRGs with o0.5%
mortalityn,w
Iowa SID, AHRQ
0.46
0.95
PSI3
Riskadjusted cases of decubitus ulcer per 1,000 discharges with a length of stay
>4 days
Iowa SID, AHRQ
6.90
11.12
PSI5
Observed rate of foreign body accidentally left in during procedure per 1,000
medical and surgical dischargesn,w
Iowa SID, AHRQ
0.043
0.073
PSI6
Riskadjusted rate of iatrogenic pneumothorax per 1,000 dischargesn
Iowa SID, AHRQ
0.47
0.18
PSI7
Riskadjusted rate of selected infections due to medical care per 1,000
dischargesn
Iowa SID, AHRQ
0.83
0.27
PSI15
Riskadjusted rate of accidental puncture or laceration during procedures per
1,000 dischargesn
Iowa SID, AHRQ
2.4
1.8
Composite score of 4PSIs
Weighted sum of PSI5, PSI6, PSI7, and PSI15, weighted by numerator of
each indicator
Iowa SID, AHRQ
1.93
1.34
Composite score of 6PSIs
Weighted sum of PSI2, PSI3, PSI5, PSI6, PSI7, and PSI15, weighted by
numerator of each indicator
Iowa SID, AHRQ
4.74
5.92
CAH
CAH status51, rural PPS status50
IHA
0
0.74
CAHmv
The moving average of CAH status of the year t, t?1, and t?2 lags
IHA
0
0.62
% Medicare days
Medicare patient days/total patient days ? 100%
Iowa SID
66.68%
65.71%
% Medicaid days
Medicaid patient days/total patient days ? 100%
Iowa SID
6.72%
7.20%
% surgical discharges
Surgical dischargesz/total discharges ? 100%
Iowa SID
12.56%
11.07%
Charlson comorbidity index Sum of Charlson comorbidity scores for all inpatient patients/total discharges
Iowa SID
0.80
0.80
Market concentration (HHI) Sum of the squares of each hospital’s market share in the county
Iowa SID
0.89
0.89
Per capita income
Countylevel per capita income (using CPI to adjusted to 1997 dollar)
ARF
$21,180 $22,432
Population density
Number of population in the county/area of the county (square miles)
ARF
31.80
31.80
nPlease refer to Guide to Patient Safety Indicators 2003 Version 3.0 (AHRQ 2006a) for the detailed definition of each indicator.
wNo riskadjusted rate is available for PSI2 or PSI5 in AHRQ PSI V3.0.
zSurgical discharges were identified based on the DRG variable in SIDs.
ARF,AreaResourceFile;SID,StateInpatientDatabase;IHA,IowaHospitalAssociation;CAH,criticalaccesshospitals;AHRQ,AgencyforHealthcare
Research and Quality; PSI, patient safety indicators; DRG, diagnosisrelated group; HHI, Herfindahl–Hirschman Index.
2092 HSR: Health Services Research 42:6, Part I (December 2007)
Page 5
prevention quality indicators (PQIs), inpatient quality indicators (IQIs), and
PSIs. PSIs screen for complications or adverse events ‘‘that patients experi
ence as a result of exposure to the healthcare system and that are likely ame
nable to prevention by changes at the system or provider level’’ (AHRQ
2006a,c). Using PSI software, 29 providerlevel PSIs can be computed. Each
observed rate of PSI can be defined as ‘‘outcome of interest/population at
risk.’’ThevariationsinobservedrateofPSIscomenotonlyfromthequalityof
care of hospitals, but also from the different patient case mix in hospitals.
Hospitals with more severe cases are more likely to have higher observed
rates. Riskadjusted PSIs are ‘‘the estimated performance of providers on the
PSIs if those providers had an ‘average’ case mix’’ (AHRQ 2006a,c). The
average case mix reflects the distribution in ‘‘age, sex, modified DRG, and co
morbidity categories’’ among the providers in the 35 states with SID da
tabases. Riskadjusted PSIs can be used for comparing patient safety among
different providers and across different years. To eliminate unstable estimates
based on too few cases, AHRQ recommends suppressing the estimates if
fewer than 30 cases are in the denominator (AHRQ 2006a,c). Only five PSIs
(PSI5, foreign body left during procedure; PSI6, iatrogenic pneumothorax;
PSI7, selected infections due to medical care; PSI15, accidental puncture or
laceration;andPSI16,transfusion reaction),whichuse almostallmedical and
surgical discharges defined by specific diagnosisrelated groups (DRGs) as
denominators, are able to generate PSI measures for all rural Iowa hospitals
without being subject to the small denominator issue. The observed rate of
PSI16 (transfusion reaction) is too rare to provide variability to differentiate
hospitals in Iowa. Thus,we identify fourPSIs(PSI5, PSI6, PSI7, and PSI15)
as our measures of patient safety for all hospitals. In addition, to measure a
broaderscopeofpatientsafety,weincludetwomorePSIs(PSI2,deathinlow
mortality DRGs and PSI3, decubitus ulcer), which have fairly large denom
inators for most hospitals. Suitable PSI2 measures are available for 630
observations among 64 hospitals. Suitable PSI3 measures are available for
624 observations among 86 hospitals. AHRQ PSI software version 3 (AHRQ
2006a,c) was used to compute the riskadjusted rates for PSI3, PSI6, PSI7,
and PSI15, and observed rates1for PSI2 and PSI5 for each hospital in our
sample for each year from 1997 to 2004. We built two composite PSIs, ‘‘com
posite score of four PSIs’’ and ‘‘composite score of six PSIs.’’ Composite score
of four PSIs, which is available for all 89 hospitals and 712 observations, is the
weighted average of PSI5, PSI6, PSI7, and PSI15. Composite score of six
PSIs, which is available for 594 observations among 82 hospitals, is the
weighted averageofPSI2,PSI3,PSI5,PSI6,PSI7,andPSI15.Theweights
Effect of CAH Conversion on Patient Safety2093
Page 6
arebased on the frequency ofthenumeratorof eachPSI inoursample(8year
inpatient discharges among 89 Iowa rural hospitals). A numerator weight puts
more weight on indicators with higher frequencies of adverse events. We also
developed two composite PSIs weighted by the denominator of each indi
cator. We examine the impacts of CAH conversion on these two denomi
natorweighted composite PSIs in our sensitivity analyses. We also created a
binary variable for each PSI. If the value of a PSI is higher than the median of
the PSI in our sample (8year 89 Iowa rural hospitals), the binary variable for
the PSI (poor performance) is equal to 1; otherwise, it is equal to 0.
CAH Conversion Measurement
Hospital CAH status was extracted from IHA records. Two variables were
used to measure hospital CAH status in the analyses. First, we examined the
effects of CAH conversion using a dummy variable, where CAH was equal to
oneifthehospitalwasCAHstatusthatyear.Secondwefollowedtheapproach
recommended by Antel, Ohsfeldt, and Becker (1995) of using a moving av
erage of CAH conversion. The moving average CAH variable was defined as
the average of CAH status of the year t, t?1, and t?2 lags. The moving
average CAH variable was used to examine the longrun effects of CAH
conversion by putting less weight on hospitals during the first 2 years of
conversion.2
Descriptive Analyses
In the descriptive analyses, we computed crosssectional comparisons be
tween rural PPS and CAH hospitals for PSIs by year. Given that the distri
butionsforPSIsareskewed,a Wilcoxon ranksum test (Paganoand Gauvreau
2000) was used to conduct crosssectional comparisons of PSIs between PPS
and CAH hospitals by year. We also conducted pre and postconversion
comparisonsonPSIsforthe66hospitalsinCAHstatusin2004.Wecalculated
pre and postconversion means and used Wilcoxon signedrank test to test
whether there were statistically significant improvements in PSIs for hospitals
that converted to CAHs.
Generalized Estimating Equations (GEE) Logit Model
The value of PSIs is zero inflated and nonnegative, which suggests that or
dinary least squares estimations may be problematic ( Jones 2000). Thus, we
employed a GEE logit model (Fitzmaurice, Laird, and Ware 2004) to estimate
the effects of CAHconversionon the binaryPSI variables(i.e., the association
2094 HSR: Health Services Research 42:6, Part I (December 2007)
Page 7
of CAH conversion with odds of poor performance for a specific PSI). The
unit of analysis is hospitalyear. The GEE logit model has the following three
part specifications (Fitzmaurice, Laird, and Ware 2004):
(a) Theconditionalexpectationoftheresponseisassumedtodependon
the independent variables through a logit link function.
logðpit=1 ? pitÞ ¼ b0þ b1CAHitþ
X
bnXitþ
X
bmZtþ eit
ð1Þ
where pitis the expected probabilityof hospital i having poor performance for
the PSI in year t. CAHitis the value of the CAH variable for the ith hospital at
year t. Xitis a vector of other explanatory variables that presumably are as
sociated with patient safety for the ith hospital at year t. Xitincludes the per
centage of Medicare patient days, the percentage of Medicaid days, the
hospital mean of the Charlson comorbidity score (Charlson et al. 1987; Deyo
et al. 1992), the percentage of surgical discharges, market concentration (Her
findahl–Hirschman Index [HHI]), countylevel per capita income, and coun
tylevel population density. Ztis a vector of the year dummy variables, which
will adjust the effects of unmeasured, timespecific factors. b0, b1, b2, and b3are
parameters associated with these variables.
(b) The variation function is pit(1–pit).
(c) We assume the withinsubject association among repeated measures
is a firstorder autoregressive correlation pattern.
By using ‘‘sandwich’’ estimation, GEE yields consistent and valid esti
mates of coefficients and their standard errors even if withinsubject associ
ations are not correctly specified (Fitzmaurice, Laird, and Ware 2004). The
dependent variables consist of eight binary quality functions with six individ
ual PSIs and two composite PSIs. CAH conversion has been shown to be
associated with significant changes in hospital characteristics, such as bed size
and total number of discharges (e.g., Hagopian and Hart 2001;Rural Hospital
Flexibility Program Tracking Team 2002, 2003). Thus, including hospital
characteristics in multiple regression models may yield a multicollinearity
issue with the CAH variables. Consequently, we only adjusted for patient
characteristics and market variables in the multiple regression models.
Sensitivity Analyses
We undertook several sensitivity analyses to examine the robustness of our
findings. First, we examined the impact of CAH conversion on the two
Effect of CAH Conversion on Patient Safety2095
Page 8
denominatorweighted composite PSI scores. Second, one limitation of this
study is the potential endogeneity of CAH variables given that CAH conver
sion is a decision made by rural hospitals themselves, rather than random
assignment. In addition, some rural PPS hospitals may never choose to con
vert to CAH status. Using these hospitals as the comparison group may in
troduce some selection bias to the extent that these hospitals may be different
from hospitals that convert to CAH. It is possible that the difference in PSI
performance between rural PPS and CAH hospitals is attributable to heter
ogeneity in hospital characteristics rather than CAH conversion. In order to
deal with this issue, we repeated the original GEE analyses based on a sample
with 81 hospitals, which eventually converted to CAH. All of these hospitals
were in rural PPS status in 1997 and in CAH status by the end of 2005.
In addition, to conduct sensitivity analyses of hospital characteristics, we
estimated GEE models, which included hospital bed size and number of
discharges.
Sensitivity analyses also included panel data randomeffects Tobit
models to estimate the effects of CAH conversion on continuous PSIs (using
Stata 7.0; see StataCorp. 2001b). The distributions of PSIs are skewed. The
value of PSIs is nonnegative with a mass at 0. Tobit models have a conceptual
advantage in analyzing data where the distribution of the dependent variable
is normal above a limiting value (e.g., Woolridge 2002; Salkever, Slade, and
Karakus 2006). The Tobit model assumes a continuous latent variable, de
noted as PSIn, with normal distribution (equations [2] and [3])
PSIit?¼ b0þ b1CAHitþ
X
b2Xitþ
X
b3Ztþ eit
ð2Þ
PSIit¼ maxð0; PSIit?Þð3Þ
In this expression, we observe PSIitfor the ith hospital in year t. Equation (3)
indicates that we observed PSIitnif it is larger or equal to 0. If PSIitnis smaller
than 0, we observe 0. CAHit, Xit, and Ztare the same as we defined in the GEE
logit model (equation [1]). The terms b0, b1, b2, and b3are parameters asso
ciated with these variables. The error term is expressed as eit.
The randomeffects Tobit models were estimated using Gauss–Hermite
quadrature to compute the log likelihood and its derivatives (StataCorp.
2001a). We used the quadchk command in Stata 7.0 to check the numerical
soundness of the quadrature approximation. The quadchk command enables
examination of the extent to which changing the number of quadrature points
affects the results. If the relative difference in coefficients is smaller than 0.01
percent, the results may be confidently interpreted. However, if the relative
2096HSR: Health Services Research 42:6, Part I (December 2007)
Page 9
difference is 41 percent, the result is uncertain (StataCorp. 2001a). We ran
randomeffects Tobit models for both riskadjusted rates of PSIs and observed
rates of PSIs. One assumption for randomeffects Tobit models is that there is
no correlation between independent variables and omitted variables. This
assumption may not be valid. By adding hospital IDs as dummy variables in
crosssectional Tobit models, the unconditional fixedeffects Tobit models
reduce the omitted variable bias by ruling out the impact of hospitalspecific
omittedvariables.3Alldataanalysisprogramsareavailableuponrequestfrom
the corresponding author.
RESULTS
ThemeanvaluesofPSIsforCAHsandruralPPShospitalsareshowninTable
2. Wilcoxon rank sum tests indicated that CAHs had statistically significantly
(p ? .05) higher performance (i.e., lower PSI value) than rural PPS hospitals
Table2:
PPS and CAHs, 1997–2004
CrossSectionalComparisonofMeansofPSIszbetweenIowaRural
Year
Hospital
Categories
Number of
Hospitals PSI2wPSI3wPSI5 PSI6 PSI7 PSI15
Composite
Score of Four
PSIs
Composite
Score of Six
PSIsw
1997 Rural PPS
1998 Rural PPS
1999 Rural PPS
CAH
2000 Rural PPS
CAH
2001 Rural PPS
CAH
2002 Rural PPS
CAH
2003 Rural PPS
CAH
2004 Rural PPS
CAH
89
89
88
1
78
11
57
32
45
44
34
55
23
66
0.46
1.53
0
NA
0
0
1.11
1.62
0.93nn
0.40nn
0.42
0.85
1.04
0.92
6.9
9.18
10.93
NA
9.31
22.94
11.28
9.17
9.62
8.31
10.49nn0.01
7.38nn0.06
10.98
11.19
0.04
0.05
0.02
0
0.06
0
0.07
0
0.04
0.21
0.47
0.27
0.3
0
0.36
0
0.21nn0.69
0.07nn1.17
0.23
0.34
0.46nn0.88nn2.38nn
0.26nn0.41nn1.89nn
0.12n0.29nn0.54nn2.68nn
0.06n0.14nn0.17nn1.46nn
0.83
0.36
0.69
0
0.6
0.41
2.44
2.72
2.99
0
3.09
2.12
2.65
2.24
1.93
2.04
2.29
0
2.35n
1.58n
2.03
1.8
1.56nn
1.41nn
1.89nn
1.45nn
2.04nn
1.09nn
4.74
5.76
6.9
NA
5.72
7.85
6.9
5.61
5.95
5.02
6.38n
4.47n
7.01
5.41
0.64nn2.00nn
0.14nn1.89nn
nSignificant difference in PSIs between rural PPS and CAH at p ? .10 (Wilcoxon ranksum test).
nnSignificant difference in PSIs between rural PPS and CAH at p ? .05 (Wilcoxon ranksum test).
wMeans are based on those hospitals with nonmissing values.
zLower PSI value is better performance.
PSI, patient safety indicators; CAH, critical access hospitals; PPS, prospective payment system.
Effect of CAH Conversion on Patient Safety 2097
Page 10
on PSI6 in 2001, on PSI2, PSI7, PSI15, and composite score of four PSIs in
2002, on PSI3, PSI6, PSI7, PSI15 and composite score of four PSIs in 2003,
and on PSI6, PSI7, PSI15 and composite score of four PSIs in 2004. Cross
sectionalcomparisonsshowedthatCAHsgenerallyhadbetterperformancein
patient safety than rural PPS hospitals.
In the pre and postconversion comparisons, as shown on Table 3, after
conversion to CAH status, hospitals experienced statistically significant im
provement in performance as measured by PSI7, PSI15, and composite
score of the four PSIs. There were no significant changes in hospital perfor
mance in PSI2, PSI3, PSI5, and PSI6. In general, following conversion,
there are more hospitals exhibiting better performance than those exhibiting
worse performance in all PSIs.
The descriptive comparisons in Tables 2 and 3 do not control for the
effects of independent variables, selection bias, or history bias. To address
theseissues,GEElogitmodelswereexamined.TheGEElogitmodelsshowed
that CAH conversion yielded significant improvement in hospital perfor
mance (or lower odds of poor performance) in PSI6, PSI7, PSI15, and
composite score of the four PSIs (Table 4). The odds ratios of poor perfor
mance in CAH hospitals compared with rural PPS hospitals are 0.30 (con
fidence interval [CI]: 0.14–0.64) for PSI6, 0.29 (CI: 0.15–0.56) for PSI7, 0.40
Table3:
Hospitals, 1997–2004
Changes in PSIsnafter Converting to CAHs for Iowa Rural
PSI2 PSI3PSI5 PSI6 PSI7PSI15
Composite
Score of Four
PSIs
Composite
Score of Six
PSIs
N
Mean value of change in PSI
(after–before)
Number of hospitals having
better performance in PSIs
after conversion (reducing
PSI value)
Number of hospitals having
worse performance in PSIs
after conversion
(increasing PSI value)
pvalueforWilcoxonsigned
rank test
59 57 66 66 6666 66
?0.623
54
?0.3100.187 ?0.0050.030 ?0.090 ?0.250 ?0.800
15 328 1829 37 4130
11 22588 16 1624
.68.60 .64.34 .01
o.01
o.01 .41
nLower PSI value is better performance.
PSI, patient safety indicators.
2098HSR: Health Services Research 42:6, Part I (December 2007)
Page 11
(CI:0.24–0.67)forPSI15,and0.49(CI:0.31–0.80)forcompositescoreoffour
PSIs.WealsoobservedhigherscaleimprovementforCAHmvinPSI6,PSI7,
PSI15, and composite score of four PSIs.2CAH conversion did not yield
significant changes in the performance in PSI2, PSI3, PSI5, and composite
score of six PSIs. GEE logit models also showed that a higher percentage of
surgical discharges in a hospital was associated with a higher odds of poor
performance in observed rates of PSI5, and riskadjusted rates of PSI6, PSI
15, and the two composite PSI measures. Also, the hospital mean for the
Charlson comorbidity score was positively associated with poor performance
in riskadjusted PSI3, PSI5, PSI15, and two composite PSIs. These results
Table4:
mance for the PSI) in Iowa Rural Hospitals, 1997–2004
GEE Logit Models of the Performance of PSIs (15Poor Perfor
PSI2z
PSI3 PSI5 PSI6PSI7 PSI15
Composite
Score of
Four PSIs
Composite
Score of
Six PSIs
CAH
% Medicare days
% Medicaid days
% of surgical discharges
Charlson Index
Market concentration
(HHI)
Per capita income
($1,000)
Population density
y1998
y1999
y2000
y2001
y2002
y2003
y2004
Year
Intercept
Observations
?0.54
0.09nn
0.09nn
0.01
?2.60nn
0.11
?0.24
0.00
0.03
0.01
1.36n
?0.42
?0.80
?0.01
?0.01
0.09nn
2.01nn
?2.11
?1.19nn?1.26nn?0.92nn?0.70nn?0.30
?0.01
0.02 0.03 0.02
0.05nn
0.01 0.18nn
0.69 1.05 1.51nn
0.370.24
?0.21
?0.01n?0.01
?0.01
0.03
0.16nn
1.37nn
?0.15
0.00
0.04
0.04nn
2.14nn
?0.44
0.090.16nn
0.00
?0.030.090.060.07nn
0.15
0.02nn
0.00
0.04
0.30
0.34
0.68nn
0.34
0.21
0.16
?0.01
?0.16
?0.63
?0.53
0.07
0.41
?0.21
0.84
0.02 0.010.00 0.000.00
0.05
0.53
0.16
0.44
0.24
?0.43
?0.29
0.18
0.19
?0.34
0.53
?0.20
?0.52
?0.24
?0.41
0.33
?0.24
0.21
?0.38
?0.16
0.11
0.23
0.66n
?0.08
0.27
0.10
?0.49
?0.23
?0.32
0.21
?0.54
?0.13
?0.50
?0.02
0.27
0.09
?185.30
630
?4.81nn?3.23
624
?2.67n?3.73nn?3.87nn?3.54nn?5.76nn
712 712712 712712594
wUnit of analysis is hospitalyear. y1998–y2004 were dummy variables for year 1998–2004. The
year of 1997 is the reference category.
nStatistically significant at .10 level.
nnStatistically significant at .05 level.
zGEE logit models for PSI2 using categorical year variables were not converged. Continuous
year variable was used.
PSI, patient safety indicators; CAH, critical access hospitals; HHI, Herfindahl–Hirschman Index.
Effect of CAH Conversion on Patient Safety2099
Page 12
indicate that hospitals with more severe cases might have worse performance
in some riskadjusted PSIs.
In our sensitivity analyses, we found that CAH conversion was asso
ciated with lower odds of poor performance in denominatorweighted
composite score of four PSIs. There is no significant impact on denominator
weighted composite score of six PSIs. The GEE logit models using a sample
excluding eight hospitals, which were in rural PPS status in 2005, showed a
similar result to those shown in Table 4. Sensitivity analyses using random
effects Tobit models showed that CAH conversion led to significantly lower
values in PSI6, PSI7, PSI15, and composite score of four PSIs. Fixedeffects
Tobit models showed that CAH conversion was associated with significantly
lower values of PSI7, PSI15, and composite score of four PSIs. In general,
estimates for the moving average measure of CAH status had a larger scale
than CAH estimates. In addition, we estimated a simulation using a random
effects Tobit model to examine the magnitude of CAH conversion impact on
both composite scores of PSIs. CAH conversion is associated with 60 and 16
percent reduction in composite score of four PSIs and composite score of six
PSIs,respectively.Aswediscussedabove,duetotheissueofmulticollinearity,
hospital characteristic variables were not included in the GEE logit models in
Table 4.
DISCUSSION
WeshowastrongandconsistenteffectofCAHstatusonPSIs.Inthesampleof
Iowa rural hospitals, the crosssectional comparisons showed that CAHs had
better performance than rural PPS hospitals. The pre and postconversion
comparisons showed that postconversion hospitals had better performance in
PSIs than preconversion hospitals. The results are robust to a number of
appropriate estimation strategies, and hold up to rigorous sensitivity analyses.
To address the concern that difference in PSIs might reflect primarily differ
encesinpatientmix,timetrends,anddifferencesinmarketsandenvironment,
we added two additional analyses. First, we used riskadjusted PSIs4as the
measures for the performance of hospital patient safety. Riskadjusted rates
reflect the performance of providers on the PSIs if those providers had an
‘‘average case mix’’ (AHRQ 2006a,c). Thus, it greatly reduces the effects of
patient case mix on patient safety measures. Second, we used GEE logit
models, randomeffects Tobit models and other sensitivity analyses to control
for the impact of patient case mix, market variables, and time trend. Owing to
2100HSR: Health Services Research 42:6, Part I (December 2007)
Page 13
multicollinearity, hospital characteristics were not included in the original
models.
A possible explanation for these findings could be that hospitals might
change their coding behaviors in response to the change in Medicare reim
bursement scheme;that is,observedlower rates of adverse events may be due
toachangeinhospitalcodingbehaviorwhentheyconvertedfromPPStocost
basedreimbursement.Ingeneral,underPPS,hospitalshavemoreincentiveto
code severe cases, which are associated with higher reimbursement rates. If
there is no change in patient case mix after conversion, a significant change in
coding behavior after converting from PPS to costbased would result in a less
severe observed case mix in rural hospitals. To examine this possibility, we
conducted separate analyses to examine the relationship between CAH con
version and hospitallevel average number of patient diagnoses, average
number of Elixhauser comorbidities (Elixhauser et al. 1998), and mean score
of the Charlson comorbidity index. We found that there was no significant
association between CAH conversion and these measures.
Another alternative explanation for the positive effects of CAH con
version on patient safety could be that it reflects a trend toward improvement
in patient safety for all hospitals. However, according to AHRQ, PSI2, PSI3,
PSI5, PSI6, PSI7, and PSI15 in the Nationwide Inpatient Sample (NIS) did
not exhibit an overall trend toward improvement over the 1997–2003 time
period (AHRQ 2006b). Likewise, our computations of the Iowa SIDs from
1997 to2004usingAHRQPSIsoftwareindicatedthattherewasnosignificant
overallimprovementtrendforPSI2,PSI5,PSI6,and PSI15inIowa’surban
and rural referral hospitals. In addition, we controlled for time effects by
including year dummy variables in the GEE and tobit models. The 8year
paneldatadesignhelpsadjustforselectionmaturationthreats(Shadish,Cook,
and Campbell 2002), regression to the mean biases5(Antel, Ohsfeldt, and
Becker 1995), and delay causation (Shadish, Cook, and Campbell 2002).
These analyses consistently showed that CAH conversion led to improved
performanceof PSI6, PSI7, and PSI15. In the sensitivity analyses,GEE logit
models and Tobit models also showed that movingaverage estimates of the
CAH effect had a larger scale than the dichotomous CAH measure, which
indicates that CAH conversion may lead to larger improvement in patient
safety in the long run.
CAH conversion has been shown to significantly improve rural hospi
tals’ financial condition. Previous studieshaveshown that preconversion rural
hospitals faced serious financial pressure (Cameron, Zelman, and Stewart
2001; Stensland, Davidson, and Moscovice 2004) and that over half of the
Effect of CAH Conversion on Patient Safety 2101
Page 14
hospitals that converted to CAH in FY1999 or FY2000 were losing money
before conversion (Stensland, Davidson, and Moscovice 2004). Stensland,
Davidson, and Moscovice (2004) reported that hospitals that converted to
CAH in FY1999 experienced an average increase in Medicare payment of
36 percent, around $500,000 in FY2000 inflation adjusted dollars after con
version, and that CAH conversion increased hospital profit margins by 2–4
percentage points.
Improved financial conditions and lower risk sharing associated with
CAH conversion are likely to contribute to improvements in patient safety.
After conversion, under costbased reimbursement, risk sharing decreased
substantially. Under PPS, the marginal costs associated with QI are not re
imbursed, and the hospital has to bear all the cost incurred by the increased
intensity and quality (Cutler 1995). Under costplus reimbursement, marginal
costs associated with increased quality are fully reimbursed. Hospitals tend to
have higher intensity and produce a higher level of quality under lower risk
sharing (Hodgkin and McGuire 1994; Cutler 1995; Chalkley and Malcomson
2000).
To the extent that more resources are needed to bring about meaning
ful improvements in quality of care, patient safety is inextricably linked to
the financial condition of hospitals (Encinosa and Bernard 2005). It is
harder for hospitals with financial problems to make investments in patient
safety improvements (e.g. errorreducing information technology system), or
to attract or retain highcost specialists. Likewise, hospitals with financial
problems might cut nurse staffing which may adversely affect patient safety
(Encinosa and Bernard 2005). Cutler (1995) found that the fiscal pressures
from the PPS in Medicare in the 1980s were associated with higher mortality.
Shen (2003) also has shown that financial pressure was associated with in
creased mortality rates after treatment of acute myocardial infarction. En
cinosa and Bernard (2005) found that a withinhospital erosion of hospital
operating margins was associated with an increased rate of adverse patient
safety events.
The Rural Hospital Flexibility Program Tracking Team found that QI
or QArelated activities were widely undertaken in CAHs and have been
reinforced over time since CAH conversion (Moscovice and Gregg 2001;
Moscovice, Gregg, and Klingner 2002). Staffing improvement was one of the
most significant factors contributing to progress in quality of care and
increased reimbursement was cited as the reason for improved staffing
(Moscovice, Gregg, and Klingner 2002). Consistent with these findings, and
expanding them to outcomes, we found that CAHs strengthened their scores
2102 HSR: Health Services Research 42:6, Part I (December 2007)
Page 15
on PSIs after conversion, at the time that they would have been experiencing
higher reimbursement.
Apart from the change in reimbursement method and the resulting fi
nancial relief, other factors may also contribute to better quality of care in
CAHs. These factors may include the establishment of a network relationship
with affiliated hospitals, improvement in case management and discharge
planning, expansion in qualified QA and QI staff, and enhancement in equip
ment (Moscovice and Gregg 2001; Moscovice, Gregg, and Klingner 2002).
There are several limitations in our study. First, there are limitations
inherenttoadministrativedatabasesincludingthepossibilityofmissingcodes,
coding errors, and variation in coding practices across hospitals (Simborg
1981; Hsia et al. 1988; Iezzoni 1997; Zhan and Miller 2003). Although we did
not find a significant association between CAH conversion and various mea
suresofpatientseverity,itispossiblethelowerratesofadverseeventsinCAH
hospitals are simply due to a reduced incentive to code certain diagnoses. In
this study, we are not able to completely rule out this possibility.
A second issue of concernis endogeneityand omittedvariable bias. The
CAHvariableislikelytobeendogenousgiventhathospitalschoosetoconvert
to CAH status. However, endogeneity is to a large extent mitigated through
the fixedeffects panel models with year dummy variables, which capture the
effects of unmeasured hospitalspecific and timespecific factors (Woolridge
2002). Furthermore, the results of the sensitivity analysis using a sample of 81
Iowa rural hospitals (all of which were in rural PPS status in 1997 and in CAH
status at the end of 2005) suggests that endogeneity is not an exceptionally
large concern. The consistency between the sensitivity analyses and our main
results indicate that our findings are robust.
A third limitation involves using AHRQ software to calculate PSIs and
evaluate hospital patient safety performance. The AHRQ software is not able
to fully identify all the preventable adverse events due to the limited clinical
information available in administrative data (AHRQ 2006a). In addition, only
sixofthe29PSIshadadequatecasestomeasurepatientsafetyinoursampleof
rural hospitals. The six PSIs are less comprehensive than the complete pack
age of PSIs created by AHRQ PSI software. Furthermore, the assumption for
the Tobit models is that the distribution is a censored normal distribution.
However,wewerenotabletotestthisassumptiongiventhatthecensoreddata
are unobservable (Duan 1983). We only include the results of Tobit models in
the sensitivity analyses. The findings of Tobit models were consistent with
GEElogitmodels.Finally,oursampleonlyincludesIowaruralhospitals,thus,
our results may not generalize to other states.
Effect of CAH Conversion on Patient Safety2103