Page 1

RESEARCH ARTICLE

Hospitals

Effect of Critical Access Hospital

Conversion on Patient Safety

Pengxiang Li, John E. Schneider, and Marcia M. Ward

Background. The Medicare Rural Hospital Flexibility Program of the 1997 Balanced

Budget Act allowed hospitals meeting certain criteria to convert to critical access hos-

pitals (CAH) and changed their Medicare reimbursement mechanism from prospective

payment system (PPS) to cost-based.

Objective. To examine the impact of CAH conversion on hospital patient safety.

Data Source. Secondary data on hospital patient safety indicators (PSIs), hospital

CAH status, patient case-mix, and market variables, for 89 Iowa rural hospitals during

1997–2004.

Study Design. Weemployedquasi-experimentaldesignsthatusebothcontrolgroups

and pretests. The hospital-year was the unit of analysis. We used generalized estimating

equationslogitandrandom-effectsTobitmodelstoassesstheeffectsofCAHconversion

on hospital patient safety. The models were adjusted for patient case-mix and market

variables. Sensitivity analyses, which varied by sample and statistical model, were used

to examine the robustness of our findings.

DataExtractionMethods. PSIswerecomputedfromIowaStateInpatientDatabases

(SIDs) using Agency for Healthcare Research and Quality indicators software. Hospital

CAH status was extracted from Iowa Hospital Association. Patient case-mix variables

wereextractedfromIowaSIDs.MarketvariablescamefromAreaResourceFile(ARF).

Principal Findings. CAH conversion in Iowa rural hospitals was associated with

better performance of risk-adjusted rates of iatrogenic pneumothorax, selected infec-

tionsduetomedicalcare,accidentalpunctureorlaceration,andcompositescoreoffour

PSIs, but had no significant impact on the observed rates of death in low-mortality

diagnosis-related groups (DRGs), foreign body left during procedure, risk-adjusted rate

of decubitus ulcer, or composite score of six PSIs.

Conclusion. CAH conversion is associated with enhanced performance of certain

PSIs.

Key Words. AHRQ patient safety indicators, critical access hospital, policy

evaluation, observational data quasi-experiments

The Institute of Medicine (IOM) definition of patient safety is ‘‘freedom from

accidentalinjuryduetomedicalcare,ormedicalerrors’’(Kohn,Corrigan,and

r Health Research and Educational Trust

DOI: 10.1111/j.1475-6773.2007.00731.x

2089

Page 2

Donaldson 1999). More than 44,000 Americans die as a result of preventable

medical errors each year and national costs of preventable adverse events

were approximately $17 billion in 1996 (Kohn, Corrigan, and Donaldson

1999).

Ruralhospitalsplayakeyroleinprovidinghealthservicesinruralareas.

Quality of care is one of the main issues that rural hospitals face. In improving

quality of care, rural hospitals face several problems, including financial con-

straints (Moscovice, Gregg, and Klingner 2002), limited resources and knowl-

edge (Moscovice 1989), and staffing (Moscovice and Stensland 2002). It was

difficultformanyofthesmallestruralhospitalstorecovertheirMedicarecosts

under the prospective payment system (PPS) rates (Moscovice and Stensland

2002; Dalton et al. 2003; Stensland, Davidson, and Moscovice 2004). In order

to protect these small, financially vulnerable rural hospitals, the Medicare

Rural Hospital Flexibility Program (Flex Program) of 1997 allowed some

small hospitals to convert to critical access hospitals (CAH) and changed their

Medicare reimbursement mechanism from prospective to cost based (Came-

ron, Zelman, and Stewart 2001). One objective of the Flex Program is to

improve quality of care for those financially vulnerable rural hospitals.

The Rural Hospital Flexibility Program Tracking Team conducted a

phone survey of 217 CAHs in 2001 and found that most CAHs were involved

in a broad range of quality assurance (QA) or quality improvement (QI)

activities. Almost all CAHs (99–100 percent) used continuing education pro-

grams for staff, medical error reporting policy systems, and hospital QA or QI

training initiatives. Eighty-seven percent of CAHs reported the use of data

feedback to staff and 38–52 percent of CAHs reported that they strengthened

QA/QI activities after converting to CAH status. Only 1–2 percent of CAHs

reported that these activities became weaker (Moscovice and Gregg 2001). In

their second year survey, they found that all of these activities were stronger

over time after CAH conversion (Moscovice, Gregg, and Klingner 2002).

Compared with preconversion, 52 percent of hospitals increased pooling

or coordinating resources; 31 percent of hospitals increased the number of

available QA or QI staff; 19 percent of hospitals increased appropriateness of

Address correspondence to Pengxiang Li, Ph.D., Division of General Internal Medicine, Uni-

versity of Pennsylvania, 1215 Blockley Hall, 423 Guardian Drive, Philadelphia, PA 19104. John

E. Schneider, Ph.D., Assistant Professor, and Marcia M. Ward, Ph.D., Professor, are with the

Department of Health Management and Policy, College of Public Health, University of Iowa,

Iowa City, IA. Dr. Schneider is also with the Center for Research in the Implementation of

Innovative Strategies in Practice, Iowa City VA Medical Center, Iowa City, IA.

2090 HSR: Health Services Research 42:6, Part I (December 2007)

Page 3

credentials;35percentofhospitalsincreaseduseofcriticalpathwayprotocols;

38percentofhospitalsincreaseduseofadmissionprotocols;and43percentof

hospitalsincrease theuseoftransferprotocols.Lessthan5percentofhospitals

decreased QA/QI-related characteristics (Moscovice and Gregg 2001). While

these reports document an increase in QA and QI activities after conversion,

no studies have been published to date that examine the effect of conversion

on outcomes. The aim of our study is to examine indicators of patient safety

over time in Iowa rural hospitals to explore changes that occurred after hos-

pitals converted from PPS to CAH status.

METHODS

Sample and Data Sources

We employed quasi-experimental designs that use both control groups and

pretests. Iowa has 96 nonfederal hospitals, which are located in non-MSA

areas. These hospitals are classified as Rural Referral Hospitals, Rural PPS

Hospitals, or CAHs. The seven Rural Referral Hospitals are located in non-

MSAareas,buthave similar operatingcharacteristicsto urban hospitals (Iowa

Hospital Association [IHA] 2005). The other 89 rural hospitals (rural PPS

hospitals and CAHs) are the sample of the study. All 89 rural hospitals were

rural PPS in 1997. By the end of 2005, 81 hospitals had converted to CAHs

and only eight hospitals remained as rural PPS hospitals. The study period

from 1997 to 2004 includes 712 observations (hospital-years).

We developed a longitudinal database for 89 Iowa rural hospitals, span-

ning the 8-year period, which merged together: (1) patient safety indicators

(PSIs)computed fromIowa State Inpatient Databases (SIDs) using Agencyfor

Healthcare Research and Quality (AHRQ) PSIs software (AHRQ 2006a,c),

(2) Hospital CAH status extracted from IHA records, (3) market variables

extracted from the Area Resource File (ARF), and (4) other variables com-

puted from Iowa SIDs. The definitions and sources of the variables are de-

scribed in Table 1. Ordinary least squares regression techniques were used to

impute temporal gapsin theARF data (Alexander, D’Aunno, and Succi 1996;

Little and Rubin 2002). Information on observed years in a county’s time

series was used to estimate data in missing years.

Patient Safety Measurement

AHRQ provides software to compute quality indicators for hospitals using

inpatient discharge data. The quality indicators include three modules:

Effect of CAH Conversion on Patient Safety2091

Page 4

Table1:

Sources of Variables and Means for 1997 and 2004

Variable

Definition

Data Sources

1997

2004

PSI-2

Observed rate of in-hospital deaths per 1,000 patients in DRGs with o0.5%

mortalityn,w

Iowa SID, AHRQ

0.46

0.95

PSI-3

Risk-adjusted cases of decubitus ulcer per 1,000 discharges with a length of stay

>4 days

Iowa SID, AHRQ

6.90

11.12

PSI-5

Observed rate of foreign body accidentally left in during procedure per 1,000

medical and surgical dischargesn,w

Iowa SID, AHRQ

0.043

0.073

PSI-6

Risk-adjusted rate of iatrogenic pneumothorax per 1,000 dischargesn

Iowa SID, AHRQ

0.47

0.18

PSI-7

Risk-adjusted rate of selected infections due to medical care per 1,000

dischargesn

Iowa SID, AHRQ

0.83

0.27

PSI-15

Risk-adjusted rate of accidental puncture or laceration during procedures per

1,000 dischargesn

Iowa SID, AHRQ

2.4

1.8

Composite score of 4PSIs

Weighted sum of PSI-5, PSI-6, PSI-7, and PSI-15, weighted by numerator of

each indicator

Iowa SID, AHRQ

1.93

1.34

Composite score of 6PSIs

Weighted sum of PSI-2, PSI-3, PSI-5, PSI-6, PSI-7, and PSI-15, weighted by

numerator of each indicator

Iowa SID, AHRQ

4.74

5.92

CAH

CAH status51, rural PPS status50

IHA

0

0.74

CAHmv

The moving average of CAH status of the year t, t?1, and t?2 lags

IHA

0

0.62

% Medicare days

Medicare patient days/total patient days ? 100%

Iowa SID

66.68%

65.71%

% Medicaid days

Medicaid patient days/total patient days ? 100%

Iowa SID

6.72%

7.20%

% surgical discharges

Surgical dischargesz/total discharges ? 100%

Iowa SID

12.56%

11.07%

Charlson comorbidity index Sum of Charlson comorbidity scores for all inpatient patients/total discharges

Iowa SID

0.80

0.80

Market concentration (HHI) Sum of the squares of each hospital’s market share in the county

Iowa SID

0.89

0.89

Per capita income

County-level per capita income (using CPI to adjusted to 1997 dollar)

ARF

$21,180 $22,432

Population density

Number of population in the county/area of the county (square miles)

ARF

31.80

31.80

nPlease refer to Guide to Patient Safety Indicators 2003 Version 3.0 (AHRQ 2006a) for the detailed definition of each indicator.

wNo risk-adjusted rate is available for PSI-2 or PSI-5 in AHRQ PSI V3.0.

zSurgical discharges were identified based on the DRG variable in SIDs.

ARF,AreaResourceFile;SID,StateInpatientDatabase;IHA,IowaHospitalAssociation;CAH,criticalaccesshospitals;AHRQ,AgencyforHealthcare

Research and Quality; PSI, patient safety indicators; DRG, diagnosis-related group; HHI, Herfindahl–Hirschman Index.

2092 HSR: Health Services Research 42:6, Part I (December 2007)

Page 5

prevention quality indicators (PQIs), inpatient quality indicators (IQIs), and

PSIs. PSIs screen for complications or adverse events ‘‘that patients experi-

ence as a result of exposure to the healthcare system and that are likely ame-

nable to prevention by changes at the system or provider level’’ (AHRQ

2006a,c). Using PSI software, 29 provider-level PSIs can be computed. Each

observed rate of PSI can be defined as ‘‘outcome of interest/population at

risk.’’ThevariationsinobservedrateofPSIscomenotonlyfromthequalityof

care of hospitals, but also from the different patient case mix in hospitals.

Hospitals with more severe cases are more likely to have higher observed

rates. Risk-adjusted PSIs are ‘‘the estimated performance of providers on the

PSIs if those providers had an ‘average’ case mix’’ (AHRQ 2006a,c). The

average case mix reflects the distribution in ‘‘age, sex, modified DRG, and co-

morbidity categories’’ among the providers in the 35 states with SID da-

tabases. Risk-adjusted PSIs can be used for comparing patient safety among

different providers and across different years. To eliminate unstable estimates

based on too few cases, AHRQ recommends suppressing the estimates if

fewer than 30 cases are in the denominator (AHRQ 2006a,c). Only five PSIs

(PSI-5, foreign body left during procedure; PSI-6, iatrogenic pneumothorax;

PSI-7, selected infections due to medical care; PSI-15, accidental puncture or

laceration;andPSI-16,transfusion reaction),whichuse almostallmedical and

surgical discharges defined by specific diagnosis-related groups (DRGs) as

denominators, are able to generate PSI measures for all rural Iowa hospitals

without being subject to the small denominator issue. The observed rate of

PSI-16 (transfusion reaction) is too rare to provide variability to differentiate

hospitals in Iowa. Thus,we identify fourPSIs(PSI-5, PSI-6, PSI-7, and PSI-15)

as our measures of patient safety for all hospitals. In addition, to measure a

broaderscopeofpatientsafety,weincludetwomorePSIs(PSI-2,deathinlow-

mortality DRGs and PSI-3, decubitus ulcer), which have fairly large denom-

inators for most hospitals. Suitable PSI-2 measures are available for 630

observations among 64 hospitals. Suitable PSI-3 measures are available for

624 observations among 86 hospitals. AHRQ PSI software version 3 (AHRQ

2006a,c) was used to compute the risk-adjusted rates for PSI-3, PSI-6, PSI-7,

and PSI-15, and observed rates1for PSI-2 and PSI-5 for each hospital in our

sample for each year from 1997 to 2004. We built two composite PSIs, ‘‘com-

posite score of four PSIs’’ and ‘‘composite score of six PSIs.’’ Composite score

of four PSIs, which is available for all 89 hospitals and 712 observations, is the

weighted average of PSI-5, PSI-6, PSI-7, and PSI-15. Composite score of six

PSIs, which is available for 594 observations among 82 hospitals, is the

weighted averageofPSI-2,PSI-3,PSI-5,PSI-6,PSI-7,andPSI-15.Theweights

Effect of CAH Conversion on Patient Safety2093

Page 6

arebased on the frequency ofthenumeratorof eachPSI inoursample(8-year

inpatient discharges among 89 Iowa rural hospitals). A numerator weight puts

more weight on indicators with higher frequencies of adverse events. We also

developed two composite PSIs weighted by the denominator of each indi-

cator. We examine the impacts of CAH conversion on these two denomi-

nator-weighted composite PSIs in our sensitivity analyses. We also created a

binary variable for each PSI. If the value of a PSI is higher than the median of

the PSI in our sample (8-year 89 Iowa rural hospitals), the binary variable for

the PSI (poor performance) is equal to 1; otherwise, it is equal to 0.

CAH Conversion Measurement

Hospital CAH status was extracted from IHA records. Two variables were

used to measure hospital CAH status in the analyses. First, we examined the

effects of CAH conversion using a dummy variable, where CAH was equal to

oneifthehospitalwasCAHstatusthatyear.Secondwefollowedtheapproach

recommended by Antel, Ohsfeldt, and Becker (1995) of using a moving av-

erage of CAH conversion. The moving average CAH variable was defined as

the average of CAH status of the year t, t?1, and t?2 lags. The moving

average CAH variable was used to examine the long-run effects of CAH

conversion by putting less weight on hospitals during the first 2 years of

conversion.2

Descriptive Analyses

In the descriptive analyses, we computed cross-sectional comparisons be-

tween rural PPS and CAH hospitals for PSIs by year. Given that the distri-

butionsforPSIsareskewed,a Wilcoxon rank-sum test (Paganoand Gauvreau

2000) was used to conduct cross-sectional comparisons of PSIs between PPS

and CAH hospitals by year. We also conducted pre- and postconversion

comparisonsonPSIsforthe66hospitalsinCAHstatusin2004.Wecalculated

pre- and postconversion means and used Wilcoxon signed-rank test to test

whether there were statistically significant improvements in PSIs for hospitals

that converted to CAHs.

Generalized Estimating Equations (GEE) Logit Model

The value of PSIs is zero inflated and nonnegative, which suggests that or-

dinary least squares estimations may be problematic ( Jones 2000). Thus, we

employed a GEE logit model (Fitzmaurice, Laird, and Ware 2004) to estimate

the effects of CAHconversionon the binaryPSI variables(i.e., the association

2094 HSR: Health Services Research 42:6, Part I (December 2007)

Page 7

of CAH conversion with odds of poor performance for a specific PSI). The

unit of analysis is hospital-year. The GEE logit model has the following three-

part specifications (Fitzmaurice, Laird, and Ware 2004):

(a) Theconditionalexpectationoftheresponseisassumedtodependon

the independent variables through a logit link function.

logðpit=1 ? pitÞ ¼ b0þ b1CAHitþ

X

bnXitþ

X

bmZtþ eit

ð1Þ

where pitis the expected probabilityof hospital i having poor performance for

the PSI in year t. CAHitis the value of the CAH variable for the ith hospital at

year t. Xitis a vector of other explanatory variables that presumably are as-

sociated with patient safety for the ith hospital at year t. Xitincludes the per-

centage of Medicare patient days, the percentage of Medicaid days, the

hospital mean of the Charlson comorbidity score (Charlson et al. 1987; Deyo

et al. 1992), the percentage of surgical discharges, market concentration (Her-

findahl–Hirschman Index [HHI]), county-level per capita income, and coun-

ty-level population density. Ztis a vector of the year dummy variables, which

will adjust the effects of unmeasured, time-specific factors. b0, b1, b2, and b3are

parameters associated with these variables.

(b) The variation function is pit(1–pit).

(c) We assume the within-subject association among repeated measures

is a first-order autoregressive correlation pattern.

By using ‘‘sandwich’’ estimation, GEE yields consistent and valid esti-

mates of coefficients and their standard errors even if within-subject associ-

ations are not correctly specified (Fitzmaurice, Laird, and Ware 2004). The

dependent variables consist of eight binary quality functions with six individ-

ual PSIs and two composite PSIs. CAH conversion has been shown to be

associated with significant changes in hospital characteristics, such as bed size

and total number of discharges (e.g., Hagopian and Hart 2001;Rural Hospital

Flexibility Program Tracking Team 2002, 2003). Thus, including hospital

characteristics in multiple regression models may yield a multicollinearity

issue with the CAH variables. Consequently, we only adjusted for patient

characteristics and market variables in the multiple regression models.

Sensitivity Analyses

We undertook several sensitivity analyses to examine the robustness of our

findings. First, we examined the impact of CAH conversion on the two

Effect of CAH Conversion on Patient Safety2095

Page 8

denominator-weighted composite PSI scores. Second, one limitation of this

study is the potential endogeneity of CAH variables given that CAH conver-

sion is a decision made by rural hospitals themselves, rather than random

assignment. In addition, some rural PPS hospitals may never choose to con-

vert to CAH status. Using these hospitals as the comparison group may in-

troduce some selection bias to the extent that these hospitals may be different

from hospitals that convert to CAH. It is possible that the difference in PSI

performance between rural PPS and CAH hospitals is attributable to heter-

ogeneity in hospital characteristics rather than CAH conversion. In order to

deal with this issue, we repeated the original GEE analyses based on a sample

with 81 hospitals, which eventually converted to CAH. All of these hospitals

were in rural PPS status in 1997 and in CAH status by the end of 2005.

In addition, to conduct sensitivity analyses of hospital characteristics, we

estimated GEE models, which included hospital bed size and number of

discharges.

Sensitivity analyses also included panel data random-effects Tobit

models to estimate the effects of CAH conversion on continuous PSIs (using

Stata 7.0; see StataCorp. 2001b). The distributions of PSIs are skewed. The

value of PSIs is nonnegative with a mass at 0. Tobit models have a conceptual

advantage in analyzing data where the distribution of the dependent variable

is normal above a limiting value (e.g., Woolridge 2002; Salkever, Slade, and

Karakus 2006). The Tobit model assumes a continuous latent variable, de-

noted as PSIn, with normal distribution (equations [2] and [3])

PSIit?¼ b0þ b1CAHitþ

X

b2Xitþ

X

b3Ztþ eit

ð2Þ

PSIit¼ maxð0; PSIit?Þð3Þ

In this expression, we observe PSIitfor the ith hospital in year t. Equation (3)

indicates that we observed PSIitnif it is larger or equal to 0. If PSIitnis smaller

than 0, we observe 0. CAHit, Xit, and Ztare the same as we defined in the GEE

logit model (equation [1]). The terms b0, b1, b2, and b3are parameters asso-

ciated with these variables. The error term is expressed as eit.

The random-effects Tobit models were estimated using Gauss–Hermite

quadrature to compute the log likelihood and its derivatives (StataCorp.

2001a). We used the quadchk command in Stata 7.0 to check the numerical

soundness of the quadrature approximation. The quadchk command enables

examination of the extent to which changing the number of quadrature points

affects the results. If the relative difference in coefficients is smaller than 0.01

percent, the results may be confidently interpreted. However, if the relative

2096HSR: Health Services Research 42:6, Part I (December 2007)

Page 9

difference is 41 percent, the result is uncertain (StataCorp. 2001a). We ran

random-effects Tobit models for both risk-adjusted rates of PSIs and observed

rates of PSIs. One assumption for random-effects Tobit models is that there is

no correlation between independent variables and omitted variables. This

assumption may not be valid. By adding hospital IDs as dummy variables in

cross-sectional Tobit models, the unconditional fixed-effects Tobit models

reduce the omitted variable bias by ruling out the impact of hospital-specific

omittedvariables.3Alldataanalysisprogramsareavailableuponrequestfrom

the corresponding author.

RESULTS

ThemeanvaluesofPSIsforCAHsandruralPPShospitalsareshowninTable

2. Wilcoxon rank sum tests indicated that CAHs had statistically significantly

(p ? .05) higher performance (i.e., lower PSI value) than rural PPS hospitals

Table2:

PPS and CAHs, 1997–2004

Cross-SectionalComparisonofMeansofPSIszbetweenIowaRural

Year

Hospital

Categories

Number of

Hospitals PSI-2wPSI-3wPSI-5 PSI-6 PSI-7 PSI-15

Composite

Score of Four

PSIs

Composite

Score of Six

PSIsw

1997 Rural PPS

1998 Rural PPS

1999 Rural PPS

CAH

2000 Rural PPS

CAH

2001 Rural PPS

CAH

2002 Rural PPS

CAH

2003 Rural PPS

CAH

2004 Rural PPS

CAH

89

89

88

1

78

11

57

32

45

44

34

55

23

66

0.46

1.53

0

NA

0

0

1.11

1.62

0.93nn

0.40nn

0.42

0.85

1.04

0.92

6.9

9.18

10.93

NA

9.31

22.94

11.28

9.17

9.62

8.31

10.49nn0.01

7.38nn0.06

10.98

11.19

0.04

0.05

0.02

0

0.06

0

0.07

0

0.04

0.21

0.47

0.27

0.3

0

0.36

0

0.21nn0.69

0.07nn1.17

0.23

0.34

0.46nn0.88nn2.38nn

0.26nn0.41nn1.89nn

0.12n0.29nn0.54nn2.68nn

0.06n0.14nn0.17nn1.46nn

0.83

0.36

0.69

0

0.6

0.41

2.44

2.72

2.99

0

3.09

2.12

2.65

2.24

1.93

2.04

2.29

0

2.35n

1.58n

2.03

1.8

1.56nn

1.41nn

1.89nn

1.45nn

2.04nn

1.09nn

4.74

5.76

6.9

NA

5.72

7.85

6.9

5.61

5.95

5.02

6.38n

4.47n

7.01

5.41

0.64nn2.00nn

0.14nn1.89nn

nSignificant difference in PSIs between rural PPS and CAH at p ? .10 (Wilcoxon rank-sum test).

nnSignificant difference in PSIs between rural PPS and CAH at p ? .05 (Wilcoxon rank-sum test).

wMeans are based on those hospitals with nonmissing values.

zLower PSI value is better performance.

PSI, patient safety indicators; CAH, critical access hospitals; PPS, prospective payment system.

Effect of CAH Conversion on Patient Safety 2097

Page 10

on PSI-6 in 2001, on PSI-2, PSI-7, PSI-15, and composite score of four PSIs in

2002, on PSI-3, PSI-6, PSI-7, PSI-15 and composite score of four PSIs in 2003,

and on PSI-6, PSI-7, PSI-15 and composite score of four PSIs in 2004. Cross-

sectionalcomparisonsshowedthatCAHsgenerallyhadbetterperformancein

patient safety than rural PPS hospitals.

In the pre- and postconversion comparisons, as shown on Table 3, after

conversion to CAH status, hospitals experienced statistically significant im-

provement in performance as measured by PSI-7, PSI-15, and composite

score of the four PSIs. There were no significant changes in hospital perfor-

mance in PSI-2, PSI-3, PSI-5, and PSI-6. In general, following conversion,

there are more hospitals exhibiting better performance than those exhibiting

worse performance in all PSIs.

The descriptive comparisons in Tables 2 and 3 do not control for the

effects of independent variables, selection bias, or history bias. To address

theseissues,GEElogitmodelswereexamined.TheGEElogitmodelsshowed

that CAH conversion yielded significant improvement in hospital perfor-

mance (or lower odds of poor performance) in PSI-6, PSI-7, PSI-15, and

composite score of the four PSIs (Table 4). The odds ratios of poor perfor-

mance in CAH hospitals compared with rural PPS hospitals are 0.30 (con-

fidence interval [CI]: 0.14–0.64) for PSI-6, 0.29 (CI: 0.15–0.56) for PSI-7, 0.40

Table3:

Hospitals, 1997–2004

Changes in PSIsnafter Converting to CAHs for Iowa Rural

PSI-2 PSI-3PSI-5 PSI-6 PSI-7PSI-15

Composite

Score of Four

PSIs

Composite

Score of Six

PSIs

N

Mean value of change in PSI

(after–before)

Number of hospitals having

better performance in PSIs

after conversion (reducing

PSI value)

Number of hospitals having

worse performance in PSIs

after conversion

(increasing PSI value)

p-valueforWilcoxonsigned-

rank test

59 57 66 66 6666 66

?0.623

54

?0.3100.187 ?0.0050.030 ?0.090 ?0.250 ?0.800

15 328 1829 37 4130

11 22588 16 1624

.68.60 .64.34 .01

o.01

o.01 .41

nLower PSI value is better performance.

PSI, patient safety indicators.

2098HSR: Health Services Research 42:6, Part I (December 2007)

Page 11

(CI:0.24–0.67)forPSI-15,and0.49(CI:0.31–0.80)forcompositescoreoffour

PSIs.WealsoobservedhigherscaleimprovementforCAHmvinPSI-6,PSI-7,

PSI-15, and composite score of four PSIs.2CAH conversion did not yield

significant changes in the performance in PSI-2, PSI-3, PSI-5, and composite

score of six PSIs. GEE logit models also showed that a higher percentage of

surgical discharges in a hospital was associated with a higher odds of poor

performance in observed rates of PSI-5, and risk-adjusted rates of PSI-6, PSI-

15, and the two composite PSI measures. Also, the hospital mean for the

Charlson comorbidity score was positively associated with poor performance

in risk-adjusted PSI-3, PSI-5, PSI-15, and two composite PSIs. These results

Table4:

mance for the PSI) in Iowa Rural Hospitals, 1997–2004

GEE Logit Models of the Performance of PSIs (15Poor Perfor-

PSI-2z

PSI-3 PSI-5 PSI-6PSI-7 PSI-15

Composite

Score of

Four PSIs

Composite

Score of

Six PSIs

CAH

% Medicare days

% Medicaid days

% of surgical discharges

Charlson Index

Market concentration

(HHI)

Per capita income

($1,000)

Population density

y1998

y1999

y2000

y2001

y2002

y2003

y2004

Year

Intercept

Observations

?0.54

0.09nn

0.09nn

0.01

?2.60nn

0.11

?0.24

0.00

0.03

0.01

1.36n

?0.42

?0.80

?0.01

?0.01

0.09nn

2.01nn

?2.11

?1.19nn?1.26nn?0.92nn?0.70nn?0.30

?0.01

0.02 0.03 0.02

0.05nn

0.01 0.18nn

0.69 1.05 1.51nn

0.370.24

?0.21

?0.01n?0.01

?0.01

0.03

0.16nn

1.37nn

?0.15

0.00

0.04

0.04nn

2.14nn

?0.44

0.090.16nn

0.00

?0.030.090.060.07nn

0.15

0.02nn

0.00

0.04

0.30

0.34

0.68nn

0.34

0.21

0.16

?0.01

?0.16

?0.63

?0.53

0.07

0.41

?0.21

0.84

0.02 0.010.00 0.000.00

0.05

0.53

0.16

0.44

0.24

?0.43

?0.29

0.18

0.19

?0.34

0.53

?0.20

?0.52

?0.24

?0.41

0.33

?0.24

0.21

?0.38

?0.16

0.11

0.23

0.66n

?0.08

0.27

0.10

?0.49

?0.23

?0.32

0.21

?0.54

?0.13

?0.50

?0.02

0.27

0.09

?185.30

630

?4.81nn?3.23

624

?2.67n?3.73nn?3.87nn?3.54nn?5.76nn

712 712712 712712594

wUnit of analysis is hospital-year. y1998–y2004 were dummy variables for year 1998–2004. The

year of 1997 is the reference category.

nStatistically significant at .10 level.

nnStatistically significant at .05 level.

zGEE logit models for PSI-2 using categorical year variables were not converged. Continuous

year variable was used.

PSI, patient safety indicators; CAH, critical access hospitals; HHI, Herfindahl–Hirschman Index.

Effect of CAH Conversion on Patient Safety2099

Page 12

indicate that hospitals with more severe cases might have worse performance

in some risk-adjusted PSIs.

In our sensitivity analyses, we found that CAH conversion was asso-

ciated with lower odds of poor performance in denominator-weighted

composite score of four PSIs. There is no significant impact on denominator-

weighted composite score of six PSIs. The GEE logit models using a sample

excluding eight hospitals, which were in rural PPS status in 2005, showed a

similar result to those shown in Table 4. Sensitivity analyses using random-

effects Tobit models showed that CAH conversion led to significantly lower

values in PSI-6, PSI-7, PSI-15, and composite score of four PSIs. Fixed-effects

Tobit models showed that CAH conversion was associated with significantly

lower values of PSI-7, PSI-15, and composite score of four PSIs. In general,

estimates for the moving average measure of CAH status had a larger scale

than CAH estimates. In addition, we estimated a simulation using a random-

effects Tobit model to examine the magnitude of CAH conversion impact on

both composite scores of PSIs. CAH conversion is associated with 60 and 16

percent reduction in composite score of four PSIs and composite score of six

PSIs,respectively.Aswediscussedabove,duetotheissueofmulticollinearity,

hospital characteristic variables were not included in the GEE logit models in

Table 4.

DISCUSSION

WeshowastrongandconsistenteffectofCAHstatusonPSIs.Inthesampleof

Iowa rural hospitals, the cross-sectional comparisons showed that CAHs had

better performance than rural PPS hospitals. The pre- and postconversion

comparisons showed that postconversion hospitals had better performance in

PSIs than preconversion hospitals. The results are robust to a number of

appropriate estimation strategies, and hold up to rigorous sensitivity analyses.

To address the concern that difference in PSIs might reflect primarily differ-

encesinpatientmix,timetrends,anddifferencesinmarketsandenvironment,

we added two additional analyses. First, we used risk-adjusted PSIs4as the

measures for the performance of hospital patient safety. Risk-adjusted rates

reflect the performance of providers on the PSIs if those providers had an

‘‘average case mix’’ (AHRQ 2006a,c). Thus, it greatly reduces the effects of

patient case mix on patient safety measures. Second, we used GEE logit

models, random-effects Tobit models and other sensitivity analyses to control

for the impact of patient case mix, market variables, and time trend. Owing to

2100HSR: Health Services Research 42:6, Part I (December 2007)

Page 13

multicollinearity, hospital characteristics were not included in the original

models.

A possible explanation for these findings could be that hospitals might

change their coding behaviors in response to the change in Medicare reim-

bursement scheme;that is,observedlower rates of adverse events may be due

toachangeinhospitalcodingbehaviorwhentheyconvertedfromPPStocost-

basedreimbursement.Ingeneral,underPPS,hospitalshavemoreincentiveto

code severe cases, which are associated with higher reimbursement rates. If

there is no change in patient case mix after conversion, a significant change in

coding behavior after converting from PPS to cost-based would result in a less

severe observed case mix in rural hospitals. To examine this possibility, we

conducted separate analyses to examine the relationship between CAH con-

version and hospital-level average number of patient diagnoses, average

number of Elixhauser comorbidities (Elixhauser et al. 1998), and mean score

of the Charlson comorbidity index. We found that there was no significant

association between CAH conversion and these measures.

Another alternative explanation for the positive effects of CAH con-

version on patient safety could be that it reflects a trend toward improvement

in patient safety for all hospitals. However, according to AHRQ, PSI-2, PSI-3,

PSI-5, PSI-6, PSI-7, and PSI-15 in the Nationwide Inpatient Sample (NIS) did

not exhibit an overall trend toward improvement over the 1997–2003 time

period (AHRQ 2006b). Likewise, our computations of the Iowa SIDs from

1997 to2004usingAHRQPSIsoftwareindicatedthattherewasnosignificant

overallimprovementtrendforPSI-2,PSI-5,PSI-6,and PSI-15inIowa’surban

and rural referral hospitals. In addition, we controlled for time effects by

including year dummy variables in the GEE and tobit models. The 8-year

paneldatadesignhelpsadjustforselection-maturationthreats(Shadish,Cook,

and Campbell 2002), regression to the mean biases5(Antel, Ohsfeldt, and

Becker 1995), and delay causation (Shadish, Cook, and Campbell 2002).

These analyses consistently showed that CAH conversion led to improved

performanceof PSI-6, PSI-7, and PSI-15. In the sensitivity analyses,GEE logit

models and Tobit models also showed that moving-average estimates of the

CAH effect had a larger scale than the dichotomous CAH measure, which

indicates that CAH conversion may lead to larger improvement in patient

safety in the long run.

CAH conversion has been shown to significantly improve rural hospi-

tals’ financial condition. Previous studieshaveshown that preconversion rural

hospitals faced serious financial pressure (Cameron, Zelman, and Stewart

2001; Stensland, Davidson, and Moscovice 2004) and that over half of the

Effect of CAH Conversion on Patient Safety 2101

Page 14

hospitals that converted to CAH in FY1999 or FY2000 were losing money

before conversion (Stensland, Davidson, and Moscovice 2004). Stensland,

Davidson, and Moscovice (2004) reported that hospitals that converted to

CAH in FY1999 experienced an average increase in Medicare payment of

36 percent, around $500,000 in FY2000 inflation adjusted dollars after con-

version, and that CAH conversion increased hospital profit margins by 2–4

percentage points.

Improved financial conditions and lower risk sharing associated with

CAH conversion are likely to contribute to improvements in patient safety.

After conversion, under cost-based reimbursement, risk sharing decreased

substantially. Under PPS, the marginal costs associated with QI are not re-

imbursed, and the hospital has to bear all the cost incurred by the increased

intensity and quality (Cutler 1995). Under cost-plus reimbursement, marginal

costs associated with increased quality are fully reimbursed. Hospitals tend to

have higher intensity and produce a higher level of quality under lower risk

sharing (Hodgkin and McGuire 1994; Cutler 1995; Chalkley and Malcomson

2000).

To the extent that more resources are needed to bring about meaning-

ful improvements in quality of care, patient safety is inextricably linked to

the financial condition of hospitals (Encinosa and Bernard 2005). It is

harder for hospitals with financial problems to make investments in patient

safety improvements (e.g. error-reducing information technology system), or

to attract or retain high-cost specialists. Likewise, hospitals with financial

problems might cut nurse staffing which may adversely affect patient safety

(Encinosa and Bernard 2005). Cutler (1995) found that the fiscal pressures

from the PPS in Medicare in the 1980s were associated with higher mortality.

Shen (2003) also has shown that financial pressure was associated with in-

creased mortality rates after treatment of acute myocardial infarction. En-

cinosa and Bernard (2005) found that a within-hospital erosion of hospital

operating margins was associated with an increased rate of adverse patient

safety events.

The Rural Hospital Flexibility Program Tracking Team found that QI-

or QA-related activities were widely undertaken in CAHs and have been

reinforced over time since CAH conversion (Moscovice and Gregg 2001;

Moscovice, Gregg, and Klingner 2002). Staffing improvement was one of the

most significant factors contributing to progress in quality of care and

increased reimbursement was cited as the reason for improved staffing

(Moscovice, Gregg, and Klingner 2002). Consistent with these findings, and

expanding them to outcomes, we found that CAHs strengthened their scores

2102 HSR: Health Services Research 42:6, Part I (December 2007)

Page 15

on PSIs after conversion, at the time that they would have been experiencing

higher reimbursement.

Apart from the change in reimbursement method and the resulting fi-

nancial relief, other factors may also contribute to better quality of care in

CAHs. These factors may include the establishment of a network relationship

with affiliated hospitals, improvement in case management and discharge

planning, expansion in qualified QA and QI staff, and enhancement in equip-

ment (Moscovice and Gregg 2001; Moscovice, Gregg, and Klingner 2002).

There are several limitations in our study. First, there are limitations

inherenttoadministrativedatabasesincludingthepossibilityofmissingcodes,

coding errors, and variation in coding practices across hospitals (Simborg

1981; Hsia et al. 1988; Iezzoni 1997; Zhan and Miller 2003). Although we did

not find a significant association between CAH conversion and various mea-

suresofpatientseverity,itispossiblethelowerratesofadverseeventsinCAH

hospitals are simply due to a reduced incentive to code certain diagnoses. In

this study, we are not able to completely rule out this possibility.

A second issue of concernis endogeneityand omittedvariable bias. The

CAHvariableislikelytobeendogenousgiventhathospitalschoosetoconvert

to CAH status. However, endogeneity is to a large extent mitigated through

the fixed-effects panel models with year dummy variables, which capture the

effects of unmeasured hospital-specific and time-specific factors (Woolridge

2002). Furthermore, the results of the sensitivity analysis using a sample of 81

Iowa rural hospitals (all of which were in rural PPS status in 1997 and in CAH

status at the end of 2005) suggests that endogeneity is not an exceptionally

large concern. The consistency between the sensitivity analyses and our main

results indicate that our findings are robust.

A third limitation involves using AHRQ software to calculate PSIs and

evaluate hospital patient safety performance. The AHRQ software is not able

to fully identify all the preventable adverse events due to the limited clinical

information available in administrative data (AHRQ 2006a). In addition, only

sixofthe29PSIshadadequatecasestomeasurepatientsafetyinoursampleof

rural hospitals. The six PSIs are less comprehensive than the complete pack-

age of PSIs created by AHRQ PSI software. Furthermore, the assumption for

the Tobit models is that the distribution is a censored normal distribution.

However,wewerenotabletotestthisassumptiongiventhatthecensoreddata

are unobservable (Duan 1983). We only include the results of Tobit models in

the sensitivity analyses. The findings of Tobit models were consistent with

GEElogitmodels.Finally,oursampleonlyincludesIowaruralhospitals,thus,

our results may not generalize to other states.

Effect of CAH Conversion on Patient Safety2103