Hospital Quality Indicators in Iowa Rural Hospitals

1
Hospital Quality Indicators in Iowa Rural
Hospitals

• Pengxiang (Alex) Li, Marcia M. Ward, Paul James,
  John E. Schneider
• 2008 AHRQ Annual Meeting
• Bethesda, Maryland
• Support grant Agency for Healthcare Research and
  Quality Grant HS015009

2
Background

• Hospital quality indicators were used to provide
  a perspective on hospital quality of care
• AHRQ Inpatient Quality Indicators (IQIs)
• AHRQ Patient Safety Indicators (PSIs)
• Our analyses focus on
• Acute Myocardial Infarction (AMI) in-hospital
  mortality (IQI-15)
• Four PSIs (PSI-5, PSI-6, PSI-7, and PSI-15)

3
Outline

• Comparison of Iowa urban and rural hospitals on
  AMI in-hospital mortality
• James PA, Li P, Ward MM. Myocardial infarction
  mortality in rural and urban hospitals
  Rethinking measures of quality of care. Annals of
  Family Medicine, 5105-111, 2007
• Association between Critical Access Hospital
  (CAH) conversion and patient safety indicator
  performance
• Li, P., Schneider, J. E. Ward, M. M., (2007)
  Effect of Critical Access Hospital Conversion on
  Patient Safety. Health Services Research, 42 (6)
  2089-2108
• Exploration of a potential reason of patient
  safety change associated with CAH conversion
• Li, P., Schneider, J. E. Ward, M. M., Effects
  of Critical Access Hospital Conversion on the
  Financial Performance of Rural Hospitals Inquiry
  (in press)

4
How do Iowa urban and rural hospitals compare on
AMI in-hospital mortality?

• James PA, Li P, Ward MM. Myocardial infarction
  mortality in rural and urban hospitals
  Rethinking measures of quality of care. Annals of
  Family Medicine, 5105-111, 2007

5
Introduction

• Observational studies find that the quality of
  care for myocardial infarction (MI) patients
  admitted to rural hospitals is substandard
  (Sheikh 2001, Baldwin 2004)
• Lower volumes of MI patients in rural hospitals
• Lacking cardiologists
• Lacking support services

6
Introduction

• Validity of these observational studies has been
  questioned
• Unbalanced comparison groups
• Patients admitted to rural hospitals tend to be
  older, poorer, in poorer health, and have greater
  number of comorbidities (Baldwin 2004, Chen 2000,
  Frances 2000)
• Referral patterns of rural provider
• Empirical study showed that less severe patients
  were referred to urban hospitals (Metha 1999)
• Unmeasured confounding may account for
  differences in patient outcomes

7
Objectives of the study

• To compare characteristics of MI patients
  admitted to rural and urban hospitals
• To examine in-hospital mortality between rural
  and urban hospitals among MI patients
• Using traditional risk adjustment techniques
  (Logistic regression)
• Using instrumental variable methods (IV)

8
Methods Data

• Discharge data from Iowa State Inpatient Dataset
  (2002 2003)
• Inclusion criteria
• A principal diagnosis of MI (ICD-9-CM
  410.01-410.91)
• Eighteen years or older
• Exclusion criteria
• The hospital identification number was missing
  (n9)
• Patients whose home county was not in Iowa
  (n1,248)
• Patients zip code was missing (n14)
• Patients sex was missing (n1)
• Our primary analyses also excluded patients
  discharged or transferred to another short term
  general hospital for inpatient care (n1,618)
• Most of our analyses are based on 12,191 MI
  patients

9
Methods Variables

• Dependent variable
• In-hospital mortality
• Independent variables
• Urban vs Rural hospitals that patients admitted
  to
• Urban 27 hospitals
• Rural 89 hospitals
• Payer e.g. Medicare, private insurance, self-pay
  
• Admission type e.g. emergency
• Race
• Risk adjustment index
• Charlson comorbidity index
• All Patient Refined DRGs (APR-DRGs) risk index

10
Methods Traditional Analytic Approach (Logistic
Regression)

• Univariate analyses of group comparisons
• Chi-square tests for dichotomous data
• ANOVAs for continuous data
• Logistic regressions for multiple regression
  analyses

11
Methods Pitfalls with Logistic Regression

• Using administrative inpatient data, one cannot
  control all patients risk factors (e.g. severity
  of illness)
• If unmeasured variables are related to selection
  of the hospital, the estimates of the
  hospital-specific contribution to mortality will
  be biased.
• For example, elderly MI patients with severe
  comorbid conditions, which are unmeasured in
  administrative data, might prefer to remain in
  the rural hospitals.
• As a result, a higher risk-adjusted mortality
  rate in rural hospitals might simply be due to
  more severe patients in rural hospitals.

12
Approaches to Minimize Bias

• Collect all the relevant patient-level variables
  very costly
• Randomized controlled trial
• Not feasible to this study
• Instrumental variable (IV) estimation
• An econometric technique which enables us to
  obtain unbiased estimates of treatment effects in
  observational studies
• An example Wehby (2006) found that using the
  logistic regression model, early initiation of
  prenatal care is associated with a higher
  probability of low birth weight (LBW)
• Unmeasured confounders women at a higher risk
  demand more (or early) prenatal care compared to
  those at lower risk.
• IV estimations showed that early time to prenatal
  care initiation is associated with a lower
  probability of LBW.

13
The Instrumental Variable (IV) estimation

• IVs are used to achieve a pseudo-randomization
• The instrumental variable technique can extract
  variation in the focal variable (rural hospital
  selection) that is unrelated to unmeasured
  confounders, and employ this variation to
  estimate the causal effect on an outcome
• Assumptions for IV(s)
• IV(s) should correlate with treatment variable
  (choice of rural hospital)
• IV(s) should not be correlated with the
  unmeasured confounders

14
Methods Instrumental Variable Technique

• Instrumental Variable Patients distance to
  the nearest urban hospital
• The distances between each patients home and all
  urban hospitals in Iowa were obtained by
  calculating the distances between the centroids
  of each patients resident zip code and all urban
  hospitals zip codes.
• Similar to Brooks (2003) approach, instrumental
  variables in the study are dummy variables that
  group patients based on the their distance to the
  nearest urban hospital.

15
Methods IV Technique First assumption

• Patients who live closer to an urban hospital are
  more likely to choose an urban hospital than
  those who live farther away.
• Partial F-statistics for the IVs in the first
  stage regression
• Small values of first-stage F-statistics imply
  failure of assumption 1
• Rule of thumb Fgt10 indicates good association
  (Staiger 1997)

16
Methods IV Technique Second Assumption

• Distance to the nearest urban hospital is not
  associated with the severity or pre-morbid risks
  of patients with MI
• Descriptive comparison between two groups of
  patients classified by IV
• If the instrument is independent of the
  unmeasured confounders, it should also be
  independent of observed risk factors (e.g. age,
  and comorbidity index).
• Over-identifying restrictions tests
• The null hypothesis is that the IV is not
  correlated with unmeasured confounders

17
Methods IV Technique

• To examine the robustness of our findings
• We used a range of patients groups for the
  instrumental variable (2, 4, 8, and 12 groups).
• We varied the independent variables.
• The syslin two-stage least squares (2SLS)
  procedure in SAS 9.1 was used to do IV estimation.

18
Results Table 1 Baseline characteristics of MI
patients admitted to rural and urban hospitals
Variables Rural (N 1,426) Urban (N 10,765) p-value
Age 82.35 68.89 lt.0001
Male () 45.02 59.76 lt.0001
Black () 0.14 1.13 0.0004
Number of secondary diagnoses 5.66 5.61 0.43
Charlson comorbidity index 0.96 0.69 lt.0001
APR-DRG risk index 0.09 0.06 lt.0001
In-hospital Mortality 0.14 0.06 lt.0001
Excluding patients discharged or transferred to
another short term general hospital for inpatient
care.
19
Results Table 2 Baseline characteristics of MI
patients transferred out of rural hospitals or
staying in rural hospitals
Variables Stay in rural hospitals (N1,426) Transfer out of rural hospitals (N730) p-value
Age 82.35 71.46 lt.0001
Male () 45.02 56.71 lt.0001
Black () 0.14 0.14 0.99
Number of secondary diagnoses 5.66 4.24 lt.0001
Charlson comorbidity index 0.96 0.67 lt.0001
APR-DRG risk index 0.09 0.04 lt.0001
Patients discharged or transferred to another
acute care hospital for inpatient care
20
Results Table 3 Odds ratios of in-hospital
mortality among MI patients admitted to urban
hospitals or to rural hospitals, using logistic
regression models (n12,191)
Model components Odds ratio (Urban vs Rural) 95 CI p-value c-statistic
Unadjusted 0.42 0.36-0.50 lt.0001 0.56
Adjusted for demographic variables (age, sex, race, admission type and source of payment) 0.70 0.59-0.84 lt.0001 0.71
Adjusted for demographic variables and Charlson comorbidity index 0.70 0.59-0.84 0.0001 0.71
Adjusted for demographic variables and APR-DRG risk index 0.68 0.56-0.82 lt.0001 0.86
Excluding patients discharged or transferred to
another short term general hospital for inpatient
care
21
Results Table 4 Characteristics among MI
patients grouped by distance to the nearest urban
hospital
Variables Distance to nearest urban hospital lt14.08 miles (N 6,097) Distance to nearest urban hospital gt14.08 miles (N 6,104) p-value
Mean Distance to the nearest urban hospital (miles) 4.94 34.20 lt0.0001
Percent of patients admitted to urban hospitals () 99.54 77.07 lt0.0001
Age 68.89 72.02 lt0.0001
Male () 58.65 57.45 0.18
Black () 1.95 0.08 lt0.0001
Number of secondary diagnoses 5.72 5.53 lt0.0001
Charlson comorbidity index 0.72 0.72 0.67
APR-DRG risk index 0.07 0.07 0.48
In-hospital mortality rate () 7.07 7.52 0.34
14.08 miles is the median distance from
patients home to the nearest urban hospital
22
Results Table 5 Instrumental variable
estimates of the difference of in-patient
mortality between urban and rural hospitals
IV models (n12,191)    Number of groups for instrumental variable Tests for instrumental variables Tests for instrumental variables IV estimates of mortality difference IV estimates of mortality difference
IV models (n12,191)    Number of groups for instrumental variable Instrument P-value for overidentifying restrictions tests Coefficients P-value
IV models (n12,191)    Number of groups for instrumental variable F-statistic P-value for overidentifying restrictions tests Coefficients P-value
Unadjusted 2 1540.16 - -0.0199 0.34
Unadjusted 4 642.65 0.65 -0.0269 0.16
Unadjusted 12 184.31 0.13 -0.0288 0.13
Adjusted for demographic variables 2 1568.24 - 0.0127 0.58
Adjusted for demographic variables 4 652.86 0.80 0.0081 0.69
Adjusted for demographic variables 12 187.14 0.10 0.0065 0.75
Adjusted for demographic variables and Charlson comorbidity index 2 1539.9 - 0.0090 0.69
Adjusted for demographic variables and Charlson comorbidity index 4 642.51 0.92 0.0053 0.80
Adjusted for demographic variables and Charlson comorbidity index 12 184.29 0.12 0.0040 0.84
Adjusted for demographic variables and APR-DRG risk index 2 1694.27 - -0.0034 0.87
Adjusted for demographic variables and APR-DRG risk index 4 640.61 0.92 -0.0069 0.72
Adjusted for demographic variables and APR-DRG risk index 12 202.50 0.01 -0.0063 0.74

If a F-statistic is less than 10, the
instrumental variables are weak. If p-value
is less than 0.05, one of the instrumental
variables correlated with unmeasured confounders
23
Results Sensitivity analyses

• Repeat analyses in different samples
• Excluding transferred in MI patients
• Three-year state inpatient datasets (2001 to
  2003)
• Different IV estimation method
• Two-stage residual inclusion method to account
  for the endogeneity in nonlinear (logistic) model
  
• Bivariate Probit model (using Stata 9.0)
• The results are consistent with IV estimation in
  Table 5

24
Discussion

• This study confirms earlier studies
• MI patients admitted to rural hospitals were
  older and sicker than their urban counterparts
• Traditional models all indicate significantly
  higher in-hospital mortality for those admitted
  to rural hospitals

25
Discussion

• Our findings suggest that the traditional
  logistic regression models are biased
• Admissions to rural or urban hospitals are likely
  to be confounded by unmeasured patient variables
• Referral patterns in rural hospitals
• Younger and less sick patients are transferred to
  urban hospitals
• The clinical judgment about transfer of rural
  senior patients with MI may rely on different
  criteria

26
Discussion

• Patient preferences are likely to play a
  significant role in transfer decisions for older
  MI patients
• May reflect personal choice or existing serious
  comorbidities
• Serious cases may choose to remain close to home
• The transfer patterns may reflect rural doctors
  respecting their patients wishes
• Using in-hospital MI mortality to measure quality
  of care in rural hospitals is problematic.

27
Limitations of the study

• The results of the IV estimation can only be
  generalized to patients for whom distance affects
  their choice
• The conclusion cannot be applied to MI patients
  bypassing rural hospitals and seeking care in
  urban hospitals
• The findings for hospitals in one state may not
  generalize to other states .
• Analyses of in-hospital mortality rates may not
  generalize to mortality rates after
  hospitalization.

28
Conclusions

• Mortality from MI in rural Iowa hospitals is not
  higher when controlled for unmeasured
  confounders.
• Current risk-adjustment models may not be
  sufficient when assessing hospitals that perform
  different functions within the healthcare system.
  
• Unmeasured confounding is a significant concern
  when comparing heterogeneous and undifferentiated
  populations.

29
Did conversion to Critical Access Hospital (CAH)
status affect patient safety indicator
performance?

• Li, P., Schneider, J. E. Ward, M. M., (2007)
  Effect of Critical Access Hospital Conversion on
  Patient Safety. Health Services Research, 42 (6)
  2089-2108

30
Background

• In order to protect small, financially vulnerable
  rural hospitals, the Medicare Rural Hospital
  Flexibility Program of the 1997 Balanced Budget
  Act allowed hospitals meeting certain criteria to
  convert to critical access hospitals (CAH)
• This changed their Medicare reimbursement
  mechanism from prospective (PPS) to cost-based
• One objective of the policy was to increase the
  quality of care in these hospitals

31
Timeframe for Conversion to CAH
32
Patient Safety
33
4 PSIs and Composite

• AHRQ recommends suppressing the estimates if
  fewer than 30 cases are in the denominator
• Only five patient safety indicators are able to
  provide PSI measures for all rural Iowa hospitals
  
• PSI-5 foreign body left during procedure
• PSI-6 iatrogenic pneumothorax
• PSI-7 selected infections due to medical care
• PSI-15 accidental puncture or laceration
• PSI-16 transfusion reaction
• Too rare to provide variability to differentiate
  hospitals in Iowa
• A composite patient safety variable was created
  by summing the four PSIs (PSI-5, PSI-6, PSI-7,
  and PSI-15).

34
Number of Hospitals Having Better or Worse
Performance after CAH Conversion
35
Cross-sectional Analyses

• Cross-sectional comparisons showed that CAHs had
  better performance than rural PPS hospitals on 4
  of the 5 PSI measures.
• However, the difference in patient safety
  indicators might be due to differences in patient
  mix, hospital characteristics besides CAH
  conversion, and differences in markets and
  environment.

36
Multivariable Analyses

• We used multivariable Generalized Estimating
  Equations (GEE) models and sensitivity analyses
  to control for the impact of patient case mix,
  market variables, and time trend.
• GEE models showed that CAH conversion was
  associated with significant better performance in
  PSI-6, PSI-7, PSI-15 and composite PSI.
• Findings were robust among sensitivity analyses
  using different samples and different methods

37
Conclusions

• CAH conversion in rural hospitals resulted in
  enhanced performance in PSIs
• We speculate that the likely mechanism involved
  an increase in financial resources following CAH
  conversion to cost-based reimbursement for
  Medicare patients

38
How did Critical Access Hospital conversion
affect rural hospital financial condition?

• Li, P., Schneider, J. E. Ward, M. M., Effects
  of Critical Access Hospital Conversion on the
  Financial Performance of Rural Hospitals Inquiry
  (in press)

39
Objectives

• To study the effects of CAH conversion on Iowa
  rural hospitals operating revenue, cost, and
  profit margin

40
Study Sample and Study design

• Sample
• Eight year (1997-2004) panel data for 89 Iowa
  rural hospitals (rural PPS hospitals and CAHs)
• Unit of analysis is hospital-year
• Study design
• Quasi-experimental designs that use both control
  groups and pretests
• Panel data regression with fixed hospital effects

41
Models

• Ad hoc models
• Revenueitf(CAHit,Pjt,Yit,Xit)
• Costitf(CAHit,Wjt,Yit,Xit)
• Marginitf(CAHit,Wjt, Pjt,Yit, Xit)
• Variables
• CAHit hospital status (CAH or rural PPS) for ith
  hospital in year t
• Pit output prices for ith hospital in year t
• Wit input prices for ith hospital in year t
• Yit output volume for ith hospital in year t
• Xit other variables for ith hospital in year t
  that empirically affect dependent variables

42
CAH variables

• One dummy variable
• CAH1, if the hospital is in CAH status
• Three dummy variables
• CAH1it1, if the hospital is in the first year of
  CAH status, otherwise CAH1it0
• CAH2it1, if the hospital is in the second year
  of CAH status, otherwise CAH2it0
• CAH3it1, if the hospital is in CAH status for
  more than 2 years, otherwise CAH3it0
• Comparison group Rural PPS

43
Other covariates

• Pit output prices for ith hospital in year t
• Medicare Part A (hospital) adjusted average per
  capita cost (AAPCC) as proxy of hospital output
  price (county level)
• Wit input prices for ith hospital in year t
• Hourly wages for registered nurses (county level)
• Yit output volume for ith hospital in year t
• Total number of acute discharges, total number of
  outpatient visits, and average length of stay of
  acute discharges
• The squared and cubed output measures and
  interaction terms will be included

44
Others

• Xit other variables for ith hospital in year t
  that empirically affect dependent variables
• Hospital size (number of beds)
• Hospital case-mix
• Hospital mean DRG weight, percent of emergency
  visits, and percent of Medicare and Medicaid days
  among acute inpatient days
• Variables reflecting the hospital market (we
  assumed the county to be the relevant geographic
  market of hospital services.)
• Herfindahl-Hirschman Index (HHI), per capita
  income, and population density in the county in
  which the hospital is located
• Year dummy variables which will adjust the
  effects of unmeasured, time-specific factors
• Revenue and expense functions were log
  transformed

45
Data Sources

• Iowa Hospital Association Profiles
• Iowa State Inpatient datasets
• Area Resource File
• Centers for Medicare and Medicaid Services
• American Hospital Association Annual Survey
  Database
• Bureau of Labor Statistics

46
ResultTable 1 Changes in rural hospital
patient care revenue, expense, and operating
margin associated with CAH conversion, 1998-2004
P-valuelt 0.1 P-valuelt 0.05
47
Table 2 Changes in rural hospital patient care
revenue, expense, and operating margin during the
first, second and third plus years of CAH
conversion, 1998-2004
P-valuelt 0.1 P-valuelt 0.05
48
Results

• Operating revenue
• No change in the first year of conversion (paid
  an interim rate)
• Significant increases since the second year of
  CAH conversion
• Operating expenses
• CAH conversion is associated with significant
  increase in hospital operating expenses
• Hospitals increase expenses in the first year of
  conversion
• Operating Margin
• Significant drop in the first year of conversion
• Significant increase since the second year of
  conversion
• Sensitivity analyses showed similar results

49
Conclusions

• CAH conversion in rural hospitals resulted in
  better patient safety.
• Rural hospital CAH conversion was associated with
  significant increases in hospital operating
  revenues, expenses and margins

50
Summary Limitations of measures

• In-hospital mortality
• Substantial unmeasured confounders
• Patient Safety Indicators
• Only small number of indicators can be applied to
  rural hospitals
• Changes of indicators might reflect changes in
  coding or reporting in administrative data
• We need hospital quality indicators specifically
  for rural hospitals

51

• Thank you
• Questions?
• Contact information
• Pengxiang (Alex) Li
• University of Pennsylvania
• penli_at_mail.med.upenn.edu