Hospital Leadership Quality Assessment Tool

1
Hospital Leadership Quality Assessment Tool

• Presentation to Hospital-LQAT Advisory Group
• September 12, 2007
• 300 pm

2
Agenda

• Welcome Intro Jim Conway and Mark
  Koepke
• Pilot Summary Chris Hatcher
• Field Testing Hospital-LQAT Shannon Archer
• Data Analysis Plan Steve Mayfield
• Affecting Practice Tom Vaughn
• Next Steps Mark Koepke

3
Welcome Intro

• Welcome New Members
• Two Phases of Project
• Field testing and validation of Hospital-LQAT
• Affecting performance with the Hospital-LQAT,
  2008
• Thank you to Sanofi-Aventis
• Related Initiatives

4
The Hospital-LQAT is designed to help hospital
boards and executives determine where a hospital
stands regarding leadership efforts that foster
changes for improved quality of care.

• The Hospital-LQAT includes descriptors of
  structures, processes and activities of hospital
  leaders related to their engagement in clinical
  quality.

5
Development of the Hospital Leadership and
Quality Assessment Tool Hospital-LQAT.

• OK Hospital Interventions QIOSC
• Publications on Leadership and Quality
  (Iowa/CMS/CareScience)
• Hospital Leadership Summit
• LQAT Draft Focus Groups
• LQAT Survey Refinement
• LQAT Alpha test to NY and Penn Hospitals
• LQAT ready for Field Test
• LQAT Administered to 15 States
• Final Analysis Field tested, validated, and
  finalized.

2005
Mar. 2006
Sept. 2006
Nov. 06 June 07
June - July 2007
Aug. 2007
Sept. 2007
Oct. Nov. 2007
December 2007
6
Pilot Summary

• Purpose of Pilot
• Supporting Partners and Participating Hospitals
• Online Tool
• Pilot Data Analysis

7
Pilot Summary By TitleNew York and
Pennsylvania Hospitals
8
Pilot Summary By Section New York and
Pennsylvania Hospitals
9
Feedback from Focus Groups

• Overall Impression of Hospital-LQAT
• Easy to use
• Time to complete average 10 minutes
• Logical flow of questions
• Benefit of having 2 types of media available
  (paper and online)
• Barriers Identified
• Some redundancy of questions in some areas
• Did not have the information available/ not aware
  of information to answer questions (ie. Board
  Member)

10
Feedback from Focus Groups

• Opportunities/Suggestions for Improvement
• Standardize scoring throughout tool
• Clarify that participant is giving their opinion
  about various leaders attributes
• Ranking type questions need improvement
• Integration/Complement With Other Survey Tools
  (i.e., Leapfrog, AHRQ Safety Culture Survey)
• Feel that Hospital-LQAT will complement current
  tools already in use
• Focus on leadership that other surveys do not
  provide

11
Required Steps for Field Test

• Establish Methodology
• Recruit Hospitals and Administer Hospital-LQAT
• Analysis

12
Establish Methodology

• How to Aggregate/Analyze Responses
• Segmentation tests
• Consistency tests
• How to Test Hypotheses
• Structural relationship between leadership and
  performance attributes (quantal response)
• Multivariate modeling vs. nonparametric tests
• How to Validate Findings Sensitivity Analysis
• Robustness of results to specification
• Sensitivity to segmentation of hospitals and
  respondent type

13
Hospital Recruitment for Field Testing Phase

• Hospital recruiters will be provided with
  material outlining the project.
• Hospitals will be targeted for recruitment from
  both high and non-high performers as identified
  by CMS measures portfolio. (time frame TBD)
• Recruitment sampling will use multivariate
  matching based on hospital characteristics
• Bed size
• Rural or Urban
• Teaching status
• Region
• And other demographic considerations allowing for
  matched pairs

14
Data Analysis Team

• Steve Mayfield, AHA, Chair
• Tom Vaughn, Sam Levey, University of Iowa
• Eugene Kroch, Penn-LDI CareScience
• Andrea Silvey, Health Services Advisory Group
• Dale Bratzler, OK Foundation for Medical Quality
• Shannon Archer, OK Foundation for Medical Quality
  
• Chris Hatcher, Dotcomments
• Mark Koepke, CMS
• Others, TBD

15
Data Analysis Plan Overview

• Participating study hospitals will take the
  Hospital-LQAT assessment to establish a point of
  measurement for each question.
• Study hospitals will have their assessment data
  correlated with CMS measures portfolio.
• Data will be analyzed to identify hospital
  structures, processes, and characteristics
  strongly associated with high performers.
• Hospital-LQAT will be modified to reflect
  findings that support the elements of the tool
  that are correlated with high performance in
  clinical quality.

16
Using the Hospital-LQAT to Affect Practice.
17
As a logical next step to the development of the
Hospital-LQAT, we propose a New Collaborative to
work with hospitals seeking clinical improvement.

• The new collaborative will take a deeper look
  into the cultures of both higher and lower
  performing hospitals and will develop evidence
  based practices that will help those hospitals
  seeking improvement.

18
Goals for the New Collaborative

• Identify how specific leadership characteristics,
  structures, processes, and activities at the
  hospital level correlate with performance on
  clinical measurements of quality.
• Identify and document specific leadership
  attributes that are associated with high
  performance. 
• Develop, implement and test effectiveness of
  tailored intervention initiatives that will
  assist hospitals in addressing deficiencies
  identified by the Hospital-LQAT.

19
Hospitals have a great opportunity to use the
Hospital- LQAT in an applied setting in their own
community and environment.  The organizational
constructs provide a structured framework to
support meaningful leadership conversations among
management, governance and physician leadership. 
Any hospital may use the instrument to gauge its
readiness and to chart its progress over time.

• Stephen Mayfield, MBA, MBBSenior Vice President
  for Quality and Performance ImprovementThe
  American Hospital Association

20
Affecting practice with Hospital-LQAT

• Once validated and available for public use,
  hospitals will be able to use the Hospital-LQAT
  to identify leadership strengths as well as any
  gap that may serve to inhibit desired change or
  success.

21
Steps of the New Collaborative 24 Month Plan

• Step I Assessment and Toolkit Development
  
• Step II Consultation and Evaluation

22
Step I Assessment and Toolkit Development

• Identify and document evidence-based practices
  of effective leadership and barriers to
  performance improvement from hospitals at various
  levels of performance on the CMS Quality Measures
  and the Hospital-LQAT.

23
Step I Assessment and Toolkit Development

• In Depth Assessments
• Analyze Data
• Document Evidence-Based Practices

24
Step II Consultation and Evaluation

• Support hospitals seeking to improve their
  clinical quality based on their Hospital-LQAT
  self-assessment.

25
Step II Consultation and Evaluation

• Identify Hospitals
• In Depth Assessments
• Determine Appropriate Interventions
• Implement Interventions
• Evaluate Success

26
New Collaborative SummaryTotal Time 24 Months
Total Budget TBD
Consulting Resources
Evaluate Success
Step II 18 Months
Step I 6 Months
27
Assignments and Next Steps

• Field Testing/Recruitment Teleconference
• Data Analysis Team
• Review draft documents on project at
  http//www.ofmq.com/hospital-leadership-collaborat
  ive
• Next meeting of Hospital-LQAT Advisory Group
  October 17 _at_ 300pm EST