Conducting a PIP: The Performance Improvement Project Manual

1
Conducting a PIP The Performance Improvement
Project Manual

• Dow A. Wieman, Ph.D.
• The Evaluation Center _at_HSRI
• Robert Egnew, M.S.W., M.P.H
• National Association of County Behavioral Health
  Directors
• Ed Diksa, Sc.D.
• California Institute for Mental Health

2

• The Evaluation Center _at_HSRI
• Human Services Research Institute
• Cambridge, MA
• A National Technical Assistance Center for
• Evaluation of Mental Health Systems
• Funded by the Center for Mental Health
  Services,Substance Abuse and Mental Health
  Services Administration

3
Conducting a PIP The Performance Improvement
Project Manual

• Judy Ashley Sheila Baler
• Eydie Dominguez Fred Hawley
• Darcy Johnson Gail Kinnamon
• Diane Koditek John Lessley
• Beth Martinez Rita McCabe-Hax
• Kenneth Meibert Timothy Mullins
• Sandra Naylor Goodwin Madelyn Schlaepfer

4
Objectives

• Gain familiarity with the PIP manual
• Understand basic principles of quality
  improvement
• Gain practical (hands-on) understanding of PIP
  concepts and methodology
• Supplements, not supplants, DMH contract and EQRO
  protocol, guidance, training, and oversight

5
Basic Principle I

• Experienced managers, clinicians, consumers and
  others in the field usually know instinctively
  when behavioral health care is of the highest
  quality and, alternatively, when there are gaps
  between what is possible and what is currently
  provided. This Manual describes methods for
  converting this intuitive knowledge into
  effective action by means of Performance
  Improvement Projects.

6
Basic Principle II Quality Improvement

• Addresses gaps between what is possible and what
  is currently provided
• Not exhortation, blaming and trying harder.
• Data-driven and cyclical

7
Basic Principle IIIError

• Error as defects (Quality gaps)
• Measurement error

8
The PIP Process

• How do I
• Plan for a PIP?
• Develop the data collection plan?
• Decide whether and how to sample?
• Decide whether and how to conduct a survey?
• Develop the data management and analysis plan?
• Develop the improvement strategy?
• Implement the improvement intervention?
• Report the results of the PIP?
• Sustain the improvement?
• Evaluate the PIP?

9
How do I identify study topics?

• A study topic is an area of concern.
• May be administrative or clinical (satisfaction
  and process or mh and functional status)
• May be one required by DMH contract (capacity,
  access, satisfaction, service system, continuity
  of care, provider relations)
• Continuous data collection.

10
How do I identify study topics? (cont.)

• Continuous collection of data
• Socio-demographic characteristics
• Utilization by subpopulations
• High-volume, high-risk services
• Known problem areas
• Sentinel events (death, serious injury)

11
How do I identify study topics? (cont.)

• Other sources
• Previous initiatives
• Consumers/families
• Providers
• Compliance
• UM data
• Conferences and literature
• Other organizations

12
IOM Quality Chasm 6 Aims

• Safe Freedom from accident or injury.
• Effective Disciplined use of systematically
  acquired evidence
• Patient-centered Health care that establishes a
  partnership among providers, patients, and their
  families
• Timely Individuals are able to obtain needed
  care and delays in obtaining care are minimized.
• Efficient Continual reduction of waste in health
  care.
• Equitable The health care system should work to
  improve the health status of all, reduce health
  disparities among subgroups.

13
How do I prioritize study topics?

• Relevance to population
• Prevalence, volume or need
• Extent of risk
• Meaningful difference
• Improvement potential

14
Practical considerations

• Ease of implementation
• Burden
• Acceptability
• Available data
• Available standards

15
Ethical considerations

• Requirements less stringent than research
• BUT
• Some recommend higher standards (risk)

16
How do I define the study question?

• Clear Write it out
• Simple Keep digging (RCA)
• Specific Population, setting, problem, outcomes
• Answerable Available and usable data

17
How do I choose study indicators?

• Defined, measurable variables which reflect a
  discrete event or a status, used to measure
  performance.
• (EQRO Protocol)

18
How do I choose study indicators?

• Often a rate (quality gap)
• Sources Practice guidelines, EBPs (fidelity),
  established process measures (www.cqaimh.org),
  benchmarks
• Also Controllable outcomes

19
What makes a good indicator?

• Objective (unbiased)
• Clear and unambiguous
• Current
• Reliable
• Valid
• Actionable

20
How do I establish indicator criteria?

• The set of rules (measure specifications)
  describing how the indicator is to be applied.
• Study population (diagnosis, enrollment, etc.)
• Service definition (criteria for receiving)
• Units of measure
• Data collection methods
• Numerator and denominator

21
II. What should the data collection plan
include?

• What type of data do I need?
• How should the data be collected?
• When should the data be collected?
• Who should collect the data?

22
What type of data do I need?

• Availability and quality of data to
• Address the study question
• Meet indicator criteria
• Ready made is best
• Many sources

23
How should the data be collected?

• Keep it simple
• Keep it easy (data entry)
• Keep it clear (instructions)

24
When should the data be collected?

• Baseline
• Post-intervention
• Follow-up

25
Who should collect the data?

• Type of data
• Chart reviews
• MIS
• Survey
• Available skills

26
Pilot Test

• What type of data do I need?
• How should the data be collected?
• When should the data be collected?
• Who should collect the data?

27
Phase III To Sample Or Not?

• May be less costly, more current
• May be any unit of information
• Must be representative (error)

28
Phase IV To Survey Or Not?

• Required for satisfaction
• Response rates Must be representative (error)
• Should be actionable

29
Phase V What should I include in the data
management/analysis plan?

• What is a data archive?
• What is required for the analysis?
• Simplicity
• Case-mix adjustment
• Standards, means, norms, benchmarks

30
Phase VI How do I develop the improvement
strategy?

• An intervention designed to change behavior at an
  institutional, practitioner or beneficiary level
• (EQRO Protocol)

31
Phase VI How do I develop the improvement
strategy?

• Barrier analysis
• Avoid confounding (KISS)
• Sources for interventions
• Stakeholders
• High performers
• Research studies
• Disease management models
• Promising practices

32
Phase VII How do I implement the improvement
intervention?

• Practical
• Based on RCA, stakeholder input
• Addresses human factors
• Organizational readiness
• Leadership
• Buy-in
• Data packaging and presentation

33
Phase VII How do I implement the improvement
intervention (cont)?

• Pilot test, modify
• Assess requirements for sustainability

34
Phase VIII How do I report results of my PIP?

• What should I include in the Report?
• What should I consider in Report design?
• How should I disseminate the Report?

35
What should I consider in the report?

• Credibility and Utility
• Consider your audience (multiple formats, media)
• Consider templates (e.g. NCQA)

36
What should I consider in the design of the
Report?

• Focus on study question
• Hierarchy of importance (may vary among
  stakeholders)
• Multiple formats, media
• Pilot test

37
What should I consider in disseminating the
Report?

• Develop strategy early
• Consider goals, target accordingly
• Demonstrate Compliance
• Facilitate change
• Sustain improvement
• Achieve buy-in to PIP process

38
IX How do I evaluate the Pip process?

• Dont neglect PIP PIP
• Process (implementation) and Impact Evaluation
• Dont forget satisfaction

39
Excerpt Logic Model from National Goals and
Core Measures of Cancer Care Quality (McGlynn, EA
and J. Malin, 2002)
40
Study Topic

• Racial disparities in the psychopharmacological
  treatment of persons with schizophrenia

41
Racial disparities in the psychopharmacological
treatment of persons with schizophrenia

• Modeled after
• Dickey, B. Normand, S-L. Hermann, RC, Eisen, SV.
  Cortes, DE.
• Cleary, PD, Ware, N. Guideline Recommendations
  for
• Treatment of Schizophrenia The Impact of Managed
  Care.
• Archives of General Psychiatry. 60 (Apr), 2003
• (NOTE The following examples are not intended to
  represent the design or results of this study)

42
Study Question(s)

• How does the appropriateness of
  psychopharmacological treatment compare for
  whites and minorities?
• How do white and minorities compare in rating the
  interpersonal aspects of their pharmacological
  treatment?
• How do whites and minorities compare in
  assessment of outcomes of their pharmacological
  treatment?

43
Study Questions

• How does the appropriateness of
  psychopharmacological treatment compare for
  whites and minorities?
• or
• How does adherence to treatment guidelines
  (Schizophrenia PORT) compare for whites and
  minorities?

44
How does adherence to treatment guidelines
(Schizophrenia PORT) compare for whites and
minorities?

• Relevance to population?
• Prevalence, volume or extent of need?
• Extent of risk?
• Meaningful difference?
• Can PIP achieve improvement?

45
Dosage for Acute Episodes

• How does adherence to PORT guidelines for daily
  medication dosage for acute episodes (300-1000
  CPZ units) compare for whites and minorities?

46
Dosage for Acute Episodes on Inpatient Service

• How does adherence to PORT guidelines for daily
  medication dosage for acute episodes (300-1000
  CPZ units) compare for whites and minorities
  treated on the inpatient service?

47
Dose within PORT Range White vs. Non-White
48
Focus groups

• Barrier analysis What is the problem?
• 1)
• 2)
• 3)
• 4)

49
Possible study questions 1) lack of knowledge

• Will initial and annual training, with reminders,
  for physicians and nurses on PORT guidelines and
  disparities in treatment improve the proportion
  of patients with schizophrenia receiving
  appropriate dose of medication for inpatient
  treatment of acute episode?

50
Possible study questions 2) Lack of time to
assess

• Will additional information on the intake
  screening form, and earlier scheduling of
  psyhcopharm assessment improve the proportion of
  patients with schizophrenia receiving appropriate
  dose of medication for inpatient treatment of
  acute episode?

51
Proceed with PIP

• Decide on indicators PORT Guidelines
• Define indicator criteria
• Denominator age 21-64 with schizophrenia
  eligible for psychopharm admitted to inpatient
  during period, stratified by race.
• Numerator number discharged on dose within PORT
  guidelines stratified by race
• Dose at intake and discharge
• Data Chart reviewrandom sample for baseline,
  again six months later. Data analysis methods.
• Implementation Present data to clinical
  leadership, line staff, develop plan
• Present results, decide on follow-up action, etc.

52
Contact Information

• Dow Wieman
• The Evaluation Center_at_ HSRI
• www.tecathsri.org
• dwieman_at_hsri.org
• 978 283 3245

53
Questions and discussion