Implementing EvidenceBased Practices in PubliclyFunded Clinics

1
Implementing Innovative Mental Health Practices
4 Models from the Innovation Diffusion and
Adoption Research Project (IDARP)
Phyllis C. Panzano
Dee Roth Decision
Support Services, Inc. Ohio
Department of Mental Health
The Ohio State University
Program Evaluation Research
Florida Mental Health Institute University of
South Florida Implementation Research Series May
22, 2008
2
IDARP Background

• Impetus Research to practice gap
• Purpose Support ODMHs Quality Agenda by
  studying CCOEs success in promoting EBP
  adoption
• Rationale Systematic data is needed to
  understand outcomes achieved and to improve upon
  this complex, statewide undertaking

3
Concerns Driving EBP Selection/CCOE Formation
Evidence Base
Salience
4
8 CCOEs

• IDDT

• MST

• Family Psychoeducation

• Cluster-Based Planning

Evidence Base

• OMAP

• MH/Criminal Justice

• MH/Schools

• Advance Directives

Salience
5
Coordinating Centers of Excellence (CCOE)

• University or local partnership
• One Best Practice per CCOE
• Statewide service area

6
Role of CCOEs

• Promotion of Best Practices
• Education Training
• Capacity Development
• Fidelity measurement
• Cross-system sharing/ TA around problem solving

7
Research Question 1
What variables explain the adoption of
evidence-based and other innovative practices by
a mental health provider organization?
8
Research Question 2
What variables explain the successful
implementation of evidence-based and other
innovative practices by a mental health provider
organization?
9
IDARP
CLIFFNotes
10
Four Practices Studied

• Selection criteria
• Expected variability on explanatory variables
• Maximization of generalizability
• 2 Team delivery models
• IDDT
• MST
• 2 Individual delivery models
• OMAP
• Cluster based planning

11
Design, Methods, Sample

• Design
• Longitudinal
• Multi-level model
• Methods
• Interview
• Survey
• Archival
• Sample
• 90 projects
• Multiple key informants
• Analyses
• Project level

12
Four IDARP Models

• 1 Adoption Decision (RQ1)
• 3 Implementation Success (RQ2)

13
Do the data support the 4 models?
14
Model 1 Adoption Decision is a Decision Made
under Risk

• Panzano Roth, 2006
  Seffrin, Panzano Roth (in review)

15
A Decision Under Risk
Perceived Risk of Adopting
ANTECEDENTS
-
DECISION STAGE

• IMPLEMENTING
• RECENTLY DECIDED
• TO IMPLEMENT
• STILL CONSIDERING
• NEVER WILL ADOPT

Capacity to Manage or Absorb Risk

Risk-taking Propensity
16
A Decision Under Risk
Perceived Risk of Adopting
ANTECEDENTS
---
DECISION STAGE

• IMPLEMENTING
• RECENTLY DECIDED
• TO IMPLEMENT
• STILL CONSIDERING
• NEVER WILL ADOPT

Capacity to Manage or Absorb Risk

Risk-taking Propensity
17
Some Antecedents to Risk Perceptions
Perceived Risk of Adopting
Scientific evidence
- .31
Field-based evidence
- .42
. 61
Dedicated resources
Capacity to Manage Risk
Ease of use
. 45
In-house expertise
. 54
Risk Taking Propensity
Learning culture
.56
Mgr. Attitude re change
.32
18
More about the 3 core risk constructs.
Capacity To Manage Risk
Perceived Risk
DECISION STAGE
Past Propensity to Take Risks
19
Defining Implementation Success
20
Success Defined in Terms of Two Classes of
Outcomes
Measures of Implementation effectiveness
Accurate, consistent and committed use of
practice by targeted employees (e.g.,
fidelity)
21
Two Classes of Outcomes

• Measures of Innovation effectiveness
  Benefits that accrue to an organization
  and its stakeholders as a result of implementing
  a practice (positive consequences for clients,
  practitioner satisfaction, etc.)

22
Expected Link Between Two
Classes of Outcomes
INNOVATION EFFECTIVENESS
IMPLEMENTATION EFFECTIVENESS
23
Success Measures Linked
Implementation Effectiveness
Innovation Effectiveness
.64
T2 Fidelity
Positive Outcomes
Positive Outcomes
T2/3 Fidelity
.54
24
Models 2 4 Understanding Implementation
Success
Panzano, Roth, Crane-Ross et al, 2005 Massatti,
Sweeney, Panzano Roth, 2008 Vaidyanathan, 2005.
25
Model 2 Multi-level factors influence
implementation success
26
Model 2
Level 5 Environment
Level 4 Inter-organizational (dyad)
Level 3 Organization
Level 2 Project
Level 1 Innovation

• Dependent Variables
• Implementation effectiveness
• Innovation effectiveness

27
Some Observed Links With Positive Outcomes at
Time 2
Level
Example
28
Model 3 Cross Phase effects found on
implementation success
29
Model 3 Cross-phase effects
Decision
Implementation
Outcomes
Initiation
Time
30
Model 3 Factors from earlier stages impact
success
Decision
Success
INITIATION
IMPLEMENTATION
Time 2
Time 1
Time 1
Time
31
Initiation-Phase Effects
SUCCESS
Expected Benefits Relative Advantage
Results Demonstrability Trust in CCOE
(purveyor)


Assimilation scale Global positive outcome
scale
32
Decision-Phase Effects
SUCCESS
Objective decision Information access
Internal influence Commitment



Assimilation scale Global positive outcome
scale
33
Model 4 Climate for Implementation and Success
Massatti, Sweeney, Panzano et al, 2008. Panzano,
Roth, Crane-Ross, et al. 2005) Vaidyanathan, 2005
34
Components of Climate for Implementing the
Practice

• Top management support
• Access to training TA
• Freedom to express doubts
• Goal clarity
• Rewards/recognition for implementing
• Removal of obstacles
• Performance monitoring
• Dedicated resources

35
CLIMATE AND SUCCESS
IMPLEMENTATION EFFECTIVENESS

Climate for Implementation


NS
INNOVATION EFFECTIVENESS

based on 1st and 2nd round data only
36
Four Take-Home Messages
37
1 The adoption decision is made in
consideration of risk

• Perceived risk
• Capacity to manage risk

38
2 Its important to pay attention to factors
operating at multiple levels

• LEVELS
• Environment
• CCOE Organization Dyad (IOR)
• Organization
• Project
• EBP

39
and to consider two classes of success measures

• Implementation Effectiveness
• Innovation Effectiveness

40
3 Be aware that aspects of earlier phases of
the process can have enduring impacts on success

• Initiation/Exploration phase
• Decision phase

41
4 The climate for implementing a particular
practice is important to success
42
Sustainer Model
Fit
Climate
Assimilation
f (implementation effectiveness ?
innovation effectiveness -? commitment)
43
Next Steps

• Submit Model 2, 3 and 4 manuscripts for
  publication (based on complete dataset)
• Test Sustainer Model Fit Climate ?
• Implementation effectiveness ?
• Innovation effectiveness ?
• Assimilation plans to persist
• Continue to work with ODMH and CCOEs to use
  findings to shape policy and action

44

• Thank You.