Toolkits

1
Toolkits
Developing a Clinician Resource for
Evidence-Based Treatment Delivery
Deni Carise, Ph.D. Tom McLellan, Ph.D. Adam
Brooks, Ph.D. Robert Forman, Ph.D
Supported by NIDA Grant (R21DA-015977)
2
The Research-Practice Gap

• 25 years of heavy research investment in
• Medication Development
• Behavioral Therapies
• Numerous treatments have
• Demonstrated efficacy in clinical trials
• Demonstrated effectiveness in dissemination
  studies

3
The Research-Practice Gap

• Research shows very low rates of adoption of
  these evidence-based practices
• Possible factors leading to non-adoption
• Financial
• Complexity of interventions
• Provider organizational and policy concerns

4
The Research-Practice Gap

• We interviewed counselors and program directors
  in the Delaware Valley Research Practice
  Collaborative (PA, DE, NJ)
• Initial Focus Groups revealed two barriers as
  greatest contributor to the gap
• Finance
• Training/Supervision

5
The Research-Practice Gap

• New therapies wont be adopted if
• the treatments cannot be supported by managers
  and funding agencies
• the training and supervision burdens of the
  treatments are overwhelming

6
The Financial Barrier

• It is a challenge to implement evidence-based
  treatment practices within the severely
  challenged infrastructure
• Addiction treatment system has experienced a
  20-year period of declining funds

McLellan, Carise and Kleber, 2003. Mark,
Levit, et. al. 2007.
7
The Financial Barrier

• General healthcare funding declined 12 between
  1988 and 1998
• During the same period, addiction treatment
  funding declined 75

Galanter, Keller et. al. 2000.
8
Financial Barrier Consequences

• Program closures or re-organizations
• Increased counselor and management turn-over
• In a survey of 450 treatment programs
• 32 faced the threat of closure
• 5 actually closed during the year of the survey

Gallon, Gabriel et. a. 2003 McLellan, Carise et
al. 2003. Knudsen, Ducharme et. al. 2004.
Roman, Blum 1997.
9
Financial Barrier Outcomes

• Increased Reliance on Group Therapy
• Reimbursement for individual therapy is
  under-funded or not funded
• Offered as primary treatment modality

10
Group Psychotherapy

• Can be as effective as individual therapy
• Are a top priority for bridging the gap
• Evidence-based approaches have often not been
  adapted for group treatment

Weiss, Jaffee et al. 2004.
11
Supervision Barrier

• Research shows that clinical supervisors
• spend less time mentoring and training
• spend more time addressing human resource or
  regulatory compliance issues
• often have ongoing case loads of their own

12
Training Barrier

• Of 400 national substance abuse treatment
  centers, 20 had no staff training budget

Decreased funding Increased licensing,
accrediting, and funding compliance requirements
Decreased clinical training and Increased
attention to administrative issues
?
Johnson 2000.
13
Training Demands of EBPs

• Research on Training EBPs shows
• Manuals (if read) are useful for learning
  interventions, but not enough to change counselor
  practice
• To be proficient in conducting new
  interventions, training needs to be ongoing
• Requires more time than a standard 2-3 day
  workshop
• Requires ongoing, expert supervision

McCarty, Fuller et al. 2007
14
The Big Problem

• We have complex treatments
• Content heavy
• Require intensive training
• Proven efficacy when delivered correctly
• We have a resource starved environment
• We have heavy turn-over in the field

15
Looking to Other Fields

• The field of education has experienced similar
  problems
• Minimal resources
• Stressed workforce
• Complicated interventions (lessons) with little
  time to prepare

16
Possible Solution

• Education field Toolkits -
• An evidence-based approach to addressing the
  challenge of upgrading skills in a resource
  restricted environment

17
Possible Solution

• Teachers use packaged lesson plans to assist
  teachers in conveying complex concepts
• They can be taught effectively, require little
  supervision and have been constructed, tested and
  refined through a scientific process

18
Possible Solution Examples

19
Possible Solution Examples
20
Possible Solution Examples
21
Existing Treatment Curricula

• Currently, there are some existing prepackaged
  curricula for substance treatment
• Vary to the degree that they are informed by
  evidence based content
• Few, if any, have been rigorously tested to
  assess their impact on treatment or outcomes

22
Testing a Curriculum Sample

• We conducted significant background work to
  determine provider interest in a curriculum-based
  approach
• We developed and pilot-tested a single-session
  curriculum Toolkit

23
Surveying Interest

• 21-item survey designed for the Clinical Trials
  Network projects called What Do You Need?
• Administered in 18 community-based treatment
  programs in the Delaware Valley Practice Research
  Collaborative (PA, NJ, DE)

24
Surveying Interest

• 269 treatment program staff completed the survey
• 195 employees identified themselves as treatment
  providers
• Other options researcher, managed care or
  faith-based organizations, support staff, or
  consumers were not included in the analyses (n74)

25
Surveying Interest

• Self-report survey included
• Demographic questions Age, gender, ethnicity,
  education, certification, years in the field
• Forced choice of preferred Training Topics (n26)

26
Sample Demographics

• Overall demographics
• Age (Mean42 SD11)
• Years of Experience (Mean9 SD7)
• Gender Female 57
• Ethnicity Caucasian 71, African American
  22, Hispanic/Latino 3, Other 4

27
Analyses 1

• Which Topics were most frequently endorsed?
• Was endorsement related to counselor background?
• Certification was defined as having any of the
  following certifications
• Certified Drug/Alcohol Counselor (PCACB, PCB,
  etc)
• ASAM Certification or APA Qualified in Addiction
  Medicine/Psychiatry
• Licensed Health Professional
• Ordained Clergy
• Experience was defined as having been in the
  field for at least 6 years (Range 0-35 yrs)

28
Analyses 2

• Years of Experience and Certification were
  examined, creating 4 groups
• Note Participants without both Experience and
  Certification data were omitted from analysis
  (n25 14)

Uncertified Certified
Inexperienced (n34, 17) (n51, 26)
Experienced (n61, 31) (n24, 12)
29
Results 1
Percent Endorsed Training Topic 62
-Increasing Client Motivation 17 -Obtaining
Project Funding 49 -Relapse Prevention
Techniques 17 -Finding Treatment Information
47 -Co-Occurring Disorders 15 -Harm Reduction
Strategies 47 -Group Therapy Techniques 15
-Applying Research in Practice 45 -Spirituality
Recovery 14 -Age Specific Treatment 27
-12-Step Oriented Approaches 14 -Sexuality
Related Treatment 25 -Ethnicity Diversity
12 -Documentation Software 25 -Biological
Basis of Addiction 11 -Tobacco Addiction 21
-Addictions Medications 11 -Use of Incentives
Rewards 19 -Sexual Addiction 11 -Internet
Addiction Resources 18 -Gender Specific
Treatment 9 -Research Evaluation Methods 18
-Alternative Therapies 5 -Using Research
Databases 4 -Statistics Research
Findings
Most frequently endorsed Research-related Topics
analyzed
30
Results 2

• Training Topic Preferences by Experience and
  Certification
• Note 1 Main Effect (ME) for Experience, 2 ME
  for Certification, 3 Interaction, at p lt .05
• Note Participants can choose more than one
  category
• Binary Logistic Regression

Inexperienced Uncertified (n34) Inexperienced Certified (n51) Experienced Uncertified (n61) Experienced Certified (n24)
Increasing Client Motivation 68 65 57 58
Relapse Prevention 56 59 36 50
Co-Occurring Disorders 1 44 31 56 58
Group Therapy Techniques 3 59 49 33 58
31
Results 3

• Training Topic Preferences by Experience and
  Certification
• Note 1 Main Effect (ME) for Experience, 2 ME
  for Certification, 3 Interaction, at p lt .05
• Note Participants can choose more than one
  category
• Binary Logistic Regression

Inexperienced Uncertified (n34) Inexperienced Certified (n51) Experienced Uncertified (n61) Experienced Certified (n24)
Spirituality Recovery 32 49 48 50
12-Step Oriented Approaches 1 27 39 20 17
Addiction Medications 1,2 35 18 23 4
Applying Research in Clinical Practice 21 10 13 25
32
Results 4

• Out of the 26 possible Training Topics, the
  majority of counselors chose the same 5, making
  up 45 of the responses
• These top 5 topics were most related to the
  treatment counselors were already providing

33
Survey Staff Beliefs about Addiction Treatment
and Clinical Trials

• NIDA Clinical Trials Network (CTN) DE Valley
• 22-items assessing 317 counselors
• Beliefs about addiction treatment
• Willingness to try new treatment approaches
• Endorsed by 80 of surveyed counselors
• Willingness to try research-based innovations
• Endorsed by 82 of surveyed counselors

34
Survey Videotape Use in Addiction Treatment
Settings

• 129 counselors were asked about their clinics
  videotape usage for group sessions
• 100 used videotapes with patients
• 97 agreed videotapes were a useful aid in
  educating patients about addiction and recovery

35
Survey Videotape Use in Addiction Treatment
Settings

• 96 wanted a library of up-to-date,
  scientifically accurate videotapes...
• 63 agreed only showing brief sections of
  videotapes with a discussion following them was
  best
• 58 were dissatisfied with their clinics
  current education tools

36
Toolkit Development

• We incorporated provider feedback into the
  development of a single Toolkit
• Goal was to
• assess provider satisfaction
• assess client satisfaction
• assess continued provider use of the Toolkit

37
Developing the Toolkit
Step 1 Choosing Content Examples Relapse
prevention HIV risk reduction Biology of
Addiction Medications Motivational
Interviewing Toolkit Prototype MI Component -
Decisional Balance
38
Developing the Toolkit

• Step 2 Designing the Toolkit
• Multiple formats (Digital Hand-Outs)
• Created Toolkit prototype

Components
Toolbox Clinician Guide DVD Worksheet
Wallet Cards
39
Toolkit Components
Worksheet
Clinicians Guide
Wallet Card
DVD
40
Toolkit Video
Good Things and Not So Good Things about Recovery
41
Methods - Counselors
Counselors - 26 Counselors from 6 clinics -
Experienced Counselors (gt 10 years) Procedures
- Counselors consented oriented - Use the
Toolkit in Group Complete Clinician survey -
One month later - Asked to use 2nd time - Next 3
months Record use of Toolkit components
42
Methods - Patients
Group Members/Patients N230 Procedures -
Consented prior to group - Attended Toolkit
group - Completed Patient Satisfaction
Survey
43
Research Questions

• Will clinicians be able to use the Toolkit after
  only a brief orientation?
• Will clinicians continue to use it, even when not
  required?
• How will patients evaluate the toolkit group?

44
Will Clinicians Be Able To Use It After Only a
Brief Orientation?
  Mean Mean Mean (SD)
Was the Toolkit orientation adequate? 4.5 4.5 4.5 0.8
Were you satisfied with the Toolkit group? 4.2 4.2 4.2 0.7
Was the Toolkit "User-Friendly?" 4.4 4.4 4.4 0.7
Do you think other Clinicians would like to use the Toolkit? 4.4 4.4 4.4 0.7
         
Scale Scale Scale    
1Not at all, 2Somewhat, 3Moderately, 4Quite a Bit, 5Very Much 1Not at all, 2Somewhat, 3Moderately, 4Quite a Bit, 5Very Much 1Not at all, 2Somewhat, 3Moderately, 4Quite a Bit, 5Very Much    
N26 (counselors)
45
Counselor Ratings
46
Patient Satisfaction
47
Will Clinicians Continue To Use It After the
Study Ends?

• 96 of counselors have reused at least one
  component of the Toolkit
• On at least one occasion, 63 of clinicians used
  all 5 components of the Toolkit as directed
• The most frequently and broadly used were the
  core components
• Counselor Guide Patient Worksheet

48
Will Clinicians Continue To Use It After the
Study Ends?

• Over a 3-month tracking period
• Counselor Guide 83 used it again (M 2.5
  times)
• Patient Worksheet 96 used it again (M 3.2
  times)

49
Times Used Toolkit Components
Toolkit component Clip- board DVD Work- sheet Guide Wallet Cards
Mean (sd) 5 (9) 1 (1) 3 (3) 3 (3) 2 (2)
Range 1 - 42 1 - 3 1 - 12 1 - 12 1 - 9
50
Conclusions

• Preliminary results in community-based settings
  support the Toolkits
• feasibility
• acceptability
• sustainability

51
Future Directions

• Will curriculum impact clinician behavior in
  treatment?
• Will curriculum beneficially impact client
  outcomes?

52
Translating a Complete Treatment

• We will translate a full treatment intervention
  into a Toolkit format
• Starting with 6 Toolkits on Relapse Prevention
  Techniques
• Content and didactic heavy
• Popularly selected by providers
• Evidence Based
• Useful in group treatment

53
Testing a Complete Curriculum

• In a larger sample, we will randomly assign
  sites to receive
• 6 training manuals
• OR
• 6 training manuals
• 6 Toolkits Curriculum Supplements

54
Future Directions, Contd

• Toolkits for other evidence-based treatments
• (12-step, etc)
• Toolkits to address comorbidity, HIV prevention,
  etc
• Arming clinicians with enough evidence-based
  content to fill 50-75 of group sessions

55
Future Directions

• Your Thoughts?