Kathryn L' Braun, DrPH

1
Ensuring participant retention fidelity
assessment in CDSMP

• Kathryn L. Braun, DrPH
• Michiyo Tomioka, MS, PhD (c)
• Noemi Pendleton, MBA

2
CDSMP and EnhanceFitness
Chronic Disease Self-Management Program (Stanford)
3
Hawaiis CDSMP participation
Hawaii N199 All other states N 2416
4
Strategies to overcome program participation
barriers

• Before
• Partners strategize how to meet needs of
  community and fulfill grant and CDSMP agreements.
• Flyers, brochures and presentation script were
  created.
• Master Trainers made presentations to existing
  groups (senior centers, churches, health groups).
  
• Shared resources by partners (location,
  transportation, networking).
• During
• Healthy snacks provided at each session.
• As cycle continued, participants begin to bring
  healthy snacks.
• At the end of workshop, certificates of
  Completion healthy pot luck.
• After
• Reunion is conducted to share stories and action
  plan.

5
6-step protocol to assure fidelity

• Deconstruct EB program using tracking-changes
  tool.
• Prepare step-by-step plan for replication.
• Review plan with the parent program.
• Coordinate excellent training to EB program
  deliverers.
• Monitor delivery, using checklists, and take
  corrective action.
• Review checklists and data collection forms to
  identify areas for improvement.

6
Step 5. Monitor delivery and recommend corrective
action.

• Use Leader Evaluation Form, measuring 10 items
  each on a 4-point scale.
• (1 poor to 4excellent)
• Originally, we measured two ways
• External observation
• Self-evaluation
• Later added Peer-eval

7
CDSMP Fidelity monitoring

• CDSMP

• 10 Items
• Arrived on-time prepared
• Followed manual
• Modeled session activities
• Worked as partner
• Used brainstorming
• Used problem-solving
• Encouraged participation
• Modeled action planning
• Used positive reinforcement
• Handled problem people

Mean of 10 items, scored from 1poor to 4
excellent.
External eval 222 times Self eval 248 times
8
Reviewing individual fidelity scores
Mean of each item, scored from 1poor to 4
excellent.
9
Using data for improvement

• External, peer, and self evals shared after each
  session
• Those scoring lt 3 on any item
• Asked to retake CDSMP training and repeat
  fidelity monitoring protocol
• Team of Certified MT may be appointed to conduct
  remedial/review training
• Poor leaders discussed at monthly partner
  meeting and could be asked to stop training

10
Sustainability issues were raised

• Limited number of evaluators
• High cost of travel especially for neighbor
  islands
• The requirement was too high
• Potential risks to burden evaluators
• External observation Every session for 1st
  cycle
• All CDSMP partners gathered to discuss what would
  be the best approach
• Administer Peer evaluation (instead of evaluator
  goes to every session)

11
Revised Fidelity Protocol
12
Continue discussion about fidelity monitoring

• Unresolved issues
• How are we going to treat people who do not want
  to stick with the protocol?
• How are we going to approach the people who are
  not under our grant?
• How long are we going to continue fidelity
  monitoring?

13
Step 6. Review data to identify areas for
improvement and follow-up
Monthly Steering Committee and Evaluation
Sub-Committee meetings
14
Compare Local and National Findings
Communication with physician (0-5, 5 best)
Self-rated Health (1-5, 1 best)
No. of hospital stays
Compared study Sobel, D. S., Lorig, K. R.,
Hobbs, M. (2002). The Permanente Journal, 6, 2,
15-22
Hawaii N 156 Sobels study N 561
15
Conclusions

• Recruitment and retention strategies need to be
  built into programming.
• Maintaining high fidelity leads to good results.
• Both can be done with proper resources and
  support.