Los Angeles County Evaluation System:

1

Site Report Presentation to Contract Program
Auditors

• Los Angeles County Evaluation System
• An Outcomes Reporting Program
• 2008

2
Purpose of this Presentation

• To provide a brief overview of the Los Angeles
  County Evaluation System (LACES).
• To review the content and purpose of the Site
  Reports.
• Review those content areas that are the focus of
  the evaluation.

3
Overview of LACES

• LACES is a partnership between ADPA, Treatment
  Providers, and UCLA/ISAP to conduct an evaluation
  - LACES is not an external study.
• Treatment programs are contractually required to
  comply with the evaluation efforts.
• Evaluate the alcohol and other drug treatment
  system.
• On-going evaluation, not a temporary study.

4
First Some Questions

• How many programs are your responsible for
  monitoring?
• How often do you visit the programs?

5
Whats a Site Report?

• Have you heard of a site report?
• Have you seen a site report?
• Have you had the opportunity to compare the
  information in the site reports with ADAM?

6
A Site Report Is

• Brief six pages, double-sided.
• Information is gathered from the admission
  discharge LACPRS.
• Information is included on your site and for all
  sites of the same program type.

7
What is the Purpose of the Site Report?

• To provide information concerning the key
  performance/outcome areas.
• To provide feedback to sites concerning how they
  are performing with regards to the focus areas of
  the evaluation.
• To provide feedback on how other similar programs
  are performing.

8
Contents of the Site Report

• Executive Summary
• Includes overall participant admission and
  discharge numbers.
• Includes information on key outcomes.
• This will be the focus of this presentation as it
  includes the outcome information that will be
  used to evaluate programs.
• Full Report
• Includes detailed information on all
  participants.
• Includes more details on changes in key outcomes.

9
Executive Summary

• Single page, stand alone document.
• Covers the admission to discharge changes for
  your site compared to all sites of the same
  program type.
• Key Areas include
• Participant Flow chart.
• Performance Measures.

10
Important Performance Outcome Measures

• Completion of treatment/Positive Compliance.
• Abstinence from/Reduction in drug/alcohol use.
• Exit Interviews
• Length of Stay/Enrollment in Treatment
• (2 benchmarks)
• 30 Days Engagement.
• 90 Days Retention.
• All important aspects of the treatment plan.

11
Why Completion of Treatment?

• Most outside of this field are interested in
  completion figures.
• Positive Compliance
• Completed treatment.
• Left treatment prior to completion with
  satisfactory progress.
• Others have adopted this mindset, but many have
  not.

12
Why Reduction/Abstinence?

• Silly picture, for a silly question.
• That is our business.
• If we cannot demonstrate abstinence or reductions
  in use, funders will move the money to those who
  can.

13
Interviews

• Responses to all discharge LACPRS questions.
• One of the most important of the evaluation foci.
• Information from both admission and discharge is
  required to measure treatment effectiveness.
• No exit interviews No measurement of changes
  that occurred during treatment.

14
Why Engagement Retention?

• First 30 days are critical to the outcome of
  treatment.
• Research shows that without an adequate amount of
  time in treatment, few improvements are found.
• Funding organizations are beginning to take note
  of engagement and retention rates.

15
Note on Documentation

• Accuracy is extremely important.
• Length of stay/enrollment.
• Example length of stay/enrollment.
• In 2004-2005, average length of stay was 105
  days.
• 40 were in treatment for 30 days.
• If providers are not inputting discharge date
  accurately what other information is being
  entered into LACPRS inaccurately?

16
Max LOS/LOE by Funding Source

• All funding sources support 30- and 90-day length
  of stay/enrollment benchmarks.

FUNDING Outpatient LOE Residential LOS Day Care LOE
CalWORKS 365 Days 365 Days 365 Days
Drug Court 730 Days 90 Days N/A
Female Offender N/A 180 Days N/A
General Relief 180 Days 180 Days 180 Days
PPN 180 Days 180 Days N/A
Prop 36 300 Days 90 Days 204 Days
Drug Medi-Cal No max length - approval is need for 180 days. No max length - approval is need for 180 days. No max length - approval is need for 180 days.
17
Average Provider Performance

• During the last 3 fiscal years, average
  performance in these areas was
• Does this make sense to you?

PERFORMANCE AREA Outpatient Residential Day Care

Reduction in Primary Drug Use 75.7 97.7 77.9
30-Day Retention 71.8 64.1 68.9
90-Day Retention 48 38.1 40.6
Exit Interviews 48.6 73.7 47
18
Roles of the Contract Monitor

• Answer questions regarding site reports.
• Review reports for accurate information.
• Provide suggestions on how to improve outcomes
  to come.
• Seek assistance from ADPA or UCLA for responses
  to questions, concerns, or additional assistance.

19
How to Improve Exit Interview Rates

• Some ways to increase rates of exit interviews
  include
• Informing the participant of the exit interview
  when admitted.
• Include completion of the exit interview as part
  of the individualized treatment plan.
• Remind the participant when they are nearing
  completion of their plan of the exit interview.

20
Improve Engagement Retention?

• Engage them from first contact
• Make same day appointments if possible.
• Appointment cards
• To encourage forgetful clients.
• Phone calls
• To remind clients of appointments.
• To follow-up with clients who may have dropped
  out.

21
Other Suggestions?
22
Challenges

• LACPRS
• Provides a lot of good information.
• Revisions and monitoring of information will be
  needed to ensure quality validity.
• Site Reports
• Also requires revisions to format content.
• Not getting past Executive Directors eyes.

23
Your Views on the Site Reports

• What do you think about the site reports?
• What have you heard from your providers?
• Do you have suggestions or recommended changed to
  the reports?

24
Future of the Site Reports

• The Site Reports will be used to asses individual
  sites in the key areas described in this
  presentation.
• Over time peer performance standards will be
  developed.
• Programs that fall short of the standards will be
  offered information, training and other forms of
  assistance in order to bring their sites
  performance up to the standard.

25
The End