Evaluation of Ohios Title IVE Waiver Demonstration Project ProtectOhio

1
Evaluation of Ohios Title IV-E Waiver
Demonstration Project ProtectOhio

• Summary of Findingsfrom the 5-year study
• September 2003
• HSRI, Westat, Chapin Hall, IHSM

2
Green demonstration PCSAYellow comparison
PCSA
Ohios Evaluation Counties
3
LOGIC MODEL
ImprovedOutcomesfor children families
Waiver
SystemsReform
Interventions

• Whats done for what families and children
• How they go through the system
• Services received
• Use of relatives

• Any service
• Any clients
• up front
• Keep savings

• Internal organization
• Services available
• Financing patterns
• MC strategies

? placement days ? permanency ?functioning ? well
being
4
Demonstration Counties Response to the Waiver

• Compared to their own pre-Waiver actions and
  comparison counties concurrent actions,
  demonstration sites made these changes
• Greater targeted focus on prevention
• More attention to outcomes
• Overall greater use of managed care strategies
• More interagency collaboration
• Slower growth in foster care spending
• More growth in other child welfare expenditures

5
Demonstration sites focused more on prevention

• Demonstrations and comparisons had similar levels
  of insufficiency in array of available services
  moderate differences in area of prevention
• Demonstrations targeted new prevention activities
  to areas identified as insufficient
• Demonstration sites expressed stronger commitment
  to prevention

6
(No Transcript)
7
(No Transcript)
8

One demonstration and one comparison counties
had incomplete fiscal data so they could not be
categorized in this table.
9
Demonstration sites gave more attention to
outcomes

• Gathering more outcomes data 10 demo and 6 comp
  reported systematically gathering outcome
  information from staff (Y4)
• Using outcomes data in decision-making (Y3)
• Integrating fiscal program outcomes information
  (Y4)

10
(No Transcript)
11
(No Transcript)
12
Ohios Managed Care Strategies

• Service array (care criteria) new services,
  shifting focus
• Financing (capitation risk) capitated case
  rate contracts, use of IV-E
• Targeting (eligibility) special initiatives,
  specialized units
• Case management unit structure, transfers
• Provider competition provider affiliations, rate
  changes
• Utilization review placement reviews, service
  reviews
• Information management automated MIS, mgrs use
  info
• Quality assurance control enhancement, outcome
  focus

13
Demonstration sites employ more managed care
strategies

• In Year 2, significantly more use of managed care
  strategies by demos
• In Year 4, demos made more use of 7 of 8 managed
  care strategies
• Overall rankings more demos in the high group,
  fewer in the low group

14
(No Transcript)
15
(No Transcript)
16
Counties Grouped by Level of Managed Care
Activity, Year 4
17
Demonstration sites have stronger collaboration

• Overall, PCSA partners say collaboration with
  PCSA has become stronger
• BUT
• Demo and comparison PCSAs have similarly strong
  relationships with juvenile court, mental health
  board

18

Difference is significant at 0.058
19

Difference is significant at 0.000
20
Relationship between PCSA and Main Partners (Year
4)

• PCSA relationship with Juvenile Court
• Majority of demo comp counties have strong
  relationship (9D, 7C)
• Inappropriate referrals continue to be issue in
  some demo and comp counties
• PCSA relationship with Mental Health
• Relationships remain strong in most counties, no
  significant change over Waiver period (9D, 7C)
• Inadequate mental health services for children
  and families, so PCSAs (both demo and comp)
  develop and purchase own mental health services

21
However, demonstration sites are no different
from comparisons in

• Family involvement in case decisions slow
  growth in family role and team conferencing
• Use of managed care contracting 3 demo
• Increased competition in foster care growth in
  foster homes and per diem rates
• Utilization review and QA moderate use of
  formal review processes.

22
Utilization Review (Year 4)

• Placement review processes
• Pre-placement 9D, 8C
• During placement 8D, 5C
• Oversight of non-placement resources
• Pre-service review 4D, 4C
• Subsequent review 2D, 3C

23
Changes in Patterns of Child Welfare Spending
Over Time

• Both groups of counties experienced growth in
  paid placement days and in the average daily cost
  of foster care
• BUT
• Demonstration sites appeared to contain growth in
  foster care spending more than comparison sites
• Demonstration sites appeared to increase other
  child welfare spending more than comparison sites.

24
Change in Placement Day Utilization
25
Changes in Foster Care Spending
26
Additional Waiver Revenue

• Most demonstration counties received more
  revenue through the Waiver than they would have
  received through normal Title IV-E reimbursement
  for foster care board and maintenance expenses.

27
Additional Revenue Received (1000s)
28
Additional Revenue Purchased

• All but one of the demonstration counties spent
  its additional Waiver revenue on child welfare
  services other than board and maintenance
  payments.
• As a result, spending on all other child welfare
  services increased significantly more among the
  group of demonstration counties than among the
  group of comparison counties.

29
All Other Child Welfare Expenditures
30
Overall, demonstration counties have made more
changes in practice than comparison sites because
the Waiver

• allows PCSAs to spend in different ways
• makes it easier to change management practices
• may grease the wheels of collaboration
• But how did these changes affect child family
  outcomes?

31
Outcomes Resulting from Organizational and Fiscal
Changes

• Changes in Caseload Trends
• Changes in Case Mix
• Waiver effects on Permanency

32
Abuse/Neglect Incidents Fell During Waiver Period
for Both Groups
33
During Waiver Period, Number of Children in
Caseloads Increased in Demonstration Counties
34
Number of Children Entering First Placements
Increased in Large Demonstration Counties
35
Early Findings Permanency Trends

• Both demonstration and comparison counties
  experienced an increase in the number of children
  in permanent custody over the first three years
  of the Waiver.
• The comparison counties experienced a much larger
  increase than did the demonstration counties in
  the number of children in PPLA.

36
Early Findings Permanency Trends

• In the first 2 years of the Waiver, both
  demonstration and comparison county groups
  increased the percentage of children who
  experienced no moves in their first placements,
  to over half of all children in first placements.
• Nearly 25 percent of children in both groups
  experienced only one move while in placement, and
  this percentage increased during the first 2
  years of the Waiver.
• The percentage of children who experienced five
  or more moves decreased in both groups

37
Early Findings Permanency Trends

• During the first 2 years of the Waiver, the
  number of teens placed in group and residential
  settings in demonstration counties did not
  decline.
• The demonstration counties did not experience
  greater success than comparison counties at
  moving children from group and residential
  settings to less restrictive settings during the
  first 2 years of the Waiver.

38
Early Findings Permanency Trends

• In the first 3 years of the Waiver, both
  demonstration and comparison county groups
  experienced an increase in the number of children
  placed with relatives.
• However, 9 of 14 demonstration counties
  experienced an increase, compared with 6 of 14
  comparison counties.

39
Use of Kinship Care Varied by County
40
Early Findings Permanency Trends

• The majority of counties in both demonstration
  and comparison groups had an increase in the
  number of new children eligible for adoption
  subsidy.
• The overall number of adoptions increased 40 in
  demonstration counties and 32 in comparison
  counties during the first three years of the
  Waiver.

41
Early Findings Safety Trends

• During the first 3 years of the Waiver, the
  recidivism rate for targeted children (those
  identified with moderate to high risk) declined
  slightly in the demonstration counties while
  decreasing considerably in comparison counties.
• Nine demonstration counties and seven comparison
  counties experienced decreases in recidivism
  rates for targeted children.

42
Early Findings Well-being

• The balance between in-home and placement
  services remained stable across the baseline and
  the Waiver periods, and demonstration counties
  proportions were similar to comparison counties

43
Changes in Placement Case MixDuring the Waiver
44
Percentage of Children Placed From Abuse/Neglect
Incidents Changed for Several Counties During
Waiver Period
45
Several Counties Increased First Placements with
Relatives During the Waiver
46
Waiver Effect on the Exit Type fromFirst
Placements Overall
47
The Waiver Shortened the Overall Median Duration
of First Placements
48
Significant County-Level Effects of Where
Children Exited from Placement

• The Waiver significantly increased the percentage
  of first placements that ended with custody to
  relatives in six of the 13 demonstration counties
  and decreased it in one county.
• The Waiver significantly increased runaways in
  three counties, although the overall effect was
  small
• The Waiver decreased reunifications in the
  largest county.
• The Waiver increased adoptions in one county,
  with no overall effect.
• The Waiver increased other exit types in one
  county (the largest) and decreased them in two
  others, for an overall increase

49
County-Level Waiver Effects on Exit Type
50
Some County-Level Waiver Effects on Median
Duration to Each Exit Type
51
No County-Level Waiver Effect on Reentry
52
Putting it all together Case Studies of 6
Demonstration sites

• Clark, Fairfield, Franklin, Lorain, Muskingum,
  Stark
• Major organizational changes adopted and outcomes
  achieved
• Offers insights into dynamics of child welfare
  practice under the Waiver, but cannot be
  extrapolated to all counties

53
Case Study Findings (1) Lorain and Muskingum

• Commitment to systemic change appears to have
  fostered a greater degree of success, in terms of
  fiscal and participant outcomes, than in other
  evaluation counties.
• More use of managed care strategies
• More focus on prevention
• Increased spending on non-foster care
• Fewer children in PPLA
• Quicker adoption, more adoption
• More exits to relative custody

54
Case Study Findings (2)

• Other four counties had less clear-cut results
  despite initiating a variety of reform efforts.
• Clark reformed front-end of the system through
  expansion of home-based services and
  collaboration with juvenile court result was
  reduced placement days.
• Franklin focus on external reform, contracting
  with private providers result was reduced LOS in
  foster care.
• Fairfield and Stark less consistent reform
  agendas during Waiver due in part to major
  organizational challenges no significant
  improvements in outcomes.

55
Overall ResultsMix of evidence of
significant impact of the Waiver, areas where
systematic impact is lacking, and signs of
substantive change in individual counties

• Confounding circumstances
• Funding context (levies, state cutbacks)
• County-administered system so county-to-county
  variation
• Lack of management tools
• Low financial risk
• Small sample sizes differences among data sets
• Suggests that the flexibility provided by the
  Waiver
• facilitates reform efforts where other factors
  are already conducive,
• but may not itself be robust enough to generate
  fundamental reform of the states public child
  welfare system.

56
The Future

• Good likelihood of Waiver extension
• Evaluation focus
• More in-depth analysis of foster care utilization
• More examination of child safety
• Further analysis of expenditure patterns
• Targeted investigation of special topics use of
  relatives, mental health services, court-referred
  children, etc.