AB1417 Performance Framework Project

1
AB1417 Performance Framework Project

• Patrick Perry
• Vice Chancellor, Technology, Research,
  Information Systems

2
What was AB1417?

• 03-04 bill (Pacheco) that charges the BOG to
  recommend the structure of a district-level
  performance evaluation system and report back to
  Legislature and Governor by March 25, 2005
• It was the bill that told us to come up with
  the new accountability system (but not enact it)

3
AB1417-Accountability

• RP Group-Center for Student Success hired as
  contractor
  
• External Panel of Experts reviewed model
• Nov. 04-Mar. 05creation, approval from
  Consultation, BOG, Finance, Leg Analystand
  Legislature
  
• Was placed into budget act for 05-06 (enacted
  it)
  
• Major accomplishment to get all parties to agree
  on model

4
AB1417 Resources

• Budget Act of 2005 gave System Office 4 new staff
  to work on report.
  
• Catharine Liddicoat, Alice van Ommeren, LeAnn
  Fong-Batkin
  
• MIS programmer to be named later
• Willard Hom, Research Director

5
AB1417 Resources

• Technical Workgroup has been assembled
• Made up of selected technical experts,
  consultation council members, DOF, LAO
  
• Will be meeting monthly until project completes

6
AB1417 Resources

• Website
• System Office, Research
• http//www.cccco.edu/divisions/tris/rp/ab_1417/ab_
  1417.htm
  

7
AB1417 Contacts

• LeAnn Fong-Batkin
• 916-327-5886
• lfongbatkin_at_cccco.edu
• We are assembling a list of district and college
  primary/secondary contacts
  
• Alias list or listserve to be created for all
  official correspondence
  

8
AB1417-Working Caveats

• Create a reporting framework that
• Improved upon PFE
• Had useful metrics and measured pertinent things
• Minimized institutional burden
• Told our story
• Didnt rely on institutional comparison
• Satisfied accountability-driven parties

9
AB1417

• The Model
• Measures 4 areas
• Student Progress Achievement-Degree/Certificate/
  Transfer
  
• Student Progress Achievement-Vocational/Occupati
  onal/Workforce Dev.
  
• Pre-collegiate improvement/basic skills/ESL
• Participation

10
AB1417

• The Model
• Has two levels of reporting
• College Core Indicators
• Originally District now by college
• Both system interests and LAO/DOF/CO parties
  decided college level reporting was most
  desirous
  
• System Indicators
• Will report system-level metrics only not broken
  out by college

11
Improving PFE

• Partnership for Excellence (98-04) used 5 metrics
  to measure system and college-level performance
  
• PFE was tied to money for improvement
• 4 of 5 PFE metrics were annual volumes (only 1
  rate)

12
PFE Metrics

• Annual volume of transfers to CSU/UC
• Annual volume of awards/certificates
• Rate of successful course completions
• Annual volume of voc. Ed. Course completions
• Annual volume of basic skills improvements

13
Volumes vs. Rates

• When you measure volumes, you are mainly
  measuring size or growth (frequently a function
  of funding)
  
• Rates mitigate size/growth issues
• All new AB1417 college metrics are rates
• Annual volume metrics are used for system
  purposes
  
• We are not measuring process!

14
Student Prog. Achievement Degree/Cert/Xfer

• District Core
• Student Progress Achievement Rate
• 1st year to 2nd year persistence rate
• System
• Annual volume of transfers
• Transfer Rate for 6-year cohort of FTFs
• Annual of BA/BS grads at CSU/UC who attended a
  CCC

15
Student Prog. Achievement Voc/Occ/Wkforce Dev

• College Core
• Successful Course Completion rate vocational
  courses
  
• System
• Annual volume of degrees/certificates by program
• Increase in total personal income as a result of
  receiving degree/certificate

16
Precollegiate Improvement/Basic Skills/ESL

• College Core
• Successful Course Completion rate basic skills
  courses
  
• ESL Improvement Rate
• Basic Skills Improvement Rate
• System
• Annual volume of basic skills improvements

17
Participation

• College Core
• None.
• System
• Statewide Participation Rate (by demographic)

18
AB1417 The mechanics

• System Office will produce the reports with MIS
  data.
  
• Draft report will be given back to Colleges for
  60-day review.
  
• College will provide brief analysis of data which
  will be included in final report.
  
• College profile will also be included.
• MIS provides some college provides statement
• No new data elements necessary.
• Integrity of existing data magnified.

19
AB1417 The Timeline

• July 05-Fall 06 Creation of extracts, refinement
  of metrics, data automation of reports
  
• Data cleanliness issues to be addressed
  resubmission of older data possible
  
• Fall 2006 First draft reports to Colleges
• Consider first report as a pilot
• 60-day local review College comments and
  analysis submitted to CCCCO
  
• April 2007 First report due to Leg.

20
AB1417 Accountability

• All District Boards will interact with the
  colleges annual reports.
  
• Can occur at any point during the year.
• CO will likely be collecting resolutions as an
  audit procedure.

21
AB1417 Performance Evaluation

• Colleges will not be ranked.
• College performance will be against that Colleges
  past performance you will see 3-yr running
  total.
  
• All parties desired the need for some type of
  meaningful comparison, however.

22
AB1417 Peer Grouping

• Peer grouping summaries will be available as
  report appendices.
  
• For each college metric, System Office will
  perform regressions to find exogenous factors
  that affect rates.
  
• Peer groupings unique to each metric will group
  multiple logical clusters so you can eyeball
  your performance within each exogenous peer group.

23
AB1417 Evaluation Triggers

• Continual decline may trigger further review or
  interaction.
  
• Not sure by who not funded for technical
  assistance.
  
• No carrots or sticks (yet).
• Most important thing use the data for
  self-evaluation and self-improvement.

24
AB1417 Name?

• Will it always be called AB1417? NO!
• ARSE Accountability Reporting for Student
  Enhancement
  
• RATS Reporting Accountability To the State
• AARGH Arnolds Accountability Reporting Grand
  Hierarchy
  
• AW-CRAP Accountability With Centralized
  Reporting About Performance