Measuring Vocational Program Outcomes: What We Know and What

1
Measuring Vocational Program Outcomes What We
Know and Whats Next

• Patrick Perry
• Vice Chancellor, Technology, Research,
  Information Systems

2
Who is this guy? Why should we listen to you?

• Brad Pitt-like looks
• Vin Diesel physique
• I collect data and measure stuff for a living.
• I have all the data.
• Information Management Institutional Research
• IMtherefore IR.

3
My Credo

• I realize that I will not succeed in answering
  all of your questions. Indeed, I will not answer
  any of them completely. The answers I provide
  will only serve to raise a whole new set of
  questions that lead to more problems, some of
  which you werent aware of in the first place.
  When my work is complete, you will be as confused
  as ever, but hopefully, you will be confused on a
  higher level and about more important things.

4
Todays Learning Outcomes

• Learn about recent impacts of fee
  increases/budget cuts on Voc. enrollments
• Learn about vocational course completion rates
• Know the trends of awards/certificates conferred
  in the system
• Learn about wages by degree type
• Learn about wages by TOP code
• Learn about the ARCC/1417 Wage Study (Pre-Post)

5
Fee History of the CCCs
Time Frame Per Unit Fees Precipitating Event(s)
Prior to Fall 1984 0  
Fall 1984 - Spring 1991 5, capped at 50 per sem. Recession in 1982-83
Fall 1991 Fall 1993 6, capped at 60 per sem. Recession/State Budget Crisis of 1991-93
Spring 1993 10, cap eliminated, differential fee of 50/unit for holders of BA/BS Recession/State Budget Crisis of 1991-93
Fall 1993 to Fall 1995 13, Differential fee of 50/unit Recession/State Budget Crisis of 1991-93
Spring 1996 to Fall 1998 13, Differential fee eliminated Period of economic boom
Spring 1999 to Fall 1999 12 Period of economic boom
Spring 2000 to Spring 2003 11 Period of economic boom
Fall 2003 Spring 2003 18 Recession/State Budget Crisis of 2001-04
Fall 2004 present 26 Recession/State Budget Crisis of 2001-04
6
Budget History of the CCCs
Fiscal Year System Funding (Billions)
1992-93 2.74
1993-94 2.729
1994-95 2.757
1995-96 2.948
1996-97 3.199
1997-98 3.530
1998-99 3.889
1999-00 4.087
2000-01 4.672
2001-02 4.956
2002-03 4.907
2003-04 4.969
2004-05 5.416
7
Fee/Budget Impacts and Voc Programs Sections
Offered
Term Total Occ Ed Occ Ed Transfer Transfer Basic Skills Basic Skills
Fall 1999 157,015 54,033 -1.70 105,182 -0.40 14,078 6.70
Spring 2000 160,728 56,834 5.20 107,948 2.60 13,717 -2.60
Fall 2000 160,697 55,422 -2.50 108,743 0.70 14,379 4.80
Spring 2001 165,906 58,988 6.40 112,242 3.20 14,090 -2.00
Fall 2001 166,735 58,385 -1.00 113,558 1.20 14,536 3.20
Spring 2002 172,811 62,678 7.40 117,260 3.30 14,259 -1.90
Fall 2002 170,373 59,528 -5.00 115,873 -1.20 15,616 9.50
Spring 2003 164,597 57,973 -2.60 112,731 -2.70 14,561 -6.80
Fall 2003 160,573 55,768 -3.80 109,975 -2.40 14,632 0.50
Spring 2004 165,261 58,209 4.40 113,162 2.90 14,831 1.40
8
Top 10 Curricular Areas Lost (FTES)
Subject Area 02-03 FTES 04-05 FTES FTES Loss
Physical Education 83,747.90 65,980.20 17,767.70
Information Technology, General 34,482.10 25,045.70 9,436.40
General Studies 123,413.20 114,348.90 9,064.30
Computer Software Development 8,848.20 5,287.50 3,560.70
Office Technology/Office Computer Applications 17,893.80 14,632.30 3,261.50
Art 32,512.60 29,656.10 2,856.50
Music 29,393.10 26,709.30 2,683.80
Administration of Justice 23,342.00 20,919.20 2,422.80
Mathematics, General 105,219.00 103,253.30 1,965.70
Electronics and Electric Technology 5,843.50 4,011.80 1,831.70
9
Top 10 Curricular Areas Gained (FTES)
Subject Area 02-03 FTES 04-05 FTES FTES Gain
Real Estate 5,065.80 7,140.40 2,074.60
Manufacturing and Industrial Technology 7,667.80 9,606.20 1,938.40
Anatomy and Physiology 6,634.00 8,044.30 1,410.30
Fire Technology 9,888.00 10,906.20 1,018.20
Chemistry, General 23,147.90 23,992.00 844.1
Nutrition, Foods, and Culinary Arts 7,882.00 8,630.20 748.2
Microbiology 1,223.60 1,887.20 663.6
Sociology 15,641.00 16,161.80 520.8
Radiologic Technology 2,460.30 2,945.40 485.1
Speech Communication 21,824.30 22,179.40 355.1
10
Voc Ed Course Completion Rates
Course Type 1995-96 2004-05 change
Transferable Courses 68.3 69.5 1.8
Basic Skills Courses 60.3 60.7 0.7
Vocational Courses 77.2 84.3 9.2
11
Annual Volume of Awards/Certificates
Degree Type 1997-98 2004-05 change
AA/AS Degrees 60,552 77,093 27.3
Certificates 23,627 21,107 -10.7
12
Effects of Degree Attainment on Wages

• Recent study done in support of ARCC/Ab1417
  Accountability Project
• Catharine Liddicoat, CCC Research Unit

13
Effects of Degree Attainment on Wages

• Methodology
• Take all award recipients in 2000-2001 who were
• Not enrolled in 01-02 and 02-03 and
• Not transferred to a 4-yr college and
• Had a record on EDDs wage file
• Cut post-attainment wages by award type and TOP
  code of award (2 years after)

14
Study Domain

• 2000-2001 Completers Cohort had 14,516 students
• 12,105 earned one award 2,411 earned 2 or more
• 78.7 had wage data for all 3 years out
• Looking at only single award earners who had 3
  years of wage data
• Study N9,490

15
Study Median and Mean Wages
Subsequent Years Median Mean Std. Dev.
Wages 01-02 31,515 35,244 24,302
Wages 02-03 35,279 39,491 26,396
Wages 03-04 37,174 41,444 28,206
16
Study Wages by Award Type
Award Type Wages 01-02 Wages 02-03 Wages 03-04
Assoc. of Arts 25,741 29,215 30,679
Assoc. Of Science 38,498 46,002 48,942
Certificate, lt6 units 46,460 48,110 48,603
Certificate, 6-18 units 24,504 26,267 27,384
Certificate 18-30 units 30,447 33,297 35,252
Certificate, 30-60 units 30,419 33,369 34,520
Certificate, 60 units 36,696 40,687 44,049
17
TOP 5 Highest Earning

• 5. Admin. Of Justice (54,158)
• 4. Plumbing Pipefitting (56,628)
• 3. Corrections (62,414)
• 2. Physicians Assistant (64,658)
• 1. Surveying (70,626)

18
Bottom 5

• 5. Fashion Design (16,206)
• 4. Animal Science (14,086)
• 3. Home Health Aide (14,081)
• 2. Cosmetology (11,616)
• 1. Spanish (9,307)

19
Wage Study Wages Pre-Post
  1994 1995 1996 1997 1998 1999 2000 2001 2002 2003
ccc median income 19,197 21,004 22,995 25,696 27,468 29,109 32,456 42,891 47,331 48,718
median household income 34,100 35,300 37,100 39,000 40,600 43,800 46,900 47,177 47,500 49,320
20
Wage Study Wages Pre-Post
21
(No Transcript)
22
ARCC/AB1417

• Is the next Accountability report of the CCC
  System
• Features College-level metrics and System-level
  metrics

23
ARCC/AB1417

• Ideally, we desired to produce program completion
  rates for all voc. Programs, but
• MIS does not collect data on student program of
  study

24
ARCC/AB1417

• Metrics Associated with Vocational outcomes
• Student Progress Achievement Rate-College
  Metric
• FTF cohort tracked over 6 years
• Tracks students who attempt transfer or
  degree-applicable level math English courses as
  behavioral intent to complete
• Outcomes Transfer, Transfer-prepared,
  transfer-directed, AA/AS, Certificate, or earned
  30 units threshhold

25
ARCC/AB1417

• Annual Successful Course Completion Rate
  (Vocational Courses)College Metric
• Same as in PFE Report

26
ARCC/AB1417

• Annual Volume of Degrees/Certificates Conferred
  (System Metric)
• Cut by TOP code
• Increase in Total Personal Income as a result of
  degree/certificate attainment (System metric)

27
ARCC/AB1417

• Draft report sent to college October
• 60-day review of draft report, data resubmission
  allowed
• Peer Groupings created for metrics
• Final draft of report to field January
• 30 day response to report and local analysis
  college comments made a part of the report
• Report due March
• Colleges have 1 year to have local board review
  report and acknowledge having interacted with it

28
Your Turn

• We are always open to hearing good research
  ideasfeel free to contact us so we may assist.
• Thanks for inviting me to your conference!