VTEA Core Indicators

1
VTEA Core Indicators Negotiated Targets
What? Who? When? How?
Vocational Technical Education Act (VTEA)

• Geri Douglas Chuck Wiseley
• March 15, 2006

2
Workshop Goals

• Understand CI as outputs outcomes.
• CI Local data - VTEA funds.
• VTEA funds - improving student success.

3
Workshop Objectives

• Participants will
• Examine Reports
• Discuss
• what meaning can be derived
• negotiated targets
• Identify
• research questions or hypothesis,
• strategies for program improvement,
• strategies for negotiating targets.

4
Data Why?

• Funding
• Outcomes Analysis
• Telling the story

5
Defining the Data

• TOP Codes
• SAM Codes
• Data Elements
• Core Indicators
• The Law
• Definitions
• Measurement Approaches/Formulas
• Negotiated Performance Levels

Funding
6
Student Accountability Model (SAM) Taxonomy of
Programs (TOP)

• Priority A - Apprenticeship
• Must have the approval of the Division of
  Apprenticeship Standards
• Priority B Advanced Vocational
• Used sparingly, no more than two courses in any
  one program
• Each B level course must have a C
  prerequisite in the same program area
• Priority "C" Clearly Occupational
• Generally taken in the middle stage of a program,
  detracts "drop-ins."

7
Student Accountability Model (SAM) Taxonomy of
Programs (TOP), Continued

• Priority "D" Possibly Occupational
• Taken by students in the beginning stages of
  their occupational programs
• Can be survey course
• Priority E Non-Occupational
• Vocational Flag on TOP code
• Designed to identify vocational Programs for
  federal reporting () - see Taxonomy of Programs,
  Sixth Edition, November 2004

8
Data ElementsMIS System

• Students, Courses, Degrees, Services
• Student VTEA Data Elements
• Economically Disadvantaged
• Single Parent
• Displaced Homemaker
• Cooperative Work Experience Education
• Tech Prep

9
Accountability Requirements Section 113(b)

• 4 core indicators of performance
• Student achievement,
• Student completion,
• Placement and retention in employment,
• Participation/completion of non-traditional
  training
• State adjusted levels of performance for each
• Levels of performance negotiated with USDE
• Results reported annually

10
Core Indicators

• Skill AttainmentStudent Achievement Successful
  course completion
• Academic
• Vocational and Technical
• Completions Student program completion
  Certificates Degrees
• Placement and Retention Placement and retention
  in employment
• Equity -- Non-Traditional Employment
  Participation/Completion of non-traditional
  training)
• Participation
• Completion

11
Cohort Definitions Used for Measurement

• Participant
• Any enrollment in a vocational course
• Concentrator Skill Attainment
• Enrollment in a course in the middle or end of a
  program (SAM A-C)
• Concentrator All other indicators
• Successful completion of at least one course in
  the middle or end of a program and 12 vocational
  units within a single discipline (two digit TOP)
  
• or
• Program completion as indicated by receipt of
  vocational certificate or degree

12
Assigning a Program

• TOP code of Certificate or Degree
• Concentrators
• Assigned to the TOP where most units occurred

13
Core Indicator 1Skill Attainment Successful
Course Completions

• Skill Attainment
• Integrated Academic and Vocational
• Academic Skill Proficiencies
• Vocational and Technical Skill Proficiencies
• G.P.A. (Compliance reports Forms)
• Grades (All other reports)
• Vocational Courses
• SAM A-C
• Vocational TOP

14
Core Indicator 1 Measurement Performance
Levels (compliance)
in SAM A - C Courses Students w/ gt 2.00
GPA Students w/ Grades A - F
Year Target Actual 2002-03
77.76 84.01 2003-04
78.76 84.73 2004-05
79.76 84.70 2005-06 84.02 ??.??
15
Core Indicator 1 Measurement Performance
Levels (Analysis)
in SAM A - C Courses Grades A-C Cr
Grades AF, Cr-NoCr, W, I
16
Core Indicator 2Program Completions

• Completers (numerator)
• Award in Current Year
• AA/AS degrees
• Certificates
• and/or
• Continued at CSU or UC
• Leavers Completers (denominator)
• Left for one year and/or
• Award in Current Year and/or
• AA/AS degrees
• Certificates
• Continued at CSU or UC

17
Core Indicator 2 Measurement Performance
Levels
Concentrator earning certificate/degree or
transferring to a CA four-year public
university All Concentrators
Year Target Actual 2002-03
59.82 76.36 2003-04
59.82 74.89 2004-05
60.82 73.42 2005-06 73.00 ??.??
18
Core Indicator 3 Placement and Retention

• Placement
• UI wage file match 1st year out
• Continuing in Four Year Institution
• CSU
• UC
• Retention in Employment UI only
• Four quarters in Academic Yr after leaving
• Three Consecutive quarters

19
Core Indicator 3 Measurement Performance
Levels -- Placement
Concentrators in UI covered employment or CA
four-year public university All concentrators
Year Target Actual 2002-03 83.19 84.08 2003-
04 83.19 82.53 2004-05
83.19 82.07 2005-06 82.07 ??.??
20
Core Indicator 3 Measurement Performance
Levels -- Retention
Concentrators in UI covered employment for three
consecutive quarters and not found at CSU or
UC Concentrators, UI any quarter not at CSU or
UC
Year Target Actual 2002-03 82.70 84.29 2003-0
4 82.75 83.77 2004-05 82.85 83.48 2005-06
83.48 ??.??
21
Core Indicator 4Gender Equity

• Training leading to Nontraditional Employment
• 75 / 25 from 1990 census employment data
• Job codes mapped to 1990 Census data
• Job titles identified by OES codes
• Job codes mapped to TOP

22
Core Indicator 4Gender Equity

• In Programs leading to Nontraditional Employment
• Nontraditional Gender Students
• All Students in Program

23
Core Indicator 4 Measurement Performance
Levels -- Participation
Nontraditional participants enrolled in a
nontraditional TOP Code All participants
enrolled in a nontraditional TOP Code
Year Target Actual 2002-03
27.98 36.98 2003-04
28.98 36.80 2004-05
29.98 35.31 2005-06 35.31 ??.??
24
Core Indicator 4 Measurement Performance
Levels -- Completion
Nontraditional completers of nontraditional
programs All completers of nontraditional
programs
Year Target Actual 2002-03 26.95
24.42 2003-04 26.95
24.21 2004-05 25.05 23.25 2005-06
23.25 ??.??
25
How evaluation looks

• Losing NT students faster than TG students
• California 2005 actual
• 35.31 NT Participation (4P1 )
• 23.25 NT Completions (4P2)
• Conclusion NT participants doing worse than TG
  in NT programs
• Conversion rates from Participant to Completer
• TG All Students minus NT
• Rate For the group (NT in 4P2) / (NT in 4P1)
• 02.60 NT Completers to NT Participants
  (numerators 4P2/ 4P1)
• 04.91 TG Completers to TG Participants (TG 4P2 /
  TG 4P1)
• Conclusion NT students are moving from
  participant to completer at about half the rate
  of TG in NT Programs

26
Do these indicators really measure what education
in CCC does?
27
Do these indicators really measure what education
in CCC does?

• They are only indicators.

28
Are they meaningless?
29
Are they meaningless?

• No. Student success matters.

30
How can we use the indicators to improve student
success?
31
How can we use the indicators to improve student
success?

• Understand What is happening to students.

32
How can we use the indicators to improve student
success?

• Understand What is happening to students.
• Why is it happening?

33
Accessing the Data

• Access
• Report Formats
• Report Structures
• Report Contents
• Report Examples

34
Negotiating Targets

• Current Law
• State negotiates targets
• Proposed Law
• State negotiates targets
• Locals either
• Accept those targets
• Negotiate local targets
• Included in Local Plans
• Targets for 2 years
• Targets renegotiated years 3-4, 5

35
Sanctions

• Current Law
• State
• Improvement plan
• Two years below, Secretary MAY withhold funds
• Proposed Law
• State same
• Except 2 years for any single target
• Locals
• Improvement Plan
• 2 years for any single target
• State SHALL provide technical assistance
• State MAY withhold funds (all or part)

36
Reports

• New reports
• Developed for negotiations
• Easier access
• Closer alignment with other states
• Increased scrutiny on use of current reports

37
Access

• http//www.cccco.edu
• Important Documentation
• Quick Reference
• Printing Instructions
• System Requirements
• Install Smart Viewer Browser Plug-in
• Need Administrative rights on PC call IT folks
• Navigate Print using Smart Viewer functions

38
Report Formats

• Print Version
• For administrators, faculty, students PIOs
• Printing from the WWW
• Local Applications and Final Reports
• For exporting to
• word processor
• spreadsheet
• Data Version
• For Researchers, IS Technophiles
• For Export to
• Excel
• SPSS, SAS or other Stat Package
• BRIO
• Access or other database software

39
Data Download
40
Using Data Download
41
(No Transcript)
42
Report Structures

• Forms
• VTEA IC - Local Application
• VTEA IC - Final Report
• Summaries
• All four Indicators on one page
• Answer Sheet Style
• Detailed
• Percents and counts
• Detailed breakouts for each Indicator component

43
Report Contents

• State, District, College
• 2-4-6 digit TOP
• Special Populations breakouts
• Identified on each report
• Performance below negotiated levels
• Performance below is highlighted
• Low numbers (denominator lt 10)
• Blue-italics

44
Contents (continued)

• Summary
• College, District State
• Identifying Low Performance Low Numbers
• Comparing Performance across Programs
• Detailed
• Program Area (6 digit TOP)
• Three Year Trends
• Identifying Low Performance areas
• Comparing Performance within Programs

45
Report Examples

• Final Report Form
• College by TOP Form
• Summary College
• Summary - 5 digit TOP
• Summary Detail
• Core 2 Completions
• Core 2 - 5 digit TOP
• Course Voc Status/SAM Code

46
Final Report Form
47
Final Report Form
48
College by TOP Form
49
Summary - College
50
Summary College 5 Digit
51
Summary - Detail
52
Core 2 - Completions
53
Core 2 - Completions 5 digit
54
Course Voc Status/SAM Code
55
Utilizing the Data

• Comparison to state-adjusted levels
• Trends over time for your college
• By program, special population, Tech Prep
• Workshops, Committees Faculty Information
• Circulate to faculty for their use input
• Factor in allocation decision-making for Local
  Plans
• Supplement MIS data with your own knowledge and
  follow-up of students

56
Utilizing the Data

• Practicum Exercise
• Review Data Analysis Practicum Worksheet Number
  One
• Review Worksheet Number Two
• Review the pre-printed report for a sample
  college
• Conduct a guided practice using selected TOP
  codes