DATA LINKING FOR OUTCOMES ASSESSMENT: DLOA

1
DATA LINKING FOR OUTCOMES ASSESSMENT DLOA

• A Training for WARP by
• Deb Stephens, SBCTC
• Carmen Stewart, SBCTC

2
PART I. ABOUT DLOA
3
Why was DLOA Created?

• Established Partnership
• Employment Security
• K-12
• WTECB
• SBCTC
• Linked participant data to external data
• Unemployment Insurance (UI) data (Washington and
  neighboring states)
• National Clearinghouse data
• Washington public University data
• Calculated placement and earnings rates in
  systematic and consistent manner

4
Whats in DLOA?

• Exiting student data
• Demographic, training, transfer and employment
  data
• Quick files for kind of student cohorts
• 3 databases
• Job Preparatory (includes quick files for 2
  cohortsjob prep and apprenticeship)
• Transfer
• Worker Retraining
• 4 cohorts

5
Who are the Cohorts?

• Exiting Job Preparatory students (F intent,
  vocational completers)
• Exiting Transfer students (kind of student T,
  academic degree completers)
• Exiting Apprenticeship students (H intent,
  completers and leavers from LI files)
• Exiting Worker Retraining students (regardless of
  kind of student or intent)

6
How Are the Tables Organized?

• Post College tables allow assessment of
  employment, wages and continuing education after
  leaving college (for all cohorts).
• Pre-Post College tables allow comparisons of
  employment information before and after college
  (all cohorts except apprenticeship).
• During-Post College tables allow comparisons of
  employment information during and after college
  (transfer and job preparatory only).

7
What Kind of Data is in the Quick Files?

• All the usual demographic data
• Employment data (3rd or 5th quarter before,
  during and 3rd quarter after college)
• Industry of employment
• Wages
• Earnings
• County of employment
• Employer
• Continuing education information
• Students are unduplicated by college

8
Dont Forget About the Documentation!

• Documentation is organized around the Quick Files
• Contains a description of all the data elements,
  including those from other databases (such as
  demographic information)
• Everything talked about today is in there
• Provides estimated employment tables (will
  discuss later)

9
The Confidentiality Agreement
10
Why Does DLOA Use Require a Confidentiality
Agreement?

• Uses data from the Unemployment Insurance
  database
• SBCTC signs a confidentiality agreement with ESD
  that we will abide by strict reporting so that
  individuals and firms cannot be identified in our
  data
• We are subject to audit by ESD

11
New Language

• Addresses concerns raised at the last WARP
  Elluminate training
• In (i).take out the words use or
• Change statistical reporting to for aggregated
  reporting
• Okay to USE the data for internal purposes, just
  cannot report it publicly so that individuals or
  firms can be identified

12
Does This Alleviate Your Concerns?

• I will not reveal any individually identifiable
  information furnished, acquired, retrieved or
  assembled by me or others for any purposes other
  than for aggregated reporting.

13
QUESTIONS and ANSWERS (we hope)
14
10-Minute Break
15
PART II. USING DLOA
16
What Kinds of Reporting Does SBCTC Use DLOA For?

• Worker Retraining Report
• Job Training Results
• EMSI Study
• Carl Perkins reporting
• Performance Measures
• Academic Year ReportMedian Wages and Earnings by
  Field of Study (will discuss later)

17
What Are Some of the Commonly Used Data Elements?

• When using Post College Files
• Educational attainment of exiting students based
  on Exit Codes EXIT_STATUS
• Summary of Exit Status GRADDROP
• Inflation adjusted wages in the 3rd Quarter after
  leaving INFLATION_ADJ_WAGE
• Inflation adjusted annual earnings
  INFLATION_ADJ_EARNINGS EST ANNUAL

18
A Few More

• More Post College Data Elements
• Whether or not in UI File 3 quarters after
  leaving college PLACEMENT_STATUS
• Industry code NAICS
• CIP Code at Completion (or last reported)
  ENDING_CIP

19
Still More Data Elements

• When using Pre-Post files.
• Wages 3rd or 5th quarter before entering college
  B_INFLATION_ADJ_WAGE
• Wages 3rd quarter after leaving college
  A_INFLATION_ADJ_WAGE
• Other elements have B and A denoting before
  and after (earnings, NAICS, etc)
• Only those students with either earnings or wages
  before AND after are included

20
And More Data Elements

• When using During-Post files.
• Wages 1 quarter before leaving college
  D_INFLATION_ADJ_WAGE
• Wages 3 quarters after leaving college
  A_INFLATION_ADJ_WAGE
• Other elements have D and A denoting during
  and after (earnings, NAICS, etc)
• Only those students with either earnings or wages
  during AND after are included

21
What the Heck are CIP_GRP_WAGE and LMH?

• Too many CIPs to talk about in a meaningful way
• CIP_GRP_WAGE clumps CIPs into fields of study
• LMH further clumps the CIP_GRP_WAGE fields into
  lower, medium and higher wage categories based on
  median wages

22
Example Academic Year Report TablesNew This
Year
23
What are some easy queries?

• To calculate Number of Completers DW_KEY,
  GRADDROP gt 0
• To calculate Number of Leavers DW_KEY, GRADDROP
  0
• To calculate number of completers employed 3
  quarters after leaving college DW_KEY, GRADDROP
  gt 0 PLACEMENT_STATUS 1

24
Example Query Calculating an Employment Rate
Step 1 (the numerator)
Calculates the number of exiting WR students who
were employed 3 quarters after leaving college
25
Step 2 Calculating the Denominator
Calculates the total number of exiting WR students
26
Speaking of Employment Rates..What Are the
Employment Rate Tables For?

• Placement Status1 is a count of ONLY those in
  the UI database
• Excludes self-employed and those working out of
  state
• Tables allow adjustments to the rates to
  accommodate those not accounted for in UI
• Adjustments based on survey results by WTECB
• Completer and Leaver Tables in the Documentation

27
Comparing Pre-Training Industries with
Post-Training Industries Step 1
28
Here is What You Get
Can do the same thing with the after NAICS
codes and compare the pre and post numbers to see
what industries WR students are shifting out of
and into
29
Get Creative!
I built a table assigning each college to one of
8 regions around the state so I could look at
where WR students worked before and after
training.did they stay in the same region?
30
Can Use the New Table to Determine if Students
Remain in their County of Training When They
Return to Work
31
Here are the Results
32
What Are the Pitfalls of DLOA?

• MUST use the appropriate tables for each cohort
• Example WR students in Job Prep tables are not
  comprehensive, must use WR table to look at WR
  students
• Only includes exiting cohorts students not
  enrolled in the CTC system the following year

33
More Questions?
34
20 Minute Break