Performance Measurement

1
Performance Measurement
For Employment And Training Programs

• Wagner-Peyser and VETS
• Reporting

2
Agenda

• Background information
• Performance updates
• Calculating outcomes on the
  performance measures
• Revisions to existing reporting
  requirements

3
General References

• ETA TEGL 28-04 (Common Measures) can be found at
• http//www.doleta.gov/Performance/guidance/tools_c
  ommonmeasures.cfm
• ET Handbook No. 406 can be found at
• http//www.doleta.gov/Performance/guidance/wia.cfm
  

4
Common Measures Concepts
Participant

• A participant is someone who has been determined
  eligible to participate and who receives a
  program funded-service in a physical location.

5
Participant (cont.)

• States and grantees are encouraged to consider as
  participants those individuals who receive
  services via the Internet and do not access those
  services through a program-funded physical
  location.

6
Participant (cont.)

• Wagner-Peyser employment services
• Universal Access
• No formal determination of eligibility is
  required
• DVOP/LVER services
• veterans, eligible persons, transitioning service
  members who meet eligibility criteria

7
Self-directed job search Clarification of
exclusion

• Application to W-P and VETS
• Exclusion is limited to the act of conducting a
  job search, job bank browsing or modifying job
  search requirements.
• Some self-directed job search activities should
  be considered reportable services (e.g. building
  resumes, accessing workforce information, and
  enrolling in online job search mini courses).

8
Exiter

• Common Measures reporting system calculates
  performance outcomes based on exit quarter.

9
Exiter (cont.)

• Exit occurs when a participant does not receive a
  service funded by the program or a partner
  program for 90 consecutive calendar days and is
  not scheduled for future services.
• The date of exit is applied retroactively to the
  last day on which the participant received a
  service.
• This is a Soft Exit approach.

10
Exiter W-P and VETS

• Significant change from the former labor exchange
  performance measures
• calculated performance outcomes on a fixed point,
  based on registration quarter
• the first or second quarter following the
  registration quarter.
• Hard Exit.

11
Exiter Calculation

• Sample Calculation Please see handout!
• It is mathematically possible for an
  individual job seeker to be a participant and an
  exiter four (4) times in one year.
• 4 X 91 364
• Although unlikely, it is possible.
• Here is a more likely scenario.

12
Global Outcome Exclusions

• Global Outcome Exclusions that apply to WIA
  Adults, Dislocated Workers, Wagner-Peyser, and
  VETS
• Institutionalization
• Health/Medical or Family Care
• Reservist called to active duty and
• Deceased.

13
Global Outcome Exclusions

• New to W-P and VETS
• In implementation of global outcome exclusions,
  states should consider
• High costs of developing system to accept and
  track supplemental data
• Negligible effects on outcome

14
Reporting Transition from PY 2004 to PY 2005

• States will continue to report receipt of
  services and outcomes using the rolling four
  quarters methodology.
• For the report due November 14, 2005, States will
  need to reach back to re-calculate number of
  participants, exiters, and outcome data.

15
Minimum Data Needed from S-S Participants

• At a minimum, data needed from self-service
  participants
• social security number (SSN)
• employment status at the point of participation.
• Upon request of a customers SSN, the customer
  must also be offered the opportunity to provide
  Equal Employment/demographic information.

16
Entered Employment Rate (EER)

• Measure Of those not employed at
  participation, the percentage of individuals
  employed in the 1st quarter after program exit.
• Includes only those not employed at entry.
• Entry to employment is based on wage record data
  and other administrative data sources.
• Employment status at the start of participation
  is self-reported by jobseekers.

17
EER former Labor Exchange measures

• Common Measures change
• Registrant -to- Participant
• Hard -to- Soft Exit
• UI Wage Record and other state data system
  records -to- all of the preceding plus.
  supplemental data
• States have to weigh benefit of developing
  follow-up case management system

18
Employment Retention Rate (ERR)

• Measure For those employed in the 1st
  quarter after exit, the percentage employed in
  the 2nd and 3rd quarter after exit
• Of those employed in the 1st quarter after exit,
  measure applies to individuals that were not
  employed at the start of participation, as well
  as those who were employed at the start of
  participation.

19
ERR former Labor Exchange measures

• Change from old labor exchange measure
  Employment in the 3rd quarter after exit is new
  measurement point.
• The common measure ERR requires employment in two
  quarters to count toward employment retention.

20
Earnings Increase (EI)

• Measure Of those employed in 1st quarter
  after exit, total post-program earnings (in Q2
  Q3 after exit) minus pre-program earnings (in Q2
  Q3 prior to participation) divided by the
  number of adults who exit during the quarter.
• Comparison of pre-program earnings to
  post-program earnings using dollar amount, not
  rate.
• Includes only those employed in first quarter
  after exit.
• Verified through wage record data only.

21
EI Wagner-Peyser and VETS

• A new outcome measure for W-P and VETS
• Unsure of the expected outcome level
• States will report the outcomes results
• Outcome goals will not be negotiated for PY 2005
  for W-P
• PY 2005 will a baseline year for W-P
• Negotiation of goals in PY 2006

22
Workforce Information

• To integrate employment and workforce
  information services, WIA Adults, Dislocated
  Workers, and Wagner-Peyser will collect and
  report data on the provision of workforce
  information to job seekers.
• Workforce information may be accessed in person
  through self-help in the resource rooms,
  individual counseling or career guidance in a
  One-Stop Career Centers, or remotely via the
  Internet, and/or in ancillary agency sites. 

23
Workforce Information (cont.)

• The W-P employment service program will report on
  receipt of workforce information as a
  self-directed and/or staff-assisted service and
  sub-aggregate outcome data for those who received
  workforce information.
• DVOP and LVER staff members will report receipt
  of workforce information only as a staff-assisted
  service.

24
W-P/VETS and TSMs

• Recently Separated Veteran
• WIA time period within last 48 months
• W-P/VETS time period within last 36 months
• Transitioning Service Member (TSM) Expects to
  retire within 24 months or separate within 12
  months and seeks employment services.
• Homeless Veteran A veteran who meets the
  statutory criterion for homelessness.
• Intensive Services - Replaces Case Management as
  a reporting category.

25
Transitioning Service Members

• The Jobs for Veterans Act authorized DVOP and
  LVER staff members to deliver employment
  participants is included in the total count
  and/or services to TSMs.
• Information on TSM separately identified on the
  ETA 9002 A, B, C and D reports and the VETS 200
  A, B and, similar to Recently Separated C
  reports.
• TSM is a reporting characteristic Veteran and
  Campaign Badge Veteran.

26
Homeless Veterans

• A homeless veteran is an individual who has
• At least one day of active duty service with an
  other than dishonorable discharge AND
• No fixed, regular, and adequate nighttime
  residence.
• Services and outcomes for homeless veterans are
  reported only on the VETS 200 C report.

27
Intensive Services

• The Jobs for Veterans Act replaced the Term Case
  Management with the term Intensive Services
  and referenced Section 134(9)(3) of WIA, which
  identifies six specific services
• Conduct of assessments
• Development of individual employment plans
• Group counseling
• Individual counseling and career planning
• Case management and
• Short-term prevocational services.

28
Intensive Services (cont.)

• Intensive Services has Replaced Case
  Management as a reporting item on the ETA 9002 B
  and the VETS 200 A-C.
• DVOP/LVER staff still must meet the two
  prerequisites for case management in order to
  deliver intensive services
• Conduct of an assessment and
• Development of an individual employment plan

29
Intensive Services (cont.)

• After the prerequisites, DVOP/LVER staff can
  provide
• Group counseling
• Individual counseling and career planning and
• Case management.
• Short-term prevocational services are NOT
  included among the intensive services to be
  provided by DVOP/LVER staff.
• Veterans are not required to receive core
  services prior to receiving intensive services
  but veterans still are expected to receive
  standard labor exchange services from
  Wagner-Peyser staff prior to being referred to
  DVOP/LVER staff for the delivery of intensive
  services.

30
Customer Satisfaction for W-P funded programs

• Because it is not a common measure, for
• the Wagner-Peyser-funded employment services, ETA
  will discontinue capturing
• job seeker customer satisfaction data using the
  American Customer Satisfaction Index and
• employer customer satisfaction data.

31
Reporting Job Openings

• States should exclude from the ETA 9002E Report
  those job openings that were initially listed
  with Americas Job Bank and then imported into
  the state job bank.

32

• Technical Changes to the
• ETA 9002 and VETS 200 Reporting
• For further information, please review the ET
  Handbook No. 406 or
• contact your ETA or VETS regional office.

33
ETA 9002 A

• Eliminations
• Reporting on job seekers who were Referred to
  Support Services (row 17 on previous report).
• Column reporting on Other Race (column 13 on
  previous report)

34
ETA 9002 A

• Minor Adjustments
• Renamed column A from Total Jobseekers to
  Total.
• Renamed column B from Employment Status at
  Registration and Unemployed to Employment
  Status at Participation and Not Employed,
  respectively.
• Renamed Total Active Jobseekers row to Total
  Participants.
• Renamed Veterans and TSM Job Seekers row to
  Veterans, Eligible Persons, and TSMs.
• Changed age breakout from Adult (19 and over)
  to Adult (18 and over).
• Changed age breakout from 19-44 to 18-44.

35
ETA 9002 A

• Additions
• Reporting row Total number of participants who
  Received Workforce Information Services.
• Reporting row Total Exiters during the
  reporting period.

36
ETA 9002 B

• Eliminations
• All column level reporting by age breakouts for
  each category of veteran. On the previous
  report, this included column B, rows 5-7, column
  C, rows 9-11, column D, rows 13-15, and column E,
  rows 17-19.
• Reporting requirements for veterans who were
  Referred to Support Services (row 9 on previous
  report).

37
ETA 9002 B

• Minor Changes
• Renamed column A from Total Veterans and TSM
  Jobseekers to Total Veterans, Eligible Persons,
  and TSMs.
• Changed age breakout in column A from 19-44 to
  18-44.
• Switched columns B and C. On new report format,
  column B is labeled TSMs and column C is
  labeled Campaign Badge Veterans.
• Renamed column F from Newly Separated Veterans
  to Recently Separated Veterans (3 yrs).
• Renamed row 1 from Total Active Veterans and TSM
  Jobseekers to Total Veterans, Eligible Persons,
  and TSMs.
• Renamed, where appropriate, case management
  services to intensive services.
• Renamed, where appropriate, the term placed in
  with entered into.

38
ETA 9002 B

• Additions
• 3 reporting rows that include the total number of
  veterans served who were ages 18-44, 45-54,
  and 55 and over for each category of veteran.
• Reporting row that includes the total number of
  veterans who Received Workforce Information
  Services.
• Reporting row that includes Total Exiters
  during the reporting period.

39
ETA 9002 C

• Eliminations
• All column level reporting based on Employment
  Status at Registration (column B on previous
  report).
• Column reporting on Other Race (column 13 on
  previous report)
• Reporting requirements for job seekers who
  Entered Employment Following Staff-Assisted
  Services (row 11 on previous report).
• Row reporting requirements for the job seeker and
  employer customer satisfaction measure (rows 12
  17 on previous report).

40
ETA 9002 C

• Minor Adjustments
• Renamed column A from Total Jobseekers to
  Total Exiters.
• Included, where appropriate, the term numerator
  and a denominator for the entered employment and
  employment retention measures.
• Changed age breakout from 19-44 to 18-44.

41
ETA 9002 C

• Additions
• Reporting row to capture Six Months Earnings
  Increase.
• 3 reporting rows to capture common measures
  outcomes for participants who receive workforce
  information services.
• Entered Employment Rate Follow Work-Info
  Services,
• Employment Retention Rate Follow Work-Info
  Services, and
• Six Months Earnings Increase Follow Work-Info
  Services.

42
ETA 9002 D

• Eliminations
• All column level reporting by age breakouts for
  each category of veteran. On the previous
  report, this includes column B, rows 5-7, column
  C, rows 9-11, column D, rows 13-15, and column E,
  rows 17-19.

43
ETA 9002 D

• Minor Changes
• Changed age breakout in column A from 19-44 to
  18-44.
• Switched columns B and C. On new report format,
  column B is labeled TSMs and column C is
  labeled Campaign Badge Veterans.
• Renamed column F from Newly Separated Veterans
  to Recently Separated Veterans (3 yrs).
• Included, where appropriate, the term numerator
  and denominator in the rows for the entered
  employment and employment retention measures.

44
ETA 9002 D

• Additions
• 3 reporting rows that includes the total number
  of veterans who entered employment for each
  category of veteran according to the following
  age breakouts 18-44, 45-54, and 55 and
  over.
• Reporting row to capture Six Months Earnings
  Increase.
• 3 reporting rows to capture common measures
  outcomes for jobseekers who receive workforce
  information services.
• Entered Employment Rate Follow WI Services,
• Employment Retention Rate Follow WI Services,
• Six Months Earnings Increase Follow WI Services

45
ETA 9002 E

• No changes - Everything stays the same!
• ?

46
VETS 200 A, B and C Reports

• Eliminations
• Services Provided
• Reporting on veterans who were Referred to WIA
  Services (row 11 on previous reports).
• Reporting on veterans who were Referred to
  Support Services (row 12 on previous reports).
• Results and Outcomes
• The 3 rows of reporting for the entered
  employment data Entered Employment, Entered
  Employment Rate Base, and Entered Employment
  Rate (rows 17-19 on previous reports).
• Placeholder for state reporting on Job Seeker
  Satisfaction Score (row 32 on previous reports).

47
VETS 200 A, B and C Reports

• Additions
• Reporting row that includes the Total Exiters
  during the reporting period.
• Services Provided
• Reporting row that includes the total number of
  participants who Received Workforce Information
  Services.
• Results and Outcomes
• Reporting row that includes an aggregate count of
  the Six Months Earnings Increase during the
  reporting period.

48
VETS 200 C - Only

• Additions
• Reporting column on the VETS 200 C only that
  includes aggregate reporting on services to
  Homeless Veterans (column I).

49
SAMPLE CALCULATION of EXITER

• An unemployed job seeker came to the One-Stop
  Center on July 1, 2005 and browsed the job bank.
  She returned on July 3rd and created and posted a
  resume. On July 5th she modified job bank search
  requirements and accessed workforce information.
  She did not receive any more services for 90
  days. Employed but interested in looking at
  other options, she returned to the One-Stop on
  January 5, 2006 to update her resume and look at
  workforce information. She did not receive any
  more services for 90 days. She remained employed
  through January 2007.
• Questions
• What is her participation date(s)?
• What is her exit date(s)?
• What quarters do you follow for entry to
  employment?
• What quarters do you follow for retention?
  Earnings gain?

50
July Aug. Sept. Oct. Nov. Dec. Jan Feb. March
April May June July Aug. Sept. Oct. Nov.
Dec. Jan2005
2006

2007

• July 1 Browsed Job Bank
• July 3 Created Resume Participation Date 1
• July 5 Modified search requirements. Looked at
  workforce information.
• Oct. 3 90 days have
  elapsed, no service. Exit program. Exit date 1
  July 5, 2005. (Exit qtr July Sept. 2005)
• Oct. Dec. 2005 qtr is the first qtr after
  Exit date 1.
• Look at this qtr for entry to employment for
  participation 1. In this case the participant
  was found to be employed
• Jan 5 Updated resume and accessed workforce
  information Participation Date 2. (still
• employed)
• April 5, 2006 90 days have elapsed, no
  service. Exit program.
• Exit date 2 Jan 5, 2006 (Exit qtr 2 Jan
  March 2006)
• April-June 2006 qtr is the first qtr after
  Exit Date 2. Because jobseeker was employed
  at this participation, not included in the EER
  for participation 2.
• Jan- March 2006 and April-June 2006 are qtrs 2
  and 3, respectively, after exit of participation
  1. Because
• participant was employed in first qtr after
  exit 1, look at these qtrs for Retention and
  Earnings Increase.

51
Time and Date Calculator
http//www.timeanddate.com/
52

• QUESTIONS?