Validating SelfReported Education: Results of a Pilot Study

1

• Validating Self-Reported Education Results of a
  Pilot Study
• Jesse Rothstein
• Cecilia Rouse
• Ashley Miller

2
How many high school graduates are there?

• Current Population Survey
• Gives status completion rate of 25-year-old
  graduates / 25-year-old population
• Common Core of Data
• Gives school-based completion rate of
  graduates in year t / of 9th graders in t-3.
• CPS estimates much higher than CCD estimates
• 2000/2 cohorts 84 CPS vs. 70 CCD

3
CPS vs. CCD-based graduation rates
4
Two easy issues

• HS diploma attainment rises with age in CPS
• Delayed completion?
• Adult education / GEDs?
• Misreporting?
• 9th grade bulge
• CCD estimates usually use 9th grade enrollment as
  denominator
• Many students are held back in 9th grade
• 9th grade enrollment consistently about 7-10
  higher than 8th grade enrollment through 1990s
• These reduce the problem, but dont eliminate it.

5
CPS vs. CCD-based graduation rates
6
Not very sensitive to definitions

• CCD-based estimates all tend to yield similar
  answers.
• Mishel and Roy (2006) Get same answers with
  CPS, Census, NELS, NLSY.

7
Alternative graduation rates, existing data
8
Not very sensitive to definitions

• CCD-based estimates all tend to yield similar
  answers.
• Mishel and Roy (2006) Get same answers with
  CPS, Census, NELS, NLSY.
• Except!
• ? Some evidence that GEDs can account for a lot
  of the discrepancy (Heckman and LaFontaine 2007)

9
Alternative graduation rates, existing data
10
Potential explanations

• Attainment increase with age is poorly understood
• Proxy responses in CPS may overstate graduation
• Respondents may overstate own graduation
• Some college may not have graduated HS
• Regular diplomas vs. GEDs, certificates of
  attendance, adult ed., etc.
• Public vs. private schools
• Immigrants/emigrants since HS
• Grade retention
• Divergent definitions in CCD
• CPS excludes armed forces, prisoners
• Poor coverage in CPS

11
Potential explanations that we will address

• Attainment increase with age is poorly understood
• Proxy responses in CPS may overstate graduation
• Respondents may overstate own graduation
• Some college may not have graduated HS
• Regular diplomas vs. GEDs, certificates of
  attendance, adult ed., etc.
• Public vs. private schools
• Immigrants/emigrants since HS
• Grade retention
• Divergent definitions in CCD
• CPS excludes armed forces, prisoners
• Poor coverage in CPS

12
Our Sample.

• Sample frame Households likely to have an 18-25
  year old member (complied by a commercial
  vendor).
• We purchased 1,000 (randomly selected) phone
  numbers.
• Our target was 50 completed inteviews.

13
Survey implementation consisted of three steps.
Step 1 Mimic the CPS

• Conduct a CPS-style interview with a HH member
  who was at least 18 years old (the household
  respondent).
• Note We ended up with 2 versions (A and B)
  where version A asked for a full HH roster
  while version B asked initially if an 18-25
  year old lived in the HH only asked questions
  about one 18-25 year old.
• Probe more on education questions, particularly
  regarding alternative types of high school
  degrees.
• Ask for the name and location of the last high
  school attended by the young adult, as well as
  year of high school graduation.

14
Step 2 Verify Proxy Report and Respondent
Understanding of CPS-style Education Question

• Ask similar CPS-style education questions of
  the young adult him or herself (initial
  report).
• Probe further on education questions (final
  report).
• Also ask for the name and location of last high
  school attended as well as year of high school
  graduation.

15
Step 3 Verify Proxy and Self-reports with
Administrative Data

• Send a letter to the high school requesting
  verification of reported high school graduation
  information.
• Follow-up with a telephone call for a subset of
  schools that did not respond within 10-12 days.

16
Table 1 Sample sizes (Part I)
17
Table 1 Sample sizes (Part II)
18
Table 1 Sample sizes (Part III)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
Conclusion

• We find a high level of agreement between proxy
  respondents, the young adults themselves and high
  school administrative records on whether
  individual has a high school diploma (or
  equivalent).
• We see less agreement between proxy respondents
  and young adults on college attendance.
• Questions about modes of high school completion
  should be asked of those with a college education
  as well as those with only a high school
  education.