Large Scale Studies of Dyslexia in Florida

1
Large Scale Studies of Dyslexia in Florida

• Richard K. Wagner and Yusra Ahmed
• Florida State University and FCRR
• NIH Multidisciplinary Learning Disabilities
  Center (P50 HD052120)

2
Alternative Approaches

• 1. Typical research study.
• 2. Meta-analysis.
• 3. Large-scale study.

3
Large-Scale Study of Incidence of Specific
Reading Comprehension Disability
4
Personal Interest in Reading Comprehension
Problems

• Comprehension errors run in my family.

5
Ellis Island
6
Question on a Form

• If admitted to this country, would you advocate
  overthrow of the government of the United States
  by force or violence?

7
Question on a Form

• If admitted to this country, would you advocate
  overthrow of the government of the United States
  by force or violence?
• violence

8
Reported Incidence of Reading Comprehension
Disability

• 10 percent of 7-11 year-olds are poor at reading
  comprehension despite being accurate and fluent
  at decoding (Nation, 2004).

9
Many Possible Causes

• There is room for lots of things to go wrong
  when comprehension fails (Perfetti, 1994, p.
  885, cited by Nation, 2005).

10
Possible Causes (Science of Reading A Handbook)

• 1. Decoding difficulties.
• 2. Difficulties with meaning (vocabulary).
• 3. Difficulty with syntax.
• 4. Limitations in working memory.
• 5. Poor inference making.
• 6. Inadequate comprehension monitoring.
• 7. Limited prior domain knowledge.
• 8. Insensitivity to text structure.

11
Reading Comprehension Task

• When the next slide appears, read the text as
  quickly as you can and summarize the passage in a
  brief sentence.
• Ready?

12
(No Transcript)
13
Answer a Simple Question

• Is necrobiosis a conceivable source of orogeny or
  of a pomiferous pompelmous?

14
Simple Question

• Is necrobiosis conceivably related to orogeny or
  to the development of a pomiferous pompelmous?
• Obviously the normal death of cells (necrobiosis)
  is not related to mountain building (orogeny),
  but is part of the process of the development of
  fruit bearing (pomiferous) trees (pompelmous).

15
Ordering Causes by Severity of Consequences for
Comprehension

• Primary
• Decoding difficulties.
• Secondary
• Difficulties with meaning (vocabulary).
• Tertiary
• Difficulty with syntax limitations in working
  memory poor inference making inadequate
  comprehension monitoring limited prior domain
  knowledge insensitivity to text structure.

16
Present Studies

• Question Addressed
• What is the incidence of reading comprehension
  disability not attributable to primary or
  secondary causes?
• Design feature
• Attempted to address issue of small samples sizes
  of typical studies by using the PMRN (progress
  monitoring and reporting network) database.

17
Study Design

• 1. Identify individuals who are poor at reading
  comprehension.
• Score at or below 5th percentile on Stanford
  Achievement Test (SAT-10) Reading Comprehension.

18
Study Design

• 2. Determine how many individuals are poor at
  comprehension yet adequate at decoding.
• SAT 10 at or below 5th percentile
• DIBELS Nonsense Word Fluency (NWF) greater than
  or equal to 25th percentile.

19
Study Design

• 2. Determine how many individuals are poor at
  comprehension yet adequate at decoding and
  vocabulary.
• SAT 10 at or below 5th percentile
• DIBELS Nonsense Word Fluency (NWF) greater than
  or equal to 25th percentile.
• Peabody Picture Vocabulary Test (PPVT) greater
  than or equal to 25th percentile.

20
Out of a First Grade-Cohort 1 (N 35,314).
21
How Many Were Poor at

• Reading comprehension (SAT 10 lt 5th)
• 1,669 (4.73) out of 35,314

22
How Many Were Poor at

• Reading comprehension (SAT 10 lt 5th )
• 1,669 (4.73) out of 35,314
• yet adequate at decoding (nonword fluency gt
  25th )?

23
How Many Were Poor at

• Reading comprehension (SAT 10 lt 5th )
• 1,669 (4.73) out of 35,314
• yet adequate at decoding (nonword fluency gt
  25th )?
• only 85 (0.24) out of 35,314!

24
How Many Were Poor at

• Reading comprehension (SAT 10 lt 5th )
• 1,669 (4.73) out of 35,314
• yet adequate at decoding (nonword fluency gt
  25th )?
• only 85 (0.24) out of 35,314!
• and in vocabulary (PPVT gt 25th )?

25
How Many Were Poor at

• Reading comprehension (SAT 10 lt 5th )
• 1,669 (4.73) out of 35,314
• yet adequate at decoding (nonword fluency gt
  25th )?
• only 85 (0.24) out of 35,314!
• and in vocabulary (PPVT gt 25th )?
• only 23 (0.07) out of 35,314!

26
Surprising Result Virtually All Had Problems in
Decoding

• Sample size was 35,314.
• But it was a single study. Results need to be
  replicated.

27
So We Did Replicate Not One, Not Two, but
28
Three First-Grade Cohorts
29
First Grade Results

• Of 150,000 6-year-olds, only .2 to .5 percent
  were poor at comprehension yet adequate at
  decoding.
• May be nature of reading comprehension at age
  6decoding explains about everything.

30
Three Second-Grade Cohorts
31
Comparing Second-Grade Results to First-Grade
Results

• Second-grade results differ a little
• Percentage of children who are poor at reading
  comprehension yet adequate at decoding is about 2
  percent, compared to .5 percent in first grade.
• But identical when adequate vocabulary is also
  imposed
• Less than .2 percent for both.

32
What about Third Grade?

• We could not do identical study because we dont
  have nonsense word fluency (NWF) as decoding
  measure for third grade.
• But we do have Gates-McGinnite reading vocabulary
  as a combined measure of decoding and vocabulary.

33
Three Third-Grade Cohorts
34
What About Less Severe Reading Comprehension
Problems

• Select if SAT 10 lt 20th ile.
• Require decoding and vocabulary 1 standard
  deviation higher (56th ile) or just somewhat
  higher (40th ile).
• Representative second-grade results for most
  lenient (40th ile) criterion.

35
Less Severe 2nd Grade Cohorts
36
What These Results Say

• Specific reading comprehension not associated
  with presence of primary (decoding) or secondary
  (vocabulary) causes is exceedingly rare
• Less than 0.1 of first-graders.
• Less than 0.2 of second-graders.
• Less than 0.3 of third-graders.

37
What These Results Dont Say

• Results dont imply that individuals poor at
  reading comprehension dont also show deficits in
  various tertiary factors.

38
What These Results Dont Say

• Results dont imply that individuals poor at
  reading comprehension dont also show deficits in
  various tertiary factors.
• decoding gains from intervention rarely translate
  into equivalent gains in comprehension.

39
Conclusions

• Individuals with tertiary causes (e.g.,
  metacognitive deficiency) in absence of primary
  (decoding) and secondary (vocabulary) causes are
  rare.

40
Conclusions

• For screening purposes, a combination of decoding
  and vocabulary should be remarkably effective.

41
Conclusions

• Its worse not to know the words (primary problem
  in decoding or secondary problem in decoding
  vocabulary) than to not know whether you know the
  words (tertiary problem in metacognition).

42
Gender Differences in Reading Disability
Reasons to Care

• 1. An active and controversial issue.
• 2. Gender bias in identification and provision
  of services may be pervasive.
• 3. Implications for theories of etiology.
• 4. New approaches to identification being
  considered potentially could mitigate referral
  bias if it exists.
• Ex. Universal screening as front end of RTI.

43
Current Controversy Two Views

• 1. Male vulnerability is a myth.
• observed ratios of 21 or 31 in clinics and
  classrooms reflect referral bias.
• true ratio is 11 or boys favored only minimally.

44
Key Study Shaywitz et al. (1990)

• Obtained both school-identified ratio and
  objective ratio for same sample.
• Statistically significant ratio of 2.21 found
  for school identified ratio.
• Non-significant ratio of 1.41 found for
  objective criteria.
• Sample size modest however (18 boys versus 13
  girls with RD).
• A 21 ratio would not even be significant.

45
Current Controversy Two Views

• 2. Male vulnerability is real.
• males roughly twice as likely to be affected.

46
Key Supporting Studies

• Liederman et al., 2005, review of the literature.
• Ratios ranged from 1.21 to 6.81.
• Concluded that true ratio was between 1.71 and
  21.

47
Key Questions

• 1. What is the magnitude of male vulnerability
  for reading disability if it exists?
• Answered by examining gender ratios for
  research-based operational definitions applied
  universally.

48
Key Questions

• 2. How accurately does school-identification
  predict research-based identification?
• Answered by classification analyses that use
  school-identification to predict research-based
  identification.

49
Key Questions

• 3. What is the magnitude of referral bias if it
  exists? Answered by three empirical analyses
• A. Magnitude of difference in gender ratios for
  school-identified versus research identified
  samples.
• B. Less overlap between school- and
  research-identification for boys than for girls.
• C. Lower mean performance for girls compared to
  boys for school-identified samples.

50
Key Questions

• 4. Do gender ratios vary as a function of
• A. Level of severity of reading problem?
• Studies differ on level used.
• B. Whether operational definition is based on
  low-achievement or IQ-achievement discrepancy?
• Some suggestion that gender differences occur for
  IQ-discrepancy but not low-achievement
  definitions.
• C. The kind of reading measure examined?

51
Sample

• Five cohorts of beginning second-grade students
  in Reading First schools in Florida (03/04,
  04/05, 05/06, 06/07, 07/08).
• N 491

52
Sample

• Five cohorts of beginning second-grade students
  in Reading First schools in Florida (03/04,
  04/05, 05/06, 06/07, 07/08).
• N 491 thousand (491,000)!

53
Measures

• Phonological decoding Nonword fluency (DIBELS).
• Reading connected test Oral reading fluency
  (DIBELS).
• IQ proxy Peabody Picture Vocabulary Test

54
Remaining Design Issues

• Compared four levels of severity
• 30th percentile.
• 15th percentile.
• 5th percentile.
• 3rd percentile.

55
Remaining Design Issues

• Low achievement operational definition determined
  by identifying the 4 target percentiles in the
  distributions.
• Discrepancy operational definition determined by
  regression reading measures of PPVT and
  identifying same target percentiles in the
  distributions of residuals.

56
Key Question 1 Male
Vulnerability?
57
Gender Ratios for Research-Based Definitions

• Ratios greater than 11 quantity extent of male
  vulnerability.

58
Gender Ratios Low-Achievement Definition,
Nonword Fluency
59
Gender Ratios Discrepancy Definition, Nonword
Fluency
60
Gender Ratios Low-Achievement Definition, Oral
Reading Fluency
61
Gender Ratios Discrepancy Definition, Oral
Reading Fluency
62
Comparing Low-Achievement and Discrepancy
Definitions Nonword Fluency
63
Comparing Low-Achievement and Discrepancy
Definitions ORF
64
Key Question 2 Accuracy of
School-Based Identification?
65
School-Identification as Learning Disabled

• 5.1 percent of all second-grade students.
• Carried out classification study using
  school-identification to predict membership in
  5th -ile research-based group.

66
Key Classification Statistics

• 1. Sensitivity.
• Proportion of research-based reading disabled
  correctly classified by school-based
  determination.
• 2. Specificity.
• Proportion of research-based reading non-disabled
  correctly classified by school-based
  determination.

67
Key Classification Statistics

• 3. Positive Predictive Value.
• Proportion of school-identified reading disabled
  who actually were according to research-based
  criteria.
• 4. Negative Predictive Value.
• Proportion of school-identified non-reading
  disabled who actually were according to
  research-based criteria.

68
Guidelines for Interpretation of Classification
Statistics

• Level 3 (highest level of support)
• Sensitivity, specificity, and positive predictive
  value gt .75.
• Level 2 (moderate support)
• Sensitivity, specificity, and positive predictive
  value gt .70.
• Level 1 (lowest but still supportive)
• Sensitivity and specificity or sensitivity and
  positive predictive value gt .70.

69
Classification Results
70
Classification Results

• Accuracy of school-based identification poor for
  research-based identification of reading
  disability.
• Accuracy is high for determination of non reading
  disability, but this reflects base rate of 5.

71
Key Question 3 Referral Bias?
72
Comparing Gender Ratios for School- and
Research-Identified Samples

• Extent to which gender ratios for
  school-identified samples exceed those for
  research-identified quantifies extent of
  referral bias.

73
Comparing Gender Ratios for School- and
Research-Identified Samples

• Gender ratio for school-identified sample 2.25
  to 1.
• Exceeds gender ratio for nonword fluency (1.51
  or 1.61 depending on definition, for same level
  of severity).
• Comparable to gender ratio for oral reading
  fluency (2.11 or 2.41).

74
Is Accuracy of School-Identification Higher for
Girls than Boys?

• If boys are referred for behavior and other
  issues and girls for reading problems, accuracy
  of school-based identification should be higher
  for girls than for boys.

75
Is Accuracy of School-Identification Higher for
Girls than Boys?
76
Lower Mean Performance for School-Identified
Girls than Boys?

• If boys are referred for behavior and other
  issues and girls for reading problems, girls
  should be poorer readers than boys in
  school-identified samples.

77
Lower Mean Performance for School-Identified
Girls than Boys?
78
Lower Mean Performance for School-Identified
Girls than Boys?
79
Conclusions

• 1. Male vulnerability is real and quantifiable.
• Increases with level of severity of reading
  problem.
• Greater for broader-based oral reading fluency
  (2.41 at 3rd -ile) than for more narrowly-based
  nonword fluency (1.71 at 3rd -ile).

80
Conclusions

• 2. Accuracy of school-identification is abysmal
  for research-based criteria.
• Sensitivities ranged from .14 to .26 (far below
  minimally acceptable value of .70).

81
Conclusions

• 3. Little support for substantial referral bias.
• Gender ratios for school-identified samples
  greater than research-based for nonword fluency
  but not for oral reading fluency.
• School identification is not more accurate for
  girls than for boys, but just the opposite.
• School-identified girls are not poorer readers
  than boys, but just the opposite.

82
Conclusions

• 4. Gender ratios not greater for discrepancy
  than for low achievement definitions for nonword
  fluency only marginally higher (from 0.21 to
  0.41) higher for oral reading fluency.

83
Questions?