WHAT IF LD IDENTIFICATION REFLECTED RESEARCH RESULTS

1
WHAT IF LD IDENTIFICATION REFLECTED RESEARCH
RESULTS?

• Daniel J. Reschly
• Vanderbilt University
• dan.reschly_at_vanderbilt.edu

Responsiveness-to-Intervention Symposium December
4-5, 2003 Kansas City, Missouri The National
Research Center on Learning Disabilities, a
collaborative project of staff at Vanderbilt
University and the University of Kansas,
sponsored this two-day symposium focusing on
responsiveness-to-intervention (RTI) issues. The
symposium was made possible by the support of the
U.S. Department of Education Office of Special
Education Programs. Renee Bradley, Project
Officer. Opinions expressed herein are those of
the authors and do not necessarily represent the
position of the U.S. Department of
Education. When citing materials presented
during the symposium, please use the following
Reschly, D. J. (2003, December). What if LD
identification reflected research results? Paper
presented at the National Research Center on
Learning Disabilities Responsiveness-to-Interventi
on Symposium, Kansas City, MO.
2
Consequences of RTI and Elimination of Severe
Discrepancy and Psychological Process in LD

• Overview
• Consider briefly changes and rationale
• Identify alternatives briefly
• Cite RTI identification results
• Estimate consequences
• Answer questions from symposium organizers

3
LD and Special Education Reality

• All students in sp ed in LD or MMR have
  significant achievement problems, often
  complicated by behaviors that interfere with
  classroom learning and performance
• Based on studies of several thousand randomly
  selected students in AZ, IA, GA, AR, DE, and FL
• Equally true of non-minority and minority
• Do all have real disabilities? Unanswerable
• Reject extreme views of political right and left

4
Criteria for Continuation/Changes in LD
Identification

• Research based decisions
• Outcomes for students-Pragmatic
• LD identification methods that enhance positive
  outcomes preferred over
• LD identification methods that interfere with or
  are irrelevant to positive outcomes
• Using these criteria, LD identification methods
  should change

5
What Changes Using These Criteria

• Disability definition Identifies conceptual
  framework and key dimensions
• Classification criteria Specifies the
  characteristics of those who are and who are not
  eligible
• Healthy diagnostic construct Consistent
• Little or no research support for cognitive
  processing or severe discrepancy
• Implement and evaluate alternatives

6
Processing Disorder Three Uses in LD

• Determine eligibility? (Scatter is normal Base
  rates Unreliability of difference scores Factor
  structures)
• Training Cognitive Processes (Independent of
  Academic Skill Instruction) Equivocal evidence
  limited (if any) transfer
• Matching teaching methodology to cognitive
  strengths-presumes an ATI-no evidence!!

7
ATI Claims Maximum Benefits from Matching
8
Results of ATI Research

• King of England describing his Danish
  brother-in-law There is nothing there.
• Cronbach, (1975). Once we attend to
  interactions, we enter a hall of mirrors that
  extends to infinity. (p. 119)
• Vaughn and Linan-Thompson (2003), There is no
  empirical support for the use of modality-matched
  instruction or learning styles as a means to
  enhance outcomes for students with LD. (p. 142).

9
Severe Discrepancy

• Controversial in mid-1970s
• Problems
• Stability (Reliability of difference scores
  Danielson Bauer, 1978)
• Validity-weak or no relationship to treatment and
  intervention results
• Easily undermined-test shopping
• Causes harm by delaying treatment

10
Multi-Tiered Academic Interventions (Donovan
Cross)

• Academics (Empirically validated instruction)
• General Education All students
• Standard Protocol Treatments Small group
  tutoring (3-4) in general education 20 of
  students at any time
• Problem Solving Individualized interventions in
  general education leading to, in some cases, sp
  ed eligibility 5 of students at any time
• Special education More intense services brought
  to student 12 of students
• Increasing intensity and measurement precision

11
Multi-tiered Behavioral Interventions (Donovan
Cross)

• Behavior-Empirically validated behavior change
  principles
• General Education School wide positive
  discipline
• Standard Protocol Treatments Classroom
  organization and management (interventions as
  needed)
• Problem Solving Targeted individual
  interventions in general education (5 of
  students at any time)
• Special education More intense services brought
  to the students
• Increasing intensity and measurement precision

12
RTI Elements

• Direct measures
• Natural settings
• Focus on interventions and intervention results
  vs. Focus on internal hypothetical constructs
  unrelated to treatment
• Frequent measurement with formative evaluation
  and intervention changes
• Outcomes criteria guide decisions from screening
  through exit from special education

13
Questions

• Does RTI find the right kids?
• Traditional System referrals, 90 are given
  pre-placement evaluations 70 are placed in sp
  ed
• Finding right kids is not difficult!! Doing
  something that changes academic and behavioral
  trajectories is the challenge
• And RTI is as accurate as the traditional system
  using traditional system criteria

14
MSP Sample of RTI Students (N106) (24 not
eligible)
15
MSP Sample of Traditional System Students (N56)
(20 not eligible)
16
Iowa Study of Traditional and Alternative
Classification Effects

• Tilly, Upah, Reschly study
• Conducted in late 1980s, prior adoption of the
  Problem Solving System and RTI eligibility
• Purpose To determine the likely effects of
  non-IQ based classification procedures on future
  LD populations
• Studied referrals, eligible for LD and not
  eligible for LD (or sp ed)

17
Characteristics of the Sample

• Students from 6 AEAs and Des Moines
• 159 Students Total 148 Usable Cases
• Grade Placement Number of Students
• Grade 2 41
• Grade 3 61
• Grade 4 46

18
Methodology

• All students were referred primarily for
  academics and all had reading problems
• 109 Referred, Tested, and Placed in SPED
• 50 Referred, Tested, and Not Placed in SPED
• Two systems of identification were used with
  these students
• Traditional IQ-Ach discrepancy using a
  regression-based method, state tables showing
  critical values for specific pairs of tests
• CBM criterion only, two times discrepant from
  peers

19
Results

• 109 of 159 referrals were diagnosed LD using
  traditional procedures
• 69 of those referred were placed in sp ed
• 80 of new LD placements met official IQ-Ach
  discrepancy requirements
• 80 of 109 students placed in LD met the CBM two
  times discrepant criterion

20
Implications

• Traditional system decisions have relatively high
  error rates, 1 in 5 to as high as 50
• Alternative criteria are as accurate AND have
  other benefits e. g., relevance to
  intervention, etc.

21
Professional Roles

• Roles change (see slides that follow re school
  psychologists in Iowa
• Continuing education needs
• Manualized assessments and treatments in direct
  assessment (CBM and CBE), problem solving,
  standard protocol treatments, and special
  education interventions

22
Current Roles of School Psychologists in the U.S.
and Iowa
22.6
Estimated Hours Per Week
14.6
12.2
9.2
7.3
6.6
3.6
2.6
1.0 0.8
Direct Intervention
Problem Solving Consultation
Research/ Evaluation
Systems Organizational Consultation
Assessment
School Psychology Role
23
School Psychology Assessment in Traditional and
Alternative Delivery Systems
28.69
20.44
18.16
17.59
Times Per Month
12.89 12.30
10.64
10.49
7.11
1.76 0.81
0.44
0.00
0.04
Ability Educational Behavior Projectives
V-M Pre-Sch Social/ Observation Fam
Emotional A.B
24
Assessment of Educational Skills U.S. and Iowa
12.12
4.04
3.51
2.88
1.49
1.43
1.61
0.69 0.00
0.68 0.03
0.52
0.0
0.42 0.03
0.00
0.00
0.03
K-TEA Key-Math PIAT WRMT
WRAT CBM/CBE W-J ACH
WIAT Other
25
School Psychologists Job Satisfaction in the
U.S. and Iowa
4.64
4.01 4.13
3.81
3.54
High Job Satisfaction Low
3.34
3.03 2.95
2.95
2.29
Job Satisfaction Dimension
26
Does Use of Local Norms Produce Non-comparable LD
Groups Across School Districts?

• Local norms are used informally in current
  system Determines who is referred
• Peterson Shinn Suburban vs St. Paul
• Gottlieb et al. Suburban vs New York City
• LD currently is low achievement relative to a
  peer group
• LD groups vary significantly now across LEAs
  within the same state

27
Will LD Criteria Become More Variable Across
States?

• SEAs LD criteria are highly variable now
• 48/50 require severe discrepancy
• 20/50 do not provide guidance about the magnitude
  of the discrepancy
• Size of discrepancy required varies for 15 to 30
  points
• Correlation between prevalence and size of
  discrepancy is statistically not significant

28
LD Prevalence? (KY 2.96 to RI 9.46) (Factor of
3 times)
29
RTI Implementation Issues

• RTI-implementation problems solutions
• Manualized problem solving protocols
• Manualized treatment protocols
• Manualized direct assessment procedures
• Technology-based supports for formative
  evaluation
• Compliance monitoring protocols
• RTI can be implemented successfully

30
Traditional System Implementation Issues

• Traditional system-implementation problems exist
• High proportions of ineligible students
  classified as LD
• Few links between classification and treatment
• Treatment often does not implement
  scientifically-based principles
• So why change?

31
So Why Change? Identification Methods Compared
32
LD Identification in Middle and High School

• Same procedures
• Large difference(s) from peers
• Insufficient response to high quality
  interventions
• Documented adverse effect
• Documented need for special education
• For example Reading fluency problem

33
LD Differentiated From Other Disabilities?

• Is this important for students?
• NRC Report on overrepresentation
• PCESE-Combined high incidence disabilities
• To some, LD differentiation is very important
• Screen for MR and other disabilities
• New AAMR-MR criteria-adaptive behavior
• LD Researchers screen for MR (Bradley et al.,
  2002)

34
LD and Average Ability

• Assumption of average or normal ability
• IQ above MR level?
• IQ in average range of 90-110 or above?
• Studies of LD populations
• Mean IQ is 2/3 SD to 1 SD below population
  averages, that is, mid to high 80s
• Stable result since mid-1970s
• RTI unlikely to do anything to average ability
  assumptions

35
Unexpected/Unexplained Low Achievement Assumption?

• Problems with IQ to set expectations
• Alternatives to IQ
• Unexpected low learning rate with high quality
  interventions
• Unexpected low performance across academic areas
• Unexpected low performance in relation to peers
• Unexpected low performance in view of other
  explanations being excluded (sensory, MR, ED, OTI)

36
Is It Premature to Abandon Process and Severe
Discrepancy

• Blunt answer
• It is never premature to stop causing harm
• Must change at least in early grades
• Right time for systematic, planned implementation
  of RTI
• Not all states are interested or ready
• Move toward national RTI implementation over
  period of years, pending evaluation of
  implementation and child outcomes

37
Will LD Diagnostic Construct Survive? Will Sp Ed
Survive?

• Largest threats to both are undocumented positive
  effects
• LD and special education will flourish IF we
  produce and document positive outcomes, BUT
• Death grip on process and severe discrepancy may
  be death of both LD and special education as we
  know it
• RTI is best answer to LD identification