Research on Test Accommodations

1
Research on Test Accommodations

• Lawrence Lewandowski, Ph.D.
• Syracuse University
• Paper presented at
• NYSDC conference LeMoyne College
• 11-1-07

2
Special Thanks

• Cassie Berger, M.S.
• Dr. Robin Codding
• Rebecca Gathje, M.S.
• Dr. Michael Gordon
• Dr. Ava Kleinmann
• Dr. Ben Lovett
• Dr. Rosanne Parolin

3
Dr. Miller, may I be excused my brain is full.
4
Test Accommodations

• Were all different, but some have disabilities
• The law allows for equal access for persons
  with a disability
• Were obligated to make accommodations to afford
  equal access, not to maximize potential or
  guarantee outcome
• Accommodations should circumvent or mitigate the
  specific impairment (i.e., blindness) so as to
  provide equal access
• Accommodations should only benefit the persons
  with disability should improve measurement of
  skills

5
The Need for Accommodations
The physical/sensory disability
model Qualitatively different needs Clear
inability to access tests Very little overlap
with nondisabled individuals
6
The Long and Winding Road

• Once upon a time I was a psychologist performing
  diagnostic evaluations for LD, ADHD, etc.
• In 1994 I was asked by the National Board of
  Medical Examiners to consult on ADA requests for
  test accommodations
• I was faced with all kinds of new issues
  questions
• How fast do adults read? Are unimpaired people
  disabled? Is extended time a valid accommodation?
  How much time is reasonable? What about laptops?
• I found little research to guide decision making,
  so I decided to do the research

7
Research Context

• Students with LD ADHD frequently seek/receive
  test accommodations
• Extended time (ET) is most common accommodation
• Various studies on LD show mixed findings most
  show that ET helps all students to a degree, if
  the test involves speed and you prevent ceiling
  effects
• ETS shows that most extended time is not used
• Applicants frequently say that they are slow
  readers, take more time to finish, work harder
  than others to succeed, need an accommodation to
  reach their potential

8
What do most college students think about their
abilities?

• We conducted a survey of SU students
• We listed all ADHD symptoms and common LD
  complaints then asked students to rate those
  items that pertained to them
• Students with disabilities endorsed more ADHD
  items (9 of 18), however typical students also
  endorsed an average of 4.5 items
• 48.8 of nondisabled have to reread to understand
  text 42.6 work harder than others for grades
  51 fidget and squirm in seat 50.4 are easily
  distracted
• So symptom complaints may be sensitive, but they
  are not specific to disability

9
(No Transcript)
10
Everyone claims to be a slow reader

• How does one know s/he reads slower than everyone
  else?
• Clinicians vary widely in what they think normal
  reading speed is (200, 300, 400 wpm)
• We tested 90 college students on NDRT, PSI, CBM
• Average reading speed was _at_189 wpm
• NDRT Reading Rate (.22) did not predict
  Comprehension but WJ III Reading Fluency score
  did (.49)
• WCPM reading was most reliable and predictive
  score

11
How important is speed?
12
Relationship of reading measures and processing
speed

• 125 college students given NDRT, WAIS PSI, WJ III
  PSC, WJ RF, SEPTAR, speed tasks
• ND RR poor predictor of anything
• Speed measures mildly related to ND Comp
  (.14-.26)
• Reading Fluency correlated moderately with PSI
  PSC (.53 .51)
• Best predictors of ND Comp is RF (.45) and Self
  Perception of Testing and Reading (.39)

13
Sup with Processing Speed

• There is no such thing as a Processing Speed
  Disorder
• No research that shows PS to be sensitive and
  specific to a particular disability
• PS as we measure it rote visual motor speed
  tasks
• PS is not a good predictor of outcome measures
  like NDRT Comp and probably any high stakes test
• No research yet on PS and need for test
  accommodations
• Clinicians might want to be careful making
  inferences about PS.

14
Extended Time
15
Validity of Extended Time
Maximum potential Differential boost
Interaction hypothesis
16
Study of Extended Time on Math Performance for
Children w/wo ADHD

• If we create a speeded task
• Use math 3-digit addition that all can do
• Provide unlimited test items to avoid ceiling
  effects
• How would students differentially perform given a
  standard time and then time and one-half?

17
Participants

• 27 per group
• All 5th-7th grades
• 10-13 yrs M 11yrs
• Control 15 males, 12 females
• ADHD 18 males, 9 females
• 20 Combined ADHD 7 Inattentive ADHD
• 21 of ADHD participants were taking medication

18
Sample of Math Calculation Test(MCT) and Other
Measures

• 156 772 588
• 978 664 613
• WJ III Math Fluency
• WISC IV Processing Speed Index
• Behavior Rating Inventory of Executive Function
  (BRIEF)
• ADHD Symptom Rating Scale

19
Group Comparisons

• ADHD
• Math Fluency 92.6
• Processing Speed 96.8
• Exec Function 69.9
• DSM IV Symptoms 14.2
• All different at least p lt .05

• Controls
• 102.1
• 105.0
• 42.7
• 3.0

20
Math Performance at Standard and Extended (1.5x)
Time
21
Math Performance Items Correct
22
Study Conclusions

• We did not find a differential boost for ADHD
  with extended timein fact, control participants
  benefited more from the accommodation.
• ADHD participants attempted fewer MCT items, made
  more errors and got fewer correct.
• ADHD had poorer math fluency, processing speed
  and executive functioning.
• ADHD performance was stable over time and was
  relatively accurate (90).
• Equal outcomes could be engineered on this test
  by giving ADHD students 50 extended time.

23
Implications

• Speeded tests and time accommodations favor the
  nondisabled.
• ET is not an accommodation specific to those with
  a disability.
• We can set a test time or extend the time to
  achieve any end result we desire.
• Perhaps we need to rethink our love affair with
  timed, high stakes tests instead of guessing who
  gets what amount of extra time.

24
RD Study
25
Extended Time on a Reading Comprehension Test for
Adolescents w/wo Reading Disabilities

• 32 RD and 32 Controls from high school
• Given IQ, WJ III RF, and NDRT Comp at standard
  and time and one-half
• Examined items attempted, correct, and correct

26
Measure Control RD ----------------------
------------------------------------------- Raven
IQ 98.76 97.8 Reading Fluency 107.1
88.88 Reading Comprehension 13 Min
Correct 17.72 7.0 19.5 Min
Correct 26.91 12.38 13 Min Attempt 22.88 12
.0 19.5 Min Attempt 35.16 21.25 13 Min
Correct 77.48 59.2 19.5 Min
Correct 76.72 58.37 --------------------------
---------------------------------------
27
Study Conclusions

• Groups were similar in IQ but not Reading
• RD group performed fewer items, got less correct,
  and had less accuracy (77 vs. 59)
• Both groups improved significantly, yet Controls
  improved more
• RD at extended time attempted 21.3 items to
  Controls at standard time (22.9 items)
• RD at extended time got 12.4 correct, whereas
  Controls at standard time got 17.7 correct

28
Implications

• Extended time is not a specific accommodation
• Everyone will benefit from ET on a speeded test
• ET can be used to equate work output ( of items
  attempted) but it wont change skill accuracy.
• The RD group appeared to be truly impaired and
  might need ET to get a better measure of their
  learning

29
Future Work

• Evaluate predictive validity of extended time
• Examine the effects of standard, time and
  one-half, and double time
• Compare handwritten vs laptop essays w/wo LD
• Study the utility of a private testing room
• Develop a reading and test taking profiling
  system for 16-30 year olds

30
Handwritten vs. Laptop Essays

• 140 students assigned to 4 essay writing groups
• HW 10 min, HW 15, Laptop 10, Laptop 15
• Scored for length, TOWL scores, writing speed,
  and WJ Writing Fluency
• No differences on WF, speed, or quality
• Difference in of words per story
• HW 10 215, HW 15 216, Laptop 10 261, Laptop
  15 356
• Laptop serves as a time saver for most, adding
  extended time has a multiplicative effect

31
For Additional Information

• Feel free to call 315-443-1015
• or write to ljlewand_at_syr.edu
• Thanks for your attention!