Dorothy V' M' Bishop

1
How to choose between diagnostic
tools?Discussion a plea for efficiency

• Dorothy V. M. Bishop
• University of Oxford
• http//psyweb.psy.ox.ac.uk/oscci/

2
Purpose of diagnosis

• In clinical contexts differential diagnosis,
  establish intervention targets, identify possible
  causal factors, assess suitability for
  educational placement, etc.
• In research
• a) to confirm that participant meets standard
  criteria for comparability with other studies
• and/or
• b) to quantify areas of impairment

3
Clinical context

• Requirement for comprehensive coverage of
  symptoms
• ADI, DISCO, 3Di all seem well-suited
• Need for direct observation
• ADOS only contender.
• Inherent problems
• Process of assessment may affect childs
  behaviour
• Limited assessment setting may give atypical
  picture
• Self-report for adults
• Possible problems with literacy/insight
• But AQ and EQ from AAA seem to work well
  

4
Research context

• NIH national database for autism research regards
  ADI-R and ADOS-G as essential tools
• Goal to help accelerate scientific discovery
• But are these the best tools to meet that goal?

5
Postgrad project on autism

• Comparison of 30 children with autism and 30
  IQ-matched controls
• Experimental tasks of executive function
• Also assessment of IQ and language level
• Time per child for travel/testing/scoring 4 hr
• Total hours 4 x 60 240 hr
• Achievable in 12 weeks (20 hr per week)

6
Additional time for ADI-R

• Travel to see parents, 1 hr (low estimate!)
• ADI-R administration, 3 hr
• ADI-R scoring, 20 mins
• Extra hours 4.33 x 60 260
• Equates to additional 13 weeks
• Likely underestimate because
• a) Harder to schedule seeing parents than
  children
• b) Longer scoring time for inexperienced person

7
Additional time for ADOS-G

• ADOS-G administration, 45 mins
• ADOS-G scoring, 15 mins
• Extra hours 1 x 60 60 hr
• Equates to additional 3 weeks

8
Acceleration of scientific discovery?

• Duration of study goes from 12 weeks to 28 weeks
  more than 2-fold increase
• But this is without including time for training
  on ADI-R and ADOS-G!
• Estimated time to find a course 6 months
• Estimated time to complete each course, 1 week
• Estimated time from course to validation 6
  months
• Study that was achievable in 3 months now
  estimated to take minimum of 1 yr 6 mo.
• This excludes time for consensus coding

9
Matson et al, 2007

• Some measures emphasize the fact that they are
  very detailed. We would argue that detail equals
  time. From a pragmatic perspective, our view is
  that a major priority should be to develop the
  balance between obtaining relevant information to
  make a diagnosis, while parsing out items that do
  not enhance that goal. (p. 49)

10
Is a 2-stage procedure viable?

• Many cases of autism are clearcut, and should be
  readily identifiable on a brief assessment
• Evidence that Social Communication Questionnaire
  (SCQ) as effective as ADI-R in categorising such
  cases
• STEP ONE use brief instrument to select cases
  all would agree on
• Should be sufficient for many studies

11
Is a 2-stage procedure viable?

• For more marginal cases, will we ever reach
  agreement?
• No evidence from studies to date may be because
  autism is inherently dimensional (on multiple
  dimensions) categorical divide wont work
• Suggests STEP TWO should be measurement of the
  dimensions of interest quantitatively as in
  DISCO/3di

12
Recommendation for evaluation of new research
instruments

• EFFICIENCY IS IMPORTANT!
• As well as reporting data on reliability and
  validity, authors should be required to identify
  the minimum set of items that can be used without
  losing sensitivity and specificity
• Dont assume interview is better than
  questionnaire do an empirical test!

13
Questions for discussion

• Why does autism diagnosis take so much longer
  than diagnosis of ADHD, dyslexia or SLI?
• Advantages/disadvantages of direct assessment of
  child vs. parental report vs. self-report
• What are positive reasons and/or problems for
  retaining current focus on ADI-R and ADOS-G?
• When is a dimensional approach preferable?
• Gold standard is expert clinical judgement
  legitimate? What about statistically defined
  categories?
• Do we need more normative data?