Initial validation of the Schedule of Growing Skills (SGS)

1
Initial validation of the Schedule of Growing
Skills (SGS)

• Margiad Elen Williams
• Bangor University

2
Content

• Background
• Validation process
• Step one
• Step two
• Conclusions

3
Background

• Screening tools are used to identify children
  with possible developmental delay to enable
  subsequent more rigorous assessment.
• Quick, inexpensive, and easy to use.
• Should be as accurate as possible.

4
American Academy for Pediatrics (2006)

• Published recommended psychometric criteria that
  all screening tools should meet.
• Sensitivity proportion of correctly identified
  children in need of further assessment.
• Specificity proportion of correctly identified
  children who are developing typically.
• Both need to be at least 70.

5
The SGS in Wales

• Welsh Assembly Government introduced Flying Start
  (FS) Initiative.
• SGS chosen as the developmental screening tool to
  evaluate FS Initiative.
• Problems with scoring identified during IY
  Toddler trial (Hutchings et al., 2011).

6
Problems with SGS scoring

• Windows of assessment vary.
• Score highest item within scale regardless of
  performance on other items.
• Cannot compare between groups or across time.
• Problems can be solved by developing way of
  scoring to yield a Developmental Quotient (DQ)
  score.

7
SGS Profile Form
8
Aims

• To validate both the original and new DQ way of
  scoring the SGS.
• Two step validation process.
• Use of two data sets, the RCT of the IY Toddler
  programme and MRes project comparing the SGS and
  GMDS.

9
Step one
Step two
Aim 1 Estimate appropriate cut-off for new SGS
scoring method Aim 2 Determine concurrent
validity of both SGS scoring methods against GMDS
Aim 1 Determine concurrent validity of both SGS
scoring methods against Ages and Stages
Questionnaire (ASQ)
10
Step one Sample Measures

• Participants
• 39 children
• Mean age 31 months
• 61 male

• Measures
• Griffiths Mental Development Scales (GMDS)
• Schedule of Growing Skills (SGS)

11
Step two Sample Measures

• Participants
• 94 children
• Mean age 22 months
• 61 male

• Measures
• Ages and Stages Questionnaire (ASQ)
• Schedule of Growing Skills (SGS)

12
Subscale comparisons
Griffiths Mental Development Scales (GMDS) Schedule of Growing Skills (SGS) Ages and Stages Questionnaire (ASQ)
Locomotor Gross motor Gross motor
Performance Eye-Hand Coordination (fine motor) Manipulative Visual (fine motor) Fine motor
Language Hearing, Speech, Language Communication
Personal-Social Interactive Self-care
13
Results Step one

• Aim 1 Establishing cut-off point

• Receiver Operating Characteristic (ROC) Curves
• Explored three potential cut-off points
• - DQ lt 90
• - DQ lt 85
• - DQ lt 80

14
Results Step one
Aim 1 Establishing cut-off point
SGS cut-off AUC Sensitivity Specificity
DQ lt 90 .794 90.83 67.88
DQ lt 85 .779 74.18 81.55
DQ lt 80 .789 65.83 91.90

• Most accurate cut-off is DQ lt 85.
• Both sensitivity and specificity levels gt 70

15
Results Step one

• Aim 2 Concurrent validity with GMDS

• Calculated
• - Sensitivity
• - Specificity
• - Over-referral rates
• - Under-referral rates

16
Results Step one

• Aim 2 Concurrent validity with GMDS

Development area SGS scoring Sensitivity () Specificity () Over-referrals () Under-referrals ()
Locomotor Original 16.67 100 0 12.80
New (DQ lt 85) 83.33 51.52 41.02 2.56
Personal-Social Original 0 100 0 7.69
New (DQ lt 85) 66.67 100 0 2.56
Language Original 20 100 0 10.26
New (DQ lt 85) 80 94.12 5.13 2.56
Fine motor Original 0 100 0 7.69
New (DQ lt 85) 66.67 80.56 17.95 2.56
Overall Original 9.17 100 0 9.61
New (DQ lt 85) 74.17 81.55 16.03 2.56
17
Results Step two

• Aim 1 Concurrent validity with ASQ

• Calculated
• - Sensitivity
• - Specificity
• - Over-referral rates
• - Under-referral rates

18
Results Step two

• Aim 1 Concurrent validity with ASQ

Development area SGS scoring Sensitivity () Specificity () Over-referrals () Under-referrals ()
Locomotor Original 10 98.81 1.06 9.57
New (DQ lt 85) 70 92.86 6.38 3.19
Language Original 50 93.02 6.38 4.26
New (DQ lt 85) 75 83.72 14.89 2.13
Fine motor Original 26.67 92.41 6.38 11.70
New (DQ lt 85) 66.67 77.22 19.15 5.32
Overall Original 28.89 94.75 4.61 8.51
New (DQ lt 85) 70.56 84.60 13.47 3.55
19
Discussion 1

• New SGS scoring method shows increased concurrent
  validity.
• Better sensitivity, comparable specificity,
  higher over-referrals, lower under-referrals.

20
Discussion 2

• Limitations
• Small sample sizes
• GMDS training
• Implications
• Increased detection rates
• Greater use in clinical practice and research

21

• Thank you for listening
• Diolch am wrando
• Any questions??