Consistency of Grading Standards

1
Consistency of Grading Standards

• Steve Aldington
• GHNHSFT Cheltenham
• Imperial College London

2
Early evidence from QA process

• First Annual Reports and EQA visits
• Excellent programme development
• Highly motivated skilled staff
• Increasingly high coverage
• BUT
• Non-standard classes/groups
• Wide variety of data forms
• Wide variety of interpretations

3
Minimum standard dataset

• Retinopathy The R level
• R0 no diabetic retinopathy detected
• R1 presence of any DR but not R2 or R3
• R2 presence of IRMA, VB, loops, multiple deep
  round haemorrhages (CWS)
• R3 presence of proliferative DR
• Every record must map to one of these 4
• OR is Unclassifiable / Unobtainable / Ungradeable

4
Minimum standard dataset (cont..)

• Maculopathy The M level
• M0 no referable maculopathy detected
• M1 presence of referable maculopathy
• Exudate within 1DD of centre of fovea
• Circinate or group of HE within the macula
• Thickening within 1DD of centre of fovea
• MA or haem within 1DD of centre of fovea ONLY if
  associated with a best VA of lt 6/12 (0.3)
• Every record must map to one of these 2

5
Within 1DD of the centre of the fovea?
6
Minimum standard dataset (cont..)

• Photocoagulation The P level
• P0 no scars of photocoagulation detected
• P1 presence of photocoagulation scars
• Evidence of focal/grid laser to macula
• Evidence of peripheral scatter laser
• Every record must map to one of these 2

7
Valid combinations of R, M P
R0 M1 is NOT valid!
8
Inconsistency examples

• R0 M1 (i.e. maculopathy but no DR)
• M1a (i.e. non-referable maculopathy)
• M1 / M2 (i.e. non-/referable maculopathy)
• M1nr (i.e. non-referable maculopathy)
• R2 by CWS (i.e. gt4 cotton wool spots)
• Microaneurysms but R0 (i.e. just a few MA)
• .

R0 M0 or R1 M1 M0 M0 / M1 M0 R1 (or R0?) R1
These are NOT sub-sets of the minimum standard
dataset
9
Inconsistencies and oddities

• Use of non-standard data groups makes
• baseline data impossible to interpret
• between-programme comparisons impossible
• retinopathy prevalence/incidence inaccurate
• referral characteristics/proportions inconsistent
• reduces overall sample size of data groups
• lessens potential for good evidence base
• It is your programme which may lose out

10
Inconsistent forms and data

• Many different data recording forms are used
  even from the same software house
• Not all graders within a programme are
  interpreting the form(s) in the same way
• Not all graders/programmes are detecting or
  classifying lesions in the same way
• Definition of retinopathy varies
• Referral characteristics vary
• Interpretation of Unclassifiable varies

11
(not) The data Police

• Wide variation in many aspects
• Few variations can be explained by patient
  demographic differences
• The National Programme Team are not saying they
  are always correct
• We need as much consistent and common evidence as
  possible to support changes for the future

12
Some crucial final questions

• Would a patient seen by Programme A receive
  exactly the same result/outcome if seen by
  Programme B?
• Do we need to develop a minimum standard
  English National grading form?
• Do we need to develop an enriched form to meet
  local needs whilst remaining common?
• Is there a need for centralised education?
• How do we apply the lessons from EQA?

13
Thank you for your very kind attention