External Quality Assurance

1
ExternalQualityAssurance

• Keith GoatmanUniversity of Aberdeen

2
Outline

• What is EQA?
• Whats it for?
• Whats it not for?
• How did it work?
• How many graders took part in the pilot?
• Whats the answer?
• A final thought Monitor your monitor

3
What is EQA?
4
What is EQA?

• Whats it for?
• Making sure there are no differences between
  centres.
• Whats it not for?
• Its not a replacement for IQA, it doesnt QA
  individuals.

5
How did it work?

• Web-based thin client application like Soarian
  hosted at Aberdeen University.
• Works with a variety of web browsers (Microsoft
  IE6, IE7, Mozilla Firefox, Opera, Apple Safari,
  Google Chrome).

Webserver
6
How did it work?

• Web-based thin client application like Soarian
  hosted at Aberdeen University.
• Works with a variety of web browsers (Microsoft
  IE6, IE7, Mozilla Firefox, Opera, Apple Safari,
  Google Chrome).
• Controls and menus designed to look and behave
  like Soarian.

7
(No Transcript)
8
How did it work?

• Internal QA requires 500 images from each grader
  to be re-graded.
• EQA uses fewer images (100 images from each
  grader) but including a much higher proportion of
  pathology and technical problems.
• Images presented in random order to each grader.

9
How many took part?

• 64 / 87 (74) graders enrolled completed all 100
  images.
• 7333 / 8700 (84) images graded.

10
How many took part?
At 7 out of 10 centres all graders completed the
image set.
11
Whats the answer?
Good agreement (100)
12
Whats the answer?
Poor agreement (equal numbers R0 and R4!?!)
13
Whats the answer?
Levels 1 and 2 90 of L1 and L2 graders agree on
75 of technical failures.
Levels 3 90 of L3 graders agree on 85 of
technical failures.
14
Monitor your monitor

• EQA system stores information about the grading
  display
• Pixel resolution (height x width of screen)?
• Colour resolution (how many colours can screen
  display)?

15
Monitor your monitor

• For 19 displays most common pixel resolution
  either 1280x960 (CRT) or 1280x1024 (flat screen).
• Incorrectly set flat screen displays do not give
  a sharp image.

16
Monitor your monitor

• Colour resolution should be set to highest
  setting (24 or 32 bit colour).
• 16 bit colour is not enough (only gives you 32
  levels of greyscale for red-free image).
• Every display in the EQA was 24 or 32 colour.

17
Monitor your monitor

• Vertical refresh rate (not recorded by the EQA
  system).
• Only an issue for older CRT monitors.
• Often defaults to 60Hz, which results in visible
  flicker.
• Refresh rate ideally between 75Hz and 85Hz.