Revisions for a sceptical audience

1
Revisions for a sceptical audience

• David Marder, ONS, UK

2
George Orwell, Nineteen Eighty-Four

• But actually, he thought, as he
  re-adjusted the Ministry of Plentys figures, it
  was not even forgery. It was merely the
  substitution of one piece of nonsense for another
  . . .

3
The statistical trade-off
Stan the Statistician and his juggling act
4
A PR problem
Financial Times, 3 July 2006
5
Media put the boot in
Stan the Statistician gets a kick from the
sceptical media
6
Public view of official statistics
7
Len Cook

• Users naively expect statistics to remain
  comparable over time in a fast changing world
• In public policy, there is rarely the luxury of
  waiting until the final statistical measure is
  available
• There are major policy risks in delaying key
  decisions because timely information is not
  available
• Challenge for statisticians not only what we
  measure is changing but so are sources, systems
  standards and the methods we use to obtain raw
  data

8
Three types of revisions

• Unplanned and regular for example, regular
  revisions that fall outside the expected range
• Planned and unusual such as the introduction of
  methodological changes
• Unplanned and unusual including errors. These
  inevitably get the media attention

9
Statisticians motto

• Clarity
• Communication

10
National Statistics Revisions Protocol
11
Statement of principle

• Substantial methodological changes will be
  announced before the release of statistics based
  on the new methods

12
Additional principles

• A general statement describing its practice on
  revisions from each department
• Key outputs subject to scheduled revisions will
  have a published policy covering those revisions
• A statement explaining the effect of revisions
  will accompany the release of all key outputs
  subject to scheduled revisions
• Revisions comply with the same principles as
  other new information

13
Additional principles

• Timeliness balanced against the need to avoid
  frequent revisions
• Minimal unexpected revisions but, if they occur,
  they will be released as soon as practicable and
  in an open and transparent manner
• Substantial revisions accompanied by explanation
  of nature and extent
• Long-term effects of revisions on key outputs
  monitored to improve quality

14
Code of Practice Statement on revisions
15
Recognising the challenge

• Communication
• Likelihood
• Reasons
• The expert filter
• City analysts
• Media correspondents

16
Communication innovations

• More information on average past revisions and
  planned future revisions
• Economic statistics forum
• Horizon scanning

17
Other lessons

• Proactive explanation
• Dont oversell
• Think about the language
• New information updates
• Methodological changes improvements
• Errors corrections
• Be less defensive
• Promote statistical literacy

Happy Stan