Londons Population Growth

1
Data deliveryEileen Howes10 April 2006
2
Summary

• What we wanted
• What we got
• What we want from 2011 Census

3
What we wanted

• Test data
• Early release of data for pre-processing
• To help with QA as data experts
• CSV files
• Data that was correct first time
• Table layouts as agreed in advance

4
Test data

• Got some eventually
• Had to fight for it

5
Early release for pre-processing

• No
• Got the Supertable data on publication date
• Had to wait a bit longer for CSV files
• Thousands of users had to wait a lot longer

6
User help with QA

• No
• So found the errors as soon as we loaded the data

7
CSV files

• Some Supertable
• Some CSV files
• Some Excel spreadsheets

8
Data that was right first time

• Not always but a lot to ask
• Expert users would have found some of the errors
  but not all

9
Table layouts as agreed in advance

• No
• Some were as published
• Others not known until the data arrived
• So even more delays in processing
• Some were different for different areas

10
A few points

• Almost all the right numbers
• Not necessarily in the right order
• Some re-releases of data extra work but
  generally OK
• Re-releases of just the cells that were wrong
  please, NO

11
A few more points

• Commissioned tables
• Some files with no area codes
• Because we didnt specify them
• But we expected them to be there

12
Nightmare at City Hall

• Commissioned table C0310 in Supertable
• Output area of workplace to ward of residence
• Part of SWS Table 301, means of travel to work
• ONS offered to split large file for us

13
Nightmare at City Hall

• To export csv files from Supertable
• Tried to export csv but computer crashed
• So ran it until it crashed, retrieved it it parts
• Started next run from where it crashed

14
Nightmare at City Hall

• CSV file with 15,200,000 records
• Then had to add area codes
• Area names already there

15
And another thing

• Area names were not always there
• But if the names were not there
• Then the codes were
• Extra work adding them back in
• We need both

16
What we want from 2011 Census

• Standard datasets
• Nationally comparable data
• Test data
• Early release of data for pre-processing
• To help with QA as data experts
• CSV files
• Data that is correct first time
• Table layouts as agreed in advance

17
What we want from 2011 Census

• ONS to ditch Supertable
• Use something that produces usable csv files
• Is easy to use
• Is easy to print from

18
What we want from 2011 Census

• Basic standards ALWAYS adhered to
• Unique area codes always included
• Area names always included
• All numbers in the same order for different areas
• Agreed table layouts not changed at the last
  minute

19
What we want from 2011 Census

• Better method of disclosure control
• Rounding only if pre-tabulation
• Consistent database
• Data for administrative areas wards and
  parishes
• etc