Improving the quality of PSI

1
Improving the quality of PSI
2
Steven Ramage ePSIplus Thematic Meeting,
London 15th July 2007 steven.ramage_at_1spatial.com
3
AGENDA

• Data Quality
• Introduction
• Focus on spatial data quality
• Aspects of spatial data integrity
• Ideas to consider
• Conclusion

4
ePSIplus Update No 2, May 2007, Enabling PSI
re-use the Need for Standards
The first step is for government bodies
themselves to know what information they have and
to organise it in such a way that the information
can be easily discovered and retrieved. The
second step should be to determine the state of
the information and its fitness for use, i.e. the
data quality elements!
5
Monday 02 July 2007 Quality Data? Christian
Lister
ePSIplus Forum
We have witnessed in the UK, restrictive data
practices due to poor record keeping, data in no
workable format and of exceptional poor quality.
http//www.epsiplus.net/epsiplus/forum__1/epsiplu
s_forum/is_germany_losing_out Experiences
similar with spatial data quality. This
highlights a major cultural issue.
6
Audit Commission Consultation on Data Quality,
April 2007

• Thank you for your email. The consultation was
  supportive of the Commission's propose
• standards. We are making some adjustments in
  response to the comments received, but the
  overall tone and content will remain the same. We
  will not be focusing specifically on spatial
  data,but on data in a more general
• sense.
• Senior Manager, Audit Policy Practice,
• The Audit Commission, July 2007

7
INTRODUCTION

• Data Quality
• Define data quality
• Is it fit for purpose?
• Define stakeholders
• Why be concerned?

8

• 'Re-use' is defined as the use of information
• held by a public sector information holder 'for
• a purpose other than the initial purpose within
• that public sector body's public task for which
• the document was produced' (The Re-use of
• Public Sector Information Regulations 2005,
• SI 2005 No.1515, 4(1), 'the Re-use
• Regulations').

9

• Data Quality
• Determine what already exists
• Very timely ITT/call for tender from European
  Commission
• The assessment of the reuse of public sector
  information (PSI) in the geographical
  (cartographic-mapping and cadastral) information,
  meteorological information and legal information
  sectors.

10
INTRODUCTION

• Data Quality
• Awareness and definitions
• Input, verification, output
• Systems and presentation

11
INTRODUCTION

• Data Quality Management Activities
• Data Augmentation Enhance data using
  information from internal/external data sources
• Data Integration Match, merge or link data
  from a variety of disparate sources

12
INTRODUCTION

• Data Quality Management Activities
• Data Profiling Inspect for errors,
  inconsistencies, redundancy and incomplete
  information
• Data Validation Correct, standardise and
  verify data
• Data Monitoring Check and control data
  integrity over time

13
INTRODUCTION

• Data Quality - vision
• Audit Commission Data Quality Consultation,
  Section 8
• Producing data that is fit for purpose should
  not be an end in itself, but an integral part of
  an organisations operational, performance
  management, and governance arrangements.
  Organisations that put data quality at the heart
  of their performance management systems are most
  likely to be actively managing data in all
  aspects of their day-to-day business, in a way
  that is proportionate to the cost of collection,
  and turning the data into reliable information.
• Audit Commission consultation
• Improving Information to Support Decision Making
  Standards for Better Quality Data, 15 Mar 2007

14
INTRODUCTION

• Data Quality risk management
• Audit Commission Data Quality Consultation,
  Section 10
• The risk in not identifying and addressing
  weaknesses in data quality, or the arrangements
  that underpin data collection and reporting
  activities, is that information may be
  misleading, decision making may be flawed,
  resources may be wasted, poor services may not be
  improved, and policy may be ill-founded. There is
  also a danger that good performance may not be
  recognised and rewarded.
• Audit Commission consultation
• Improving Information to Support Decision Making
  Standards for Better Quality Data, 15 Mar 2007

15
INTRODUCTION

• Data Quality accountability (1)
• Audit Commission Data Quality Consultation,
  Section 6
• Public bodies are accountable for the public
  money they spend they must manage competing
  claims on resources to meet the needs of the
  communities they serve, and plan for the future.
  The financial and performance information they
  use to account for their activities, both
  internally and externally, to their users,
  partners, commissioners, government departments
  and regulators, must be accurate, reliable and
  timely.
• Audit Commission consultation
• Improving Information to Support Decision Making
  Standards for Better Quality Data, 15 Mar 2007

16
INTRODUCTION

• Data Quality accountability (2)
• Audit Commission Data Quality Consultation,
  Section 9
• Ultimate responsibility for ensuring that data
  is fit for purpose can only rest with public
  bodies themselves. This responsibility should not
  be confused with the role of government
  departments in setting a policy framework,
  including defining national performance measures
  and issuing standards and guidelines, or the role
  of regulators in providing assurance and
  identifying improvements.
• Improving Information to Support Decision Making
  Standards for Better Quality Data, 15 Mar 2007

17
INTRODUCTION

• Data Quality fitness for purpose
• Audit Commission Data Quality Consultation,
  Section 11
• There are many audiences for the data collected
  by public services. This in itself can cause
  problems with the reliability of reported
  information, because the need to aggregate and
  analyse raw data in a variety of ways to suit a
  variety of purposes (Table 1) may not be
  understood by all those involved in the data
  collection and reporting processes. Data
  collected for a specific local purpose may
  ultimately be used or reported in ways not
  envisaged, intended or understood by its
  originators.
• Audit Commission consultation
• Improving Information to Support Decision Making
  Standards for Better Quality Data, 15 Mar 2007

18
FOCUS AREA

• Spatial Data in Public Sector
• PIRA report 2000, Euros 36 billion
• 40 of public sector data is geographic
• Replacement cost today, Euros 100 billion
• Everything happens somewhere
• Public Sector Information reuse essential

19
Before
After
20
FOCUS AREA

• Spatial Data in Public Sector
• Consider, as an illustration, a land
  management/property registration application used
  to record ownership rights. In such a system, the
  business rules concerning the spatial data are
  well understood
• every piece of land (parcel) has owners
• land parcels do not overlap
• land parcels do not have gaps between them.
• Generic Concepts Once rules are adopted it
  becomes possible to monitor them and to quantify
  the impact of drift. Importantly, rules state the
  formal set of conditions that should be met
  before data can be said to be fit for purpose.

21
SPATIAL DATA INTEGRITY

• Spatial Data in Public Sector
• Spatial data used for decisions relating to
  education, health, land and property, roads and
  networks, etc.
• Issues associated with updating the real
  world, changes in technology and organisational
  change
• Accurate data for accurate decisions

22
SPATIAL DATA INTEGRITY

• Spatial Data in Public Sector
• There are 410 LAs in England and Wales at the
  County, or District/Borough levels (Unitary
  Authorities are responsible for the duties of
  both). Together they employ over two million
  people and each authority undertakes an estimated
  700 different functions.2 Examples of the types
  of raw information held by LAs include policy and
  strategy documents,details of services, annual
  reports, budget plans, statistics, public
  consultations, meeting minutes, performance data,
  and the location of council buildings, land and
  other assets http//www.oft.gov.uk/shared_oft/repo
  rts/consumer_protection/oft861e.pdf

23
SPATIAL DATA INTEGRITY

• Spatial Data in Public Sector
• Business issues
• cannot share data or sharing poor data
• data must be recaptured frequently
• cleansed frequently
• high offshore and maintenance costs
• analysis of inaccurate information

24
SPATIAL DATA INTEGRITY

• Spatial Data in Public Sector
• Technical issues
• cannot load data into existing software or
  database systems
• once loaded errors can prevent electronic
  processing
• manual intervention takes much longer, diverts
  resources and is subject to visual inspection
  only

25

• Data Quality
• Independent verification important
• use tools or services from 3rd parties
• Quantitative measures necessary
• have a baseline to review
• set the target level
• monitor progress
• build into planning

26
CONSIDERATIONS

• Spatial Data in Public Sector
• Work according to organisational Vision
• Strategic Assessment up front
• Data Audits within Data Management Plan
• Use Information Strategy as guideline
• Decisions based on quantifiable and measurable
  results iterative process
• Future proof or sustain data quality

27
CONSIDERATIONS

• Spatial Data in Public Sector
• Is it quality assessed or measured?
• Do you or can you share spatial data?
• Can you integrate standard business
  information with spatial data?
• How much has been invested in those spatial
  data? What is the required return on that
  investment?

28
KEY LEGISLATION

• Spatial Data in Public Sector
• INSPIRE (Infrastructure for Spatial
  Information in Europe)
• European Spatial Data Infrastructure
• Impacts public sector in Europe
• Limited budget allocated to GI projects in
  eContentplus programme

29
INITIATIVES ACROSS EUROPE

• Spatial Data in Public Sector
• FOT ID, Denmark
• AAA, Germany
• IntesaGIS, Italy
• RGI, Netherlands
• SPIRE, England and Wales

30
CASE STUDIES

• Spatial Data in Public Sector
• AdV, Germany
• City of Amsterdam, Netherlands
• City of Oslo, Norway
• East Sussex County Council, England
• Environment Agency, EnglandWales

31
CASE STUDIES

• Spatial Data in Public Sector
• IGN Belgium
• IGN France
• KMS Denmark
• London Borough of Enfield, England
• Ordnance Survey of Northern Ireland
• Transport for London

32
CASE STUDIES City of Amsterdam
Spatial Data in Public Sector It boils down to
us being able to satisfy our customer's need for
high quality data, and in the same time being
able to deliver and link data conforming national
standards (andtherefore being able to re-use
public service information).
33
CASE STUDIES City of Amsterdam

• Spatial Data in Public Sector
• A government that doesnt ask the same thing
  twice
• is customer oriented
• cannot be misled
• knows its facts
• doesnt spend more than necessary
• Must have access to reliable
• and high quality data

34
CASE STUDIES City of Amsterdam
Civil Registry Office
GEMEENTELIJKE BASISADMINISTRATIE
BASISREGISTRATIE ADRESSEN
GEOGRAFISCH KERNBESTAND
PERSONS
ADDRESSES
Large scale maps Small scale maps Aerial
photographs
Geo en Vastgoedinformatie
Tax department
BASIS BEDRIJVEN REGISTER
BASIS GEBOUWEN REGISTER
KADASTRALE REGISTRATIE
Businesses
RESIDENCE-UNITS BUILDINGS
PARCELS
35
CASE STUDIES City of Amsterdam
Spatial Data in Public Sector It boils down to
us being able to satisfy our customer's need for
high quality data, and in the same time being
able to deliver and link data conforming national
standards (andtherefore being able to re-use
public service information).
36
CASE STUDIES AdV germany

• Spatial Data in Public Sector
• Automated model generalisation for 7 German
  States
• Demonstrated significant savings through a fully
  automated workflow without manual edits
• Spatial data quality played a major role in the
  success of this large project
• Changing data models across Europe their current
  data against that model.

37
CASE STUDIES OSNI GeoHub

• Spatial Data in Public Sector
• Departmental data is usually fit-for-purpose
  within its own setting. Combining data from
  different sources is often the first indication
  of inconsistencies.
• There was no mechanism for testing
  fit-for-purpose of the combined data. Now
  using Radius Studio.
• The overheads in correcting combined data
  (particularly when the original data is fit for
  its original intended purpose) discourage
  official data sharing.
• Data can be created at different
  scales/resolutions but expected to work together.
  Often the end-user has little understanding of
  the accuracy of the data should we expect the
  end user to know this?

38
CASE STUDY LESSONS

• Spatial Data in Public Sector
• Improving spatial data quality to enable PSI
  re-use
• Automation key to success
• Applicable across local, regional/central and
  national government
• Rationalising the supply chain

39
CASE STUDY LESSONS

• Spatial Data in Public Sector
• Points to address
• Fragmentation of datasets and sources
• Gaps in spatial data availability
• Diverse collection and preservation practices
• And lack of harmonisation between datasets at
  different geographical scales

40
CASE STUDY LESSONS

• Spatial Data in Public Sector
• Aggregate core geographic information across
  borders to guarantee its interoperability for
  seamless data integration.
• Easily accessible and re-useable datasets
• Use open, non-proprietary standards (where
  appropriate)
• Assessments of pricing models and their effects
  on re-use
• Consider multilingual access
• Provide performance indicators (specific,
  realistic, measurable)
• Involve relevant stakeholders (including
  co-ordinating bodies)

41
TESTIMONIALS
The open sharing of information and monitoring of
performance are key to the success of Local
Partnerships, LAAs and LSPs.Now for the very
first time, we are going to be talking the same
language but also talking about it at the same
time.  I believe that data and that accuracy of
data will provide us with the power we never had
but also provide us with much more confidence
that the activities we are suggesting as the way
forward are the right activities Dr Angela
Lennox, Chair, Leicester Partnership
42
STANDARDS

• Data Quality Working Group
• Approved December 2006
• Build on ISO 1911n
• Standard way of describing and communicating
  spatial data quality
• Chaired by 1Spatial

43
STANDARDS
Ascertain what organisations involved in the
market place understand and mean when using the
term spatial data quality. The WG will attempt to
define a framework and a grammar for the
certification and communication of spatial data
quality. This method to describe and communicate
data quality measures will reference, but not be
limited by, a number of categories such as
completeness, accuracy, scale, consistency and
validity. Reference shall be made to the
standards defined in ISO 19113, 19114, and 19138
when published. DQ WG Charter
44

• "Prediction is very difficult, especially if it's
  about the future."
• -- Nils Bohr, Nobel laureate in Physics

45
Steven Ramage ePSIplus Thematic Meeting,
London 15th July 2007 steven.ramage_at_1spatial.com