57th ISO TC184SC4 Framework of Data Quality Management

1
57th ISO TC184/SC4Framework of Data Quality
Management

• May 15-22, 2009
• Sunho Kim, Myongji University
• Changsoo Lee, Gangnung University
• Jinwoo Lee, 2e Consulting
• Changhan Lee, KDPC

2
DQM Processes in ISO 8000-1

• Proposed on 2008-09-05 by Matthew West
• ISO 8000 promotes the adoption of a process
  approach when developing, implementing and
  improving the effectiveness of an information
  quality management system, to enhance internal
  and external information customer satisfaction by
  meeting all internal and external customer
  requirements.
• The model of a process-based information quality
  management system in Part 1

3
Why are DQM Processes important?

• Data quality secured by temporary management
  doesnt last long
• Data quality can be preserved or improved when
  managed continuously by processes
• Need to take both data and process into account

Issues of DQM
DQM Target
Management Process
Data
Action
Improve and preserve DQM processes
Measure data quality and improve data of bad
quality
Strength

• It can reduce fundamental causes of data errors
• It is possible to improve data quality
  continuously and step by step by process
  improvement

• Its possible to respond quickly against data
  errors by measurement criteria
• Data quality is measured quantitatively

Weakness

• It is not possible to respond quickly to data
  errors
• Data quality is measured qualitatively

• It is not easy to measure quality levels.
• After a certain level of quality, the quality
  declines as time goes by

4
DQM Objectives Establishment
DQM Framework
Data Quality Management
DQM Resources Allocation
DQM Performance Management
Data Governance
DQM Role/Responsibility Authorization
5
DQM Framework
Management Layer
Data Quality Criteria Management
Data Design
Control Layer
Data Error Cause Analysis
Realization Layer
6
DQM Processes
7
Enterprise Integrated Architecture Management

• Data design is performed in compliance with the
  enterprise integrated architecture
• Data traceability is managed based on the
  enterprise integrated architecture
• Activities
• Definition and change management of enterprise
  integrated architecture
• Definition and observation of enterprise data
  standards
• Sharing of enterprise integrated architecture
• Definition of data items
• Mapping between the enterprise data architecture
  and data architectures of individual systems
• Identification of duplicated data and
  establishment of unification methods

8
Data Quality Planning

• The data quality criteria management is performed
  based on data quality planning.
• Enterprise integrated architectures are used in
  the process of data quality objects
  determination.
• Data traceability is used in the analysis of
  influential factors which affect data quality.
•  Activities
• Identification of data objects for DQM
• Analysis of influential factors which affect data
  quality
• Establishment of DQM methods
• Identification of roles and responsibilities for
  DQM activities
• Performance management of DQM activities

9
Data Traceability Management

• Data traceability can be done with ease when the
  enterprise integrated architecture management
  precedes this process
• Data error causes are analyzed based on this
  process.
• Activities
• Data authorization management
• Data ownership (persons in charge of data
  quality)management
• Management of relationships between data and
  applications
• Management of flows among data
• To secure an ability to trace data manipulators
• Analysis of data error effects

10
Data Design

• This process is implemented based on the
  enterprise integrated architecture management.
• Data manipulation is done based on the data
  design process.
• Data quality criteria is defined based on this
  process.
• When data is created or changed in this process,
  the data should have a traceability
• A number of data errors are generated by wrong
  data design and they are corrected by cause
  tracing.
• Activities
• Identification of data manipulation requirements
• Data architecture design
• Definition of data elements in detail
• Connection to enterprise integrated architectures
• Realization of databases
• Change management of data architecture

11
Data Quality Criteria Management

• This process is implemented as a lower level
  process of data quality planning.
• Data quality criteria are defined based on data
  design.
• Data quality criteria can be reinforced as a
  result of data error cause analysis.
• Activities
• Establishment of data quality criteria
• Definition of requirement levels of data quality
  criteria
• Cooperation with people concerned for data
  quality criteria and requirement levels
• Definition of checking items against DQM objects
• Change management for data quality criteria

12
Data Error Cause Analysis

• Data error causes can be partially identified in
  the process of data error correction.
• Data traceability should be secured to analyze
  data error causes.
• Data quality criteria can be reinforced as a
  result of data error cause analysis.
• Activities
• Classification of data errors
• Tracing of data error causes
• Plan to remove data error causes
• Removal of data error causes and continuous
  management
• Accumulation of data errors and treatment details
• Performance of a priori error prevention
  activities

13
Data Manipulation

• This process is performed based on data design.
• Data quality monitoring is done against the
  result of data manipulation.
•  Activities
• Data input, search, change, transfer, deletion,
  etc.
• Provision of data manipulation guidelines for
  high quality data.
• Logging of data manipulation details

14
Data Quality Monitoring

• The process is performed based on data quality
  criteria.
• Data error correction is done as a result of this
  process.
• Activities
• Real-time or periodical data quality monitoring
• Sharing of monitoring results
• Accumulation of monitoring results and
  statistical analysis

15
Data Error Correction

• The process is performed as a result of data
  quality monitoring.
• Data error causes can be analyzed in this
  process.
• Activities
• Cooperation among persons concerned with
  solutions to data errors
• Identification of data error items and related
  items
• Correction of data errors and related data items
• Sharing of data errors and correction details
• Accumulation of data errors and correction details

16
DQM Process Flows
Enterprise Integrated Architecture Mgmt
Data Quality Planning
Data Design
Data Quality Criteria Mgmt
Data Manipulation
Data Quality Monitoring
Data Error Correction
Data Error Cause Analysis
Data Traceability Mgmt
17
Relationship with ISO 8000-100 series
ISO 8000-100Overview
ISO 8000-102Vocabulary
ISO 8000-110Syntax, semantic encoding, and
conformance to data specification
ISO 8000-120Provenance
ISO 8000-130Accuracy
ISO 8000-140Completeness
ISO 8000-150 Master data quality management
systems
18
Relationship with ISO 9000
PQMS in ISO 9000
19
Conclusion

• DQM Processes should be included in the scope of
  ISO 8000-1.
• DQM Processes should be defined in detail in ISO
  8000-X00 series.
• DQM Processes will be a basis for the extension
  to the DQM maturity model which has been
  mentioned in draft Part 1.