Case Study

1
Case Study Australian Bureau of Statistics
A brief history of metadata at the ABS
Simon Wall Metadata Team Leader simon.wall_at_abs.gov
.au
2
Premetazoic era

• 1905-1973
• before the term 'metadata' existed
• some metadata management anyway
• but no metadata management strategy.

3
Protometazoic era

• 1973-1990
• created data dictionaries
• limited reuse of common data elements
• not fully corporate in scope
• driven by technology, not by strategy

4
Mesometazoic era

• 1991-2000
• focus on data warehousing for output
• beginning of a strategic view
• Towards a Unified Data and Metadata System at the
  ABS (1991)
• developed core metadata repositories
• statistical activities (ABS collections),
  datasets, data items (data elements/variables),
  classifications, terms
• seen as overhead by subject matter areas
• limited return for effort
• quality questionable, not actively maintained

5
Neometazoic Era

• 2001-2007
• reorganised as Corporate Metadata Repository
• adding services interfaces
• legacy repositories both asset and liability
• vision greater than implementation
• Formal strategy adopted
• Strategy for end-to-end management of ABS
  metadata (2003)
• Some new development work
• 11179-based Data Element Registry
• Quality Declarations

6
Holocentric epoch

• 2008
• ABS executive seek clear and compelling picture
  of long term metadata aspirations.
• not provided by existing strategy.
• How would it improve the way the organisation
  operates?
• What new capabilities would it deliver?
• Are they the capabilities we need most?
• Is the strategy actually achievable?
• Is it the most appropriate future for us to be
  investing in?
• now developing "2020 Vision"
• encapsulating longer term ABS aspirations for
  metadata

7
Drivers for 2020 vision

• Uptake of Corporate Metadata Respository services
  very slow
• existing processing systems "monolithic"
• not easy to modify
• existing processing systems have "special needs"
• customisations required
• lack of funds and business drivers
• disconnect between user view and description for
  systematic manipulation.
• processing systems unable to provide metadata to
  describe transformations

8
Drivers for 2020 vision

• Frame of reference for ABS metadata management
  requirements is no longer defined by the
  boundaries of the organisation
• Developing infrastructure that can be shared with
  other producers of statistical data within
  Australia
• Other government agencies now have their own
  focus on interoperability, including metadata.
• Software collaboration / open source software
• Interoperate with agencies with a focus on
  administrative, geospatial, or research-oriented
  data content.

9
Where are we going?

• One The practical complexities and difficulties
  of using metadata in an "end to end" context to
  actually drive actual ABS processes is becoming
  clearer.
• Two There is pressure to make our metadata
  capabilities less ABS specific
• more flexible, interoperable and generic - .
• These factors suggest that charting a successful
  way forward in regard to metadata management for
  the ABS will require a "paradigm shift", a new
  era.