OECD Expert Group on Statistical Data and Metadata Exchange

1
OECD Expert Group on Statistical Data and
Metadata Exchange

• Geneva 10-11 May 2007
• OECD.Stat SDMX Web Service (II)
• Jens Dossé, OECD

2
OECD SDMX web service

• Fully conforms to SDMX standard v2
• Currently two methods? extract data in
  GenericData message? extract Data Structure
  Definitions (Key Families), concepts and code
  lists
• Method planned for the future extract Metadata
• Secured access integrates authorization to
  OECD.Stat
• Still needs a few additional conventions bridging
  SDMX-ML limits

3
Additional conventions I

• For time series data, independently of the
  presence of a FREQUENCY dimension, the requested
  time frequency must be specified using a
  TIME_FORMAT attribute ltqueryAndgt ltqueryAtt
  ribute name"TIME_FORMAT"gtP1Mlt/queryAttributegt
  ltqueryTimegt ltqueryStartTimegt1960-01lt/querySt
  artTimegt ltqueryEndTimegt2005-12lt/queryEndTimegt
  lt/queryTimegt lt/queryAndgtFormat values
  P1Yannual, P6Msemi-annual, P3Mquarterly and
  P1Mmonthly data Time formats yyyy, yyyy-mm or
  yyyy-mm-dd with digits yyear, mmonth
  and dday, e.g. '2005-09-01'. This attribute is
  then also present in the resulting generic data
  message at the Series level of time-series data.
• Non-time-series data omit all time
  specifications in the query message ? The
  response messages will use 9999 as (obligatory)
  time periods
• NEW If the ltqueryStartTimegt or ltqueryEndTimegt
  nodes are not present then all the available
  values are returned

4
Additional conventions II

• OECD.Stat data warehouse? Each dataset
  (multidimensional cube) has its own Data
  Structure Definition.? Datasets can share
  common dimensions but which can have additional
  non-common codes
• Several datasets within one message? Query
  message can request data from several datasets?
  NEW Different datasets can now be returned
  within one SDMX generic message using
  MessageGroups

5
Additional conventions III

• For simplicity during the start-up of this
  generic web service, the returned message headers
  only contain required information
• Maximum size of generic data message 5 MB?
  message is truncated if necessary? Requests
  need to be split if this limit is exceeded

6
(Still) open challenges for SDMX web services

• !!! Big message sizes for any data format need
  standard for compression
• !!! Security need standard for
  authentication/encryption
• !!! Need for another message format generic
  cross-sectional format ? remove the special
  status of the Time dimension (as not all
  statistical data are time series)
• Registering SDMX web services in the SDMX
  registry
• Multilaterally connecting organisations SDMX web
  services to build international networks

7
OECD SDMX web service client example

• Simple web application
• Provides example query messages
• Source code freely available
• Secured access to all data in data warehouse
  (integrates authentication and authorization to
  OECD.Stat)

8
Contacts/Links

• jens.dosse_at_oecd.orgdouglas.paterson_at_oecd.org
• http//stats.oecd.org/OECDStatWS_SDMX/SDMX.asmx
• http//stats.oecd.org/OECDStatWS_SDMX/Test_SDMX.as
  px