CountryData SDMX for Development Indicators

1
CountryDataSDMX forDevelopment Indicators

• MDG DSD vs. the Di Database
• Using the Mapping Tool

2
MDG Data Structure Definition (DSD) background

• Developed by SDMX Task Team of IAEG on MDGs
• Supports exchange of MDG Indicator data between
  international agencies (UN, UNICEF, UNESCO, )
• Implemented in SDMX 2.0
• Latest version (2.4) finalised in Feb 2013

3
DevInfo (Di) background

• Data dissemination software supported
• and promoted by UNICEF
• DevInfo7 (Di7) launched in Nov 2012
• SDMX 2.1 2.0 compliant
• Web base software
• 9 out of 11 project countries using DevInfo
• Stable version compare to previous releases

4
Simple relation between Di DSD
Di Database
MDG DSD

• Frequency (Default Annual)
• Reference Area
• Series
• Units of measurement
• Unit multiplier (Default 0)
• Location (Default Total)
• Age group (Default All Ages)
• Sex (Default Both Sexes)
• Source Type (Default NA)
• Source details
• Time Period
• Time period details
• Nature of data points (Default C)
• Footnotes

• Area
• Indicator
• Unit
• Subgroup (i.e. Sex, Age, Location etc.)
• Source
• Time Period
• Footnotes

5
Mapping to the DSD

• DSD dimensional structure means
• values are mandatory for LOCATION, SEX AGE
  GROUP.
• Due the nature of this domain (i.e. MDGs), not
  obvious which values should be used in these
  dimensions
• For example, what is SEX for Births attended by
  skilled personnel
• Not Applicable? Total? Female?

6
Mapping to the DSD

• Inconsistent mappings lead to duplications and
  other anomalies
• In CountryData, mappings for indicators/ time
  series are agreed before data exchange (see
  mapping for MDGs from 1st workshop)
• However, this is just one side of the story

7
Mapping to the DSD

• Understanding the structure and contents of the
  origin database is fundamental to the mapping
  process
• Mapping to the DSD requires the data to enter
  into certain restrictions its not bound by in
  the database (and vice versa).

8
Mapping to the DSD

• The mapping tool in di software is
• designed to work with the di database as simply
  as possible
• the tool is based on mapping between the
  codelists of the DSD and origin database
• certain situations require some further manual
  effort to map a time series
• and sometimes a fix is required to the
  database where the data simply isnt valid or
  its duplicated.
• Therefore its good to review di structure to
  understand where these issues usually occur.

9
Di Data Architecture

• Area, hierarchical dimension
• IUS Indicator, Unit and Subgroup
• Time series data are stored with the combination
  of the 3 dimensions
• Indicator
• Unit
• Subgroup Combination of one or more
  sub-dimensions
• Source Time Period
• Together with IUS uniquely defines each data
  value
• Footnote
• Free text field stored with data value

10
Di INDICATOR

• IUS Indicator Unit Subgroup
• Indicator, for example
• Infant Mortality
• AIDS Death
• Malaria Death
• Similar to SERIES in the DSD
• Contains only Indicator specific values

11
Di UNIT

• IUS Indicator Unit Subgroup
• Unit
• Percentage
• Number
• USD
• Square KM
• Similar to UNIT of Measurement in DSD
• Contains only Unit specific values

12
Di SUBGROUP

• IUS Indicator Unit Subgroup
• SubGroup Dimension
• Combination of one or more sub-dimensions
• Age, Sex, Location and Other
  sub-dimensions are set initially in database
• Specific values can be created under each
  sub-dimension
• Relate to SEX, AGE GROUP and LOCATION in DSD.

13
Di SUBGROUP

• IUS Indicator Unit Subgroup
• Formation Logic

Sub-Dimension
Age
Sex
Location
Other
Sub-Dimension values
lt 1 Year lt 5 Year 5 10 Year
Male Female
Urban Rural
Total Rice Wheat
SUBGROUP (Combination)
lt1 Year Male lt5 Year Female Rural Urban
14
Di SUBGROUP
15
Di SUBGROUP
16
Di SOURCE
17
Di TIME PERIOD
18
Di Mapping Tool Introduction

• Once data exists in di7 web-based software then
  data can be mapped and published which conforms
  with the MDG DSD.
• This is all done online through the di7 web-based
  repository through the administration profile, so
  lets begin

19
Getting Started
20
Scroll down to Registry menu
21
Log onto administrative profile
22
Log onto administrative profile
23
Full access to Registry features
24
Prepare the Dbase for mapping
25
Prepares the SDMX artefactes
26
Ready to Upload the DSD
27
Choose a DSD from your folders
28
Choose a DSD from your folders
29
DSD Upload is a success
30
Now you are ready to map
31
1st Step Codelist mapping
32
1st Step DSD Codelists
Indicator CodeList Indicator CodeList
SH_HIV_INCD HIV incidence rate
SH_MLR_MORT Notified cases of malaria
SE_ADT_1524 Literacy rate
SE_PRM_CMPL Primary completion rate
SEX CodeList SEX CodeList
NA Not applicable
F Female
M Male
T Both sexes
Location CodeList Location CodeList
T T Total (national level)
U U Urban
R R Rural
UNIT CodeList UNIT CodeList
NA Not applicable
CUR_LCU Local currency
USD USD
NUMBER Number
RATIO Ratio
PERCENT Percent
KM2 Square kilometers
T Metric Tons
PER_100_LIVE_BIRTHS Per 100 live births
PER_100_POP Per 100 population
PER_1000_LIVE_BIRTHS Per 1,000 live births
PER_1000_POP Per 1,000 population
PER_100000_LIVE_BIRTHS Per 100,000 live births
PER_100000_POP Per 100,000 population
AGE CodeList  AGE CodeList 
NA Not applicable
000_099_Y All age ranges
000_006_M under 6 month olds
000_005_Y under 5 year olds
000_001_Y under 1 year olds
000_018_Y under 18 year olds
000_006_Y under 6 year olds
010_005_Y 10-14 year olds
015_005_Y 15-19 year olds
015_010_Y 15-24 year olds
015_035_Y 15-49 year olds
006_054_M 6-59 months old
006_009_Y 6-14 year olds
005_013_Y 5-17 year olds
015_050_Y 15-64 year olds
33
1st Step (A) Map Indicator codes
34
1st Step (B) Map Unit codes
35
1st Step (C) Map Subgroup codes
36
1st Step (C) Choose Subgroup list
37
1st Step (C) Map Age subgroup
38
1st Step (C) Map Sex Location
39
1st Step (D) Map Area
40
1st Step Save codelist mappings
41
1st Step Ignore warning
42
1st Step Confirm mapping saved
43
Exercise 1 Codelist mapping

• Use unstats.un.org/unsd/demodiweb1-6
• Username webmaster_at_xyz.com
• Password support_at_2012
• Map the codelists (where possible) for
• Unit
• Age
• Sex
• Location
• Area
• And just one indicator, Antenatal care coverage
  for at least one visit

44
1st Step Complete
45
2nd Step Confirm IUS mapping
46
2nd Step Save IUS Mappings
47
Exercise 2 mapping time series

• Use unstats.un.org/unsd/demodiweb1-6
• Username webmaster_at_xyz.com
• Password support_at_2012
• Map the time series for
• Antenatal care coverage for at least four
  visits
• Employment to population ratio
• Literacy rate of 15-24 year-olds
• Death rate associated with malaria
• Proportion of population using solid fuels

48
2nd Step Complete
49
Final Step Register the mappings
50
Final Step Select mappings
51
Final Step Generate SDMX-ML
52
Final Step Complete
53
Exercise 3 Publish time series

• Use unstats.un.org/unsd/demodiweb1-6
• Username webmaster_at_xyz.com
• Password support_at_2012
• Publish/ register the time series for
• Antenatal care coverage for at least four
  visits
• Employment to population ratio
• Literacy rate of 15-24 year-olds
• Death rate associated with malaria
• Proportion of population using solid fuels

54
Why the 2nd step?

• The default values for SEX, LOCATION or AGE GROUP
  mapping may not be applicable to all mappings
• The codelist mapping may only provide a partial
  mapping of the time series (i.e. more information
  is required)
• These changes are made in the 2nd step.
• This is all done online through the di7 web-based
  repository through the administration profile, so
  lets begin

55
Where are the default values?
56
Admin panel Application settings
Insert screens shot/details of admin panel and
default value storage
57
Application settings has all mapping default
values
58
Manual mapping of SUBGROUP

• Where a subgroup value is missing the default
  values will apply, for example

Unit
Indicator

• Default Values
• Location T

?
59
Manual mapping of SUBGROUP

• So subgroups coverage affects the number of
  manual changes which have to be made

Unit
Subgroup for Age and Sex?
Indicator

• Default Values
• Age Group 000_099_Y
• Sex T

60
Manual mapping of SUBGROUP

• Common example of where default subgroup mapping
  do not apply

Subgroups?
Indicator
Unit

• Default Values
• Location T
• Age Group 000_099_Y
• Sex Both sexes

61
Manual mapping of SUBGROUP

• Common example of where default subgroup mapping
  do not apply

Subgroup for Sex?
Indicator
Unit

• Default Values
• Sex T

62
Manual mapping of SUBGROUP

• Common example of where default subgroup mapping
  do not apply

Indicator
Unit
?

• Default Values
• Location T
• Age Group 000_099_Y
• Sex T

Subgroup for Location, Age and Sex?
63
Manual mapping of SUBGROUP

• If the subgroups are sorted more simply, this
  also helps with the mapping

Unit
Indicator

• Default Values
• Location T

?
64
Manual mapping of SUBGROUP

• Common example of where default subgroup mapping
  do not apply

Unit
Indicator
?

• Default Values
• Location T
• Age Group 000_099_Y
• Sex T

Subgroup for Location, Age and Sex?
65
Back to mapping
66
2nd Step Amend Indicator
When using the check box to tick the mapping, you
are fixing the mapped DSD values. If the box is
unchecked again and the mappings saved, then DSD
values revert to those mapped at codelist/
default values (i.e. any manual changes are
undone.)
67
Final Step Register new mappings
68
Exercise 4 Amend time series

• Use unstats.un.org/unsd/demodiweb1-6
• Map/ amend/ publish the time series for
• Antenatal coverage rate
• Children orphaned by AIDS
• Children under-five sleeping under
  insecticide-treated net (ITN)
• Proportion of births attended by skilled health
  personnel
• Share of women in wage employment in the
  non-agricultural sector
• Proportion of urban population living in slums

69
More complex mappings under the 1st and 2nd
mapping step?

• The most common changes made to mappings are
  between subgroups and the Sex, Age Group and
  Location dimensions
• But sometimes manual changes are required between
  di and DSD indicator and unit, either
• More than one di code relates to a single DSD
  code OR
• More than one DSD code relates to a single di
  code

70
Many-to-one mapping for Indicator codelist
(Example 1)
Indicator
Indicator
Indicator
71
Many-to-one mapping for Indicator codelist
(Example 2)
Indicator
Indicator
72
Many-to-one mapping for Indicator codelist
(Example 3)
Indicator
Indicator
73
Many-to-one mapping for Unit codelist (Example 1)
Unit
Unit
74
Manual mapping of INDICATOR
?
Indicator
Unit
Manual change
75
Manual mapping of INDICATOR
Unit

Indicator
?
Manual change
76
Manual mapping of UNIT

Indicator
Unit

• Manual change
• Unit Ratio

77
Back to mapping
78
1st Step many di to 1 DSD code
79
2nd Step 1 di to many DSD codes
80
Final Step Register new mappings
81
Exercise 5 Complex time series

• Use unstats.un.org/unsd/demodiweb1-6
• Map/ amend/ publish the time series for
• Contraceptive prevalence rate
• Primary completion rate
• Gender parity index in primary education
• Seats held by men in national parliament
• Seats held by women in national parliament
• Telephone lines

82
Other issues encountered with generating SDMX
from DevInfo

• The MDG DSD requires any data point to be
  uniquely described by the following dimensions

• However, DevInfo allows data to be stored in
  overlapping time intervals and with multiple
  sources. These issues need to be resolved to
  conform to the uniqueness required by the MDG
  DSD.

83
Multiple sources
Allowable in DevInfo but not in the DSD
84
Overlapping time

• This issue is only a problem where overlapping
  periods begin from the same year, as the mapping
  tool takes the first year in the period as the
  value for the Time Period dimension.

85
Targets in the database
Targets are also an issue when found in the
database since they should not be exchanged as
observed values
86
Target in database (Example 1)
Sometimes stored as subgroup which can be ignored
at the 2nd stage
87
Target in database (Example 2)
But other times can be found as a time period
among observed values
88
Use of filters at registration

• To deal with the issues of
• multiple sources for a given time period,
• overlapping time period beginning at the same
  year
• And targets presented alongside observed values
• The mapping tool provides a feature to filter out
  data from a generated SDMX message associated
  with specific time periods and source references.

89
Back to mapping
90
Final Step Filter by time/ source
91
Final Step Select source filter
92
Final Step Select time filter
93
Final Step Register new mappings
94
Final Step Complete
95
Exercise 6 Filter time series

• Use unstats.un.org/unsd/demodiweb2-6
• Map/ amend/ publish the time series for
• Under-five mortality rate
• Maternal mortality ratio (MMR)
• Net enrolment ratio in primary education (NER)
• Orphans primary school enrolment
• Tuberculosis prevalence rate
• Proportion of the population using improved
  sanitation facilities

96
DSD Maintenance

• The mapping and registry tool allows users to
  edit and delete the DSD as well as upload.
• For when the DSD is updated, it is recommended
  to edit the DSD rather than delete
• DSD deletion has the effect of removing all the
  mappings and subscriptions used for that DSD

97
DSD Maintenance
98
DSD Header