Assessing M

1
Assessing ME Systems for Data Quality
USAID Mini University George Washington
University Washington, DC, October 5th, 2007
2
Why is data quality important?

• Governments and donors collaborating to fight
  HIV/AIDS, TB, and malaria- Three Ones
• Accountability for funding and results reported
  increasingly important
• Quality data needed at program level for
  management decisions

Unreliable data can impact on the appropriateness
of management decisions Office of Inspector
General, 2006
3
Data quality and the Program Cycle
Data Quality
Results Reporting
4
Data Quality
Linking your services and your data
Services the REAL world In the real world,
project activities are implemented in the field.
These activities are designed to produce results
that are quantifiable.
Data INFORMATION SYSTEM An information system
represents these activities by collecting the
results that were produced and mapping them to a
recording system.
but how well does it all work?
5
Aggregated Data from PEPFAR Countries
Where did these OVC numbers come from?
Do you think they are good numbers? Why or why
not?
Source 3rd Annual PEPFAR report
Make a flow chart of the data from sites to PEPFAR
6
Data Quality
REAL WORLD In the real world, project activities
are implemented in the field. These activities
are designed to produce results that are
quantifiable.
INFORMATION SYSTEM An information system
represents these activities by collecting the
results that were produced and mapping them to a
recording system.
Data Quality How well the information system
represents the real world
Data Quality
1. Accuracy 2. Reliability 3. Completeness 4.
Precision 5. Timeliness 6. Integrity 7.
Confidentiality
Real World
Information System
7
Dimensions of Data Quality

• What are some elements of data quality?

8
Dimensions of Data Quality
9
Information Flow
Results
Indicators
Data collection/reporting systems
Data sources
10
Support to PLWHA and/or familiesHow are these
data collected?

• Community health workers daily record of
  households visited
• Facility-level register
• Client intake forms
• How aggregated?
• How forwarded to next level?
• How forwarded to next level?
• How forwarded to national level?
• How forwarded to international level?

11
Program/project ME Plan and List of Indicators
Levels of the ME System
What is the role of the central ME Unit in the
ME system and data quality?
What is the role of the intermediate aggregation
level?
What is the role of sites?
12
Brainstorming

• .what needs to be in place at each level of the
  ME system to ensure good quality data?

13
DQ Systems Questions

• How well does your information system function?
  Does your project/program link to the national
  information system?
• Are the definitions of indicators clear and
  understood at all levels?
• Do individuals and groups understand their roles
  and responsibilities?
• Does everyone understand the specific reporting
  timelinesand why they need to be followed?

14
DQ Systems Questions, cont.

• Are data collection instruments and reporting
  forms standardized and compatible? Do they have
  clear instructions?
• Do you have documented data review procedures for
  all levelsand use them?
• Are you aware of potential data quality
  challenges, such as missing data, double
  counting, lost to follow up? How do you address
  them?
• What are your policies and procedures for storing
  and filing data collection instruments?

15
What is the Link between ME Systems and Data
Quality?
Dimensions of Data Quality
1. Accuracy 2. Reliability 3. Completeness 4.
Precision 5. Timeliness 6. Integrity 7.
Confidentiality
16
Strengthening ME Systems through data Quality

• How can we promote good quality data?

17
Multiagency Tools
Under draft
18
Three Multi-agency complementary DQ Tools
INDICATOR APPROACH
SYSTEMS APPROACH
AUDITING APPROACH (also for capacity building)
19
Functional Areas of an ME System that Affect
Data Quality
Dimensions of Data Quality
1. Accuracy 2. Reliability 3. Completeness 4.
Precision 5. Timeliness 6. Integrity 7.
Confidentiality
20
ME Systems Strengthening Tools

• Systems approach
• Facilitator moderated
• Three levels
• ME Plan
• ME Coordination Unit
• Data Reporting Systems

21
Data Quality Assurance Tools

• Auditing approach
• Two components
• Systems Assessment
• Data Quality Verification
• Verification factors

22
DQA Protocol 1 Functional Areas of an ME
System that Affect Data Quality (from DQA
Systems Assessment Protocol)
23
DQA Protocol 1 Functional Areas of an ME
System that Affect Data Quality (from DQA
Systems Assessment Protocol)
24
(No Transcript)
25
DQA Summary of ME System Functional Areas
Interpreting the findings The larger the score,
the stronger the component
26
Data verification DQA Protocol 2
27
DQ Dimensions, Levels of the ME System and
Functional Areas
Dimensions of Data Quality
Levels of the ME System
Functional Areas of an ME System Needed to
Ensure Quality
28
Data Quality Plan Template

• Country Time Period
  Program or Partner (e.g. National Program
  or Partner)

DQ Plan Table 2. Functional Areas, Strengths and
Weaknesses and Strengthening Measures
Description of Strengthening Measure
TA needs
Functional Areas (8)
Funding
Timeline
Responsibility
29
Data Quality Plan Template

• Country Time Period
  Program or Partner (e.g. National Program
  or Partner)

DQ Plan Table 2. Functional Areas, Strengths and
Weaknesses and Strengthening Measures
Description of Strengthening Measure
TA needs
Functional Areas (8)
Funding
Timeline
Responsibility
30
MEASURE Evaluation is funded by the U.S. Agency
for International Development (USAID) through
Cooperative Agreement GPO-A-00-03-00003-00 and is
implemented by the Carolina Population Center at
the University of North Carolina in partnership
with Constella Futures, John Snow, Inc., ORC
Macro, and Tulane University. Visit us online at
http//www.cpc.unc.edu/measure.