Assessing Governance

1
The Tools of the Trade An Overview of The
World Banks Governance Diagnostic and Assessment
Instruments
Presented to PREM WBI Core Course on Public
Sector Governance Anticorruption
Presented by Francesca Recanatini Senior
Economist World Bank Institute (WBIGP) www.worldba
nk.org/wbi/governance
February 14-17, 2005

2
Outline of the Session

• A working framework to select among tools
• Conceptual design
• Empirical tools
• Implementation process
• Sampling and Field work
• Analysis and use of the data
• A few country-specific illustrations

3
Objective
How to choose among governance tools?

• Which are the key elements for a governance
  assessment?
• Which empirical tools and approaches are already
  available?
• How can we select among them?
• How can such assessments be used for policy
  purpose?

4
Key starting points

• 1. What is the purpose of the assessment?
• Research and analysis
• Awareness raising
• Policy and Action planning
• Capacity building
• Monitoring

5
Key starting points

• 2. What is the focus of the assessment?
• Governance as a whole
• Corruption
• Performance of a specific agency/sector
• Quality of a specific public service delivered

6
Suppose we have determined ...

• The final purpose of the assessment
• The focus of the assessment
• What next?

7
Existing Empirical Tools

• BEEPS
• IGR
• Public Official surveys
• PETs
• QSDS
• Score Cards
• Investment Climate Surveys
• EC Audits

• PER
• CFAA
• CPAR
• GAC
• Case Studies
• HIPC Exp. Tracking
• ROSC

www.worldbank.org/wbi/governance/assessing
8
Which are the key elements of a Governance
Assessment?

• Four dimensions
• Conceptual
• Empirical
• Process / Capacity Building
• Analytical and Policy

9
Use a working framework
Conceptual dimension
- Analytical Framework -
- Analytical Framework -
Governance Assessment
Implementation process
Empirical tools sample
Analysis use
10
Conceptual dimension

• Clear definition of the variable we focus on and
  its manifestations
• Translation of the definition into observable and
  measurable components
• Selection of methodological approach
• Understanding of the links between governance and
  
• Performance outcomes
• Development outcomes

11
Linking the Tools to the Blueprint
12
Conceptual dimension, cont.

• Finding answers may require single or multiple
  methods and data forms
• The methodological approach can be a combination
  of different methods (for example, qualitative,
  quantitative or mixed)
• To each method corresponds a set of empirical
  tools that we can use
• Data can also be qualitative and/or quantitative

For more information on alternative methods
www.worldbank.org/wbi/governance/assessing
13
Purely Qualitative

• Methods
• - Focus Group Discussions, interviews, case
  studies
• Problems
• Non-representative
• Lack of counterfactuals, causality is unclear
• Small Samples
• Advantages
• Open-Ended
• Context, History

14
Purely Qualitative

• Problems
• Structured Questions
• Top-Down
• Reflect Biases of Researcher
• Advantages
• Large Samples
• Representative Samples
• Clear Methods for Inferring Causality

15
Mixed Methods

• Take Best of Both Worlds - Advantages
• Quantitative Questions Informed by Qualitative
  Investigation.
• Hypotheses Generated by Qualitative tested for
  Generalizability by Quantitative.
• Depth supplemented by breadth thick
  understanding with generalizability.
• History, Context, Process and Identifying Causal
  Links
• Participation

Remember!
16
Mixed Methods Problems

• High Cost
• Time Consuming
• Large Teams Coordination Problems
• Usually poorly done more research required to
  understand how methods compare

17
Examples of Existing Empirical Tools

• Quantitative method
• Investment Climate Surveys
• QSDS
• Public official surveys
• PETs

• Qualitative method
• Budget use monitoring
• Video Observations
• Judicial Investigations

• Mixed method
• Governance Diagnostic Surveys
• Score Card approach
• PER

18
Empirical dimension

• Focus on institutions vs. individuals
• Experiential vs. perception data
• One vs. many types of respondents
• Standard vs. customized empirical tools
• Definition of sample and field work details
• Open end vs. close end questions

19
Linking the Tools to the Respondents
20
Process/Capacity Building dimension

• To increase impact and sustainability
• Consultative and participatory approach to
  discuss purpose, use and features of the
  assessment
• Engage local NGOs and academic institutions to
  adapt/revise tools
• Public dissemination of results
• Joint design of policy recommendations

21
Measuring governance possible outcomes
Broader awareness
Internet, radio
Focus groups
Enhanced local capacity
Governance Assessment
Greater consensus
Workshops
Policy dialogue
Knowledge for policy and/or research
22
Where we are

• We defined our objective
• We identified a tool and a method
• We defined the sample
• We specified the details of the field work
• We defined the process
• What next?

23
Analytical and Policy dimension

• The use of data to
• identify an issue
• measure a phenomenon
• understand a process
• compare experiences
• evaluate constraints
• quantify costs and benefits
• link effect to a cause
• evaluate policy choices

Focus on links between governance manifestations
and - Quality of services - Growth - Public
sector characteristics
24
Empirical analysis a word of caution

• The data has to handle with care to interpret
  correctly the results
• Key elements
• Characteristics and size of sample
• Modus of data collection
• Objectiveness of the data
• Rigor of the approach
• Margin of error
• Significance

25
Key dimensions for governance analysis

1. Identify both weak institutions (in need of
   reform) and strong institutions (example of good
   governance)
2. Unbundle corruption by type administrative,
   capture of the state, bidding, theft of goods and
   public resources, purchase of licenses and
   regulations

26
Key dimensions (cont.)

• Assess the cost of each type of corruption on
  different groups of stakeholders
• Identify key determinants of good governance
• Develop policy recommendations

27
An example Peru 2002

• Issue the government wanted to monitor progress
  in terms of
• Transparency of public administration activities
• Civil society participation and voice
• Quality of public services

28
Peru 2002, cont.

• Purpose of assessment monitoring
• Focus of the assessment
• Transparency
• Citizens Participation and Voice
• Quality of public services
• What next?

29
Peru 2002, cont.

• Purpose of assessment monitoring
• Final users government and civil society
• Key feature
• Comparability across time
• Ability to identify progresses
• Type of information needed agency-specific
• Approach objective, and based on citizens
  feedback

30
Peru 2002, cont.

• Conceptual dimension
• Transparency in the management of resources
• Quality of basic health and education services
• Quality of complaint and feedback mechanisms
• Empirical Tool
• Score card/Questionnaire to households
• Focus on agency-specific information
• Objective, experiential data
• Close-end questions

31
Peru 2002, cont.

• Process/Capacity building
• Partnership between WBI and with National
  Statistical Office on methodological issues
• Data and results publicly available
• Analytical dimension
• Monitoring of indices performance over time
• Link between indices of performance and measures
  of poverty

32
Peru 2002 Decisions taken

• To develop the following yearly indicators
• Index of transparency and civil society
  participation
• Index of quality of public services
• To focus on households/users only
• To promote a partnership between the National
  Statistical Agency and citizens

33
Governance and A-C diagnostic surveys

• A demand-driven process to improve governance,
  build local capacity and consensus
• Key elements participation, transparency and
  analytical rigor
• Outcomes greater local capacity, new policy
  actors, baseline governance data, and action plan
  for policy reform

34
The process
Honduras CNA report and strategy to newly
elected gov (January 2001) integration of
strategy in the 2002-2006 government plan
Challenge poor governance and corruption
Country Implemented
Guatemala Highly fragmented civil society Joint
effort to build consensus and focus on rigorous
approach
Sierra Leone Strong commitment (civil society,
state, donors) gt surveys and report within a
year. Results will be used for Institutional
Reform Loan
35
Examples of variables measured

• USERS/HOUSEHOLDS
• Quality of specific public services
• Cost and time to obtain a service
• Information available on basic rights
• Quality of public agencies
• Experience with inappropriate procedures and
  behavior

36
Examples of variables measured

• ENTERPRISES
• Quality of specific services and procedures
• Cost and time to comply with permits and licenses
• Information available on basic rights
• Quality of public agencies
• Experience with inappropriate procedures and
  behavior

37
Examples of variables measured

• PUBLIC OFFICIALS
• Quality of the rules and procedures
• Transparency of budget and employment decisions
• Information available on procedures
• Quality of management
• Experience with inappropriate procedures and
  behavior within their office

38
Governance Variables for Selected Countries
(View of the Firm, EOS 2003) - Percentage Firms
Reporting Low Governance
Bad
Good
Source EOS (firm survey), 2003. Y-axis
displays percentage of firms who reported low
Governance (1-3) in each governance dimension.
39
Extent of corruption
(Various countries, 2001)
40
National and municipal agencies are ridden by
different types of corruption(based on public
officials' responses, a Latin American country,
2001)
41
Bribes and Quality of Service in Public Sector
(as reported by public officials in Honduras,
2001)
42
Quality of education by district(Sierra Leone
2003)
43
Obstacles to using courts in Zambia (as reported
by households and firms)
Businesses
Households
44
Corruption increases inequality
The thin lines represent margins of error (or 95
confidence intervals) for each
45
The costs of corruption(Investment Climate
Survey, Guatemala, Honduras and Nicaragua, 2003)
As reported by managers
46
The costs of corruption(Investment Climate
Survey, Guatemala, Honduras and Nicaragua, 2003)
47
Corruption imposes barriers to households to
access basic social services
Sierra Leone Roads Transport Authority
48
Mechanisms to participate to policy process
of households reporting to use the following
channel to participate in the policy process
49
The determinants of governance a set of new
indicators

• Internal Transparency
• Meritocracy
• Quality of norms and rules
• Accountability citizen voice
• Rule application and supervision
• Salary Satisfaction
• Agency Missions
• Politicizing
• Quality of services

50
Index of Quality of Rules (to manage personnel
and budget resources)
High
Low
As reported by public officials
51
Index of Voice and Accountability
High
As reported by public officials
Low
52
Governance corruption indicators by province
South North East West WHOLE
Prov. Prov. Prov. Area COUNTRY
Corruption in budget 35 43 48 39 40
Overall corruption 22 32 35 33 32
Corruption in public contracts 18 35 29 33 30
Corruption in personnel (2) 39 44 55 53 49
           
Accessibility for poor 85 74 87 74 78
Audit Mechanisms 55 59 66 58 58
Enforcement of rules 70 67 80 73 71
Politicization 21 34 22 34 32
Quality of rules 62 62 70 61 63
Resources 54 51 47 55 52
Transparency 51 55 53 51 55
Citizen voice 70 59 65 66 66
Meritocracy 66 65 70 69 68
53
Governance corruption indicators by province
Enforcement Citizen Wage
Audit of rules Resources Transparency Voice Satisfaction
Presidential Affairs 52 74 51 55 59 17
Ministry of Finance 57 68 52 54 60 25
Quasi-independent institutions 69 85 41 63 78 32
Ministry of Trade and Industry 64 84 60 59 63 19
Ministry of Internal Affairs 60 76 57 61 76 0
Ministry of Energy and Power and NPA 72 92 62 61 69 25
Ministry of Foreign Affairs International Cooperation 50   74 54 50 30
Ministry of Development Economic Planning 54 72 69 57 58 25
Ministry of Social Welfare, Gender Children's Affairs 59 71 53 56 58 13
Ministry of Health Sanitation 54 71 53 55 65 21
Ministry of Information and Broadcasting 55 68 51 51 60 18
Ministry of Education, Science and Technology 54 69 51 55 67 26
Ministry of Youth and Sports 56 76 50 52 72 25
Ministry of Labor and Industrial Relations 61 71 48 56 64 25
Ministry of Agriculture and Food Security 60 69 46 58 66 16
Ministry of Local Government Municipal and district councils 58 72 54 57 63 15
Ministry of Transport Communications 64 77 46 57 66 31
Ministry of Works and Public Maintenance 53 67 52 53 68 5
Ministry of Lands, Housing Country Planning     54 57 56 13
Ministry of Mineral Resources 60 67 68 55 55 21
House of Parliament 54 53 41 59 50 13
Office of the Attorney General, Minister of Justice and Director of Public Prosecutions 60 75 49 55 67 25
Traffic Police and Sierra Leone Road Transport Authority 41 74 64 54 63 34
Supreme Courts 64 78 55 56 72 43
Local Courts 61 68 41 53 77 36
SALWACO/GVWC and SIERRATEL 56 73 53 57 58 35
SALPOST 61 66 46 54 60 28
Sierra Leone Ports Authority 53 66 56 54 59 42
SL National Tourist Board and SL Standards Bureau 44   47 49 51 63
Police and Prisons 59 68 57 53 66 25
Post-conflict institutions 60 71 47 54 64 44
WHOLE COUNTRY 58 71 52 55 66 25
54
WBI Governance on the Web

• About Governance Diagnostics and Statistical
  Capacity Building http//www.worldbank.org/wbi/go
  vernance/capacitybuild/about.html
• Governance Diagnostic Surveys Country Sites
  http//www.worldbank.org/wbi/governance/capacitybu
  ild/d-surveys.html
• Worldwide Governance Indicators 1996-2002
  http//www.worldbank.org/wbi/governance/govdata200
  2/index.html
• The Business Environment and Enterprise
  Performance Survey (BEEPS) 1999-2000
  http//info.worldbank.org/governance/beeps/
• The Business Environment and Enterprise
  Performance Survey (BEEPS II) 2002
  http//info.worldbank.org/governance/beeps2002/
• Courses and Surveys Governance Diagnostic
  Capacity Building http//www.worldbank.org/wbi/go
  vernance/capacitybuild/courses.html
• Step by Step Guide to Governance Diagnostic
  Empirical Tools Implementation
  http//www.worldbank.org/wbi/governance/capacitybu
  ild/diagnostics.html

55
QA