GTZ Approach to BIC Reform and Impact Chains

1
GTZ Approach to BIC Reform and Impact Chains

• GTZ Capacity Development Workshop
• Strategies and Practices Toward a More Enabling
  Business and Investment Climate
• May 23, 2006
• Dr. Julius Spatz

2
Structure

• Conceptual Approach
• Impact Chains
• Monitoring Approach
• Steps Forward

3
Conceptual Approach (1)
BIC (in the broader sense)
Access to business Know-how, BDS and technology
Access to loans and other financial services
BIC (in the narrow sense)
Access to land and property
Improving the legal, political and institutional
framework conditions
Access to qualified labor
Access to productive infrastructure (water,
energy, telecomm., transport)
4
Conceptual Approach (2)
5
Conceptual Approach (3)
2. Designing Technical advice PPD
1. Assessing Use of diagnostic tools and
processes(RIA, BCS, TiM)
3. Implementing reforms Capacity development,
training
Lifecycle of Reform
4. Monitoring and evaluating Measuring the
outcomes of reforms and their impact
5. Documentation Replication of the Lessons
Learnt
6
Conceptual Approach (4)
Public Sector
Private Sector
National Level
NationalPublic-Private Dialogue
Interministerial Coordination
National Business Association
Ministry
Program Enabling Environment
Provincial Level
Local provincial Government
Local provinc. Business Assoc.
Local Public-Private Dialogue
7
Impact Chain (1)
8
Impact Chain (2)
9
Impact Chain (3)
10
Impact Chain (4)
11
Challenges to Impact Monitoring

• Complexity of BIC Programs
• Length of the Results Chain
• Isolating Individual Reform Measures in Embedded
  Programs or Multi-Donor Settings
• Time Lags between Activities and Outcomes
• Lack of Counterfactual
• Validity and Sustainability of the Survey

12
Monitoring Approach (1)

• Two Levels of Monitoring

• Monitoring up to outcome level
• ? Measuring how far the assumed outcomes have
  been achieved
• ? Identifying external factors that have
  positively or negatively affected the process of
  achieving the outcomes

• Monitoring beyond the attribution gap
• ? Giving a plausible description of possible
  contributions of the project towards the observed
  changes at impact level

13
Monitoring Approach (2)

• Dual Purpose of Monitoring

14
Monitoring Approach (3)

• Tools and Instruments
• Target group surveys (quant./qual.)
• Intermediaries surveys (quant./qual.)
• Policy impact analyses (PSIA, RIA)
• Expert interviews
• Peer review (formalized or non-formalized)
• Evaluation according to international standards

15
Where Do We Stand Local BIC

• Improving the Local Investment Climate in Ormoc
  and Bacolod (Philippines)

• Awareness raising for the importance of an
  enabling investment climate at local level and
  promotion of PPD
• Streamlining the business registration process
  and capacity development to implement the
  reforms

• Monitoring Approach

• Biannual City Competitiveness Ranking Survey of
  the Asian Institute of Management
• Questionnaire includes 70 indicators on the
  local investment climate
• Treated group SME in Ormoc and Bacolod
• Control group SME in other cities of the
  Visayas

16
Local BIC Results (1)

• 2003

• 2005

17
Local BIC Results (2)

• Complementary Evidence

18
Where Do We Stand National BIC

• Development of a National Industrial and Trade
  Policy (Mongolia)

• Technical advice for drafting a national
  industrial and trade policy to comply with WTO
  rules and to develop an export-friendly
  environment
• Capacity development to facilitate the
  implementation of the reforms

• Monitoring Approach

• Annual export climate survey
• Questionnaire includes indicators on export
  duties, unofficial charges, restrictions,
  bureaucratic procedures and export promotion
• No control group
• First round July 2004, second round April 2005

19
National BIC Results
Perceived Obstacles to Exports
20
Steps Forward (1)

• Build impact monitoring into design and
  implementation of new projects (e.g., baseline
  studies, selection of treated and control group)
• Methodologically sound monitoring through Ph.D.
  students in pilot projects (esp. in the field of
  local investment climate)
• Combine quantitative evidence (surveys) and
  qualitative evidence (structured interviews)

21
Steps Forward (2)

• Regional Cooperation in Impact Monitoring
• Harmonization of indicators and survey design
• Mutual learning by sharing experiences and
  information
• Greater economic efficiency by making use of
  existing tools and instruments and/or by
  developing them together
• Gaining a regional perspective of the business
  and investment climate

22
Bridging the Attribution Gap

• Two Levels of Monitoring

• Monitoring up to outcome level
• ? Measuring how far the assumed outcomes have
  been achieved
• ? Identifying external factors that have
  positively or negatively affected the process of
  achieving the outcomes

• Monitoring beyond the attribution gap
• ? Giving a plausible description of possible
  contributions of the project towards the observed
  changes at impact level

23
The Rationale for Monitoring Beyond the
Attribution Gap (1)

• Millennium Development Goals draw the attention
  to poverty and other highly aggregated impacts
• At the 2002 International Conference on Financing
  for Development in Monterrey, donors agreed on
  Managing for Development Results
• BIC programs do not yield quick wins

24
The Rationale for Monitoring Beyond the
Attribution Gap (2)

• What we would like to attribute to the program
• impact on poverty reduction
• impact on GDP growth
• impact on employment
• What we can attribute to the program
• perception of the legal, political and
  institutional aspects of the business and
  investment climate
• implementation of legal, political and
  institutional reforms

25
Steps Towards Bridging the Attribution Gap

• Provide consistent results chains and
  substantiate them with empirical results
• Possible starting points
• Cost of Doing Business in 2005
• WDR 2005 A Better Investment Climate for
  Everyone
• Further evidence
• Case studies from other BIC programs in Africa

26
Thank you very much for your attention!
27
Backup Survey Design (1)

• Counterfactuals
• They cannot be observed, they can only be
  simulated by comparing program participants
  (Treated Group) with a control group

28
Backup Survey Design (2)

• Experimental
• A set of units of analysis who are equally
  eligible and willing to participate in the
  program are ex ante randomly divided in two
  groups

• Quasi-Experimental
• The non-equivalent control group is selected ex
  post using matching techniques or statistical
  methods are used to account for observed
  differences between treated group and control
  group

• Non-Experimental
• Creation of a control group is not possible or
  not ethical

29
Backup Survey Design (3)

• The Sustainability of the Monitoring Evaluation
  System Has to Be Built into the Survey
• Indicators and Survey Design Should be Agreed
  Upon by Donor and Counterparts
• Data Collection and Analysis Should be Gradually
  Transferred to the Counterparts
• The Survey Should Provide Value Added Beyond the
  Project
• The Results Should be Widely Disseminated to
  Raise Awareness