Governmentwide monitoring and evaluation system GWM

1
Government-wide monitoring and evaluation system
(GWME)

• Presentation to the Public Service and
  Administration Portfolio Committee
• 18 October 06

2
Presentation outline

• Aim of GWME
• GWME Results
• Implementation phases
• Management structures
• Achievements

3
Government-wide monitoring and evaluation system
(GWME)

• The aim of the GWME system is to contribute to
  improved governance, promote learning and enhance
  the effectiveness of public sector organisations
  and institutions and accountability reporting
• The system objectives are the collection and
  collation, analysis and dissemination and the
  application of information on the progress and
  impact of programmes in order to
• Ensure transparency and accountability
• Promote service delivery improvement
• Ensure compliance with statutory and other
  requirements and
• Promote the emergence of a learning culture in
  the public sector

4
Results to be achieved

• Result one Accurate and reliable information on
  progress in the implementation of government and
  other public sector programmes are collected and
  updated on an ongoing basis
• Result two Information on the outputs, outcomes
  and impact achieved by government and other
  public bodies are periodically collected and
  presented
• Result three The quality of monitoring and
  evaluation practices in government and public
  bodies is continuously improved

5
GWMES Proposal and Schedule Implementation
Plan

• Phase One Setting the basis for implementing the
  reporting practice
• Development of Principles Practices
• Compendium of Development Indicators for SA
• OPSC assessment of departmental ME Systems
• Capacity Building and Accreditation
• Phase Two System Functionality, Alignment and
  Standardisation.
• Drafting, consultation finalization of data
  gathering templates and reporting formats
• Develop ICT platform architecture to receive,
  collate, and report on data collected.
• Ensuring the interoperability and integration
  between this ICT platform (GWMES) and existing
  departmental ICT systems.

6
GWMES Proposal and Schedule Implementation Plan

• Phase Three System Functionality, Alignment and
  Standardisation.
• Initiate a pilot with a few departments to test
  system functionality.
• Develop a detailed project plan to serve as a
  template for later roll-out.
• Independent evaluation of the pilot system
  roll-out using the evaluator's findings and
  recommendations to make system adjustments.
• Gradual extension of the system to all government
  bodies, starting with national departments, then
  provincial departments, local authorities, and
  all SOEs.

7
GWMES Proposal and Schedule Implementation
Plan
Phase One Setting the basis for implementing the
reporting practice 1. Development of
Principles Practices

• What are principles and practices?
• Government is adopting an implementation approach
  based on shared principles and common practices
• This is a different strategy to one based on
  rules and prescriptions (old-style public
  administration)
• Basing our emergent system on principles provides
  a clear framework while it encourages adaptive
  thinking, consultation and communication

8
Purpose of ME Principles Practices

• The document sets out the national ME
  principles and practices, including minimum norms
  and standards for monitoring and evaluating
  government policies, projects and programmes.
• It also focuses on procedures to be adopted by
  national, provincial, local and public entities
  in the monitoring and evaluation of
  implementation of government policies and
  programmes/projects

9
Core principles

• Principles are the unchanging, non-negotiable
  values that underlie the system
• Although proposed, the GWMESs core principles
  have not yet been determined
• The system will have different kinds of
  principles at its heart

10
Types of principles

• The GWMES must be built on
• Ethical principles Rights-based transparent and
  accountable
• Managerial principles A balance of efficiency,
  economy, effectiveness service delivery oriented
• Technical principles Indicator based,
  participatory, people-centered, developmental,
  inclusive etc.
• Proprietary Principles are intended to ensure
  that an evaluation will be conducted legally,
  ethically, and with due regard for the welfare of
  those involved in the evaluation, as well as
  those affected by its results
• Feasibility Principles The feasibility
  principles are intended to ensure that an
  evaluation will be realistic, prudent, diplomatic
  and frugal The evaluation procedures should be
  practical, to keep disruption to a minimum while
  needed information is obtained Be cost effective

11
GWMES Proposal and Schedule Implementation
Plan
Phase One Setting the basis for implementing the
reporting practice 1. Development of
Principles Standards

• Milestones Achieved
• Initial Draft completed May 2006
• Workstream established June 2006
• TORs for the work stream approved by ME Task
  Team July 2006
• Conference to deliberate on the matter held
  19-23 June 2005
• Comments/Input from conference incorporated July
  2006
• Consultations with Relevant Stakeholders
  completed August 2006
• Ongoing progress on the activity reported to
  Cluster Cabinet via POA Bi-monthly

12
GWMES Proposal and Schedule Implementation
Plan
Phase One Setting the basis for implementing the
reporting practice 1. Development of
Principles Practices (including Norms
Standards)

• Milestones Achieved
• Draft Guidelines for ME being discussed at Work
  Stream level
• Key Glossary of ME Terms completed August 2006

13
GWMES Proposal and Schedule Implementation
Plan
Phase One Setting the basis for implementing the
reporting practice 2. Compendium of
development Indicators for SA

• Milestones Achieved
• Based on the Ten Year Review, appropriate set of
  the indicators for five clusters have been
  defined, which comprise of 163 indicators and a
  total of 335 subset indicators that would be used
  to measure the impact of government performance
  in the long term.
• The matrix constitutes eight significant
  variables namely
• Index (Category) Index Description Indicator
  Subset Indicators frequency of data collection
  data source data Sets and Level of
  disaggregation
• Deliberated upon at a workshop in November 2005
  Conference in June 2006
• Indicator refinement definitions by PCAS/NSSD
  Ongoing work
• Discussed at Cluster Level January/March 2006
• Draft baseline document completed circulated
  for comments May Sept 2006
• Baseline data collection completed Sept- 2006
  (for quantitative indicators)

14
GWMES Proposal and Schedule Implementation
Plan
Phase One Setting the basis for implementing the
reporting practice 2. Compendium of
development Indicators for SA (Cont)

• Milestones Achieved
• Data has been collected and has been captured in
  several computer packages (MS Access, MS Excel,
  MS Word and SPSS)
• Integrated all the available databases into one
  usable database - DevInfo in order to achieve a
  
• a common understanding of indicators programme
• Analysis, assessment and evaluation of the
  validity and reliability of SA information
• Ability to collate, disseminate and present both
  government, millennium development goals and
  other international commitments
• The software is available to SADC countries at no
  cost

15
GWMES Proposal and Schedule Implementation
Plan
Phase One Setting the basis for implementing the
reporting practice 3. OPSC assessment of
departmental ME Systems

• Description
• The OPSC was requested to undertake an audit of
  departmental ME systems and reporting
  requirements. The purpose of this report is to
  provide an assessment of what exists in terms of
  ME at a departmental level and an understanding
  of how departments undertake their reporting
  requirements.
• The report is currently being finalised and will
  be presented to various stakeholders soon.
• In the context of the above, it should be noted
  that the PSC is implementing its governance
  monitoring system across departments. To-date
  over 35 such reports have been completed.

16
GWMES Proposal and Schedule Implementation
Plan

• Phase One Setting the basis for implementing the
  reporting practice
• 4. Capacity Building and Accreditation
• Approach
• One core training manual will be developed
  (based on the Oxfam model) in the GWMES, the
  training will be based on the system in
  particular, not just on ME in general.
• Adopting this approach will ensure that many
  self proclaimed ME experts will not be allowed
  to market their training as GWMES training,
  this will be done for government, by government
  agencies.
• Different facilitation guides will ensure that
  the level and complexity target beneficiaries
  will be addressed, in other words, if an ME
  manager is being trained, the complexity and
  depth of the training will be commensurate with
  the ME responsibilities that she has to fulfill.
  The same applies to frontline staff who have very
  basic ME responsibilities, for instance,
  recording information on client contact or
  services rendered.

17
The Proposed Capacity Building Strategy
Assessment of ME capacity needs at provincial
level

• Does the province have an ME Strategy that is
  clearly articulated?

• Has the province conducted departmental ME
  Audits?

• Does the province have an ME Strategy that is
  clearly articulated?

• Does the province have an ME Learning Network?

• What provincial ME capacity building
  initiatives have taken place thus far?

• Is the provincial ME Strategy clearly aligned
  with principles and standards of the GWMES?

• Does the provincial ME Strategy accommodate
  cluster reporting

• Does the provincial ME Strategy provide useful
  information on the Service Delivery Objectives of
  the Office of the Premiere?

• Has the province launched its IDP nerve centre?

18
Conceptual learning framework - capacity
building consist of three dimensions

• Training and development
• Competency-based training - Knowledge, skills and
  values
• Additional seminars/ workshops more detail eg
  Advanced Statistics
• Accredited training to be recognised at tertiary
  institutions further studies
• Morning sessions for Chief directors and above
  advocacy for GWME
• Organisation development
• Consultation and support with development of
  systems, data and reporting
• Change management
• Community of practice
• Principles and practices, learning networks to
  support.
• SAMEA to support the ME Forum for Government
  officials and recognition for officials declared
  competent on training

19
Development of training material

• Focus group engagement to determine learning
  outcomes / topical issues and methodology as
  design framework for training material
• Consultation workshop with Premiers offices and
  key stakeholders on the Curriculum framework
• Approval from ME task team and SAMDI Curriculum
  Review Committee
• Submit funding proposal for development of
  training material (and implementation in key
  departments)
• One training manual differentiate through
  facilitators guidelines and target groups
  identified (Application of learning exercises
  action plans)
• Appointment of persons (internal or external) to
  develop training material and obtain approval
  from Curriculum Committee.
• Quality Assurance and Research on capacity
  development to enhance training and development
  adding value to relevance and implementation.

20
GWMES Proposal and Schedule Implementation
Plan
Phase Two System Functionality, Alignment and
Standardisation. 1. Drafting, consultation
finalization of reporting formats

• Milestones Achieved
• The joint team of NT and DPSA have commenced
  with the development of the requirements of a
  system that would ensure the monitoring of the
  departmental strategic plans The requirements
  will be finalised by the end of October 2006
• The work stream has commenced with the
  development of the framework of an end of year
  report that would complement the current report
  with information currently within the transversal
  systems The framework will be finalised at the
  end of November 2006

21
Annual State of the Nation Report(End of year
report)

• This report will inform/support the January
  Lekgotla and the SONA
• Content
• Cluster annual progress reports based on POA
• Combination of transversal systems information
• Aggregation of departmental strategic plans,
  annual reports and departmental individual
  performance plans
• The reporting format, indicators, collection
  reporting system are still being developed

22
Cluster Performance
Strategic Plan
Admin Manag
Human Resource
Batho Pele
Outcome/Impact
Outputs
Financial
Sector Performance
Dept/Prov Performance
Annual Reports Quarterly Rep.
Reporting Requirements
Reporting Formats 2009
HR Utilisation
Public Man Watch
OPSC
GWME Database
National Treasury
DPLG
POA
23
Current Transversal ME Systems
24
Public Service ME System (OPSC)

• SA Constitution lists 9 basic values and
  principles that govern the public service
• The PS ME System looks at the extent to which
  public service departments comply with these
  principles
• The system aims to
• Gather info that can be used to identify areas
  needing attention, ultimately contributing to
  improved public service performance
• This information is published on an annual basis
  in the State of the Public Service Report

25
Cluster progress Presidency

• This system is based on the matrix of indicators
  which is a set of indicators for each of the
  clusters
• The aim of the system is to track the progress of
  the different clusters in reaching their outcomes
• The system indicates
• Those areas where progress is slow and
  intervention is needed
• Those areas of good progress as examples of best
  practices
• This system is supported by the NSS and reports
  are updated on a quarterly basis

26
Programme of Action (Presidency)

• The system is based on the Government Programme
  of Action that flows from the Presidents State
  of the Nation Address that provides the strategic
  direction for the Public Service
• The aim of the system is to provide an indication
  for progress in implementing the priorities of
  government.
• The system indicates
• Those areas where progress is slow and
  intervention is needed
• Those areas of good progress as examples of best
  practices
• The system is electronic database Excel templates
  and are updated every quarter with a two-month
  lag per quarter as due date for reporting
• Feedback on progress is given on a quarterly
  basis by means of consolidated reports on the
  intranet of Treasury once the quality of the data
  is credible it will be available on the internet

27
In Year Monitoring (National Treasury)

• Monthly monitoring reports
• Focus on performance against budget
• Alert managers where remedial action is required
• Reports consolidated and published
• National Departments monthly publication in the
  national Government Gazette
• Provinces Monthly reporting and monitoring BUT
  quarterly publication in the national Government
  Gazette
• Section 32 Reports

28
Non Financial Reporting (National Treasury)

• This system is based on provincial departments
  reporting against targets set in their Annual
  Performance / Strategic Plans
• The provincial indicators are set per sector to
  enable cross- provincial comparison
• The system indicates
• Those areas where progress is slow and
  intervention is needed
• Those areas of good progress as examples of best
  practices
• This system together with the In Year Monitoring
  System would facilitate cost benefit analysis
• The system is updated on a quarterly basis
• The system has been rolled out but there is still
  a problem with the quality of the data provided
  by departments

29
Public Management Watch (DPSA)

• Early warning type system with regard to
  management within departments
• Consists of the following processes
• Statistical rating based on payroll and financial
  information
• Communication to and self assessment by
  identified departments
• On site assessment of selected department
• Intervention into the management of departments
• The system is functional and will be applied
  every quarter
• The system -
• Indicates those departments/areas in need of
  intervention
• Indicates those departments that have proven to
  be examples of best practices
• Facilitate the knowledge exchange between
  departments
• This system aims to direct the interventions of
  the centre of government to those departments
  that require assistance

30
Annual HR Report (DPSA)

• Annual analysis of the status of HR in the Public
  Service
• The following would serve as information sources
  for this report
• Departmental annual reports
• Public Management Watch findings
• Payroll and Financial Systems
• Reports OPSC, AG, DPSA, etc.
• The system -
• Indicates trends within the HR environment
• Identifies those bottlenecks effecting the
  efficient and effective utilisation of HR
• Identifies those remedial steps taken to address
  the bottlenecks between departments
• This system aims to guide the centre of
  government and HR managers on the areas in need
  of attention

31
GWMES Proposal and Schedule Implementation
Plan
Phase Two System Functionality, Alignment and
Standardisation. 2. Develop ICT platform
architecture to receive, collate, and report on
data collected

• Milestones Achieved
• The development of the IT system is dependent
  upon the finalisation of the user requirements
  (data and reporting requirements)

32
GWME Proposed reporting outputs

• Bi-monthly reports against the government
  Programme of Action (POA)
• Mid-year progress report
• Annual State of the Nation Report (End of year
  report)
• Term of office PS Annual report
• 2014 Review
• Special sector ME reports eg Education and
  Health

33
Management structure of the Institutional
Framework for Implementing the GWMES
Governance Administration Cluster
The Presidency
Intellectual Leadership Strategic Direction
Project Steering Committee
Project Management, Systemic Gearing and Capacity
Building
Project Implementation Tools, Standards,
Templates, Platforms, Training Interventions
ME Task Team
Capacity Building Work Stream guided by a TOR
Principles and Standards Work Stream guided by a
TOR
Reporting and Databases Work Stream guided by a
TOR
34
Institutions Responsible for Implementing Work
Streams
Reporting and Databases
Capacity Building
Principles and Practices
Lead Agency
DPSA
Presidency
SAMDI
Partner Agencies
OPSC
Presidency
Presidency
Stats SA
Stats SA
Stats SA
National Treasury
SITA
DPSA
DPLG
DPLG
DPSA
National Treasury
35
Overall Achievements to date

• A public commitment to creating the system is an
  important first step and is shown in Cabinets
  adoption of the Strategy and Implementation Plan
• Political support (key to public service reform
  at this level) has been consistently provided
• Consistent, sustained membership and involvement
  of core task team members
• Recent agreements on appointment of task teams
  and identification of members to facilitate
  reporting on the programme of action 2006
• POA system is fully functional and promotes
  transparency and accountability
• Indicator development
• Transversal systems have been developed and are
  operational