Evaluation Plan for HRSDC 20042005 to 20082009

1
Evaluation Planfor HRSDC2004-2005 to 2008-2009

• Performance Breakfast Series
• January, 2005

2
Outline

• Changing Context for Evaluation
• Key Principles for Evaluation Planning
• Evaluation Planning Process
• The HRSDC Evaluation Universe
• Establishing the Priorities for Evaluation
• Five-Year Plan for Evaluation
• Relation to Audit Planning
• Appendices
• PAA
• Logic Model
• Definition of Evaluation Products

3
Changing Context for Evaluation

• Greater demand for accountability and reporting
  on results
• TBS Evaluation Policy (2001)
• Modern Comptrollership
• Management Accountability Framework
• Expenditure Reviews ? Timely and objective
  information on
• program relevance
• performance
• cost-effectiveness
• Increased need for strategic information,
  analysis and advice
• Policies and Programs cut across several
  departments (e.g. YES, LMDAs, )
• Performance and results information at a more
  strategic level than typically considered in past
  evaluations (e.g. Program Activity level vs
  Sub-Sub Activity)
• Relationship between Evaluation and program and
  policy development
• Contribute to an integrated social and labour
  market policy framework
• Broader role for Evaluation in meeting
  departmental wide challenges
• Completeness of performance measures and
  performance information
• Support to planning, priority setting,
  expenditure review

4
Key Principles for Evaluation Planning

• Primary client is Deputy Minister and senior
  management
• The Plan is strategically focused to serve policy
  development
• Mandatory requirements of central agencies must
  be met
• The Plan is aligned with Program Activity
  Architecture (PAA)
• Strong quality assurance process must be
  maintained
• Reporting evaluation results will be more timely
• Resources availability is an issue
• Demand for evaluation exceed resourcing
• Stable longer term funding is required
• The Evaluation need to be flexible enough to
  respond to emerging needs
• Evaluation Plan will be reviewed and updated
  annually

5
Evaluation Planning Process
6
HRSDC Evaluation Universe

• Why using Program Activity Architecture?
  (Appendix 1)
• Represents the full set of programs, policies and
  major initiatives
• Identify
• Departmental strategic outcomes
• Information requirements to address performance
  indicators
• TBS expects the department to use PAA for budgets
  and performance reporting to Parliament
• For evaluation planning purpose
• Strategic evaluations will be conducted at the
  Program Activity Level (7)
• Typical program evaluation will be concentred at
  the sub-activity level (36 sub-activities) over a
  5-7 year cycle
• Evaluations will be conducted at the sub-sub
  Activity (50) when they are mandatory

7
Establishing the Priorities for Evaluation

• Priorities will be updated annually ? Higher
  priority projects will be conducted earlier

8
(No Transcript)
9
(No Transcript)
10
Relation to Audit Planning

• Audit Universe also starts with the PAA but has
  more modifications to reflect control orientation
  
• e.g. no specific GC programs in the audit
  universe
• AU has 8 primary components comprising 27 major
  auditable units
• Audit Planning is two-pass process
• First pass (periodic) to determine inherent risk
  as a basis for determining cyclicality -- similar
  factors to evaluation
• Materiality (spending) Materiality (persons
  affected)
• Public Confidence Mandatory TB requirement
• Level of Discretion Use of Third Parties
• Second pass (annual) to determine focus and scope
  of individual projects
• Adequacy of control design Time since last audit
• Change in risk environment

11
Appendix 1PROGRAM ACTIVITY ARCHITECTURE (PAA)
12
Appendix 2LOGIC MODEL
13
Appendix 3Definitions of Evaluation Products

• Formative Evaluation
• clarity of program objectives assess operational
  and organizational issues
• adequacy of performance measures and of data
  availability to measure program performance and
  results
• Summative Evaluation
• continued need for the program
• program performance and success
• cost-effectiveness and potential alternatives
• Special Study
• provide performance information and analysis to
  serve a variety of intended uses and corporate
  needs
• Strategic-level Evaluation
• address broader horizontal issues that may cross
  several program areas or jurisdictions and serve
  to support policy or broad program strategy
  reviews