Impact and outcome evaluation

1
Impact and outcome evaluation

• Prof Hal Swerissen
• Director
• Australian Institute for Primary Care

2
Needs
Population Funding
Inputs
Activities
Outputs
Impact
Outcome
Outcome
Cost
Impact
Process-implementation
Process-output
Technical efficiency
Allocative efficiency
3
Impact/outcome evaluation
Causative factors
Population Needs
Factors mediating service seeking
Resource Allocation
Expressed Need
Latent Need
Service delivery
Waiting
Outcome
Substitution
Outcome
Direct demand management
Outcome
Prevention program
Outcome
4
Controlling external validity

• Random sampling
• Quota sampling
• Qualitative sampling
• Plausibility and program logic

5
Impact and outcome specification

• Specification of Impact and outcome
• or
• specification of program goals and objectives

6
Outcomes

• Physical/biological (e.g. mortality)
• Functional/
• behavioural (e.g. ADL, IADL)
• Psychological (e.g. cognitive function)
• Subjective utility (e.g. QoL)

• Economic (e.g. lost work time)
• Social (e.g. community capacity, equity)

7
Impact examples

• smoking rates for lung cancer reduction
• immunisation rates for Comm Disease reduction
• weight loss for diabetes morbity reduction
• adherence to asthma management plan for unplanned
  admission reduction

8
Goals and objectives

• Goals general statement of intention (e.g.
  increase immunisation rates for happy valley)
• Objectives operationalisation of a goal by
  quantity, quality, target group and time (e.g.
  increase immunisation by 10 for preschool
  children consistent with agreed schedule by
  December 2002)

9
(No Transcript)
10
Measurement

• Data sources
• quantitative measures
• qualitative measures

11
Data sources

• New measures
• surveys
• interviews
• observations

• Program measures
• Client records
• Client surveys
• Complaints etc

12
Measurement

• Quantitative
• minimum data sets and agency performance measures
• surveys
• established tools and indicators
• Structured observation

• Qualitative
• Open and semi structured interviews
• Focus groups
• Participant observation

13
Interpretation

• What were the impacts/outcomes oberved?
• Did the impact/outcome achieve the program
  objectives and goals?
• Did the program cause the impact/outcome?
• Were there any unintended positive impacts or
  outcomes?
• Were there any unintended negative impacts or
  outcomes?

14
Use

• Rational
• Decide whether to continue or discontinue the
  program
• Decide whether to modify the goals and objectives
• Decide whether to change the strategies
• Strategic
• Promote the program
• Disseminate and generalise the results

15
Controlling external validity

• Random sampling
• Quota sampling
• Qualitative sampling
• Plausibility and program logic