Evaluation

1
Evaluation

• Discovering Whether Programs Work

2
Measuring Policies Programs

• Program Inputs
• Program Outputs
• Program Impact

3
Policy Inputs

• Government resources allocated to address a
  public problem
• Agency personnel assigned (FTEs)
• Funds allocated by legislature

4
Policy Outputs

• Indicators of government activities in carrying
  out policy
• School Vouchers
• of applications for vouchers
• of vouchers distributed
• of students switching schools
• of failing schools reinvesting in facilities,
  teachers, etc.
• DARE
• of children in the program
• of school districts participating
• Acid Rain
• of SO2 scrubbers installed
• of inspections carried out

5
Policy Impacts

• Degree to which the policy is addressing the
  problem
• Amount of the problem solved
• School Vouchers
• Improved educational attainment among target
  population
• Using vouchers
• Not using vouchers
• Reduction in the of failing schools
• D.A.R.E
• Reduced drug experimentation
• Reduced substance use
• Acid Rain
• Regeneration of aquatic ecosystems
• Reduced damage to acid-sensitive structures

6
Policy Impacts

• Unintended/Unexpected Impacts
• Vouchers
• Impoverishment of public school districts
• Reduction in the educational attainment of
  superior schools
• D.A.R.E
• Improved community-police relations

7
Process Evaluation

• Is the program carrying out the actions it is
  suppose to carry out?
• Measuring program outputs
• Did the program make the public aware of the
  availability of vouchers?
• Does the application process for vouchers
  encourage participation?
• Is government distributing vouchers according to
  the rules?

8
Impact Evaluation

• Is the Program Working?
• Is it reducing the problem as intended?
• Measuring Program Impact
• Are test scores improving? Are fewer students
  dropping out?
• Voucher users
• Voucher non-users
• To what degree are previously failing schools
  attempting to compete reinvesting?

9
Uses for Evaluations

• Go/No Go Decisions
• Improve Practices Procedures
• Expand Program Scope and Reach
• Allocate Resources among competing programs
• Test Policy Theory

10
Stages of Evaluation

• Program Goal Identification
• Construction of Impact Model (Policy Theory)
• Research Design
• Data Collection
• Analysis
• Report/Presentation

11
Program Goal Identification

• Who wants the evaluation?
• Legislature
• Agency
• Policy/Program directors
• Program staff
• Clients
• Interest Groups
• Scholars

12
Policy Theory (School Vouchers)
13
Research Design

• Isolating the causal effects of the program from
  other influences
• Measures of program impact
• How would we know change if we saw it?
• Form of analysis
• Case study
• Focus groups
• Clients
• Program staff
• Statistical analysis
• Data collection (sampling) requirements