Outline

1
Outline

• Definitions of Research/Evaluation
• Purposes of Evaluation / Research
• Science and Scientific Management
• Research/Evaluation as Process
• Types of Research / Evaluation

2
Definitions

• Evaluation Process of judging the merit or
  worth of something
• Research
• application of scientific methods to answer
  questions
• controlled inquiry directed at increasing
  knowledge/establishing truth
• Evaluation Research - combine the two

3

• Research Evaluation

Evaluation Research
4
Why Evaluate - the Academic list

• To assess merits of alternative programs
• To discover whether how well objectives are
  being fulfilled
• To determine the reasons for successes
  failures.
• To uncover the principles underlying a successful
  program.
• To refine, revise, update or track a program

5
Why Evaluate- "The Real List"

• 1. Because we are required to
• 2. To make better decisions.
• 3. To learn from experience
• 4. To justify programs
• 5. To kill programs

6
Purposes of Research

• Answer management questions - applied research
• For sake of knowing - pure or basic research
• Answer research questions - methodological
• Develop/test management alternatives -
  developmental research
• Assess worth or merit of programs - evaluation
  research

7
Research Purposes - II

• Exploratory
• Descriptive
• Explanatory
• Predictive

8
Ways of Knowing or Establishing belief or truth

• Tradition
• Authority
• Repetition/Tenacity
• Science

9
Science

• Body of Knowledge
• systematic
• abstract
• general
• parsimonious

• Method of Inquiry
• logical
• induction
• deduction
• self-corrective
• empirical

10
Scientific Management

• Application of scientific principles to
  management and decision making
• systematic information gathering
• empirical, objective, self-corrective

11
Process -- Steps

• Research
• define problem
• objectives/hypotheses
• literature review
• research methods
• gather data/analysis
• conclusions

• Evaluation
• describe program
• evaluation criteria
• program scoping
• evaluation methods
• gather data/analysis
• conclusions

12
Types of Evaluationby Program Stage

• formative (conceptualization/design)
• process (implementation)
• summative (outcomes, impacts, efficiency)

13
Types - By Approach

• Standards
• norm-based
• criterion-referenced
• Goals and objectives
• Impacts or effects

14
Types- by method

• Qualitative - quantitative
• Secondary data - Primary
• Survey - Experiment
• Internal vs external
• Who - Peers, participants, superiors,
  subordinates, scientists

15
Evaluation Criteria

• Effort - qnty and qlty of inputs
• Performance - qnty and qlty of outputs
• Adequacy - meet needs?
• Efficiency - benefits/costs
• Equity - distributional issues, fairness

16
Process Evaluation

• Identifies how and why program works
• attributes
• recipients
• conditions
• effects
• single or multiple
• intended or side effects
• timing duration, long/short term
• cognitive, affective or behavioral

17
Types by program area/subarea

• Emphasis Parks, Tourism, Interp, Program Mgmt,
  Commercial Rec, ...
• Program lands, facilities, personnel, budget,
  marketing, PR, maintenance, design, policies,
  plans

18
Types of Research

• Basic - Applied
• Exploratory, Descriptive, Explanatory, Predictive
• In-house / out- house
• by discipline - sociological, psych, economics
• by topic - boating, wilderness, legal, tourism,
  ...
• by method - survey, expmt, ...

19
Examples of Evaluation Studies

• Community needs assessment
• Feasibility study, SWOT analysis
• Program Audits, Market audit
• Cost/benefit anaysis
• Impact assessment
• PPBS, MBO, GPRA, CAPRA, LAC, ...
• Peer review
• Importance-performance analysis