An Action Research Model

1
An Action Research Model of Program Evaluation
Joseph Montgomery, WSU Tri-Cities Barbara
Peterson, NLA Michael Villarrial, Warden School
District
2
Agenda

• I. Introduction/background
• II. The Action Research model
• III. Feedback mechanisms for WSU Harvest of
• Hope GEAR UP
• Using feedback data to improve services
• the NLA perspective
• V. Using feedback data to improve services
• the school district perspective

3
GEAR UP an Organizational Collaboration
4
Action Research Steps
Note This is a model of learning from your own
situation and taking actions to improve
it.
5
Why Use An Action Research Approach to Evaluating
GEAR UP?

• Provide timely feedback data to service providers
  so they
• know how it went
• Build teamwork, openness across organizations,
  between
• evaluator and providers
• Provide data to WSU Administrator, NLA for
  effective
• management of services
• Provide data to the Board to support effective
  governance,
• strategic direction of project
• Create a clear and realistic picture of project
  performance

6
Examples of Feedback Mechanisms

• Event/activity feedback (short-term)
• Board meeting feedback (quarterly)
• Student survey, focus groups (annual)
• Parent survey, focus groups (annual)
• Teacher survey, focus groups (annual)
• Service provider feedback, interviews
  (quarterly)
• Site Scorecard
• WASL/ITBS trends and predictions (annual)

See Survey Packet
7
Ex. of Board Feedback
Excellent Good Poor Problematic
April
May
8
Survey Item Means April vs May Meetings
Note Significant decrease in ratings for items
4-7 decrease for item 13 approaches
significance.
9
Incorporating Standardized Test Scores into
Evaluation Process Background

• Efforts to Improve elementary school ITBS scores
• Dissatisfaction with WASL feedback from Query
  system
• Additional analyses of WASL datagraphs, figures
• Interest in WASL/ITBS relationships
• Application of Industrial Psychology prediction
• methodologies to WASL/ITBS relationships
  the initial
• prediction algorithms (2000) for 3-4, 6-7,
  9-10
• Providing students predicted scores to teachers
• Assessing 4-7, 7-10 WASL test relationships
• Testing prediction accuracy
• Testing predictive algorithms on other districts

10
WASL Trends
Scale Score
00 01 02
00 01 02
00 01 02
11
Trends in Writing Scores
Scale Score
12
Percent Meeting Standard for each Scale
13
Percentage Meeting Standards by Year
14
Percentage of Students Meeting Standard on each
Scale
15
Percent Meeting Standards by Writing Strands (1)
16
Percent Meeting Standards by Ethnic Group (1)
Listening
Writing
Reading
Math
Hispanic
White
Hispanic
Hispanic
White
White
2002
2000
2001
17
7th WASL Predictions