Turning Well-Written Program Objectives into High-Quality Extension Impact Statements

1
Turning Well-Written Program Objectives into
High-Quality Extension Impact Statements

• Barry A. Garst, Ph.D.
• Assistant Professor and Extension Specialist
• 4-H Youth Development
• Virginia Tech
• bgarst_at_vt.edu
• 2006 VCE In-Service January 18-20, 2006
• Charlottesville, Virginia

2
Overview

• Importance of well-written program objectives
• Goals vs. objectives
• Characteristics of a well-written objective
• Practice!
• Types of impact statements
• Turning objectives into impacts

3
GETTING STARTED
What types of program evaluation have you been
involved with in the past year (in any setting,
in any position)? In those instances, what did
you evaluate? Why was evaluation important to
you?
4
Evaluation

• allows us to make decisions about our programs
  (What do I keep?, What do I change?, What do I
  discontinue?, What fits best with the 4-H
  mission?, What program makes the greatest impact
  in my community?, etc.)
• allows us to gather information in a systematic
  way using a purposeful process
• leads to decisions and action
• helps us to determine what is compared with
  what should be.

Henderson Bialeschki, 1995
5

• allows us to determine if our programs made an
  impact on participants
• allows us to improve our program effectiveness
• helps us to justify our program expenditures
• provides us with results that we need to report
  back to donors
• provides us with information for future funding
  requests

Allen, Stevens, Hurtes, Harwell, 1998
6

• enhances our credibility (Our programs make a
  difference!!!)
• increases our personal and professional
  satisfaction

Allen, Stevens, Hurtes, Harwell, 1998
7

• Begin with the end in mind.

Covey (1990), 7 Habits of Highly Effective People
8

• What are the goals of our program?
• What do we expect to happen to youth?
• What do we want youth to gain, learn, etc?

9
Goals and Objectives

• Related to the needs, skills, and
  competencies of positive 4-H youth development.
• Helps to establish a direction for the unit 4-H
  program
• Helps to define what resources are needed and how
  (and where) resources will be used
• Helps to define what 4-H programs will be
  provided
• Specifically links desired outcomes to our 4-H
  programs

10
Goals and Objectives

• Goals
• Broad, long-range statements that define the
  programs/services that are going to be provided
• Objectives
• Specific statements (about the attainable parts
  of the goal) that are measurable and have some
  dimension of time.

11
Objectives

• Specific
• Must be clear and concrete
• Measurable
• Must be some way to determine whether or not the
  desired results have been achieved
• Achievable
• Must be attainable and reality-based!!!
• Relevant
• Must be useful must have worth to your
  organization
• Time-limited/Time connected
• Must specify a time frame for accomplishment

Adapted from Edginton, Hanson, Edginton, 1980
12
Situation

• A 4-H Agent learns that many of the youth in
  his/her community lack communication skills.
  Youth are performing poorly in situations in
  which they have to speak in front of groups.
• The 4-H Agent creates a 4-H public speaking
  program to help youth to develop communication
  skills.
• The primary goal of the program is to improve
  youths communication skills.

13
Writing ObjectivesStep 1

• State the performance objective with an action
  verb
• EXAMPLE Youth will demonstrate proficiency in
  giving an oral presentation...
• In this example, the verb demonstrate clearly
  defines acceptable performance.

14
Writing ObjectivesStep 2

• State the method of performance which the young
  person can demonstrate
• EXAMPLE .by standing in front of a peer group
  and discussing a selected topic
• In this example, the phrase standing in front of
  a peer group and discussing a selected topic
  clearly defines how the behavior should be
  performed.

Edginton, Hanson, Edginton, 1980
15
Writing ObjectivesStep 3

• Identify factors or conditions that are necessary
  for success (This forces you to be more
  specific!)
• EXAMPLE which has an introduction, main
  argument, supporting facts, and conclusion
• The phrase which has an introduction, main
  argument, supporting facts, and conclusion...
  identifies the necessary components of the speech.

16
Writing ObjectivesStep 4

• Indicate how the objective will be measured
  (Note There can be more than one way!)
• EXAMPLE in 4-6 minutes.
• In this example, the phrase in 4-6 minutes
  indicates the desired duration of the speech and
  offers one way that the objective can be
  measured.
• EXAMPLE 2 making eye contact with members of
  the group at least 3 times.

17
Writing an Objective- Summary

• State the performance objective with an action
  verb (What behavior do you expect from the
  4-Her(s)?)
• State the method of performance which the young
  person can demonstrate How do you want the
  4-Her(s) to demonstrate the behavior?
• Identify factors or conditions that are necessary
  for successful achievement of the objective Is
  there a certain way (a best way?) to perform the
  behavior?
• Indicate how the objective will be measured
  (e.g., numbers, dimension of time, amount of
  something)

18
Individual or Group

• Remember that goals and objectives can be written
  for an individual (i.e., how you want one
  particular 4-Her to perform/succeed) or a group
  (i.e., how you want a group of 4-Hers to
  perform/succeed).
• Although most of your goals and objectives will
  likely be written from the perspective of group
  performance, be aware that both types can be
  developed.
• Examples of both are provided in the next slides.

19
Putting it All Together

• GOAL To improve youths communication skills.
• OBJECTIVE Each member of a 20 person in-school
  club will demonstrate proficiency in giving an
  oral presentation by standing in front of a peer
  group and discussing a selected topic, which has
  an introduction, main argument, supporting facts,
  and conclusion, in 4-6 minutes.
• (Building on the last one) OBJECTIVE 2 Bill
  will complete a 5 minute presentation to his
  class with no more than five ums and
  maintaining eye-contact with the class at least
  50 of the time.

20
Example 2

• GOAL To improve youths healthy food choices
• OBJECTIVE 1
• 85 of all 4-H club members will actively
  participateby interacting with other group
  members and asking questionsin at least four (4)
  1-hour meetings during a two month period, during
  which nutrition and healthy food choices will be
  taught.
• OBJECTIVE 2
• By the end of a two month period (60 days), 85
  of all 4-H club members will be able to
  successfully identify healthy foods from an
  overall food list and be able to correctly state
  why those foods are considered to be healthy.

21
Example 3

• Goal
• Youth campers will improve their socialization
  skills
• Objective
• At least 95 of all campers in a group of 40
  youth will demonstrate positive social skills,
  through active participation in small and large
  group activities 3-4 times per day, in which they
  are playing, interacting, and talking with other
  youth, for at least 10 minutes per interaction.

22
YOU BE THE JUDGE!
Working alone or in small groups of 2-4 people,
review the following list of objectives.
Whats wrong with each of these? How could
each one be improved?
23
ACTIVITY
Write at least one (1) objective for each of the
following goals -To improve cooperation among
4-H club members -To enhance 4-Hers
problem-solving skills
24
Impact Statements

• An impact statement is a brief narrative
  that summarizes the outcomes of your 4-H
  programming.
• Impact statements should relate to the goals and
  objectives that you developed for your program

25
Impact Statements

• Extension faculty are expected to report impacts
  annually through EFARS.
• Impact statements are used to report the
  difference that Virginia Cooperative Extension
  programs makes in the lives of Virginias
  citizens
• Impacts submitted in past years suggest that
  many Extension Agents struggle with writing
  impact statements

26
Impact Statements

• Remember the logic model!!!
• An impact statement can be related to a
• Short-term outcome
• Medium-term outcome
• Long-term outcome

27
Logic Model
Resources (Inputs)
Activities
Outputs
Outcomes
Impact
Your intended results
Your planned work
W.K. Kellogg Foundation, 2002
28
The Logic Model Outcomes

• Short-term
• (Learning)
• Awareness
• Knowledge
• Attitudes
• Skills
• Opinions
• Aspirations
• Motivations

• Medium-term
• (Action)
• Behavior
• Practice
• Decisions
• Policies

• Long-term
• (Conditions)
• Social
• Economic
• Civic
• Environmental

See handout Some Ideas for Stating Change
29
Impact Statements can be

• Quantitative (Numerical)- Number of participants
  Percentage (75) or number (170 out of 200) that
  experienced change
• Qualitative (Anecdotal)- a short narrative
  description of a program impact, usually from the
  perspective of an individual participant
• Combination of both numeric and anecdotal

30
Impact statements

• (1) Identify the community problem/issue
• (2) Make a connection between the issue and
  something that Extension/4-H is doing to address
  the issue
• (3) Identify the impact of the Extension
  program on the issue (as it relates to specific
  individuals or groups) in measurable terms.

31
Objective ? Impact

• Well-written objectives can be tweaked into
  good impact statements
• THIS SAVES YOU TIME!
• Proper planning with well-written objectives
  makes it easier to write impacts!

32
Objective ? Impact

• Include the relevant portions of your objective
  (audience, action verb, measurable elements such
  as your dimension of time)
• Use specific statements that identify how your
  4-H program made a difference (which resulted
  in (KEY WORDS which resulted in)
• Include results from your evaluation to support
  your impact (e.g., survey results, anecdotal
  statements, interview results, comments from
  parents, responses from teachers, etc.)

33
Objective ? Impact(EXAMPLE 1)

• IMPACT (numeric)
• 75 of 4-H club members (100 youth) learned how
  to give an oral presentation which enhanced their
  communication and socialization skills. According
  to 9th grade teachers at the local high school,
  this program resulted in a improvement in
  academic performance related to in-school
  presentations.

• OBJECTIVE
• 75 of all 4-H club members will demonstrate
  proficiency in giving an oral presentation by
  standing in front of a peer group and discussing
  a selected topic, which has an introduction, main
  argument, supporting facts, and conclusion in 4-6
  minutes.

34
Objective ? Impact(EXAMPLE 2)

• IMPACT (anecdotal)
• 200 high school students completed a 4-H healthy
  foods for healthy kids program. One participant,
  who learned to make healthy food choices during a
  six-week 4-H after-school program, reduced her
  weight and has controlled her diabetes. She
  attributed these results to 4-H

• OBJECTIVE
• Within 6 weeks, 85 of all 4-H after-school
  members will be able to demonstrate an
  understanding of healthy food choices by
  selecting healthy foods from a food list and
  identifying at least one reason why those foods
  are considered to be healthy.

35
Objective ? Impact (EXAMPLE 3)

• IMPACT (numeric)
• Participation by 300 students in a 4-H in-school
  character education contributed to a decrease in
  the rates of cheating within a one-year period,
  according to an in-school survey.

• OBJECTIVE
• 300 4-H in-school youth will learn and practice
  the Six Pillars of Character through
  participation in a one-year Character Counts
  program in which each student will complete at
  least two character-related activity/exercise.

36
Objective ? Impact (EXAMPLE 4)

• IMPACT (anecdotal)
• Participation in a 4-H after-school electricity
  club resulted in a 16-year old 4-Her being able
  to successfully obtain a job in a local
  electronics repair shop. This teen shared, I
  didnt know a lot about fixing electronics, but
  the basic understanding of electronics that I
  learned through 4-H helped me in my interview

• OBJECTIVE
• 90 of all 4-Hers who are enrolled in a 4-H
  electricity project book will demonstrate a basic
  competency in electronics by creating a working
  circuit within the first month participation in
  an after-school 4-H electricity club.

37
Objective ? Impact (EXAMPLE 5)

• IMPACT (numeric)
• Pretest vs. posttest scores on a citizenship
  skills assessment demonstrated that 75 of
  participants in a 4-H teen club (35 teens)
  learned parliamentary procedure. This leadership
  skill resulted in a higher functioning 4-H club
  which contributed 100 hours of community service
  within the next year.

• OBJECTIVE
• 75 of all teens will successfully demonstrate a
  knowledge of parliamentary procedure by
  successfully leading a group through the process
  of making a motion within the first 4 weeks of
  participating in a 4-H teen club.

38
Objective ? Impact(EXAMPLE 6)

• OBJECTIVE
• At least 95 of all campers (120 youth) will
  demonstrate positive social skills, through
  active participation in small and large group
  activities 3-4 times per day, in which they are
  playing, interacting, and talking with other
  youth, for at least 10 minutes per interaction.

• IMPACT (both)
• 120 campers ages 9-13 completed small and large
  group activities during 4-H camp which emphasized
  positive social skills. On a post-camp survey,
  68 of parents indicated that their children
  were demonstrating more positive social behaviors
  at home following camp. One parent reported that
  her son seemed more likely to share with his
  brother.

39
ACTIVITY
Using the objectives that you wrote earlier in
this session, write one quantitative (numerical)
impact statement and one qualitative (anecdotal)
impact statement
40
References

• Allen, L., Stevens, B., Hurtes, K., Harwell, R.
  (1998). Benefits-based programming of recreation
  services- Training Manual. Ashburn, VA The
  National Recreation and Park Association.
• American Evaluation Association. (2002). Guiding
  Principles for Evaluators. Retrieved on October
  7, 2002 from http//www.eval.org/Publications/publ
  ications.htmlGuiding Prin
• Arnold, Mary E. (2002). Program Planning and
  Evaluation A computer-aided guide for faculty.
  Unit one Introduction to program planning and
  evaluation. The Oregon State Extension
  Association. CD-ROM.
• Edginton, C.R., Hanson, C.J., Edginton, S.R.
  (1980). Leisure programming concepts, trends,
  and professional practice. Dubuque, IL Wm. C.
  Brown Communications, Inc.
• Henderson, K.A. Bialeschki, M.D. (1995).
  Evaluating leisure Services Making enlightened
  decisions. State College, PA Venture
  Publishing, Inc.
• McDonald, G.B. Valdivieso, R. (2001).
  Measuring deficits and assets How we track
  youth development now, and how we should track
  it. In Jaffe, N. (ed.), Youth development
  Issues, challenges, and directions.
  Public/Private Ventures. Retrieved on September
  10, 2002 from http//www.ppv.org/content/reports/y
  dv_pdf.html.
• Virginia Cooperative Extension. (2000). VCE
  Program Evaluation and Reporting. Retrieved on
  November 2, 2000 from http//www.ext.vt.edu/vce/ed
  ucprogspt/prgmdev/process/process.html.
• W.K. Kellogg Foundation. (2002). Logic Model
  Development Guide. Retrieved on October 7, 2002
  from ttp//www.wkkf.org/Knowledgebase/Pubs/Resourc
  eOverview.asp?CID0ID3669