Evaluation

1
Evaluation
2
Evaluating The Organization

• Effective evaluation begins at the organizational
  level. It starts with a strategic plan that has
  been carefully developed.
• The plan provides a clear set of measurable
  objectives, timelines, and activities to
  accomplish the objectives.
• Therefore organizational evaluation becomes a
  simple task of tracking the objectives and the
  related time lines and quantifying the results.

3
Timing

• The timing of organization evaluations is more
• long-term than the program/course evaluations
• because the objectives themselves are long term.
• The financial aspects of the operation may be
• evaluated monthly, quarterly, or annually.
• Professional development of the staff is usually
• evaluated annually.

4
Additional Methods

• In addition to the concept of self-evaluation
  (measuring your objectives against outcomes),
  there are other ways to conduct organizational
  evaluations. These efforts involve some form of
  peer evaluation and include
• 1. An operational audit - Similar to a financial
  audit. A third party closely examines the entire
  CE process to determine how effectively they meet
  their objectives.
• 2. A team peer review - A team of CE
  professionals performs a brief, intensive review
  of the CE operation. Accrediting agencies use
  this model. However, it does not have to be
  limited to accrediting visits. For example, ACRU
  CE professionals could be invited as teams to
  visit your program. The benefit of this type
  evaluation is that both the reviewers and the
  team reviewed learn.

5

Program/Course Evaluation

• A typical list of the purposes of course
  evaluation, provided in a 1994 LERN publication,
  (Report 1J10) includes
• 1. To find out how the class is doing. As an
  administrator, you cannot attend all your
  classes. The frequent question asked is "How are
  they doing?" An evaluation can help you answer
  this question.
• 2. To weed out the least effective teachers.
  Course evaluations are a good way to determine
  which teachers are ineffective and should not be
  teaching for you. They provide valuable feed back
  on which to base your evaluation of instructors.

6
Program/Course Evaluation

• Course Evaluation (Continued)
• 3. To improve or help teacher. This is one of the
  most valuable uses for course evaluations. Almost
  all of your teachers will perform well, but a few
  may experience some problems. Most problems can
  be solved. By helping teachers with their
  specific problems, you can significantly improve
  the quality of their classes.
• 4. To provide data on the quality of learning or
  participant satisfaction. Every educational
  organization must be able to gauge the quality of
  its programs and the level of learning that
  occurs.

7
Program/Course Evaluation

• Course Evaluations (continued)
• Again, the overall objectives of the course
  evaluations must be derived from the objectives
  defined in the strategic plan. Some of these
  general objectives may be
• 1. quality control
• 2. improvement.
• Areas that may need improvement include
• 1. marketing efforts
• 2. professional development
• 3. client communications
• 4. accountability to the institution

8
Happiness Index

• The most frequently used instrument to evaluate
  the courses is one referred to as the happiness
  index. It is usually administered at the end of
  the course and includes rating the instructor,
  rating the materials, rating the facilities,
  rating the presentation, and rating the content.
  It usually makes no effort to measure the
  progress toward accomplishing the intended
  objectives. The evaluation is positive if the
  students are "happy.

9
Suggestions

• Given the limitations described above, the
  following suggestions can help with evaluation
  efforts
• 1. Evaluations should be made as soon as possible
  after the event. In some cases, evaluations
  repeated at a later date also provide valuable
  information (Did the program actually change
  behavior?)
• 2. If the evaluation is a long one, divide it
  into two or more sections to be completed at
  different times.
• 3. For very large groups, ask participants at
  random to complete evaluations.

10
Suggestions

• Suggestions (continued)
• 4. Use brief, simple evaluation forms. More
  people are willing to respond to check-off lists,
  but more information may be obtained from
  open-ended questions.
• 5. Video taped sessions can be analyzed late for
  both content and methods.

11
Suggestions

• Suggestions (continued)
• 6. Programmers and administrators may obtain
  valuable information by informally interviewing
  participants at breaks and other convenient
  times. The following questions may be asked
• What did you expect from the event?
• How well did the event meet your expectations?
• Is anything missing from the event?
• How will you use what you have learned?
• What changes will you make based upon this
  experience?
• How would you improve the program?

12
Types of Questions

• Evaluation instruments are usually made up of
  three types of questions
• 1. Ones that may be checked off
• 2. Ones that ask participants to rank various
  elements
• 3. Ones that are open-ended and require the
  participants to provide their own answers.

13
Sequence

• The first questions should be broad-based and
  easy to answer. Their content should be about the
  overall quality of the program.
• As the participant moves further into the
  instrument, the questions may become more
  detailed such as various inquiries about the
  instructor and the program content.
• Open ended questions should be assigned to
  the end of the document. Make these questions
  optional to maximize the chances that
  participants will at least respond to earlier
  questions.

14
Typical Questions

• 1. Overall evaluation
• Were you satisfied with the course?
• Were the marketing materials for the course
  accurate?
• Was the course worth your time and money?
• What monetary value would you place on the
  course?
• Would you recommend the class to a friend?

15
Typical Questions

• 2. Teacher evaluation
• Was the instructor well prepared?
• Did the instructor know the subject?
• Did the instructor communicate the information
• well?
• Did the instructor manage the class well?
• Did the instructor encourage full class
  participation?

16
Typical Questions

• 3. Other Questions
• Were facilities comfortable and conductive to
  learning?
• Was the administrative support for the course
  positive?
• What did you like about the class?
• What, if anything, would you change?

17
Typical Questions

• 4. Testimonials
• Since no evaluation is typical, you should
  customize each
• evaluation to the extent possible and "sell" the
  
• participants on the concept that their input is
  extremely
• valuable and will be carefully reviewed and
  implemented.

18
Techniques For Return

• There are a number of techniques to encourage the
  participants to return completed evaluations.
• One example includes providing a gift to a
  participant based upon a randomly selected
  evaluation.
• Another is to provide additional subject matter
  such as a book or a writing tablet for everyone
  who completes and returns an evaluation.

19
Evaluation and Marketing

• Positive evaluations are one of the strongest
  course marketing elements that you can use to
  communicate the value of your program. When
  participants are willing to make positive
  comments that can be used in marketing materials,
  you are able to add credibility to your programs.
  It is important that they agree to be quoted. It
  is also important to identify their organization
  if they agree to provide that information.