Extension Performance Evaluations

1
Extension Performance Evaluations

• A Review and Revision Process
• Prepared by Dr. Wendy V. Hamilton, New Mexico
  State University, CES

2
Why Review/Revise Performance Evaluation Doc?

• WHY REVIEW?
• To help the organization remain on task with what
  is mission-important.
• To eliminate duplication of work (could
  performance evaluations and promotion tenure
  documents be combined?).
• To increase sound practices in the evaluation
  process.

3

• Why Review (continued)
• To ensure a fair and balanced evaluation system
  for evaluatees, evaluators, and users.
• To promote evaluation practices that help the
  organization attract, develop, and retain
  qualified faculty.
• To increase job satisfaction and performance by
  reinforcing positive accomplishments .

4

• Why Review (continued)
• To ensure evaluations are just and defensible.

5

• WHY REVISE?
• To improve the evaluation system so it can be
  useful in employee
• selection,
• assignment,
• reassignment,
• promotion,
• tenure,
• staff development,
• recognition, and other decisions.

6

• Why Revise (continued)
• To correct deficiencies in some current practices
  such as reaching diverse audiences and
  identifying program impacts.
• To reflect unique or individualized positions
  that identify assigned duties and relevant areas
  of scholarship more fully.

7

• Why Revise (continued)
• To help establish new policy initiatives
  (performance ratings, merit pay, etc).
• To ensure and provide quality assurance for
  diversity in the workplace.

8

• Guiding Assumptions for Reviewing and Revising
  Performance Evaluations
• The fundamental purpose of performance
  evaluations is to help provide effective services
  to clientele.
• Performance evaluations should be constructive
  and free of unnecessary, threatening or
  demoralizing statements.

9

• Guiding Assumptions (continued)
• Performance evaluations should be conducted in
  the context of cultural competence (paying
  attention to age, gender, ethnicity, disability,
  religion, etc).
• Performance evaluations should point the way to
  new learning directly related to the evalutees
  needs, clientele interests, and job
  responsibilities.

10

• Guiding Assumptions (continued)
• Users must define their organizations
  expectations, conditions, approaches, performance
  levels, and role definitions before applying any
  Personnel Evaluation Standards to the evaluation
  tool.
• Not all 27 personnel standards may apply to the
  organizations performance evaluations.

11

• Next Presentation Two
• The Personnel Evaluation Standards2nd Edition
• Why Use This Tool?

12
The Personnel Evaluation Standards 2nd Edition
How to Assess Systems for Evaluating Educators

• Why Use This Tool?
• Prepared by Dr. Wendy V. Hamilton, New Mexico
  State University, CES

13
Why Use This Tool?

• Its well tested and is being field tested for
  the second time in ten years.
• It allows for maximum flexibility in
  organizational tool development.
• It is based on extensive literature reviews, case
  studies, and development since 1981.

14
Why Use This Tool?

• It contributes to overcoming much of the anxiety,
  controversy, and litigation often accompanying
  performance evaluation tool development and
  implementation.

15
Why Use This Tool?

• It represents national and international
  consensus on what is most important to sound
  performance evaluation, personnel systems, and
  the requirement that performance evaluations be
  ethical, fair, feasible, and accurate.

16

• Next Presentation Three
• The History of Personnel Standards

17
The History of Personnel Standards

• Prepared by Dr. Wendy V. Hamilton, New Mexico
  State University, CES

18
The History of Personnel Standards

• 1800s Accreditation of schools movement began.
• 1950s Evaluation of student performance and
  standardized testing began.
• -Publication of Standards for Educational and
  Psychological Tests (NEA, 1955, APA, 1966, 1974,
  1985)

19
The History of Personnel Standards

• 1960s Project evaluations, especially for
  externally funded projects started.
• - The Standards for Evaluations of Educational
  Programs, Projects and Materials were published
  (Joint Committee on Standards for Educational
  Evaluation 1981).

20
The History of Personnel Standards

• As continued evaluation of programs and
  students in the 1970s and 1980s revealed
  further deficiencies in student performance and
  program quality, pressure increased dramatically
  for focusing accountability on individual
  educators (Joint Committee, 1994, pg 161).

21
The History of Personnel Standards

• As a consequence, the professional societies in
  education have increased their efforts to develop
  sound personnel evaluations and, as one measure,
  fourteen agencies are supporting the Joint
  Committee to develop standards by which to plan
  and assess personnel evaluation in .educational
  agencies (Joint Committee, 1994, pg 162).

22
The History of Personnel Standards

• NM CES is field testing the 2nd version of the
  personnel evaluation standards as a tool to
  reviewing and revising our current agent and
  specialist annual performance documents.
• Involvement with the field test began through an
  invitation from the American Evaluation
  Association of which NM CES is a member.

23
The History of Personnel Standards

• Our national contact for the field test, Dr.
  Barbara Howard, is not aware of Extension ever
  having participated in this or the previous field
  test.
• By participating in the field test, NM CES
  becomes involved with a long and distinguished
  list of universities, foundations, and
  organizations.

24

• Next Presentation Four
• The Field Test
• Why Involve Extension in the Field Test?

25
The Field Test

• Why Involve Extension in the Field Test?
• It is timely.
• Current performance evaluations create
  duplication of work.
• Documents may not adequately measure
  individualized assigned duties.
• Documents may not adequately report relevant
  areas of scholarship.

26
The Field Test

• What Will The Field Test Involve? (contd)
• A committee comprised of faculty at all levels of
  NM CES.
• The review, revisions and pilot testing of newer
  documents.
• A concentrated period of work (estimated Sept 05
  Jan 06).

27
The Field Test

• What Will The Field Test Involve? (contd)
• Collaborative input from evaluators, evaluatees,
  and users from throughout the NM CES system.
• A results report to the Joint Committee (Dr.
  Barbara Howard, Chair) describing which of the 27
  standards we incorporated into our performance
  documents and why.

28

• Next Presentation Five
• The Personnel Standards How are they organized?

29
How Are the Personnel Standards Organized?

• Prepared by Dr. Wendy V. Hamilton, New Mexico
  State University, CES

30
The Personnel Standards

• How Are They Organized?
• There are 27 different standards that can be used
  in a variety of ways in performance evaluation.
• These standards are grouped into FOUR areas
• Utility
• Propriety
• Feasibility
• Accuracy

31
The Personnel Standards

• How Are They Organized? (continued)
• Propriety The extent to which an evaluation
  will be conducted legally, ethically, and with
  due regard for the welfare of those involved in
  the evaluation as well as those affected by its
  result.
• Fairness, equity, legal viability of the system.

32
The Personnel Standards

• How Are They Organized? (continued)
• Examples of Propriety Standards
• Service Orientation
• Appropriate Policies and Procedures
• Access of Evaluation Information
• Interactions with Evaluatees
• Balanced Evaluation
• Conflict of Interest
• Legal Viability

33
The Personnel Standards

• How Are They Organized? (continued)
• 2. Utility The extent to which an evaluation
  will serve the relevant information needs of
  evaluatees and other users.
• Usefulness of the evaluation results including
  identification of users and uses, evaluation
  criteria, evaluator credibility, and follow up
  (professional development).

34
The Personnel Standards

• How Are They Organized? (continued)
• Examples of Utility Standards
• Constructive Orientation
• Defined Uses
• Evaluator Credibility
• Explicit Criteria
• Functional Reporting
• Professional Development

35
The Personnel Standards

• How Are They Organized? (continued)
• 3. Feasibility The extent to which an
  evaluation is appropriate and practical for
  implementation.
• Feasibility of use and effectiveness, including
  cost-effectiveness, especially of human resources
  needed for implementation.

36
The Personnel Standards

• How Are They Organized? (continued)
• Examples of Feasibility Standards
• Practical Procedures
• Political Viability
• Fiscal Viability

37
The Personnel Standards

• How Are They Organized? (continued)
• 4. Accuracy The extent to which an evaluation
  conveys technically adequate information about
  the performance and qualifications of an
  evaluatee.
• Justification of evaluation conclusions based on
  reliability and soundness of data collected.

38
The Personnel Standards

• How Are They Organized? (continued)
• Examples of Accuracy Standards
• Validity Orientation
• Defined Expectations
• Personal and Context Analysis
• Documented Purposes and Procedures
• Defensible Information

39
The Personnel Standards

• How Are They Organized? (continued)
• Examples of Accuracy Standards (contd)
• Reliable Information
• Systematic Data Control
• Bias Identification and Management
• Analysis of Information
• Justified Conclusions
• Metaevaluation

40

• Next Presentation Six
• Difficulties in Applying Standards
• Some Things to Consider

41
Difficulties in Applying Standards

• Some Things to Consider
• Prepared by Dr. Wendy V. Hamilton, New Mexico
  State University, CES

42
Difficulties in Applying Standards

• Some Things to Consider
• There are many unknowns yet to be solved in the
  practice of performance evaluation design.
• And, while the best thinking of many
  knowledgeable and experienced people is
  represented here, not all questions may be
  predicted. But some things to consider include..

43
Difficulties in Applying Standards

• Some Things to Consider (continued)
• There may be a common lack of understanding or
  agreement about what constitutes a good
  performance evaluation and how to measure those
  elements.
• Suggestion allow time for discussion and
  consensus building.

44
Difficulties in Applying Standards

• Some Things to Consider (continued)
• 2. There are some evaluators who are inclined to
  evaluate personal characteristics rather than
  qualifications, performance or effectiveness. Be
  certain to aim the standards at evaluations of
  qualifications, performance and/or effectiveness.

45
Difficulties in Applying Standards

• Some Things to Consider (continued)
• 3. Be sure everyone clearly understands who is to
  receive personnel evaluation reports, and how
  they are to be used. Be sure safeguards are in
  place whenever necessary to avoid misuse of
  reports.

46
Difficulties in Applying Standards

• Some Things to Consider (continued)
• 4. There may be times, such as a reduction in
  force, when the ground rules for the performance
  evaluation change. Be sure that the standards are
  reviewed whenever the system changes.

47
Difficulties in Applying Standards

• Some Things to Consider (continued)
• 5. Caution evaluators to not just look at
  achievement of performance goals but to also look
  for unintended or unexpected achievements, as
  well. The personnel standards remind us to
  broaden our view beyond the expected or intended.

48
Difficulties in Applying Standards

• Some Things to Consider (continued)
• 6. There may be feelings of frustration,
  hopelessness, or a stifling effect when the
  standards are applied and many deficiencies or
  inadequate resources prevent the evaluation
  system from
• working ideally.
• Keep in mind that few things in life are ideal
  and the standards guide the practice in the best
  possible way. Development of a high quality
  evaluation system may be a work in progress.

49
Difficulties in Applying Standards

• Some Things to Consider (continued)
• 7. There may be times when it is difficult to
  judge whether a standard has been met.
• Professional judgments are required in applying
  the standards.
• A committee may enhance the soundness and
  defensibility of those judgments.

50
Difficulties in Applying Standards

• Some Things to Consider (continued)
• 8. Can you think of other difficulties that may
  exist when applying performance evaluation
  standards in NM CES?

51

• Next Presentation Seven
• Applying the Standards as a Group
• Remind Me Again Why Were Doing This

52
Applying the Standards as a Group

• Remind Me Again Why Were Doing This
• Prepared by Dr. Wendy V. Hamilton, New Mexico
  State University, CES

53
Applying the Standards as a Group

• Remind Me Again Why Were Doing This
• Reason 1 The Performance Standards as a group
  provide guidance in developing, reviewing, and
  revising personnel evaluation systems.
• Reason 2 The Performance Standards enable
  evaluators to answer questions about design,
  implementation and effectiveness of an evaluation
  system. They address questions such as..

54
Applying the Standards as a Group

• Remind Me Again Why Were Doing This
• Is our evaluation system well designed?
• - Does it look good on paper?
• - Is the design flawed? If so, how?
• - How can it be improved?

55
Applying the Standards as a Group

• Remind Me Again Why Were Doing This
• Is our evaluation system properly operated?
• - Is it being carried out in accordance with
  its design and accepted standards?
• - If not, what corrective actions are indicated?

56
Applying the Standards as a Group

• Remind Me Again Why Were Doing This
• Is the evaluation system having the impact it
  should?
• - Does it have the intended effects?
• - Does it have other unwanted effects?
• - What has to be done to improve the outcomes?

57
Applying the Standards as a Group

• Remind Me Again Why Were Doing This
• By systematically answering these fundamental
  questions, NM CES can assure that their
  evaluation systems are well conceived, being
  implemented correctly and achieving the desired
  end results (Joint Committee on Standards, 1994,
  pg 125).

58

• Next Presentation Eight
• Steps in Evaluating the Evaluation System
• Steps in the Evaluation of a Performance
  Evaluation System

59
Steps in Evaluating the Evaluation System

• Steps in the Evaluation of a Performance
  Evaluation System
• Prepared by Dr. Wendy V. Hamilton, New Mexico
  State University, CES

60
Steps in Evaluating the Evaluation System

• Steps in the Evaluation of an Evaluation System
• Step 1 Become familiar with The Personnel
  Evaluation Standards.
• Step 2 Clarify the purposes of the evaluation
  system.
• Step 3 Describe our evaluation system.

61
Steps in Evaluating the Evaluation System

• Steps in the Evaluation of an Evaluation System
• Step 4 Apply agreed upon standards to existing
  system.
• Step 5 Review evaluation results collected and
  decide if newly revised evaluation tools are on
  effective for the organizations needs.

62
Steps in Evaluating the Evaluation System

• Steps in the Evaluation of an Evaluation System
• Details of Step 1 Become familiar with The
  Personnel Evaluation Standards.
• Ensure that committee members AND total
  organization become familiar with standards.
• Keep organization abreast of committee work.
• Offer training to assist personnel in learning
  about performance standards and how to apply
  them.

63
Steps in Evaluating the Evaluation System

• Steps in the Evaluation of an Evaluation System
• Details of Step 2 Clarify the purposes of the
  evaluation system and answer these questions.
• Whose work is to be evaluated?
• Why should the evaluation be done?
• Who will use the findings?

64
Steps in Evaluating the Evaluation System

• Steps in the Evaluation of an Evaluation System
• Details of Step 2 (continued)
• What decisions will be determined in response to
  evaluation data?
• Should evaluations focus on qualifications,
  performance and/or effectiveness?
• What impact is the evaluation system intended to
  have?

65
Steps in Evaluating the Evaluation System

• Steps in the Evaluation of an Evaluation System
• Details of Step 3
• Assemble all documents relevant to the evaluation
  system including
• Policy statements
• Job descriptions
• Letters of employment
• Personnel policies
• Procedure manuals
• Reporting tools
• Rating forms

66
Steps in Evaluating the Evaluation System

• Steps in the Evaluation of an Evaluation System
• Details of Step 3 (continued)
• Use the documents to develop a written
  description of the evaluation system. Answer
• How are evaluations staffed and what are
  qualifications of evaluators?
• What are relevant policies?
• What questions are addressed in tool?
• What data are collected?

67
Steps in Evaluating the Evaluation System

• Steps in the Evaluation of an Evaluation System
• Details of Step 3 (continued)
• How are data analyzed, interpreted and reported?
• How is the evaluation system organized,
  scheduled, managed and monitored?
• What follow-up activities occur?

68
Steps in Evaluating the Evaluation System

• Steps in the Evaluation of an Evaluation System
• Details of Step 3 (continued)