Developing and Validating an Assessment Measure

1
Developing and Validating an Assessment Measure
2
Goals, Objectives Criteria

• It is critical that employees have a clear
  understanding about what part of their
  performance is appraised and how it will be
  measured.  
• Performance can be measured according to
  previously established and mutually understood
  goals between the employee and the supervisor.
• Goals should be challenging but attainable.
• Goals must be related to the core
  responsibilities.
• After the goals and objectives are established,
  criteria against which the employee's job
  performance can be evaluated are agreed upon.
  Criteria are the indicators of successful
  performance on the job.

3
Good performance goals, objectives, and criteria
are

• job related
• challenging, but achievable
• clearly communicated mutually understood
• specific and objective
• time-oriented
• written
• subject to revision, as needed

4
Six Steps

• (1) conducting a job analysis
• (2) developing predictors
• (3) developing criterion measures
• (4) administer predictor criterion measures
• (5) conducting validation fairness analyses
• (6) implementation

5
Examples

• Lets review a few cases
• http//www.pdri.com/value/assess

6
The errors we make
7
Halo/Horn Error

• Halo error - Occurs when supervisor generalizes
  one positive performance feature or incident to
  all aspects of employee performance resulting in
  higher rating
• Horn error - Evaluation error occurs when
  supervisor generalizes one negative performance
  feature or incident to all aspects of employee
  performance resulting in lower rating

8
Leniency/Strictness

• Leniency - Giving undeserved high ratings
• Strictness - Being unduly critical of employees
  work performance
• Worst situation is when institution has both
  lenient and strict supervisors and does nothing
  to level inequities

9
Central Tendency

• Error occurs when employees are incorrectly rated
  near average or middle of scale
• May be encouraged by some rating scale systems
  requiring evaluator to justify in writing
  extremely high or extremely low ratings

10
Recent Behavior Bias

• Employees behavior often improves and
  productivity tends to rise several days or weeks
  before scheduled evaluation
• Only natural for rater to remember recent
  behavior more clearly than actions from more
  distant past
• Maintaining records of performance

11
Personal Bias (Stereotyping)

• Supervisors allow individual differences such as
  gender, race or age to affect ratings they give
• Effects of cultural bias, or stereotyping, can
  influence appraisals
• Other factors
• like an individual that may be appraised more
  harshly simply because they do not seriously
  object to results

12
Manipulating the Evaluation

• Sometimes, supervisors control virtually every
  aspect of appraisal process and are in position
  to manipulate system
• Example Want to give pay raise to certain
  employee. Supervisor may give employee an
  undeserved high performance evaluation

13
Employee Anxiety

• Evaluation process may create anxiety for
  appraised employee
• Opportunities for promotion, better work
  assignments, and increased compensation may hinge
  on results

14
Establishing Performance Criteria (Standards)

• Traits
• Behaviors
• Competencies
• Goal Achievement
• Improvement Potential

15
THREE FOCAL POINTS OF APPRAISAL

• 1. PERSONAL TRAITS AND CHARACTERISTICS
• inexpensive to develop and use
• not specialized by position one form for all
  workers
• - high potential for bias and rating errors
• - not very useful for feedback or development
• - not easily justifiable for reward/promotion
  decisions
• 2. JOB BEHAVIOR AND ACTIVITY
• can focus on specific duties listed in the
  job description
• intuitively acceptable to employees and
  superiors
• useful for providing feedback
• seem fair for reward and promotion decisions
• - are time consuming to develop and use
• - can be costly to develop
• - have some potential for rating error and bias

16
THREE FOCAL POINTS OF APPRAISAL

• 3. WORK RESULTS AND OUTCOMES
• less subjectivity bias
• acceptable to employees and superiors
• links individual performance to
  organizational objectives
• seem fair for reward and promotion decisions
• - are time consuming to develop and use
• - may encourage a short-term perspective
• - may use deficient or inappropriate criteria

17
GOALSETTING ISSUES

• GOAL DIFFICULTY How challenging should the work
  objectives be? I want an easy goal, but the
  organization wants me to stretch.
• ACCEPTANCE Will workers feel committed to work
  toward objectives that have been assigned to
  them, rather than those set participatively?
• SPECIFICITY Precise quantitative indicators may
  not exist for critical elements of the job.
  General, open-ended goals are not easily
  assessed.
• MOTIVATION Objectives should be challenging,
  yet reachable. They also need to be linked to
  desirable rewards to successfully motivate
  workers.