Establishing A Passing Standard

1
Establishing A Passing Standard

• Paul D. Naylor, Ph.D.
• Psychometric Consultant

2
Standard Setting

• The process used to arrive at a passing score
• Lowest score permitting entry to the field
• Recommended Standard

3
Validity is About Interpretation

• Evidence to support the interpretation
• What standard setting system was used?
• What was the rationale for selecting it?
• What was the qualification of those who
  participated?

4
Standards

• Mandated
• Norm-referenced
• Criterion-referenced

5
Mandated Standards

• Often used in licensing
• Difficult to defend
• Not related to minimum qualification

6
Norm-referenced Standards

• Popular in schools
• Limits entry
• Inconsistent results

7
Criterion-referenced Standards

• Wide acceptqnce in professional testing
• Determines minimum qualification
• Not test population dependant
• Exam or item centered

8
Procedures

• Angoff (modified)
• Nedelsky
• Ebell
• Others

9
Minimally Competent Performance

• Minimum acceptable performance
• Minimal qualification
• Borderline
• Its all relative

10
Angoff Method

• Judges
• Selection
• Training
• Probabilities
• Would vs Should
• Rater Agreement
• Tabulation

11
1. How many goals are on a basketball court?

• A. 1
• B. 2
• C. 3
• D. 4

12
2. How many halves are in a college basketball
game?

• A. 1
• B. 2
• C. 3
• D. 4

13
3. How many quarters are in a high school
basketball game?

• A. 1
• B. 2
• C. 3
• D. 4

14
4. How many points does a 3- point
field goal account for in a basketball game?

• A. 1
• B. 2
• C. 3
• D. 4

15
5. How many teams are in the NCAA Mens
Basketball National Championship Tournament?

• A. 48
• B. 64
• C. 65
• D. 32

16
Application

• Adjustment
• Angoff Values
• Alternative Forms of Exam
• Passing Score

17
Validity Evidence

• How was the passing score set?
• Who set the passing score?

18
Minimum Requirements

• Description of method
• Results of method used
• List of participants and qualifications
• Demographics
• Credentials
• Affiliations

19
Additional Evidence

• Data from method used
• Description of training given to participants
• Definition of minimum qualification

20
References

• Livingston, S.A. Zieky, M.J. (1982). Passing
  Scores. ETS.
• Cizek, C.J. Standard setting guidelines.
  Educational Measurement Issues and Practices, 15
  (1), 13-21, 12.
• CLEAR Exam Review (Winter 2001, Summer 2001, and
  others)