Reliability Theory

1
Reliability Theory Estimates

• The Concept of Reliability
• A Basic Model
• X T E
• Observed true error
• Error Variance and Reliability
• What is true variance? Error variance?
• Types of error variance
• How do we prevent error variance?

2
How is reliability measured?

• Whats that theoretical model again?
• Goal Estimating the amount of error
• Reliability/Correlation Coefficient
• How to evaluate? Higher better, within the
  context of theory and understanding the construct
  itself!!!
• Standard Error of Measurement
• Confidence Intervals
• Domain sampling models

3
SEM and Confidence Intervals(see handout)

• Start with T-R reliability (stability of
  measurement over time)
• Using this and standard deviation, we can
  calculate average amount of error for any given
  score (SEM)
• And, using that SEM, we can estimate the true
  score range within a certain level of probability
  (95 or 99)

4
Types of Reliability

• Test-Retest
• Alternate/Parallel Forms
• Internal Consistency
• Split Half Reliability Estimate
• Split Half Estimates
• Spearman-Brown Correction (Formula)
• Kuder-Richardson (20 21)
• Cronbachs Coefficient Alpha
• Inter-item Correlations
• Item-Total Correlations
• Inter-Rater Reliability (reliability of raters
  versus reliability of test scores)

5
For Week 4

• Go Back through ALL of the articles (old and new)
  and review how they use validity coefficients!!
• Can you restate it?
• Exercises 6, 7, 12 (5 8-11, optional)