Validity

1
Validity
2
Todays Objectives

• What is validity and what are the 4 broad
  categories of methods for establishing validity
  evidence?
• Why is the base rate of the predictor and of the
  criterion important considerations in
  establishing criterion validity?
• Define construct and name the methods for
  establishing construct validity
• What are some difficulties in interpreting
  differences between pre- and post-test scores?

3
Validity

• Validity refers to an estimate of how adequately
  a test measures what it purports to measure

4
Validation

• Validity Evidence
• Face Validity
• Content Validity
• Criterion Validity
• Construct Validity

5
Face Validity

• To the person being tested, what does the test
  appear to be measuring?

6
Content Validity

• Extent to which the measure represents all facets
  for a given construct.
• Are the test items representative with regard to
  the application context?

7
Criterion-Related Validity

• Extent to which test score can be used to infer
  an individuals most probable standing on some
  criterion (i.e., measure of interest)
• Concurrent versus Predictive

8
Criterion-Related Validity

• Validity coefficient
• Restricted range
• Incremental validity
• Degree to which additional predictor explains
  something about the criterion measure that is not
  explained by the predictors already in use

9
Criterion-Related Validity
Hit
Miss
False Alarm
Correct Rejection
10
Construct Validity

• Construct Informed scientific idea hypothesized
  to describe behavior
• Unobservable Presupposed
• Underlies differences in behavior (such as
  differences in test performance, or such as
  differences on the criterion variable)
• Useful in that proposed construct should pay its
  way by doing some work (cf. Occams razor,
  principle of parsimony)

11
Construct Validity

• Construct Validity
• Degree to which inferences can legitimately be
  made from the operationalizations in your study
  to the theoretical constructs on which those
  operationalizations were based.
• Reification Error
• If the only way you can 'prove' the existence of
  something is through language, then does that
  something exist in reality? Could it be only word
  play?

12
Construct Validity

• Construct Validity
• If empirical results are contrary to theory,
  then
• A. Test does not measure the construct
• B. Theory is flawed
• C. Method was flawed

13
Construct Validity

• Construct Validity Evidence
• Homogeneity
• Age related changes
• Pretest/Posttest changes
• Contrasted groups evidence
• Convergent Evidence
• Discriminant Evidence
• Multitrait-Multimethod Matrix

14
Construct Validity

• Homogeneity
• Item analysis Relationship between test takers
  scores on individual items and their score on the
  entire test

15
Construct Validity

• Age Related
• Some constructs lend themselves to the prediction
  that the construct will change predictably over
  time
• Reading ability versus marital satisfaction

16
Construct Validity

• Pretest/Posttest
• Compare scores on a pretest with a posttest

17
Construct Validity

• Rival explanations for significant Pre/Post test
  differences
• Regression toward the mean
• Instrumentation
• History
• Maturation
• Mortality
• Test experience

18
Construct Validity

• Method of Contrasted Groups
• Demonstrates that scores on a test predictably
  vary as a function of group membership

19
Construct Validity

• Convergent Validity Correlation of test with
  other tests that measure the construct (or
  related constructs)
• Spurious convergence Positively correlated
  measures may be related only because they share
  the same method of measurement (i.e., they share
  common method variance)

20
Construct Validity

• Discriminant Evidence
• Correlation between test scores and other
  measures that should NOT theoretically be related

21
Construct Validity

• Multitrait-Multimethod Matrix (Campbell Fiske,
  1959)
• Two or more traits, two or more methods of
  measurement examined for convergent and divergent
  validity
• Data for each trait indicator obtained by
  multiple methods and/or from multiple sources