Types of Validity

1
Types of Validity
Construct Validity
Criterion Validity
Content Validity
Predictive Validity
Concurrent Validity
Convergent Validity
Discriminant Validity
Adapted from Sekaran, 2004
2
Types of Validity

• Construct Validity
• Extent to which hypotheses about construct are
  supported by data
• Define construct, generate hypotheses about
  constructs relation to other constructs
• Develop comprehensive measure of construct
  assess its reliability
• Examine relationship of measure of construct to
  other, similar and dissimilar constructs
• E.g. Height Weight Networking career
  outcomes study Learning Style Orientation measure

3
Establishing Construct Validity

• Multi-trait multi-method matrix
• Convergent validity coefficient
• Absolute size of correlation between different
  measures of the same construct should be large
  significantly different from zero
• Discriminant validity coefficient
• Correlations between the same construct measured
  by different methods should be significantly
  different from correlations of
• Different constructs measured by different
  methods (across methods across constructs)
• Different constructs measured by same method
  (method bias)

4
Convergent Discriminant ValidityCorr b/w
Objective (O) Self-Reports (SR) of Height
Weight
O-H SR-H O-W SR-W
O-H 1.00
SR-H .98 1.00
O-W .55 .56 1.00
SR-W .68 .69 .92 1.00
 
5
Convergent Discriminant Validityof Objective
Self-Reports of Height Weight

• Convergent validity (across methods)
• Objective and subjective measures of height are
  correlated .98
• Objective and subjective measures of weight are
  correlated .92

6
Discriminant Validityof Objective Self-Reports
of Height Weight

• Discriminant validity (across constructs)
• Objective measures of height weight are corr
  .55
• Subjective measures of height weight are
  corr.69
• STRONG CASE Are the correlations b/w the same
  construct measured by different methods
  significantly different from corr b/w different
  constructs measured by same methods
• i.e., Are .92 .98 significantly different from
  .55 .69?
• Convert rs to z scores and compare

7
Discriminant Validityof Objective Self-Reports
of Height Weight

• Discriminant validity (across constructs)
• Objective height subjective weight are corr
  .68
• Subjective height objective weight are corr .56
• WEAK CASE Are the correlations b/w the same
  construct measured by different methods
  significantly different from corr b/w different
  constructs measured by different methods
• i.e., Are .92 .98 significantly different from
  .56 .68

8
Establishing Construct Validity in Networking
Study

• For convergent validity, different measures of
  the same construct should be highly correlated
• Note In networking study, diff measures diff
  subscales
• Corr b/w diff measures of career success
  (promotion, salary, perceived career success)
  range from .20 to .36 (moderate support)
• .20 .36 should not be sig diff from each other
• Corr b/w diff measures of networking (increasing
  internal visibility, socializing etc.) range from
  .03 to .43 (weak support)
• .03 .43 should not be sig diff from each other

9
Establishing Construct Validity in Networking
Study

• For discriminant validity, different measures of
  the same construct should be more highly
  correlated than different measures of different
  constructs
• Correlations b/w career success networking (.01
  to .35) should be sig different from
• Corr b/w diff measures of career success (.20 to
  .36)
• Corr b/w diff measures of networking (.03 to .43)

10
Learning Style Study

• Developed items by generating critical incidents
  (Study 1)
• N67
• Yes/no responses to statements
• Recall of learning events
• Two types of learning theoretical, practical (2)
  
• Two types of outcomessuccess, failure (2)
• 2 x 2 events per participant
• 112 items constructed in total

11
Learning Style Study

• Study 1 Part 2
• Created 112 items from critical incidents
• Administered to 154 participants
• 5-point likert scale (agree/disagree)
• Extracted 5 factor solution w/factor analyses
• 54 items loaded highly on the 5 factors
• Content validity item sorting by 8 grad students
• Also administered personality scale

12
Learning Style Study

• Item Development Study (study 1)
• Convergent Validity
• High reliabilities of subscales of Learning Style
  (.81-.91)
• Corr b/w different measures (subscales) of
  learning style .01 to.32 but 1 only corr is
  significant
• Weak support for convergent validity of new
  learning style measure
• Discriminant validity
• Corr b/w different measures of different
  constructs (Learning Style personality) .42 to
  .01 should be lower than and sig diff from corr
  b/w different measures of same construct
  (subscales of learning style) .01 to .32

13
Learning Style Orientation Measure

• Validation Study (study 2)
• N350 -193
• New learning style, Personality, old Learning
  style, preferences for instructional assessment
  methods
• Construct validity
• Confirmatory factor analysis confirms 5
  dimensions
• Reliability of new learning style subscales.74
  to .87 compared to
• Reliability of old learning style subscales.83
  to .86
• Reliability of personality subscales.86 to .95

14
Learning Style Orientation Measure

• Validation Study (study 2)
• Convergent validity
• Corr b/w similar measures of key construct Corr
  b/w diff subscales of new learning style 01 to
  .23 should be comparable to
• Corr b/w similar measures of other constructs in
  the study
• Diff subscales of old learning style .23 to .40
• Diff subscales of personality .01 to .27

15
Learning Style Orientation Measure

• Validation Study (study 2)
• Discriminant validity
• Corr b/w measures of similar constructs Corr b/w
  new learning style subscales old learning style
  .01 to .31
• Corr b/w measures of different constructs
• Corr b/w new learning style personality
  subscales is .01 to .55
• Corr b/w old learning style personality
  subscales .02 to .38

16
Learning Style Orientation Measure

• Validation Study
• Incremental validity (aka construct validity)
• Additional variance explained by old vs. new
  learning style measures in preferences for
  assessment instruction

DV LSOM (new) LSI (old)
Subjective assessment .15 .01
Interactional instruction .21 .04
Informational instruction .06 .00
17
Types of Validity
Construct Validity
Criterion Validity
Content Validity
Predictive Validity
Concurrent Validity
Convergent Validity
Discriminant Validity
Adapted from Sekaran, 2004