Self Awareness

1
Why?
How?
Self Awareness
So What?
Can we trust it?
2
Can we trust these measures?

• Reliabilityconsistency (e.g., thermometer, job
  interview)
• Types of Reliability
• Test-retest
• Consistency across time
• Alternate forms
• Consistency across versions
• Internal
• Consistency across items
• Scorer (inter-rater)
• Consistency across raters/judges

Review of B23B29/B27
3
Types of Reliability

• Test-Retest
• Consistency across time
• Administration
• Same applicants
• Same test
• Different testing periods
• How to measure reliability
• Scores at Time 1 are correlated with scores at
  Time 2
• Correlation should be above .70

Review of B23B29/B27
4
Types of Reliability

• Test-Retest Problems
• Common to most other types
• Characteristic being measured may change over
  time (e.g., skill)
• Carry over effects (e.g.,practice, memory)
• Time consuming
• Expensive
• Inappropriate for some test types (e.g., mood)

Review of B23B29/B27
5
Types of Reliability

• Alternate Forms
• Consistency across forms
• E.g., two different exams in different sections
  of same class
• Administration
• Two forms similar in content response
  characteristics
• Same applicants (if memory is an issue)
• Different testing periods
• How to measure reliability
• Scores at Time 1, Form 1 are correlated with
  scores are Time 2, Form 2
• Can counterbalance
• Need high correlations

Review of B23B29/B27
6
Types of Reliability

• Alternate Forms Problems
• Difficult to develop
• Errors in Content Sampling

Review of B23B29/B27
7
Types of Reliability

• Internal
• Consistency across items
• Administration
• Same applicants
• Same test
• Same time
• How to measure reliability
• Responses across one half of all the items are
  correlated with the other half
• Cronbach (ratio/interval data)
• Kuder Richardson (dichotomous data)

Review of B23B29/B27
8
Types of Reliability

• Inter-rater (Scorer)
• Consistency across Raters
• E.g., Panel job interview
• Administration
• Different raters
• Same target
• Same time
• How to measure reliability
• Scores across two or more raters are correlated
  with each other

Review of B23B29/B27
9
Can we trust these measures?

• Reliability
• Reliability is necessary but not sufficient for
  validity
• Using a tape to measure height is not a valid way
  to measure weight although a tape may reliably
  measure height bec. weight is related to height
• Validity

Review of B23B29/B27
10
So What?

• Validity
• Example Structured Job Interview
• How useful is my job interview?
• To what extent are my conclusions from the
  interview justified by the scores on the
  interview?
• How accurately does the interview capture all
  aspects of the job?

Review of B23B29/B27
11
So What? Types of Validity

• Face Validity
• Content Validity
• Criterion Validity
• Construct Validity

Review of B23B29/B27
12
Types of Validity

• Face Validity
• Degree to which the test appears to measure what
  it is supposed to measure
• Do your test takers perceive that the test is
  valid?
• Face Validity
• Motivates test takers
• Performance settings
• To change behavior in self development courses
• Reduces legal challenges
• Staying in applicant pool
• Accepting job offer

Review of B23B29/B27
13
Types of Validity

• Content
• Degree to which the measure samples the content
  it is supposed to assess
• Is your job interview based on what is required
  for the job? (i.e., job description)
• Do experts of the job (incumbents, supervisors)
  rate the job interview as having relevant
  questions?

Review of B23B29/B27
14
Types of Validity

• Criterion
• Degree to which the measure is correlated to some
  other, important measure (i.e., criterion)
• Does your performance on the interview predict
  your performance on the job?
• Types of Criterion Validity
• Concurrent
• Predictive
• Validity generalization
• Mostly used in hiring situations

Review of B23B29/B27
15
Types of Validity Criterion

• Concurrent
• Using current employees
• Range restriction is a problem
• Predictive
• Correlates scores with future behavior
• Can be impractical
• Removes range restriction problem
• Validity Generalization
• Measure valid for one job in one location (org)
  is valid for same job in a different location

Review of B23B29/B27
16
Types of Validity

• Construct
• Degree to which the test measures the concept it
  is supposed to measure
• Are your conclusions justified from the test
  scores?
• Correlation with a related construct (e.g.,
  intrinsic motivation and locus of control)

Review of B23B29/B27
17
Why?
How?
Self Awareness
So What?
Can we trust it?