Evaluating Measures

1
Evaluating Measures

• Validity
• Does it really measure the concept?
• Reliability
• Does the measurement give the same result each
  time? (Replicable)
• Is it precise?

2
Questions about Your Measures
  

• 1)  Validity
• Indicators of the concepts you are talking about
  really measuring the concepts you are interested
  in?
• E.G., what does "the homeless" mean in a survey
  question asking about the homeless? How would you
  measure an R's attitude toward the homeless?
• 2) Reliability
• Are the measures repeatable, precise, accurate?
  Are we off by an inch or a mile?

3
RELIABILITY AND VALIDITY DEALING WITH ERRORS

• THEORY MEASURES
• RELIABILITY, VALIDITY, BIASES
• TYPES OF ERROR RANDOM, SYSTEMATIC
• OBSERVER, STIMULUS, RESPONDENT, SAMPLING ERROR
• TEST-RETEST RELIABILITY, SPLIT-HALF RELIABILITY

4
Tests of Measurement

• Tests of Validity
• Face Validity
• Based on operationalization (concept matching)
• Tests of Reliability
• Test-Retest Reliability
• Split-Half Reliability

5
Tests of Validity

• Face validity does the test look like a
  measure that we are trying to get at?
• Content validity Does the measure contain items
  from the domain of interest?

6
Face validity

• Possible advantage can get more accurate
  answers by providing an appropriate context for a
  respondent.
• Possible disadvantage if the respondent can
  determine what we are looking for, they may bend
  shape their answers to what they think we want.

7
Content validity

• If we want a measure with low face validity (in
  the case of sensitive issue areas), we need to
  make sure the item taps into the area we are
  interested in.
• Example We want to gauge someones racial
  attitudes, but have a policy question that masks
  the racial content.

8
Tests of reliability

• Test-retest reliability you conduct the study
  at two different time points and compare the
  results for similarity.
• Split-half reliability you measure half of your
  sample one way, and use a different version of
  the same measure on the other half of the sample
  and compare results.
• Multiple measures is also a useful method (e.g.
  creating an index).

9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
Inferential Statistics

• Population the entire set of individuals of
  interest in a particular study.
• Sample a set of individuals selected from a
  population, usually intended to represent the
  population in a research study.

13

• Parameter a value, usually a numerical value,
  that describes a population.
• Statistic a value, usually a numerical value,
  that describes a sample.
• Descriptive statistics statistical procedures
  used to organize, summarize, and simplify data.
• Inferential statistics techniques that allow us
  to study samples and then make generalizations
  about the populations from which they were
  selected.
• Sampling error the discrepancy, or amount of
  error, that exists between a sample statistic and
  the corresponding population parameter.

14
(No Transcript)
15
(No Transcript)
16
HW Assignment

• Agresti and Finlay p. 29-31 problems 1-9, 15.
• SHOW YOUR WORK TO RECEIVE FULL CREDIT!!!

17
Warm Up

• The local fast food restaurant offers small,
  medium, and large soft drinks. What kind of
  scale is used to measure the size of drinks?
• The Scholastic Achievement Test (SAT) most likely
  measures aptitude on a(n) _____ scale?
• In a study on perception of facial expressions,
  participants must classify the emotions displayed
  in photographs of people as anger, sadness, joy,
  disgust, fear, or surprise. Emotional expression
  is measured on a(n) ______scale.
• A researcher studies the factors that determine
  how many children couples decide to have. The
  variable, number of children, is a __________
  (discrete/continuous) variable.
• An investigator studies how concept-formation
  ability changes with age. Age is a
  (discrete/continuous) variable.