Measurement and Observation

1
Measurement and Observation
2
Choices During Operationalization

• Researchers make a number of key decisions when
  deciding how to measure a concept
• Dimensions and sub-dimensions
• Range of variation within dimensions
• Categories to represent range
• Levels of measurement
• Nominal, ordinal, interval, ratio

3
Operationalization A deliberative process

• Not a simple, linear process
• Complicated and fraught with trade-offs
• Iterative process with cycles of consideration
• Debate over proper measurement is key

4
Dimensions of the Concept

• Creating operational measures forces realization
  about lack of conceptual clarity
• List of possible dimensions may be long
• Need to decide which ones are most relevant
• Ask which ones are central to the inquiry
• Reflect on research hypotheses or theories

5
Range of Variation

• Sense of the upper and lower limits
• How much are you willing to combine different
  people into the same category?
• Extremely high and Extremely low may be collapsed
• Eg. Income, age, height, etc.
• Opposition and support for attitudes
• Agreement and disagreement

6
Variation Between Extremes

• Degree of precision
• How detailed you need to be in measurement
• Eg. Age breaks or Exact age?
• Related to purpose of study
• Eg. Political Party ID
• Dichotomy Democrat or Republican
• Continuum 7-point scale w/ independent-leaner

7
Levels of Measurement

• Nominal Measures
• Ordinal Measures
• Interval Measures
• Ratio Measures

8
Nominal Measures

• Names for characteristics
• Do not Exist along an Explicit continuum
• Exhaustive
• Mutually Exclusive
• Eg. Religious Affiliation
• Eg. Place of Birth

9
Ordinal Measures

• Can be logically rank-ordered
• Represent relatively more of less of variable
• No consistent distance between points of
  measurement
• Not just different from one another
• More of less of some attribute
• Eg. Not very important, fairly important,
  very important Extremely important

10
Interval Measures

• Consistent distance separating attribute
• We can say how much more of an attribute
• Logical distance between attributes can be
  Expressed in meaningful standard intervals
• Eg. Temperature
• 90 degrees vs. 80 degrees 10 degree difference
• 50 degrees vs. 40 degrees 10 degree difference
• Zero-point is arbitrary

11
Ratio Measures

• In addition to all the properties of nominal,
  ordinal, and interval measures, ratio measures
  have a true zero point
• Eg. Length of time
• Eg. Number of times
• Eg. Number of affiliations
• Can actually state ratio of one to another
• X has twice as many affiliations as Y

12
Whats that scale?

• Style of music in a music video
• Number of violent acts in a music video
• Whether a music video has violence or not?
• High, Medium or Low violence in a music video
• Hair color
• Number of hairs on your head
• Sat scores
• Social Security Number

13
Whats that scale?

• A baseball player's batting average
• A baseball player's field position
• A baseball player's position in the batting
  order
• A baseball player's uniform number
• College football rankings
• IQ

14
Types of questions

• Multiple choice questions
• Agree/disagree questions
• Likert questions
• Frequency scales
• Semantic differential scales
• Forced-choice statement pairs
• Thermometer feeling scales
• Nominal checklists
• Ordinal categories
• Rank-order questions
• Filter questions
• Open-ended

15
Multiple Choice Question
16
Multiple Choice with Range Options
17
Agree/Disagree Questions
18
Likert Scale
19
Frequency Scale
20
Semantic Differential Scales
21
Forced-choice Statement Pairs
22
Thermometer Feeling Scales
23
Nominal Checklist
24
Ordinal Categories
25
Rank-order Preference Questions
26
Rank-order Evaluation Questions
27
Filter Questions
28
Open-ended Questions
29
Tips on Question Construction

• 1. Make questions clear using simple language
• 2. Keep questions concise
• 3. Provide instructions for answering questions
• Dont assume respondent knows question style
• 4. Keep research purpose in mind
• Make sure items can answer research question
• 5. Dont ask double-barreled questions
• E.g., How well do you think the current
  Presidential Administration is handling foreign
  policy and the war on terrorism?

30
More Tips

• 6. Avoid leading questions
• E.g., Like most Americans, do you read a
  newspaper every day?
• 7. Avoid negative questions
• E.g., The U.S. should not invade Iraq Agree or
  disagree?
• 8. Do not ask questions that require complicated
  mental calculus
• E.g., In the past 30 days, how many hours have
  you spent watching television with your family?
• 9. Keep ordering of questions in mind

31
Using Pre-Existing Measures

• It is okay to borrow measures
• Cite source of questions to give credit
• Benefits of using Existing measures
• Saves work
• Pre-tested for reliability/validity
• Research becomes cumulative

32
Pretesting

• Clarity in question wording
• Are categories
• Exhaustive?
• Mutually Exclusive?
• Realistic time estimate
• Preliminary empirical analysis