Scales and Measurement Chapters 11, 12

1
Scales and MeasurementChapters 11, 12
2
Constructs and Measurement

• Construct Development
• Identifying and defining what is to be measured
• A construct is a hypothetical variable composed
  of different elements that are thought to be
  related (e.g., 5 questions tapping brand loyalty)
• Measurement
• Figuring out how to measure what you want to
  measure
• Measure needs to be reliable and valid ?

3
Different Types of Reliability

• Internal Reliability
• Extent to which items on a scale hang together
  or are correlated with one another
• Cronbachs alpha (covered in last class)
• Split-half reliability (split measure into
  halves, correlate)
• Test-Retest Reliability
• Extent to which scores are stable over time
• Have people complete questionnaire twice and
  correlate scores

4
Validity Overview of Key Definitions

• Validity (in general)
• The extent to which conclusions drawn from a
  study are true
• Internal Validity
• When a researcher can clearly identify cause and
  effect relationships (i.e., there are no
  confounds)
• External Validity
• The extent to which what you find in your study
  can be generalized to your target population
• Construct Validity
• Extent to which your constructs of interest
  (e.g., sensation seeking) are accurately and
  completely identified (measured)
• In other words, the extent to which you are
  actually measuring what you say you are measuring
  (your sensation seeking scale really does measure
  the true construct of sensation seeking)

5
Other Forms of Validity

• Content Validity (Face Validity)
• Extent to which a measure is appropriate
  according to experts in the domain of interest
• Concurrent Validity (Convergent Validity)
• Extent to which one measure of a construct
  overlaps with other similar measures of that
  construct
• Discriminant Validity
• Extent to which a measure of one construct does
  not overlap with measures of different constructs
• Predictive Validity
• Extent to which a measure of a construct can
  predict theoretically-relevant outcomes
• Nomological Validity
• How a construct fits within a broader set of
  related constructs

6
Key Idea

• To develop reliable and valid measures which we
  can subsequently (and appropriately) use in
  statistical analyses, we must understand
• Properties of scales
• How to design good questions that do not lead to
  biased or inconsistent responses

7
Measurement/Scaling Properties

• Assignment
• You can assign objects to categories
• Order (Magnitude)
• You can order objects in terms of having more or
  less of some quality
• Distance (Equal Intervals)
• The distance between adjacent points on the scale
  is identical
• Origin (Absolute Zero Point)
• Zero means something (absence of a given
  quality)

8
Types of Scales

• Nominal Scale
• Has Assignment Only (Political Party)
• Ordinal
• Has Assignment, Order (Rank Order of Finish in a
  Race)
• Interval
• Has Assignment, Order, Equal Intervals
  (Temperature)
• Hybrid Ordinally-Interval Scale
• Like an ordinal scale, but researcher pretends
  it is an interval scale (e.g., assumes 1 to 7
  scale is an interval scale) commonly used in
  questionnaires
• Ratio
• Has Assignment, Order, Equal Intervals, Absolute
  Zero (Number of Cars)

9
What Type of Scale?

• Number of Sweaters Purchased This Year? _______
• What is Your Ethnicity?
• To what extent do you agree or disagree that
  Congress should have approved the 700 bailout?
  (1 strongly disagree to 7 strongly agree)
• Please rank the following issues from most to
  least important (Iraq, Health Care, Economy,
  Environment)
• What is your income? (5-10k 11-15k 16-20k
  20-25k 25-30k)

10
On the Importance of Attitudes
I believe both candidates bring strengths to the
table
I think McCain has a good grasp of national
security issues
I feel a strong connection with Obama
I think Obamas Plan for Health Care Reform is a
good one
Im planning on voting for Obama
I am inclined to vote for McCain
I feel McCain would be a good leader
11
Three Components of Attitudes

• Cognitive Component
• How a person thinks about an attitude object
  (product, issue, candidate, idea)
• Affective Component
• How a person feels about an attitude object
• Behavioral
• A persons behavioral predisposition to respond
  to an attitude object in a certain way

12
Three Components of Attitudes?
I believe both candidates bring strengths to the
table
I think McCain has a good grasp of national
security issues
I feel a strong connection with Obama
I think Obamas Plan for Health Care Reform is a
good one
Im planning on voting for Obama
I am inclined to vote for McCain
I feel McCain would be a good leader
13
Measuring Attitudes

• While attitudes not perfect predictors of
  behavior, still very important
• We need to understand how to measure attitudes
  accurately
• Today, well look at some creative ways to
  measure attitudes
• And, some standard approaches taken in marketing
  research
• As we do this, think about the types of questions
  we might want to ask in our restaurant concepts
  study

14
Creative Measures of Attitudes

• Projective techniques (partially structured)
• Physiological (GSR Heart Rate Blood Pressure)
• Functional magnetic resonance imaging (fMRI)
• Flushing toilets
• Bogus pipeline
• Lost letters and emails
• Implicit association test (IAT)

15
Youve Got Mail!

• Dear Peter Price,
• Thank you for applying for a Glassner Foundation
  Scholarship. As you know, these scholarships are
  highly competitive and are given only to a few
  select students. They cover tuition for four
  years at a state-funded university. There is also
  an additional 500 per year for academic
  supplies.
• Because of the large number of applicants this
  year we are late in sending out these notices.
  Because of the time sensitive nature of this
  material, we wanted to immediately inform you of
  the committees decision regarding your
  application.
• We are happy to inform you that you have been
  selected to receive a Glassner Scholarship.
• We ask that you respond within 48 hours. Due to
  the high number of applicants, we would like to
  extend the scholarship to others applicants if
  you do not accept the scholarship.

16
Youve Got Mail!

• Dear Mohammed Hameed,
• Thank you for applying for a Glassner Foundation
  Scholarship. As you know, these scholarships are
  highly competitive and are given only to a few
  select students. They cover tuition for four
  years at a state-funded university. There is also
  an additional 500 per year for academic
  supplies.
• Because of the large number of applicants this
  year we are late in sending out these notices.
  Because of the time sensitive nature of this
  material, we wanted to immediately inform you of
  the committees decision regarding your
  application.
• We are happy to inform you that you have been
  selected to receive a Glassner Scholarship.
• We ask that you respond within 48 hours. Due to
  the high number of applicants, we would like to
  extend the scholarship to others applicants if
  you do not accept the scholarship.

17
Youve Got Mail!

• Dear Mohammed Hameed,
• Thank you for applying for a Glassner Foundation
  Scholarship. As you know, these scholarships are
  highly competitive and are given only to a few
  select students. They cover tuition four four
  years at a state-funded university. There is also
  an additional 500 per year for academic
  supplies.
• Because of the large number of applicants this
  year we are late in sending out these notices.
  Because of the time sensitive nature of this
  material, we wanted to immediately inform you of
  the committees decision regarding your
  application.
• We regret to inform you that you have not been
  selected to receive a Glasser Scholarship.
• We ask that you respond within 48 hours. Due to
  the high number of applicants, we would like to
  extend the scholarship to others applicants if
  you do not accept the scholarship.

18
Youve Got Mail!(Method)

• Bushman Bonacci (2004, JESP)
• Pretested on Arab-American Prejudice
• 2 weeks later, receive email intended for a
  different person
• 2 IVs
• Intended recipient had European-American (Peter
  Price) vs. Arab-American name (Mohammed Hameed)
• Intended recipient won or didnt win a
  scholarship (4 years support)
• DV willingness to return the email to the
  sender to indicate it was incorrectly delivered
  (must be done in 2 days)

19
Youve Got Mail!(Results)
20
Youve Got Mail!(Results)
21
Youve Got Mail!(Results)
22
Youve Got Mail!(Results)

• GOOD NEWS! YOU WON!!
• P (returning email) went down as prejudice went
  up, especially when the intended recipient had an
  Arab-American name
• BAD NEWSYOU DIDNT WIN.
• European-American P (returning email) went down
  as prejudice went up
• Arab-American P (returning email) went up as
  prejudice went up.
• People with Arab-American prejudice were actually
  more likely to return the bad news than the good
  news email when recipient was Arab American
• WHAT MIGHT EXPLAIN SUCH ARAB-AMERICAN PREJUDICE?

23
Terror Management Theory

• Basic Assumption
• Humans have developed anxiety buffering
  mechanisms against fear of death (cultural
  worldview, self-esteem, relationships)
• Typical Method
• Write about own death (mortality salience) or
  dental pain
• Filler task
• Judge others from ingroup vs. outgroup
• Typical Result
• We judge worldview threatening others more
  harshly under conditions of mortality salience
• Support for Bush (vs. Kerry) goes up after MS
  prime

24
Implicit Association Task (IAT)

• An implicit attitude is an attitude which people
  are not conscious of (or would not want to admit
  to) but which can be assessed via the
  associations people hold in their minds between
  evaluative words (good, bad) and attitude objects
  (Caucasian, Arab)
• The IAT is a reaction time (RT) task in which
  subjects categorize words (positive, negative)
  and attitude objects (pictures of Caucasians,
  Arabs) on the left or right side of screen
• To simplify, some trials involve categorizations
  that are consistent with an implicit attitude,
  others are inconsistent with the attitude
• If there is an implicitly negative attitude
  toward Arabs, then reaction times should be
  slower to the inconsistent trials

25
The following set of trials is consistentwith
an implicitly negativeattitude toward
ArabsReaction times should berelatively fast
if thereis an implicitly negative
attitudetoward Arabs
26
Good Word or Caucasian
Bad Word or Arab
If you see a good word or a Caucasian, hit e
(left) If you see a bad word or an Arab, hit i
(right)
27
Good Word or Caucasian
Bad Word or Arab
28
Good Word or Caucasian
Bad Word or Arab
Dirt
29
Good Word or Caucasian
Bad Word or Arab
30
Good Word or Caucasian
Bad Word or Arab
Happy
31
Now Change Categorization(The following trials
areinconsistent with implicit negative attitude
toward Arabs)Reaction times should be slower on
these trials if there is an implicitly negative
attitude toward Arabs
32
Good Word or Arab
Bad Word or Caucasian
Categorization Switched
33
Good Word or Arab
Bad Word or Caucasian
34
Good Word or Arab
Bad Word or Caucasian
Joy
35
Good Word or Arab
Bad Word or Caucasian
Death
36
Good Word or Arab
Bad Word or Caucasian
37
What are we saying?!

• Sensitive attitudes (e.g., racial prejudice) are
  sometimes hard to assess with a self-report scale
  (due to social desirability concerns)
• The IAT measures associations (automatic
  attitudes)
• Implicitly negative attitudes are not uncommon,
  but
• People can also override these automatic
  responses with controlled processes (so behavior
  is not discriminatory)
• For more information, visit https//implicit.harva
  rd.edu/implicit/ where you can take an IAT for
  the 2008 election (or other issues)

38
Verbal Rating Scales
How satisfied were you with todays
meal? 1 2 3 4 5 6 7 Extremely Dissatisfied Som
ewhat Neither Somewhat Satisfied Extremely Dissat
isfied Dissatisfied Satisfied Satisfied
39
Issues in Designing Verbal Rating Scales

• Many measures taken by marketing researchers are
  verbal ratings
• What do we need to consider when we develop
  verbal rating scales?
• Number of categories
• Forced vs. unforced scale
• Balanced or unbalanced scale
• Extent of verbal description
• Should response categories be numbered or not
• Comparative vs. noncomparative scale
• Scale direction

40
Number of Response Categories?

• To what extent are you satisfied with your
  current MP3 player?
• Most researchers suggest between 5 and 7
  categories for example
• 1 2 3 4 5 6 7
• Extremely Dissatisfied Somewhat Neither Somewhat
  Satisfied Extremely Dissatisfied Dissatisfied S
  atisfied Satisfied
• Too few does not give you enough information
• Too many and it will be hard for people to
  discriminate between the options (e.g., a
  100-point scale)

41
Forced vs. Unforced Scale?

• How likely would you be to buy a car manufactured
  in Brazil?
• Forced Scale (even number of options forces the
  respondent to lean one way or the other)
• 1 2 3 4 5 6
• Very Unlikely Somewhat Somewhat Likely Very
• Unlikely Unlikely Likely Likely
• Unforced scale gives people a neutral option
• 1 2 3 4 5 6 7
• Very Unlikely Somewhat Neither Somewhat Likely Ve
  ry Unlikely Unlikely Likely Likely

42
Balanced vs. Unbalanced Scale?

• How satisfied are you with your current hair
  stylist?
• Balanced scale (same number of positive and
  negative options)
• 1 2 3 4 5 6 7
• Extremely Dissatisfied Somewhat Neither Somewhat
  Satisfied Extremely Dissatisfied Dissatisfied S
  atisfied Satisfied
• Unbalanced scale (here all options are positive)
• 1 2 3 4 5 6 7
• Somewhat Very Satisfied Satisfied
• Unbalanced scale can give biased results unless
  distribution is naturally skewed to one side of
  the scale, should use balanced scale

43
Extent of Verbal Description?

• The U.S. should invest in wind powered energy
• Label endpoints or label all options?
• 1 2 3 4 5 6 7
• Strongly Strongly Disagree Agree
• 1 2 3 4 5 6 7
• Strongly Moderately Slightly Neither
  Agree Slightly Moderately Strongly Disagree Disag
  ree Disagree or Disagree Agree Agree Agree
• Labeling all options can aid in interpretation.

44
Should Categories be Numbered?

• Toyota is an Environmentally Friendly Company
• Strongly Moderately Slightly Neither
  Agree Slightly Moderately Strongly Disagree Disag
  ree Disagree or Disagree Agree Agree Agree
• 1 2 3 4 5 6 7
• -3 -2 -1 0 1 2 3
• Numbers can help respondents understand scale
• 1 to 7 scale quite common
• But -3 to 3 can help interpretation of scale
  (disagree is negative, agree is positive) it
  may, however, overemphasize negativity
• Judgment call pretesting both scales could help
  identify problems

Should we have numbers here?
45
Comparative vs. Noncomparative?

• Noncomparative question
• How would you evaluate Mintifresh toothpaste?
• Comparative question
• Compared to your current brand, how would you
  evaluate Mintifresh toothpaste?
• Comparative questions establish the referent and
  can be useful if you need to know how your
  product compares to a specific competitor or the
  customers current brand
• Noncomparative have the advantage of allowing the
  respondent to create their own referent, which
  can potentially improve accuracy

46
Direction of Scale?

• Typical direction (lower values, negative
  connotation on left)
• Strongly Moderately Slightly Neither
  Agree Slightly Moderately Strongly Disagree Disag
  ree Disagree or Disagree Agree Agree Agree
• 1 2 3 4 5 6 7
• Some scales are not valenced, so must be careful
  about positioning. Here we see a semantic
  differential scale, with amusing positioning
• Unpleasant -2 -1 0 1 2 Pleasant
• Flimsy -2 -1 0 1 2 Sturdy
• Male -2 -1 0 1 2 Female
• Hmmmthis arrangement suggests that males are to
  be evaluated negatively must be careful in
  designing scales so as not to bias results

47
Other Types of Scales
Rank order the restaurants above where 1 most
preferred, 3 least preferred
48
Rank-Order andPaired Comparison Techniques

• Rank-order technique
• Advantages easy to understand, typically what we
  do in real-life (vs. ratings)
• Disadvantage it may be that a person dislikes
  all of the options, so ranking not that
  informative
• Paired comparisons
• Take n products, compare each one to every other
  (in each pair, pick the one you prefer)
• Gives you direct comparisons, but paired
  comparisons can be very large as number of
  products to be compared increases

49
Semantic Differential Scale

• Please provide your impression of Politician X
• Dumb _____ _____ _____ _____ __X__ Smart
• Cold _____ _____ _____ __X__ _____ Warm
• Boring _____ _____ _____ _____ __X__ Funny
• Mean _____ _____ _____ __X__ _____ Nice
• Please provide your impression of Politician Y
• Dumb _____ __X__ _____ _____ _____ Smart
• Cold __X__ _____ _____ _____ _____ Warm
• Boring _____ _____ __X__ _____ ____ Funny
• Mean _____ __X__ _____ _____ _____ Nice
• Profile Analysis of Politician X vs. Y
• Dumb _____ _____ _____ _____ _____ Smart
• Cold _____ _____ _____ _____ _____ Warm
• Boring _____ _____ _____ _____ _____ Funny
• Mean _____ _____ _____ _____ _____ Nice

X
Y
50
Stapel Scale

• Please provide your impression of Shoe Store X.
  (Use negative numbers if you feel it is
  inaccurate positive numbers if you feel it is
  accurate)
• -5 -4 -3 -2 -1 Fast Service 1 2 3 4 5 -5 -4 -3 -
  2 -1 Friendly 1 2 3 4 5
• -5 -4 -3 -2 -1 Knowledgeable 1 2 3 4 5
• Can draw comparative profile analysis (e.g., of
  various shoe stores) as we did with semantic
  differential scale (previous slide)

51
Questionnaire DesignChapter 13

• Some Key Issues

52
Questions Common Pitfalls

• Incomprehensible
• When you consider a new stadium, is it possible
  that part of your determining factors might rest
  on the fact that you sometimes choose to forgo
  entertainment in favor of more pedestrian
  activities like walking your dog?
• Unanswerable
• Will building a new stadium in Boise cost too
  much?
• Leading
• Wouldnt you agree that it is a great idea to
  build a new multiuse facility in Boise?
• Double-barreled
• Do you think the old stadium needs to be replaced
  and a new stadium should be built downtown?

53
Purpose of Questions

• Descriptive
• Describe characteristics of your sample
• Predictive (Hypothesis Testing)
• What demographic factors are associated with
  support for the stadium?
• Every question should be designed to provide
  useful information
• What is our primary goal in the stadium study?
• What questions are we trying to address?
• How do our questions stack up?
• Good, bad questions?
• Do they each answer a question? Descriptive?
  Predictive?

54
Arrangement of Questions Flowerpot Approach
Exhibit 13.5