Scaling

1
Scaling

• Measuring the Unobservable

2
Scaling

• Scaling involves the construction of an
  instrument that associates qualitative constructs
  with quantitative metric units.
• Scaling evolved out of efforts to measure
  "unmeasurable" constructs like authoritarianism
  and self esteem.

Note We are talking about constructed scales
involving multiple items, not a response scale
for a particular question.
3
Scaling

• How do we define or capture or measure a
  nebulous concept?
• By taking stabs from several directions, we can
  get a more complete picture of a concept we know
  exists but cannot see.

4
Scaling

• In scaling, we have several items that are
  intended to capture a piece of the underlying
  concept.
• The items are then combined in some form to
  create the scale.

Quite technically, we will talk about scales and
indexes interchangeably. Scales are composed of
items caused by an underlying construct, whereas
indexes are composed of items that indicate the
level of a construct and might be useful together
to predict outcomes.
5
Scaling
Graphical depiction of a scale
Latent Variable
Observed Item 1
Observed Item 2
Observed Item 3
Observed Item 4
e1
e2
e3
e4
6
Scaling
Form an Index
Observed Item 1
Graphical depiction of an index
Observed Item 2
Observed Item 3
Observed Item 4
7
Scaling

• In most scaling, the objects are text statements,
  usually statements of attitude or belief.

8
Scaling

• A scale can have any number of dimensions in it.
  Most scales that we develop have only a few
  dimensions.
• What's a dimension?
• If you think you can measure a person's
  self-esteem well with a single ruler that goes
  from low to high, then you probably have a
  unidimensional construct.

9
Scaling
10
Scaling

• Many familiar concepts (height, weight,
  temperature) are actually unidimensional.
• But, if the concept you are studying is in fact
  multidimensional in nature, a unidimensional
  scale or number line won't describe it well.
  E.g., academic achievement how do you score
  someone who is a high math achiever and terrible
  verbally, or vice versa?
• A unidimensional scale can't capture that type of
  achievement.

11
Scaling

• Factor analysis can tell you whether you have a
  unidimensional or multidimensional scalehelping
  you discover the number of dimensions or scales
  that exist among a group of variables.
• Factor analysis is typically an exploratory
  process, but it can be confirmatory.
• Exploratory factor analysis helps you reduce data
  by grouping variables into sets that tap the same
  phenomena.

12
Scaling

• Steps in factor analysis (what the computer
  does)
• Assumes one factor and checks the correlation of
  each item with the proposed factor and compares
  the proposed inter-item correlations with the
  actual inter-item correlations.

Compared with Do they Match?
Proposed Model
Actual Data
Item 1
A
Item 1
Factor Sum of 1,2
B
Correlation
Item 2
A 1s correlation with factor B 2s
correlation with factor By definition, Item 1
2s correlation is A B
Item 2
13
Scaling

• Steps in factor analysis (what the computer
  does)
• If the single concept is not a good model, the
  computer rejects one factor and forms a residual
  correlation matrix (real 1,2 proposed AB)
• Identifies a second concept that may explain some
  of the remaining correlation and checks the
  proposed inter-item correlation against the real
  correlations.
• And so on until the correlations match.

14
Scaling

• In actuality, factor analysis will give K factors
  for K variables. The last residual correlation
  matrix will result in zeros.
• So, how many factors should you use?
• You could use statistical criteria extract
  factors until matrix is not statistically
  significant from zero.
• Historically, number of factors has been
  determined by substantial needs, intuition, and
  theory.

15
Scaling

• Guideline for subjective analysis A group of
  factors should be able to explain a high
  proportion of total covariance among a set of
  items.
• Eigenvalue test
• Scree Test

16
Scaling

• Eigenvalues
• An eigenvalue represents the number of units of
  information that a factor explains in a k set of
  variables with k units of information.
• E.g., when k 10, an eigenvalue of 3 represents
  30 of information is explained by the factor.
• An eigenvalue of 1 corresponds with a variables
  worth of information. Therefore, factors with an
  eigenvalue of 1 or less do not help to reduce
  data.
• Get rid of factors with eigenvalues less than 1

17
Scaling

• Scree Test
• Most researchers are looking for stronger, fewer
  factors (they want to reduce data). Therefore,
  they tend to use the scree plot.
• Plot the eigenvalues relative to each other
• Strong factors form a steep slope, weaker factors
  form a plateau
• Retain those factors that lie above the elbow
  of the plotlike with gangrene, cut off the
  elbow!

2 1
Scree plot for 5 variables
18
Scaling

• In addition, factors should be composed of
  similar, logically linked items. This is an
  especially helpful rule when the number of
  factors is not that obvious.

19
Scaling

• Factor Rotation
• Factor rotation involves using an algorithm to
  maximize the correlation of items to a
  factormaking each item appear most relevant to a
  single factor.
• The point is to identify variables that most
  similarly form indicators of the same factoreach
  factors variables being most clearly highlighted.

20
Scaling

• Factor Rotation
• The best-scenario (never happens) is when all
  items load (correlate with) as 1 on a single
  factor and 0 on all the rest. This is called
  simple structure.
• Factor rotation mathematically takes the items as
  close as possible to simple structure.

21
Scaling

• Factor Rotation
• Orthogonal versus oblique rotation
• Orthogonal rotation makes factors completely
  independent of each other.
• This is preferred for finding the most unique
  factors. Use if factors ought not be related.
• Any items variation explained by one factor can
  be added to that of another factor to get the
  total variation explained by the two.
• If you find lots of cross-loading, you should
  consider Oblique.
• Oblique rotation makes factors that are allowed
  to be correlated with each other to some degree.
• Use if the factors ought to be related.
• There is redundancy in the variation of any item
  explained by one factor versus another, such that
  they have overlapping explanatory power.
• You might want to try both and look for simple
  structure.
• Strong loadings on two factors may indicate a
  single factor, high correlation of two factors
  may indicate a single factor.

22
Scaling

• Factor Rotation
• Items with a high loading on (high correlation
  with) a factor form the factors variable for
  research purposes.
• Common elements of the items is likely what the
  factor represents.

23
Scaling

• Type of analysis in extracting factors
• Principal components analysis produces specified
  proportion of total variance among items
  explained.
• Common factor analysis produces specified
  proportion of shared variance among items
  explained.
• Bottom line report which you used.

24
Scaling

• Exploratory versus Confirmatory Analysis
• Exploratory is that which we have been
  discussing. If using exploratory, with new
  samples you rediscover a structure in each
  sampleyou have persuasive evidence of the
  structure.
• Confirmatory typically refers to models generated
  by Structural Equation Modeling where items are
  specified to form a factor in advance. The
  question becomes, How well do the data fit a
  specified model using statistical inference?
  You have to be careful not to overproduce many
  meaningless factors.

25
Scaling

• Validity and Reliability
• Like other measures, scales and indexes must be
  valid and reliable to be useful.
• Validity Face, Content, Criterion, Construct
• A particular kind of reliability that is
  particularly useful for scales and indexes is
  inter-item reliability (internal consistency or
  high inter-item correlation)
• To the degree that the items are correlated, the
  common correlation is attributable to the true
  score of the latent variable.

26
Scaling

• Inter-item ReliabilityAlpha
• Variation in each item is caused by the latent
  variable and error (unique for each)
• Common variation is caused by the latent
  variable.
• Using the variance/covariance matrix, you can see
  total variance in the sum of components.
• The diagonal (variance) represents unique
  variation for each item.
• The off-diagonal represents co-variation of
  items. This also equals 1 (?Unique/Total)

27
Scaling

• Inter-item ReliabilityAlpha
• The off-diagonal represents co-variation of
  items. This also equals 1 (?Unique/Total)
• To correct for the ways variance/covariance
  matrices change with number of items, the formula
  above is adjusted by k/k-1, where k number of
  items. This constrains alpha to range from 0 to
  1.
• k ?Unique variance
• ? k 1 ?Total variance

28
Scaling

• Inter-item ReliabilityAlpha
• Some characteristics of alpha
• Holding correlation constant, alpha goes up with
  more scale items
• To improve a scale, look for effect on alpha if
  an item were dropped.
• Reliability is not good unless it is .65 or
  above. Best reliability would be around .9.
• Good scales require a balance between reliability
  and length.

29
Scaling

• Creating a scale
• Determine what you want to measure
• Clarity
• Specificity
• Generate an item pool
• Scales may be generated from 40 to 100 items
• Good reason to reuse scales
• Writing
• Positive and negative items
• Stay on topic
• Avoid lengthy items
• Keep wording simple
• Avoid multiple negatives
• No double-barreled items

30
Scaling

• Creating a scale
• Determine format for measurement.
• Response options
• Broad versus narrow
• Review of item pool by experts
• Include scale validation items
• Administer to a development sample
• Evaluate items
• Differences between items
• You need item variance
• Look for means in the middle of the scale
• Item-scale correlations