Measures

1
Measures
Introduction to the Principles and Practice of
Clinical Research

• Daniel S. Pine, M.D./Erin B. McClure, Ph.D.
• Mood Anxiety Disorders Program, NIMH

With thanks to F. Xavier Castellanos David
Rubinow
2
The Uncertainty Principle

• The more precisely the POSITION is determined,
  the less precisely the MOMENTUM is known.
• 
  Werner Heisenberg

3
Introduction

• Measure selection must be informed by
  well-defined, answerable questions
• Results are only as good as the worst measure and
  are never perfect Types of measures
• Types of error (random non-random)
• Non-random error (bias) ? Type I error (false
  positives)
• Would you vote for this moderately intelligent
  candidate?
• Random error ? Type II error (low power)
• What will you do on November 2nd?
• Reliability limits statements on validity
• XXXX is more likely to win on November 2nd.

4
Outline Operational Questions

• 1. What is a measure/variable?
• 2. How do you measure?
• 3. How do you test measures (Reliability/Validity)
  ?
• 4. How do you select measures?
• 5. What factors influence the measures?
• 6. When is the measure obtained and how often?

5
What is a measure?

• Something that varies in an observable,
  quantifiable fashion
• Measure true value error
• Error random error biased error
• Central limit theorem eliminates effects of
  random error
• http//www.stat.sc.edu/west/javahtml/CLT.html
• http//www.math.uah.edu/stat/sample/sample5.html

6
Central Limit Theorem in ActionSum of All Rolled
Dice
Two Dice
One Die
Three Dice
Four Dice
7
Outline Types of Scales(What You Measure With)

• Nominal or Categorical
• Classification or set of categories mutually
  exclusive and collectively exhaustive e.g.,
  gender sick vs. healthy
• Ordinal
• Mutually exclusive classes that form an ordered
  series rank order e.g., grades on a statistics
  test seriousness of a tumor
• Interval
• Ordered series of ranks with equal intervals
  between any two pairs of adjacent classes e.g.,
  temperature
• Ratio
• An interval scale with a true zero point origin
  e.g., weight

8
Categorical/Nominal Measures

• hanging chad only one corner of the almost
  completely punched-out piece of paper is still
  connected to the ballot
• swinging chad 2 corners remain attached,
  resembling a door
• tri-chad 3 corners remain attached and only one
  corner is flapping outward, but some pushed-away
  space is evident
• pregnant chad the rectangle has a bulge and
  seems to have been lightly punched, but all four
  corners are still attached
• dimpled chad the space has a slight
  indentation, but the corners remain connected

Gannett News Service
9
Categorical/Nominal Measures

• Classification or set of categories mutually
  exclusive and collectively exhaustive
• Statistical operation Counting
• Caseness
• Criteria lifetime vs. current symptoms
• Comorbid conditions
• Subthreshold non-cases
• Fundamental process in epidemiology

10
Ordinal Scales

• e.g., Rating Scales, IQ
• Subjectivity
• Floor/Ceiling effects common

11
Implications of Scale Type

• Determine which statistical operations are
  permissible
• Parametric
• Non-parametric
• Within subject comparisons
• Between subject comparisons

12
How Do You Test Measures?

• Reliability
• The consistency with which a measure assesses a
  given trait i.e., agreement between two measures
  obtained by the same or maximally similar methods
• Validity
• The extent to which a measure actually measures a
  trait i.e., agreement between two measures
  obtained by maximally different methods

13
Types of Validity

• Face (Content) Validity
• Right items performance (or response) free of
  influence of irrelevant variables
• Criterion-related Validity
• Comparison with independent, direct measures
• Construct Validity
• Measurement of the theoretical construct

14
On the methods and theory of reliabilityJ J
Bartko W T Carpenter J Nerv Ment Dis 1976,
163307-314

• Unsuitable methods
• Percent agreement
• Chi-square
• Correlation
• Suitable methods
• Kappa
• Intraclass correlation coefficient (ICC)

15
Diagnostic Reliability - Agreement
Expected Frequencies
Bartko Carpenter J Nerv Ment Dis 1976,
163307-314
16
How Do You Select Measures?

• What is the variable of interest?
• What are the dependent and independent variable
  domains?
• Disorder dependent variables
• e.g., symptoms, side effects, biochemical indices
• Disorder independent variables
• e.g., life events

17
Measure Selection

• What are the variable characteristics?
• Severity vs. frequency, vs. both?
• Absolute value vs. change
• Symptom vs. syndrome

18
Analog Scales

• Symptom specific
• Ease of completion
• Sensitive
• Replicable
• Ease of data entry and analysis

10 cm
Anchor point
Anchor Point
19
Effect of Endpoint Labels
Schwartz et al., 1991 Public Op Q 570-582
20
Effects of Response Alternatives
Schwartz et al., 1985, Public Opinion Q 49
388-395
21
Assessing the Measure

• Examine data that tests the instrument
• Range
• Sensitivity
• Discrimination of relevant, predictable
  differences
• Proportion of true cases identified

22
Assessing the Measure (II)

• What are the kinetics of the process?
• How long and how often should the measure be
  applied?

23
What Factors Influence the Measure?

• Performance variables
• Skill and care
• Practice, floor ceiling effects
• Typographical errors
• Test conditions
• Insensitivity in range of interest
• Infinite unknown factors ? RANDOMIZE

24
Measures - Summary

• Choose your measures carefully
• Know their weaknesses
• Reliability before validity
• Beware of biased error above all
• Central Limit Theorem
• At the NIH, if plt.00001 then its an artifact

25
Useful on-line Statistics Primer

• Hopkins, W. G. (1997). A new view of
  statistics Internet Society for Sport Science
• http//www.uq.oz.au/hmrburge/stats/index.html