Statistical Analysis

1
Statistical Analysis

• Professor Lynne Stokes
• Department of Statistical Science
• Lecture 18
• Random Effects

2
Fixed vs. Random Factors
Fixed Factors Levels are preselected, inferences
limited to these specific levels
Factors Shaft Sleeve Lubricant Manufacturer S
peed
Levels Steel,
Aluminum Porous, Nonporous Lub 1, Lub 2, Lub 3,
Lub 4 A, B High, Low
3
Fixed Factors (Effects)
Fixed Factors Levels are preselected, inferences
limited to these specific levels
One-Factor Model yij m ai eij Main
Effects mi - m ai Parameters
Changes in the mean mi
Fixed Levels
4
Random Factors (Effects)
Random Factor Levels are a random sample from a
large population of possible levels. Inferences
are desired on the population of levels.
Factors Lawnmower
Levels 1, 2, 3, 4, 5, 6
One-Factor Model yij m ai eij
Random Levels
5
Random Factors (Effects)
One-Factor Model yij m ai eij Main
Effects
Random ai Variability
Estimate Variance Components sa2 , s2
6
Skin Swelling Measurements
Factors Laboratory animals (Random)
Location of the measurement Back, Ear (Fixed)
Repeat measurements (2 / location)
7
Automatic Cutoff Times
Factors Manufacturers A, B (Fixed)
Lawnmowers 3 for each manufacturer (Random)
Speeds High, Low (Fixed)

MGH Table 13.6
8
Random Factor Effects
Assumption Factor levels are a random sample from
a large population of possible levels

• Subjects (people) in a medical study
• Laboratory animals
• Batches of raw materials
• Fields or farms in an agricultural study
• Blocks in a block design

Inferences are desired on the population of
levels, NOT just on the levels included in the
design
9
Random Effects Model Assumptions(All Factors
Random)

• Levels of each factor are a random sample of all
  possible levels of the factor
• Random factor effects and model error terms are
  distributed as mutually independent zero-mean
  normal variates e.g., eiNID(0,se2) ,
  aiNID(0,sa2), mutually independent

Analysis of variance model contains
random variables for each random factor and
interaction
Interactions of random factors are assumed random
10
Skin Color Measurements
Factors Participants -- representative of
those from one ethnic group, in a
well-defined geographic region of the U.S.
Weeks -- No skin treatment, studying week-to-week
variation (No Repeats -- must be able to
assume no interaction)
MGH Table 10.3
11
Two-Factor Random Effects Model Main Effects Only
Two-Factor Main Effects Model
yijk m ai bj eijk i 1, ..., a j
1, ..., b
12
Two-Factor Random Effects Model
Two-Factor Model
yijk m ai bj (ab)ij eijk i 1, ...,
a j 1, ..., b k 1, ..., r
13
Two-Factor Model Differences
Fixed Effects
mij m ai bj (ab)ij
Mean
Change the Mean
Variance
14
Expected Mean Squares

• Functions of model parameters
• Identify testable hypotheses
• Components set to zero under H0
• Identify appropriate F statistic ratios
• Under H0, two E(MS) are identical

15
Properties of Quadratic Forms in Normally
Distributed Random Variables
16
Expected Mean Squares
One Factor, Fixed Effects
yij m ai eij
i 1, ... , a j 1, ... , r
eij NID(0,se2)
17
Expected Mean Squares
One Factor, Fixed Effects
18
Expected Mean Squares
One Factor, Fixed Effects
Sum of Squares
EMSA)se2 a1 a2 ... aa
19
Expected Mean Squares
Three-Factor Fixed Effects Model
Source Mean Square Expected Mean
Square A MSA se2 bcr Qa AB MSAB se2 cr
Qab ABC MSABC se r Qabg Error MSE se2
Typical Main Effects and Interactions

• All effects tested against error

20
Expected Mean Squares
One Factor, Random Effects
yij m ai eij
i 1, ... , a j 1, ... , r
ai NID(0,sa2) , eij NID(0,se2)
Independent
21
Expected Mean Squares
One Factor, Random Effects
Sum of Squares
22
Expected Mean Squares
One Factor, Random Effects
Sum of Squares
EMSA)se2 sa2 0
23
Skin Color Measurements
Factors Participants -- representative of
those from one ethnic group, in a
well-defined geographic region of the U.S.
Weeks -- No skin treatment, studying week-to-week
variation (No Repeats -- Must be Able to
Assume No Interaction)
24
Expected Mean Squares
Three-Factor Random Effects Model
Source Mean Square Expected Mean
Square A MSA se2 rsabc2 crsab2
brsac2 bcrsa2 AB MSAB se2 rsabc2
crsab2 ABC MSABC se rsabc2 Error MSE se2

• Effects not necessarily tested against error
• Test main effects even if interactions are
  significant
• May not be an exact test (three or more factors,
  random
• or mixed effects models e.g. main effect
  for A)

25
Expected Mean SquaresBalanced Random Effects
Models

• Each E(MS) includes the error variance component
• Each E(MS) includes the variance component for
  the corresponding main effect or interaction
• Each E(MS) includes all higher-order interaction
  variance components that include the effect
• The multipliers on the variance components equal
  the number of data values in factor-level
  combination defined by the subscript(s) of the
  effect

e.g., E(MSAB) se2 rsabc2 crsab2
26
Expected Mean SquaresBalanced Experimental
Designs

• 1. Specify the ANOVA Model

yijk m ai bj (ab)ij eijk
Two Factors, Fixed Effects
MGH Appendix to Chapter 10
27
Expected Mean SquaresBalanced Experimental
Designs

• 2. Label a Two-Way Table
• a. One column for each model subscriptb. Row
  for each effect in the model -- Ignore the
  constant term -- Express the error term as a
  nested effect

28
Two Factors, Fixed Effects
yijk m ai bj (ab)ij eijk
29
Expected Mean SquaresBalanced Experimental
Designs

• 3. Column Subscript Corresponds to a Fixed
  Effect.
• a. If the column subscript appears in the row
  effect no other subscripts in the row effect
  are nested within the column subscript
• -- Enter 0 if the column effect is in a
  fixed row effect
• b. If the column subscript appears in the row
  effect one or more other subscripts in the
  row effect are nested within the column
  subscript
• -- Enter 1
• c. If the column subscript does not appear in
  the row effect
• -- Enter the number of levels of the factor

30
Two Factors, Fixed Effects
yijk m ai bj (ab)ij eijk
Step 3a
31
Two Factors, Fixed Effects
yijk m ai bj (ab)ij eijk
Step 3b
32
Two Factors, Fixed Effects
yijk m ai bj (ab)ij eijk
Step 3c
33
Expected Mean SquaresBalanced Experimental
Designs

• 4. Column Subscript Corresponds to a Random
  Effect
• a. If the column subscript appears in the row
  effect
• -- Enter 1
• b. If the column subscript does not appear in
  the row effect
• -- Enter the number of levels of the factor

34
Two Factors, Fixed Effects
yijk m ai bj (ab)ij eijk
Step 4a
35
Two Factors, Fixed Effects
yijk m ai bj (ab)ij eijk
Step 4b
36
Expected Mean SquaresBalanced Experimental
Designs

• 5. Notation
• a. f Qfactor(s) for fixed main effects and
  interactions
• b. f sfactor(s)2 for random main effects and
  interactions
• List each f parameter in a column on the same
  line as its corresponding model term.

37
Two Factors, Fixed Effects
yijk m ai bj (ab)ij eijk
Step 5
38
Expected Mean SquaresBalanced Experimental
Designs

• 6. MS Mean Square, C Set of All Subscripts
  for the Corresponding Model Term
• a. Identify the f parameters whose model terms
  contain all the subscripts in C (Note can have
  more than those in C)
• b. Multipliers for each f
• -- Eliminate all columns having the
  subscripts in C
• -- Eliminate all rows not in 6a.
• -- Multiply remaining constants across rows
  for each f
• c. E(MS) is the linear combination of the
  coefficients from
• 6b and the corresponding f parameters
  E(MSE) se2.

39
Two Factors, Fixed Effects
yijk m ai bj (ab)ij eijk
Step 6a
40
Two Factors, Fixed Effects
yijk m ai bj (ab)ij eijk
Step 6b MSAB
41
Two Factors, Fixed Effects
yijk m ai bj (ab)ij eijk
Step 6c
42
Two Factors, Fixed Effects
yijk m ai bj (ab)ij eijk
Step 6b MSB
43
Two Factors, Fixed Effects
yijk m ai bj (ab)ij eijk
Step 6c
44
Two Factors, Fixed Effects
Under appropriate null hypotheses, E(MS) for A,
B, and AB same as E(MSE) F MS / MSE