2kr Factorial Designs with Replications

1
Statistics
continued
CPS 807
2
2kr Factorial Designs with Replications

• r replications of 2k Experiments
• 2kr observations.
• Allows estimation of experimental errors
• Model
• y q0 qAxA qBxB qABxAxBe
• eExperimental error

3
Computation of Effects

• Simply use means of r measurements

Effects q0 41, qA 21.5, qB 9.5, qAB 5
4
Estimation of Experimental Errors
5
Experimental Errors Example
6
Allocation of Variation
7
Derivation
8
Derivation (contd)
9
Derivation (contd)
10
Example Memory-Cache Study
11
Example Memory-Cache Study(contd)
SSA SSB SSAB SSE 5547 1083 300 102
7032 SST
Factor A explains 5547/7032 or 78.88 Factor B
explains 15.40 Interaction AB explains
4.27 1.45 is unexplained and is attributed to
errors.
12
Review Confidence Interval for the Mean

• Problem How to get a single estimate of the
  population mean from k sample estimates?
• Answer Get probabilistic bounds.
• Eg., 2 bounds, C1 C2 ?? There is a high
  probability, 1-?, that the mean is in the
  interval (C1, C2 )
• Pr C1 ? ? ? C2 1 - ?
• Confidence interval ? (C1, C2 )
• ? ? Significance Level
• 100 (1-?) ? Confidence Level
• 1-? ? Confidence Coefficient.

13
Confidence Interval for the Mean (contd)
Note Confidence Level is traditionally
expressed as a percentage (near 100) whereas,
significance level ?, is expressed as a fraction
is typically near zero e.g., 0.05 or 0.01.
14
Confidence Interval for the Mean (contd)
15
Testing for a Zero Mean
Difference in processor times of two different
implementations of the same algorithms was
measured on 7 similar workloads. The differences
are 1.5, 2.6, -1.8, 1.3, -0.5, 1.7, 2.4 Can we
say with 99 confidence that one implementation
is superior to the other
16
Testing for a Zero Mean (contd)
17
Type I Type II Errors
18
Confidence Intervals For Effects
Effects are random variables. Errors N(0,se) gt
y N( y.., se)
Since q0 Linear combination of normal
variables gt q0 is normal with variance Variance
of errors
19
Confidence Intervals For Effects (contd)
Denominator 22(r - 1) of independent terms
in SSE gt SSE has 22(r - 1) degrees of
freedom. Estimated variance of q0
Similarly, Confidence intervals (CI) for the
effects CI does not include a zero gt
significant
20
Example
For Memory-cache study Standard deviation of
errors
Standard deviation of effects
For 90 Confidence
21
Example (contd)
Confidence intervals
No zero crossing gt All effects are significant.
22
Confidence Intervals for Contrasts
Contrast ? Linear combination with ?
coefficients 0 Variance of ? hiqi
For 100 ( 1 - ? ) confidence interval, use
23
Example Memory-cache study
u qA qB -2qAB Coefficients 0,1,1, and -2
gt Contrast Mean u 21.5 9.5 - 2 x 5
21 Variance
Standard deviation
t0.958 1.86 90 Confidence interval for u
(16.31, 25.69)
24
CI for Predicted Response

Mean response y y q0 qA xA qB xB
qABxA xB The standard deviation of the mean
of m response

neff Effective deg of freedom
Total number of runs

1 Sum of DFs of params used in y
25
CI for Predicted Response (contd)
26
Example Memory-cache Study

• For xA -1 and xB -1
• A single confirmation experiment
• y1 q0 - qA - qB qAB
• 41 - 21.5 - 9.5 5 15
• Standard deviation of the prediction

Using t0.9581.86, the 90 confidence interval
is
27
Example Memory-cache Study (contd)

• Mean response for 5 experiments in future

The 90 confidence interval is

• Mean response for a large number of experiments
  in future

The 90 confidence interval is
28
Example Memory-cache Study (contd)

• Current mean response Not for future.
• (Use the formula for contrasts)

90 confidence interval
Notice Confidence intervals become narrower.
29
Assumptions
1. Errors are statistically independent. 2.
Errors are additive. 3. Errors are normally
distributed 4. Errors have a constant standard
deviation ?e. 5. Effects of factors are
additive. gt observations are independent and
normally distributed with constant variance.