Analyzing the Results of an Experiment

1
Analyzing the Results of an Experiment

• -not straightforward..
• Why not?

2
Variability and Random/chance outcomes
3
Inferential Statistics

• Statistical analysis appropriate for inferring
  causal relationships and effects.
• Many different formulaswhich one do you use?

4
Inferential Stat selection

• -Determine that you are analyzing the results of
  an experimental manipulation, not a correlation
• Identify the IV and DV.
• The IV Will always be nominal on some level, even
  when it may seem to be continuous..low, medium
  and high doses of a drug

5
Inf. Stat Selection

• What is the scale of the DV?
• Scale of DV -Statistic to use

Nominal Chi-squared
Ordinal Mann-Whitney U-test

Continuous T-test or ANOVA



6
t-test or ANOVA?

• How many levels of the IV are there?

2 levels more than 2 levels

T-test or ANOVA ANOVA

7
There are different forms of T-tests and
ANOVAsDid the Study Use a Within Group or
Between group Experimental Design?
Between Group Within Group

Only 2 levels of the IV Unpaired t-tests (or t for independent samples). Paired t-tests ( or t for dependent samples)
Unpaired t-tests (or t for independent samples). Paired t-tests ( or t for dependent samples)
OrANOVA ( the basic ANOVA is fitted for between group designs) OrWithin group ANOVA (often referred to as a repeated measures ANOVA)

More than 2 levels of the IV ANOVA Repeated Measures ANOVA
8
In some ways all inferential Stats are similar.

• They calculate the probability that a result was
  due to the IV as opposed to random variability
• Lets focus on the Basic ANOVA since it is likely
  to be the statistic you may use most commonly.

9
ANOVA

• ANOVA produces an F-value.
• F values are the ratio of overall between group
  Variability to the Mean within group variability
• Between Var. ( chance) /Mean within grp.
  Variability ( chance)
• What does this mean?

10
Lets suppose

• Experiment- IV marijuana
• Control
• Placebo control
• Low dose
• High dose

11
Dependent Variable is

• Performance on a short term memory task measured
  number correct out of 10 test items.
• 9 subjects in each group

12
Possible out come 1
13
Possible Outcome 1Control Placebo Low
dose High dose

• 4 2 2 2
• 5 3 3 3
• 6 4 4 5
• 5 6 4 3
• 5 5 5 4
• 6 5 4 4
• 4 4 5 4
• 3 4 6 6
• 7 3 3 5

14
Distribution of scores for control sample
15
Placebo scores
16
Low dose scores
17
High dose scores
18
The population distribution of scores
19
F value relatively low
High
low
placebo
control
w/in grp. var
Between grp. Var
20
Now consider this Possible Outcome
2Control Placebo Low dose High dose

• 4 2 2 2
• 5 3 3 3
• 6 4 4 5
• 5 6 4 3
• 5 5 5 4
• 6 5 4 4
• 4 4 5 4
• 3 4 6 6
• 7 3 3 5

21
Distribution of scores for control sample
22
Placebo scores
23
Low dose scores
24
High dose scores
25
F value relatively High
High
low
placebo
control
w/in grp. var
Between grp. Var
26
The high F value reflects

• Logic!
• Distribution of score are much more obviously
  separated, and in this case are completely
  non-overlapping
• Low F values indicate highly overlapping score
  distributions

27
So how do we decide if an F value is large enough
to consider the result as causal?

• We consult a table of established probabilities
  of different F values, within the context of
  Degree of freedom terms

28
(No Transcript)
29
(No Transcript)
30
ANOVA Significance table
31
(No Transcript)
32
(No Transcript)
33
Where is/are the difference (s)?
34
Inferential Statistics
35
The story of Scratch
36
Why not jus use repeated t-tests? Probability
pyramiding

• 15 t-tests required for this data set
• Post-hocs include compensations for repeated
  testing of a large data set

37
After all this where so we stand?We can still be
wrong.
38
(No Transcript)
39
(No Transcript)
40
Factors that affect power.Sample size
41
One vs two-tailed testing
42
(No Transcript)
43

• Effect size

44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
(No Transcript)
51
(No Transcript)
52
(No Transcript)
53
(No Transcript)
54
(No Transcript)