Experiment Basics: Control

1
Experiment Basics Control

• Psych 231 Research Methods in Psychology

2
Announcements

• Due this week in labs - Group project
• Methods sections
• IRB worksheet (including a consent form)
• Recommended/required
• Questionnaires/examples of stimuli, etc. things
  that you want to have ready for pilot week (week
  10)
• Exam 2 two weeks from today

3
Experimental Control

• Our goal
• To test the possibility of a systematic
  relationship between the variability in our IV
  and how that affects the variability of our DV.

• Control is used to
• Minimize excessive variability
• To reduce the potential of confounds (systematic
  variability not part of the research design)

4
Experimental Control

• Our goal
• To test the possibility of a systematic
  relationship between the variability in our IV
  and how that affects the variability of our DV.

the variability in our IV
NRexp Manipulated independent variables (IV)

• Our hypothesis the IV will result in changes
  in the DV

NRother extraneous variables (EV) which covary
with IV

• Condfounds

Random (R) Variability

• Imprecision in measurement (DV)
• Randomly varying extraneous variables (EV)

5
Experimental Control Weight analogy

• Variability in a simple experiment

T NRexp NRother R
6
Experimental Control Weight analogy

• Variability in a simple experiment

T NRexp NRother R
Control group
Treatment group
7
Experimental Control Weight analogy

• If there is an effect of the treatment then NRexp
  will ? 0

Control group
Treatment group
Difference Detector
Our experiment can detect the effect of the
treatment
8
Things making detection difficult

• Potential Problems
• Confounding
• Excessive random variability

Difference Detector
9
Potential Problems

• Confound
• If an EV co-varies with IV, then NRother
  component of data will be present, and may lead
  to misattribution of effect to IV

IV
DV
EV
10
Confounding

• Confound
• Hard to detect the effect of NRexp because the
  effect looks like it could be from NRexp but
  could be due to the NRother

NR
other
NR
exp
Difference Detector
Experiment can detect an effect, but cant tell
where it is from
11
Confounding

• Confound
• Hard to detect the effect of NRexp because the
  effect looks like it could be from NRexp but
  could be due to the NRother

These two situations look the same
NR
other
Difference Detector
There is not an effect of the IV
There is an effect of the IV
12
Potential Problems

• Excessive random variability
• If experimental control procedures are not
  applied
• Then R component of data will be excessively
  large, and may make NRexp undetectable

13
Excessive random variability

• If R is large relative to NRexp then detecting a
  difference may be difficult

Difference Detector
Experiment cant detect the effect of the
treatment
14
Reduced random variability

• But if we reduce the size of NRother and R
  relative to NRexp then detecting gets easier

• So try to minimize this by using good measures of
  DV, good manipulations of IV, etc.

Difference Detector
Our experiment can detect the effect of the
treatment
15
Controlling Variability

• How do we introduce control?
• Methods of Experimental Control
• Constancy/Randomization
• Comparison
• Production

16
Methods of Controlling Variability

• Constancy/Randomization
• If there is a variable that may be related to the
  DV that you cant (or dont want to) manipulate
• Control variable hold it constant
• Random variable let it vary randomly across all
  of the experimental conditions

17
Methods of Controlling Variability

• Comparison
• An experiment always makes a comparison, so it
  must have at least two groups
• Sometimes there are control groups
• This is often the absence of the treatment

Training group
No training (Control) group

• Without control groups if is harder to see what
  is really happening in the experiment
• It is easier to be swayed by plausibility or
  inappropriate comparisons
• Useful for eliminating potential confounds

18
Methods of Controlling Variability

• Comparison
• An experiment always makes a comparison, so it
  must have at least two groups
• Sometimes there are control groups
• This is often the absence of the treatment

• Sometimes there are a range of values of the IV

1 week of Training group
2 weeks of Training group
3 weeks of Training group
19
Methods of Controlling Variability

• Production
• The experimenter selects the specific values of
  the Independent Variables

1 week of Training group
2 weeks of Training group
3 weeks of Training group

• Need to do this carefully
• Suppose that you dont find a difference in the
  DV across your different groups
• Is this because the IV and DV arent related?
• Or is it because your levels of IV werent
  different enough

20
Experimental designs

• So far weve covered a lot of the about details
  experiments generally
• Now lets consider some specific experimental
  designs.
• Some bad (but common) designs
• Some good designs
• 1 Factor, two levels
• 1 Factor, multi-levels
• Between within factors
• Factorial (more than 1 factor)

21
Poorly designed experiments

• Bad design example 1 Does standing close to
  somebody cause them to move?
• hmm thats an empirical question. Lets see
  what happens if
• So you stand closely to people and see how long
  before they move
• Problem no control group to establish the
  comparison group (this design is sometimes called
  one-shot case study design)

22
Poorly designed experiments

• Bad design example 2
• Testing the effectiveness of a stop smoking
  relaxation program
• The participants choose which group (relaxation
  or no program) to be in

23
Poorly designed experiments

• Bad design example 2

• Non-equivalent control groups

Independent Variable
Dependent Variable
Self Assignment
Training group
Measure
participants
No training (Control) group
Measure
Problem selection bias for the two groups, need
to do random assignment to groups
24
Poorly designed experiments

• Bad design example 3 Does a relaxation program
  decrease the urge to smoke?
• Pretest desire level give relaxation program
  posttest desire to smoke

25
Poorly designed experiments

• Bad design example 3

• One group pretest-posttest
• design

Independent Variable
Dependent Variable
Dependent Variable
participants
Pre-test
Training group
Post-test Measure
Add another factor
Problems include history, maturation, testing,
and more
26
1 factor - 2 levels

• Good design example
• How does anxiety level affect test performance?
• Two groups take the same test
• Grp1 (moderate anxiety group) 5 min lecture on
  the importance of good grades for success
• Grp2 (low anxiety group) 5 min lecture on how
  good grades dont matter, just trying is good
  enough

• 1 Factor (Independent variable), two levels
• Basically you want to compare two treatments
  (conditions)
• The statistics are pretty easy, a t-test

27
1 factor - 2 levels

• Good design example
• How does anxiety level affect test performance?

28
1 factor - 2 levels

• Good design example
• How does anxiety level affect test performance?

anxiety
80
60
Observed difference between conditions
T-test
Difference expected by chance
29
1 factor - 2 levels

• Advantages
• Simple, relatively easy to interpret the results
• Is the independent variable worth studying?
• If no effect, then usually dont bother with a
  more complex design
• Sometimes two levels is all you need
• One theory predicts one pattern and another
  predicts a different pattern

30
1 factor - 2 levels

• Disadvantages
• True shape of the function is hard to see
• Interpolation and Extrapolation are not a good
  idea

31
1 factor - 2 levels

• Disadvantages
• True shape of the function is hard to see
• Interpolation and Extrapolation are not a good
  idea

32
1 Factor - multilevel experiments

• For more complex theories you will typically need
  more complex designs (more than two levels of one
  IV)
• 1 factor - more than two levels
• Basically you want to compare more than two
  conditions
• The statistics are a little more difficult, an
  ANOVA (Analysis of Variance)

33
1 Factor - multilevel experiments

• Good design example (similar to earlier ex.)
• How does anxiety level affect test performance?
• Two groups take the same test
• Grp1 (moderate anxiety group) 5 min lecture on
  the importance of good grades for success
• Grp2 (low anxiety group) 5 min lecture on how
  good grades dont matter, just trying is good
  enough

• Grp3 (high anxiety group) 5 min lecture on how
  the students must pass this test to pass the
  course

34
1 factor - 3 levels
35
1 Factor - multilevel experiments
60
36
1 Factor - multilevel experiments

• Advantages
• Gives a better picture of the relationship
  (function)
• Generally, the more levels you have, the less you
  have to worry about your range of the independent
  variable

37
Relationship between Anxiety and Performance
38
1 Factor - multilevel experiments

• Disadvantages
• Needs more resources (participants and/or
  stimuli)
• Requires more complex statistical analysis
  (analysis of variance and pair-wise comparisons)

39
Pair-wise comparisons

• The ANOVA just tells you that not all of the
  groups are equal.
• If this is your conclusion (you get a
  significant ANOVA) then you should do further
  tests to see where the differences are
• High vs. Low
• High vs. Moderate
• Low vs. Moderate