Design

1
Design

• Negotiate expectations class norms
• Introduction
• Research what is it?
• Review of past course (201)
• Designing research
• Experimental Designs

2
Types of research

• Descriptive
• Exploratory
• Explanatory
• True Experiments
• Control
• IVs are manipulated
• Random assignment
• Counterbalance random order
• Quasi-experimental
• Ltd. control
• Non random assignment
• EX POST FACTO (pre determined groups)

3
Randomisation

• Separates the true from the quasi
• R in sampling participant allocation into
  groups
• Defn
• Unbiased assignment
• chance allocates
• Necessary
• But
• Not simply order of arrival
• Hard to implement in the field
• Ethical issues (e.g. placebo control grp)

4
Validity

• The main aim of designing a research plan
• Correspondence with reality
• Coherence
• Internal Validity
• Fundamental logic of experiment
• Rule out alternative explanations
• But
• Confounding effects of extraneous variables
• Can covarying variables be completely eliminated?
  
• Acute problems for Quasi-experiments

5
Validity cont.

• Construct validity
• Do the results support the theory?
• If HIGH construct validity
• Alternative hyps ruled out
• Reduces need for additional research
• External validity
• The extent the results can be extrapolated
  (generalised)
• But
• Problematic in Organisational research
• or Evaluation research
• E.g. drug tests on your dog

6
Validity cont.

• Statistical validity
• Can we use statistical tests?
• Implies correct (random) sampling
• Meet test assumptions (independence normality)
• Are the findings the result of chance processes?
• i.e. Randomisation controlled sampling, and
  controlled assignment

7
Threats to Validity

• Internal validity
• Covarying events
• Maturation
• Effects of testing
• Measurement unreliability
• Statistical regression
• Selection
• Participant drop-out

8
Threats to Validity

• Construct validity
• Other theories accounting for results
• Loose connection between theory and the
  experiment
• Participants understanding of the
  tasks/experiment
• Instructions
• Hawthorne effect
• Evaluation apprehension
• Social desirability

9
Threats to Validity

• External validity (generalisability)
• Participants (Ps)
• if you use different Ps in diff phases
• Selection of Ps e.g. volunteers, students
  cultural background
• Times of experiment
• E.g. morning vs afternoon the day of the week
• Changing the time during the research
• Settings
• E.g. from the laboratory to real life from one
  climate to another from one language to
  another...

10
The Social Psychology of the experiment

• Participants expectations
• Role demands
• Can try to prevent this
• deceive P
• Data from another setting/Ps/time
• P unaware of Experiment
• Use unobtrusive measures
• Placebo for control group
• Experimenter Bias
• Conscious fudging, and use, of data
• Prevention
• Blind Double Blind procedures
• Standardised procedures
• Journal/Discipline Bias
• Only significant results published
• The spirit of the times

11
Control Strategies

• Participant as own control
• Random assignment
• Matching
• Build nuisance variables into design
• Statistical control
• Order sequence effects
• Order Order control
• Sequence Sequence control
• Replication

12
Control Strategiescont.

• Notes page cont.

13

• Encore...Research designs
• What can be known? i.e. the nature of reality
  (Paradigm)
• What do we want to know? (Question)
• Why do we want to know it? (Aim)
• Who do we ask questions about? (Sample)
• Individuals
• Groups
• Within groups
• Between groups
• How can we best answer our question?
• Common research methods
• Systematic observation
• Case study
• Psychometric method
• Survey method
• Experimental method
• There is no one right way of designing research
• But remember Validity, etc relevant to all
  research

14
Encore

• Validity
• Internal validity
• A syllogism
• External
• e.g. of drug tests on your dog (dont try this at
  home)
• The conflict between Internal External validity
• Lots of control little generalisability
• How to maximise validity
• Converging methods (Triangulation)
• Hybrid methods
• Confounding variables

15
Encore...cont.

• Causality
• Comparative research
• Relationship yes, but
• Asking for cause and effect sequence
• NB correlation is not cause
• Criteria of causality
• Covariation
• Time precedence
• Non-spuriousness no rival hyps or confounding
  variables
• Generalisability
• Representativeness of
• Sample (how, and size)
• Investigative conditions
• Tasks responses

16
Experiment Designs

• One group post-test only design
• X ------- 01
• Post-test only with non-equivalent control group
• X ------- A1
• X ------- Ch1
• One group pre-test - post-test design
• 01 ------- X ------- 02
• Better designs
• Two conditions, tested within-groups
• Two conditions, between-groups
• Multiple conditions experiment
• Factorial designs
• Multiple linear Regression

17

• Two conditions, tested within-groups design

Allocation Treatment Test Group 1 All Ps
experience 1 treatment Yes Group 2
both conditions 2 no treatment Yes

• Two conditions, between-groups design

Allocation Treatment Test Group
1 Random 1 treatment Yes Group
2 allocation 2 not treatment Yes
18

• Multiple conditions experiments design

Random assignment
All Ps experience all conditions random or
counterbalanced order
(e.g.cell phone sales)
Used when a) several conditions must be compared
at the same time b) need to determine the shape
of the function relating the IV to the DV
parametric experiments (often use
graphs/curves) c) need to rule out more than
one rival hyp. in one experiment.
19

• Factorial Design
• Covers every combination/variation
• include the interaction between variables
• Covariation between each factor and DV
• need
• Fixed point for variables (a choice)
• try to exhaust main possibilities
• discrete cuts i.e. categorical data
• Fixed Effects ANOVA
• But
• Decreased generalisability
• because weakness of the expt may exist at any
  one specific point and may therefore be missed
• However
• Can build nuisance variables in

20

• Two conditions, tested within-groups design
• within-group - each P is own control
• Replication studies
• e.g. Visualisation techniques and performance
  (number of wickets taken during season)
• Test (score) before and after the intervention

Causal effect of Visualisation on
performance
21

• Two conditions, between-groups design
• Between group - matching random assignment
• e.g. men-men/women-women same sport same level
  of competition (e.g. world cup)
• Test/score before and after the intervention

Causal effect of Visualisation on
performance
22

• Multiple conditions experimental design
• e.g. Intervention of Visualisation techniques on
  performance (wickets taken) by Fast Bowlers
• Include nuisance variable Hawthorne effect
• Either within-group

Or between groups
causal effect of Visualisation Tech on
Performance
23

• Factorial Design
• Can build more into experiment
• e.g. Visualisation Technique, observation, new
  fabric shirt/shoes...

• Performance (wickets taken) during season
• each grp plays same number of matches
• Groups (bowlers) are matched

• Between group (is normal) factorial design
• e.g. 80 fast bowlers, randomly assigned to cells

• Extra Often in Psyc research
• Within group Latin Square (factorial) design
• For few subjects
• Repeated measures (problematic! Why?)

24
Quasi-experimental research

• Ltd. control
• Non random assignment
• EX POST FACTO (pre determined groups)
• Non-equivalent groups (e.g.men-women)

• How do we compare results - when not equivalent?
• So, make hyp. about relationship rather than
  difference (between measures of µ)/cause
• (interaction will occur if hyp is correct)
• e.g. developmental research kids of diff ages
• or Men - Women comparisons
• or effect of Stock market crash (or Zim
  situation) on Investor Confidence

25
Possible Test Exam Test 15 marks Format
short answers OR MC? e.g. 1) What is Internal
Validity? (2marks) - - 2) What distinguishes
Experimental Designs from other designs?
(2marks) - - 3) What design would you use to
answer the following question? (2marks) Does
Aspirin use cause Heart disease? 4) List 2
possible confounding variables
(2marks) - - 5) What 2 Control Strategies
could you use to limit the influence of these
confounding variables(4mrks) age - other
hypotheses -