Research Methods and Statistics 2

1
Research Methods and Statistics 2

• rhys.davies_at_newport.ac.uk

2
Last week

• The scientific cycle
• Hypothesis, design study, derive predictions,
  test them and conclude.
• Descriptive Statistics and Inferential Statistics
• Descriptive Statistics - Displaying data
• Variables IV, DV and confound
• Reliability and Validity
• Coding and levels of data (nominal, ordinal,
  interval, ratio)
• Central Tendency (mean median mode) and measures
  of dispersion (range, SD and variance)
• A little about the normal distribution and
  Parametric and non parametric tests

3
Sampling Methods

• Cannot make direct observations of everyone in
  the population.
• Ideally, sample population on characteristics
  of interest
• Two general approaches
• Probability sampling all elements have an equal
  chance of being selected, the chance of them
  being selected can be calculated.
• Non-probability sampling selected on
  availability or researchers judgement that they
  are representative

4
Probability sample

• Sampling frame Probability sample starts here, a
  list of everyone in population - operationally
  defines target population, possibly give everyone
  a number.
• Simple random sample sampling done in first
  stage with everyone equally likely to be picked
• Systematic sample sampling starts at a randomly
  selected participant and then takes every nth
  participant
• Cluster sample sample selected in stages,
  selecting groups (schools, towns) then randomly
  selecting participants within these (one of
  previous 2 methods)

5
Random sampling example

• Research attitudes of secondary school children
  towards Mathematics in Pembrokeshire
• Obtain sampling frame a list of all secondary
  school pupils from LEA database complete
  sample?
• Select a relatively small number from this list
• 7000 pupils, you decide you want to sample 100.

6
Random sample example cont/d

• Simple random sample random number table or
  computer used to randomly pick 100 pupils
• Systematic sample
• 7000/10070 (every 70th pupil on list).
• Rnd(70)any number between 1-70
• Then every 70th pupil after that
• Cluster sample 3 schools picked (random or
  non-random?), 33 pupils randomly selected from
  each (cluster again year groups)

7
Non-random samples

• If you dont have a sampling frame or other
  constraints (cheaper, geography, time)
• Convenience sample the most common type!
• Participants are picked due to availability
• Does not include the entire population as
  potential participants loose some ability to
  generalise the results
• Stopping shoppers or school pupils in a corridor,
  this is NOT truly random selection

8
Non-random samples

• Purposive Sampling researcher picks
  subjectively and tries to include range between
  extremes
• Quota Sampling decide on representative
  characteristics (gender, age) 20 1st years (10
  boys -10 girls), 20 2nd years researcher should
  try not to pick the ones they like, easiest to
  get etc bias
• Snowball Sampling initially contact a few, e.g.
  vegetarians, then ask if they know of others

9
Non-response

• Refusal try for a reason ok to refuse
• Unable to contact
• Not able to participate reason
• 2 major problems
• Reduction in sample size replace
• Bias get reasons, try to explain
• Good to list non response reasons

10
Sample size

• Depends on
• Methodology selected
• Degree of accuracy required
• Population characteristics, how much does what
  youre interested in vary?
• Time, money, availability
• Statistical regularity- a reasonably large sample
  will be, on average, representative of population
• Large numbers of participants show greater
  stability than small numbers, variations in data
  tend to cancel each other out with larger numbers

11
Example of sample size

• Say 50 of population are C of E, we can
  calculate our accuracy.
• 95 of the time, the true proportion in the
  population will be within /- sampling error

12
Parametric tests - probability

• With a normal distribution we can predict the
  percentage of a population at points under the
  curve

13
Type I and Type II error

• When we use a statistical test, there is a chance
  that the decision we make with it is a wrong one
• We either accept or reject our hypothesis on a
  chance score ((plt0.05 or plt0.01) a 1 in 20 chance
  or a 1 in 100 chance)
• Occasionally these chances will be wrong, either
  a Type I error or a Type II error

14
Type I and Type II error

• Type I error The possibility of rejecting the
  null hypothesis given that it is true, or
  believing in an effect where there is not one
• Type II error failing to reject the null
  hypothesis when it is not true, there is an
  effect but we have not found it in this
  experiment so we say there is not one

15
Probability

• We will often read results such as
• For the World Trade Center confidence in the
  veracity judgements was found to have reduced
  significantly (U51, Z-2.468, p0.014)
• Or
• a significant difference existed between
  self-rated personal experiences and experiences
  through the media
• (Z -6.367, ties 24, p lt 0.0001)

16
The p value gives us this probability

• P0.2 - a 1 in 5 (20) chance that the result
  is due purely to sampling error or chance
  anything plt0.2 may be indicative of an effect,
  was sample size too small
• P 0.5 - 50 of the time we will get this
  result due to sampling error we accept the null
  hypothesis as true, there is no effect

17
Summary

• Sampling techniques that are available
• Probability and non probability samples
• Sample size
• Why sampling and sample size are important
• Type I and Type II errors
• Probability and p-values

18
All of this will be on

• www.education.newport.ac.uk
• Follow the link to research
• I am going to try to put it up there tomorrow, if
  not I will do it over the weekend and it will be
  there next Monday
• Email me for things that you would like on the
  site and I will try to get the content up
  quickly, the site is under development and your
  input is appreciated
• rhys.davies_at_newport.ac.uk