Statistics and your thesis: How to approach the analysis

1
Statistics and your thesis How to approach the
analysis avoid common pitfalls.

• Postgraduate Colloquium
• Jeromy Anglim

2
Sources of assistance

• Quant Lecturers (Paul Dudgeon Phil Smith
  Richard Bell Garry Robins Pip Pattison)
• Department Stats Consultant - Jeromy Anglim
• j.anglim_at_pgrad.unimelb.edu.au
• Room 1110
• University Statistics Consulting Service
• http//www.scc.ms.unimelb.edu.au/
• At no charge, up to five hours of statistical
  advice to any Masters by research, PhD or MD
  student in 2004

3
Statistics Subjects

• 4th Year statistics (sit in)
• Excellent course that allows you to learn a whole
  range of multivariate techniques including data
  importation, cluster analysis, multidimensional
  scaling, exploratory and confirmatory factor
  analysis, correspondence analysis, optimal
  scaling, canonical correlation, ANCOVA MANOVA,
  and more.
• 512-984 Quantitative Methods for Organisational
  Psychology
• good for anyone wanting to do network analysis or
  multi-level modelling
• 512-933 Categorical Data Analysis
• as the name suggests, good for analysing
  categorical data
• chi-square, log-linear modelling, time series,
  markov chains, survival analysis, correspondence
  analysis
• 512-998 Structural Equation Modelling

4
Additional Assistance

• School of Graduate Studies
• http//www.sgs.unimelb.edu.au/services/skills/upsk
  ills/computer.html
• short courses in SPSS
• Introduction to SPSS
• May even be able to take Monash courses based on
  reciprocal program
• Private tutoring
• http//www.pen.psych.unimelb.edu.au/general/tutors
  .html

5
Books and websites

• Statistics Books
• Tabachnick Fidell - Using Multivariate
  Statistics (4th Edition)
• Hair, Anderson, Tatham, Black Multivariate
  Data Analysis
• SPSS books
• Coakes Steade - SPSS without Anguish (5th
  Edition)

6
SPSS Help

• Right click on an SPSS table and click results
  coach for an explanation of the table
• Right click an SPSS dialog box for more
  explanation about the particular option
• Press the help button
• SPSS syntax guide
• Websites
• http//www2.chass.ncsu.edu/garson/pa765/statnote.h
  tm
• Google search name of statistics technique
• Richard Bells 4th year subject page
  http//webraft.its.unimelb.edu.au/512422/pub/

7
Latest Position

• Statistical Task Force for the APA
• Effect size and power
• Honesty in reporting
• Simplicity is sometimes better

8
Making Friends with your data

• Wright, D.B. (2003) Making friends with your
  data Improving how statistics are conducted and
  reported. British Journal of Educational
  Psychology, 73(Mar), 123- 136.
• Report exact p values
• Report effect sizes
• Level of measurement is a decision made by
  researchers
• If you are worried about outliers or bad
  distributions, run the analyses with and without
  outliers removed or transformations applied. If
  you get same results, you should feel more
  confident
• Avoid using median splits
• Data model error
• Tell a story with the data

9
Some golden rules

• Stay close to the data
• Link analyses back to theories
• Try it both ways and if it doesnt make a
  different then it probably doesnt matter

10
Stage 1 experimental design

• Statistical Power
• G-Power
• http//www.psycho.uni-duesseldorf.de/aap/projects/
  gpower/
• or type g power into Google
• freeware power analysis software
• tips repeated measures increases power reduce
  within group variance, increase size of effect,
  increase sample size

11
Stage 1 Experimental Design

• Selecting good instruments (reliability,
  validity)
• Proposing analyses for mini-conference
• Ground theory in testable ideas
• Get input from your panel, if you are uncertain
• Common pitfalls
• Multiple groups and regression based models
• Poorly designed questionnaires (consider the
  factor structure from the start, have enough
  rating levels for the scale)

12
Stage 2 - Data Entry

• Create a data definition document
• Sets out columns used in raw text file SPSS
  variable name label or description missing
  values and what they mean any value labels
• Enter at the item level

13
Stage 2 - Data Entry

• Use variable names that are simple to work with
• e.g., s1 to s20 for a 20 item questionnaire
• stick to a set of conventions)
• Have a data validation strategy
• Examine frequencies of variables for plausible
  values
• Examine correlations and see whether they
  correspond to expectations

14
Stage 3 Initial Analyses

• Missing values analysis
• Take advantage of the new SPSS procedure
• Scale computation
• Remember to reverse negatively worded items (max
  min - score)
• Factor analyse
• Check reliability statistics (item if scale
  deleted)
• Basic descriptives
• Means
• Frequencies
• distributions
• correlations
• transformations, outliers, etc.

15
Stage 4 Main Analyses

• Break hypotheses down in terms of specific
  variables and level of measurement (interval,
  ordinal, nominal)
• Brainstorm list of possible analyses (decision
  trees in statistics books can be helpful)
• Develop analysis plan
• What needs to be done to prepare the data for
  this analysis?
• What do I need to learn?

16
Stage 5 Results Presentation

• Report some measure of effect size
• Statistical decisions are about having
  justifications for your decisions. Cite respected
  authors to back up statistical decisions.
• Tell a story that highlights what is interesting

17
Error checking

• Get to know your data
• Check for data entry errors
• Check that the variables are behaving as you
  would expect

18
Common Pitfalls

• Choosing analyses that are more complex than
  needed
• Too much time to master
• Not interpreting appropriately
• Violating statistical assumptions
• Not error checking data

19
Questions???