Educational Research & Statistics Research Application to Practice

1
Educational Research StatisticsResearch
Application to Practice

• Summer 2003
• Dr. Chiang

2
Major Components of the Course

• Understanding basic research principles and
  methods
• Becoming familiar with educational statistics
• Basic descriptive inferential statistics
• Using SPSS to analyze data

3
Assignments for the Course

• Completing the SPSS assignment
• www.uwosh.edu/faculty_staff/chiang/

4
The Status Quo of Educational Research

• Why do educators have so little regard for
  research?
• I have no time for research.
• research is not understandable irrelevant,
  inconclusive, contradictory
• children are not guinea pigs
• not in my job description
• any other reasons?

5
If not research, what else drive our
decision-making?

• political forces
• federal, state, and local agencies
• legislature and courts
• teacher unions
• Various advocacy groups
• publishers
• tests
• textbooks

6
-- continued

• fads or trends
• media influence
• teachers own or colleagues experience
• common sense
• any other factors?

7
Scientific Method

• deductive reasoning (from general to specific)
  Aristotles syllogisms
• inductive reasoning (from specific to general)
  F. Bacons field study
• deductive-inductive C. Darwin

8
Similarities between educational and scientific
research

• Describe
• Explain
• Predict
• Control

9
Differences between educational and scientific
research

• Complexity of subject matter
• Observability of subject matter
• Repeatability of subject matter

10
Four Scales of Measurement

• Nominal (3 not gt 2)
• Ordinal (3 gt 2, but 3-2 not 2-1)
• Interval (3-2 2-1, but 100/2 not 50)
• Ratio (100/2 50)

11
Basic vs. Applied Research

• theory development
• has long term value
• draw conclusions

• problem solving
• socially important (immediately)
• make recommendations

12
Descriptive Statistics

• Measures of central tendencies
• Mode
• Median
• Mean
• Measures of variance
• Range
• Standard deviation

13
Modern Scientific Methods

• Seven typical steps
• identification of problem
• definition of problem
• formulation of hypothesis
• development/selection of measure
• collection of data
• analysis of data
• draw conclusions

14
Where can you locate topics for research?

• any perplexing questions that you have
  encountered?
• ask your colleagues for such questions
• check over related journals/books/newspaper
  articles

15
Hypothesis

• start with a research question
• change the question into a null hypothesis
• contrast null hypothesis with alternative
  hypothesis
• compare a directional and a non-directional
  hypothesis
• a good hypothesis is concise testable

16
Examples of Research Questions

• Do women and men do equal amount of housework?
  (gender equity?)
• Definition of housework
• Data gathering procedure
• Are black drivers more likely to be ticketed for
  speeding than white drivers? (racial profiling?)
• Research method
• Validity threats
• Does hormone-replacement therapy (HRT) do more
  harm than good for women with PMS? (estrogen
  study, July 2002)
• Sample
• Statistics

17
Variables

• Independent (grouping) variable (IV)
• Dependent (test) variable (DV)
• IV precedes DV
• IV is to be experimentally manipulated
• DV is to be measured
• A hypothesis should contain only one IV and one
  DV

18
Review of literature

• Start with the most recent research
• Use Internet search engines
• Use interviews for expert comments
• Primary vs. secondary source
• Research vs. discussion articles
• Follow APA style
• Paraphrase vs. quotation

19
Using t-tests to compare the means

• Why is t-test considered an inferential
  statistics?
• When do we use independent t?
• To compare two groups that are mutually exclusive
  (e.g. experimental vs. control groups, males vs.
  females)
• When do we use dependent (correlated) t?
• To compare pretest vs. posttest (paired samples
  t)
• To compare two groups that have been matched in
  pairs (e.g. studies of identical twins)

20
Sampling

• purpose
• random sampling
• Each individual in the population has an equal
  independent chance to be selected.
• stratified sampling
• Either proportional or not proportional to
  population
• systematic sampling
• cluster sampling

21
When should we have larger N?

• For studies of significant consequence
• If the sample is very diversified
• Minute differences are expected
• For longitudinal studies
• If you are to have subgroup analyses
• Attrition of subjects are anticipated
• Test measures are unreliable
• Variables are complex and difficult to control

22
Survey research

• Why is it lowly regarded?
• How to improve return rate?
• Use captured audience
• Ensure confidentiality
• Keep it short
• Provide incentive
• What is the purpose of a pilot study?

23
Observational research

• Most useful in what situations?
• Study early childhood or infants
• Study social interactions
• Study behavior changes
• How to improve objectivity and reliability?
• Define behaviors very clearly
• Train observers
• Report inter-rater reliability

24
Recording of observational data

• event recording (frequency count)
• duration recording
• latency recording
• interval recording
• time sampling

25
Use correlation coefficient (r) to indicate
degree of relationship

• Correlation vs. causation
• Correlation coefficient ranges from 0 to 1 or 1
• Scattergram is a graph showing the spread of data
  points (or line of best fit) between variables X
  Y.
• Correlation matrix displays correlation
  coefficients among several variables
• Interpretation of r
• Pearson product-moment correlation (interval
  scale)
• Spearmen rank-difference correlation (ordinal
  scale)

26
Ethics in Educational Research

• Informed consent
• Use of human subjects
• Withdrawal from participation
• Counterbalance treatments
• Double blind procedures and use of placebo
• Report not revealing recognizable identity
• Citing sources to give credits

27
Use chi-square to analyze survey results

• Chi-square is a non-parametric test (n gt 5 per
  cell)
• The data are in frequency counts
• Compare observed frequencies vs. expected
  frequencies
• Arrange data in 2 X 2, 2 X 3 contingency tables
• Use Crosstab function in the SPSS program to
  obtain crosstabulations and chi squares

28
Research Planning A Simulation

• Why Johnny cant sleep?
• Ferberizing a baby (vs. fertilizing the plots)?
• Researchers bias
• what variables have to be controlled?
• What data to collect?
• Did you agree on one study or have ideas for
  several studies?

29
Experiment vs. Investigation

• What constitutes an experiment?
• Manipulation of the independent variable
• Its purpose is to demonstrate a functional or
  cause-and-effect relation between variables
• Investigation involves analysis of data (without
  manipulation)
• Ex post facto design
• Cross sectional vs. longitudinal studies

30
Fundamental Principles of Experiment

• Assume equivalence between the two groups, often
  by random assignment of subjects
• Maintain treatment fidelity by following
  treatment scripts
• Control extraneous variables
• Have a reasonable experimental duration (e.g. a
  quarter)
• Minimizes Pygmalion, Hawthorne, halo effects

31
Internal vs. External Validity

• Which is absolutely essential? Why?
• Relation between internal external validity is
• Sequential (internal validity first, external
  next)
• Reciprocal (too much internal validity will
  result in little or no external validity)

32
Threats to Internal Validity

• history
• maturation
• testing
• instrumentation
• regression towards the mean
• differential selection
• experimental mortality or attrition
• interactions of the above factors

33
Threats to External Validity

• Reactive or interaction effects of pre-testing
• Interaction effects of selection bias IV
• Reactive effects of experimental arrangements
• Multiple treatment interference

34
Research Design

• Pre-experimental designs
• One shot case study
• One-group pretest-posttest design
• Static group comparison
• True experimental designs
• Pretest-posttest control group design
• Posttest only control group design
• Solomon four group design
• Quasi experimental design
• Non-equivalent control group design

35
Single Subject Research

• Compare single subject and group research
• Reversal or ABAB design
• Baseline pattern
• Ethical concerns of reversal
• Irreversible behaviors
• Multiple baseline design
• Across different behaviors
• Across different subjects
• Across different settings

36
Review

• What does significant level of .05 .01 and .001
  mean?
• Can you defend the use of null hypothesis?
• Correlation is not causation. Give an example or
  two to explain why.
• What is a scattergram or scatterplot for?
• Explain what crosstabulation or crosstab is
  about?
• How is bibliography different from reference?
• Differentiate random sampling from random
  assignment.

37
Review 2

• Give examples of dependent vs. independent
  variables
• Explain internal and external validity.
• Can single subject designs demonstrate functional
  relations?
• Are survey studies typically experimental or
  investigation?
• How is time sampling different from interval
  recording?
• Is -.70 possible for an r? Is it also possible
  for a t test score? How about a chi-square test
  score?
• Is degree of freedom related to homogeneity of
  subjects?

38
Review 3

• Statistical concepts
• Median versus mean
• Inferential statistics
• Critical value
• Two-tailed test
• Inverse correlation
• Non-parametric test (continuous data vs.
  categories)
• r and r square
• Ordinal vs. Interval scale
• Making conclusion statement after hypothesis
  testing
• Hawthorne versus Pygmalion effect
• Ex post fact research design
• Cross-sectional versus longitudinal design