Educational Research

1
Educational Research

• Chapter 13
• Post-Analysis Considerations
• Gay, Mills, and Airasian
• 8th Edition (Electronic Reserves)

2
Topics Discussed in this Chapter

• Verification and storage of data
• Checking, organizing, and evaluating data
• Interpretation of results
• Hypothesized and unhypothesized results
• Statistical issues
• Methodological issues
• Statistical and practical significance
• Replication

3
Verifying and Storing Data

• Three issues
• Checking data
• Organizing data
• Evaluating the research conclusions
• Checking data for accuracy
• Check the original data
• Check the coded data against the initial uncoded
  data
• Electronic data
• Print the data sets and check them
• Check the output carefully

Objective 1.1
4
Verifying and Storing Data

• Organizing data
• Label the data
• Dates
• Important details related to the data itself and
  the study
• Store the data
• Multiple copies
• Multiple sites

Objective 1.2
5
Verifying and Storing Data

• Evaluate the research conclusion(s)
• Check the accuracy of the computations
• Check the reasonableness of the results

Objective 1.3
6
Interpreting the Results

• Interpretations are made in light of
• the purpose of the study,
• the original research hypotheses, and
• the findings of other related studies.
• Hypothesized results
• Results are interpreted relative to their support
  or lack of support of the research hypotheses

Objectives 2.1 3.1
7
Interpreting Results

• Hypothesized results (continued)
• Supported hypotheses
• Probabilistic nature of the decision
• Easily interpreted given the control and
  manipulation used in the research design
• Non-supported hypotheses
• Probabilistic nature of the decision
• Possible importance of knowing what doesnt work
• Potential methodological problems

Objectives 2.1 3.1
8
Interpreting Results

• Unhypothesized results
• Results that appear during the study
• Must be interpreted with great care
• Researchers base their studies on ideas related
  to what will happen, not what is happening
• Possible importance to and influence on later
  studies

Objective 4.1 4.2
9
Interpreting Results

• Statistical influences
• The sample must be representative of the
  population
• The assumptions of the specific statistical tests
  of significance must be met

Objective 5.1
10
Interpreting Results

• Three methodological influences
• Measurement error large amounts of error hamper
  the ability to find statistically significant
  results
• Statistical power
• The probability of avoiding a Type II error
  (failure to reject a false null hypothesis
  something is there but you say there isnt)

Objectives 5.2 6.1
11
Interpreting Results

• Three methodological factors (cont.)
• Statistical power (cont.)
• Three influential factors
• Sample size the larger the sample the greater
  the power
• Significance levels
• Increasing alpha (e.g., .01, .05, .10) increases
  power
• If you say that there is a 5 chance that the
  differences are due to chance (null) then you are
  more likely to accept a false null hypothesis
  than if you set the value at a 10 chance.

Objectives 5.2 6.2
12
Interpreting Results

• Three methodological factors (cont.)
• Statistical power (cont.)
• Effect size
• Larger effect sizes increase power
• Effect sizes can be estimated from prior studies,
  estimated by minimal levels below which the
  results are deemed unimportant, and agreed upon
  levels of small (.20), moderate (.50), and large
  (.80).

Objectives 5.2 6.2
13
Interpreting Results

• Statistical and practical significance
• Statistical significance relates to the tests of
  significance being used
• Practical significance relates to the usefulness
  of the results
• Statistical and practical significance are not
  necessarily the same
• Statistical significance depends on sample size
• Studies with large samples can be significant but
  of little value

Objective 5.3
14
Interpreting Results

• Replication
• Replication involves repeating the study
• Different subjects
• Different context
• Different times
• Extremely important when
• an unusual or new result is found, or
• the results have high levels of practical
  significance.

Objective 7.1