Session 7 Introduction to Research and Evaluation - PowerPoint PPT Presentation

1 / 120
About This Presentation
Title:

Session 7 Introduction to Research and Evaluation

Description:

Session 7 Introduction to Research and Evaluation Topic 1: Research Questions and Hypothesis Testing And Topic 2: Introduction to Statistics For Tonight Today Review ... – PowerPoint PPT presentation

Number of Views:264
Avg rating:3.0/5.0
Slides: 121
Provided by: Dea56
Category:

less

Transcript and Presenter's Notes

Title: Session 7 Introduction to Research and Evaluation


1
Session 7 Introduction to Research and
Evaluation
  • Topic 1 Research Questions and Hypothesis
    Testing
  • And
  • Topic 2 Introduction to Statistics

2
For Tonight
3
Today
  • Review the contents of the proposal
  • Topics tonight
  • Finish the research questions
  • Types of data review
  • Hypothesis Testing
  • Intro to Stats

4
The phases of a research project
  • Problem statement
  • Purpose
  • Hypothesis development / research question(s)
  • Population / Sample type
  • Results reporting (data)
  • Statistical testing
  • Conclusions Recommendations

5
Parts of the Research Report
  • Chapter 1
  • Chapter 2
  • Chapter 3
  • Chapter 4
  • Chapter 5
  • References
  • Appendix

6
Components of Chapter 1
  • Introduction
  • Background of the study
  • Problem statement
  • Significance of study
  • Overview of methodology
  • Delimitations of study
  • Definitions of key terms
  • Conclusion (optional)

7
Characteristics of Components in Chapter 1
  • Introduction 1 paragraph 3 pages
  • Gets attention - gradually
  • Brief vs. reflective opening
  • Background 2-5 pages
  • History of problem, etc.
  • Professional vs. practical use
  • Be careful of personal intrusions

8
Characteristics of Components in Chapter 1
  • Problem Statement ½ page
  • States problem as clearly as possible
  • Significance of study 1 pgh. to 1 page
  • Answers Why did you bother to conduct the
    study?
  • Be careful of promising too much

9
Ways to Convey Significance
  • Problem has intrinsic importance, affecting
    organizations or people
  • Previous studies have produced mixed results
  • Your study examines problem in different setting
  • Meaningful results can be used by practitioners
  • Unique population
  • Different methods used

10
Characteristics of Components
  • Delimitations as needed
  • Not flaws
  • Establishes the boundaries can study be
    generalized?
  • Consider
  • sample
  • Setting
  • time period
  • methods

11
Stating the Problem
  • Developing a hypothesis
  • Methods estimation and hypothesis testing.
  • Estimation, the sample is used to estimate a
    parameter and a confidence interval about the
    estimate is constructed.
  • Parameter numerical quantity measuring some
    aspect
  • Confidence Interval range of values that
    estimates a parameter for a high proportion of
    the time
  • Hypothesis Testing the most common use
  • Hypothesis an intelligent guess or assumption
    that guides the design of the study
  • Null hypothesis there is no difference or there
    is no effect
  • Alternative hypothesis there is a difference or
    there is an effect
  • Hypotheses more than hypothesis, which are
    related to the population

12
TYPES OF DATA
13
Variables
  • Two categories
  • Independent
  • Variables in an experiment or study which are not
    easily to be manipulated without changing the
    participants.
  • Age, gender, year, classroom teacher, any
    personal background data, etc
  • Dependent
  • Variables which are changed in an experiment
  • Hours of sleep, amount of food, time given to
    complete an activity, curriculum, instructional
    method, etc.

14
(No Transcript)
15
Variables
  • A variable any measured characteristic or
    attribute that differs for different subjects.
  • Two types
  • Quantitative sometimes called "categorical
    variables.
  • measured on one of three scales
  • Ordinal first second or third choice (most of
    the children preferred red popsicles, and grape
    was the second choice)
  • Interval direct time periods between two events
    ( time it takes a child to respond to a question)
  • Ratio scale compares the number of times one
    event happens in comparison to another event.
    (example the number of time a black card is
    pulled in comparison to the number of times a red
    card is pulled)
  • Qualitative
  • measured on a nominal scale.

16
Types of Data
  • Nominal Data  -- Data that describe the presence
    or absence of some characteristic or attribute
    data that name a characteristic without any
    regard to the value of the characteristic also
    referred to as categorical data. Male 1 Female
    2, blue, green, etc
  • Ordinal Data -- Measurement based on the rank
    order of concepts or variables differences among
    ranks need not be equal.
  • interval data -- Measurement based on numerical
    scores or values in which the distance between
    any two adjacent, or contiguous, data points is
    equal scale without a meaningful or true zero
  • Ratio Data  -- Order and magnitude. Measurement
    for which intervals between data points are
    equal a true zero exists if the score is zero,
    there is a complete absence of the variable.

17
  • Four levels
  • nominal assigning items to groups or categories
  • Examples Classroom, color, size
  • Ordinal ordered in the sense that higher numbers
    represent higher values
  • Examples 1 freshmen, 2 sophomore
  • Interval one unit on the scale represents the
    same magnitude on the trait or characteristic
    being measured across the whole range of the
    scale.
  • Interval scales do not have a "true" zero point,
  • it is not possible to make statements about how
    many times higher one score is than another.
  • Ratio represents the same magnitude on the trait
    or characteristic being measured across the whole
    range of the scale.
  • DO have true zero points

18
Nominal level of measurement
  • Assigns a number to represent a group (gender
    geography)
  • Numbers represent qualitative differences
    (good-bad)
  • No order to numbers
  • Statistics -- mode, percentages, chi-square

19
Ordinal level of measurement
  • Things are rank-ordered -- gt, lt
  • Numbers are not assigned arbitrarily
  • Assume a continuum
  • Examples -- classification (fr, soph, jr, sr),
    levels of education, Likert scales
  • Statistics--median (preferred), mode, percentage,
    percentile rank, chi-square, rank correlation.

20
Interval level of measurement
  • Equal units of measurement
  • Arbitrary zero point--does not indicate absence
    of the property
  • Example -- degrees, Likert-type scales
    (treatment), numerical grades
  • Statistics -- frequencies, percentages, mode,
    mean, SD, t test, F test, product moment
    correlation

21
Ratio level of measurement
  • Absolute zero
  • Interval scale
  • Examples -- distance, weight
  • Statistics -- all statistical determinations

22
Which are these?
  • Never married
  • Lower middle Class
  • Divorced
  • Age
  • Separated
  • Middle class
  • Widowed
  • Weight
  • Religious Affiliations
  • Height
  • Political Affiliations
  • Distance
  • freshmen

23
Which are these?
  • Never married
  • Lower middle Class
  • Divorced
  • Age
  • Separated
  • Middle class
  • Widowed
  • Weight
  • Religious Affiliations
  • Height
  • Political Affiliations
  • Distance
  • freshmen
  • Minutes

N
I/R
O
N
N
I/R
O
I/R
N
I/R
O
N
I/R
N
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
Key Point
  • Statistical Significance must be distinguished
    from practical significance
  • Even a small difference in a large sample might
    be significant if the sample is large
  • No p-value of a .0001 means that 1 in 10000 times
    the difference observed will occur by chance (no
    real difference between groups)

31
Example Hypothesis
  • There will be no significant difference in the
    EOC scores for schools that use CAERT and those
    that dont.
  • The EOC exam scores for schools using Caert and
    those that dont will not be significantly
    different.
  • The EOC exam scores for schools using Caert and
    those that dont will be significantly different.

32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
Statistics for Teachers
37
StatisticsIf you can assign a number to it,
you can measure it Dr. W. Edward Demming
  • Statistics
  • refers to calculated quantities regardless of
    whether or not they are from a sample
  • is defined as a numerical quantity
  • Often used incorrectly to refer to a range of
    techniques and procedures for analyzing data,
    interpreting data, displaying data, and making
    decisions based on data. Because that is the
    basic learning outcomes of a statistics course.

38
(No Transcript)
39
(No Transcript)
40
What is the mean medium and the mode in this
example?
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
Descriptive statistics
  • Descriptive statistics
  • summarize a collection of data in a clear and
    understandable way.
  • Example Scores of 500 children on all parts of a
    standardized test.
  • Methods numerical and graphical.
  • Numerical more precise- uses numbers as accurate
    measure
  • mean the arithmetic average which is calculated
    by adding a the scores or totals and then
    dividing by the number of scores.
  • standard deviation. These statistics convey
    information about the average degree of shyness
    and the degree to which people differ in shyness.
  • Graphical better for identifying patterns
  • stem and leaf display a graphical method of
    displaying data to show how several data are
    aligned on a graph
  • box plot. Graphical method to show what data are
    included. The box stretches from the 25th
    percentile to the the 75th percentile
  • historgrams.
  • Since the numerical and graphical approaches
    compliment each other, it is wise to use both.

46
Inferential statistics
47
For choosing a statistical test variables fall
into 2 groups
  • Continuous variables are numeric values that can
    be ordered sequentially, and that do not
    naturally fall into discrete ranges.
  • Examples include  weight, number of seconds it
    takes to perform a task, number of words on a
    user interface
  • Categorical variable values cannot be
    sequentially ordered or differentiated from each
    other using a mathematical method.
  • Examples include  gender, ethnicity, software
    user interfaces

48
Tools for Measuring
  • Measurement is the assignment of numbers to
    objects or events in a systematic fashion.
  • Four levels
  • nominal assigning items to groups or categories
  • Examples Classroom, color, size
  • Ordinal ordered in the sense that higher numbers
    represent higher values
  • Examples 1 freshmen, 2 sophomore
  • Interval one unit on the scale represents the
    same magnitude on the trait or characteristic
    being measured across the whole range of the
    scale.
  • Interval scales do not have a "true" zero point,
  • it is not possible to make statements about how
    many times higher one score is than another.
  • Ratio represents the same magnitude on the trait
    or characteristic being measured across the whole
    range of the scale.
  • DO have true zero points

49
Data Analysis
  • Explaining and interpreting the data
  • Data are plural
  • You are looking at more than one number or group
    of numbers subject-verb agreement is important
    when writing.
  • Central Tendency measures of the location of the
    middle or the center of the whole data base for a
    variable or group of variables
  • Frequency the number of times a number appears
  • Mean the arithmetic average
  • Mode the number that appears most often
  • Median the number in the middle when numbers are
    arranged by value
  • Skew A distribution is skewed if one of its
    tails is longer than the other. Data may be
    skewed positively or negatively.
  • Standard deviation the amount of variance
    between each sigma

50
Inferential statistics
  • Inferential statistics
  • Infers or implies something about population from
    a sample.
  • Population A total group
  • Sample A few from the whole group
  • Representative sample a sample that is equally
    propionate to the population
  • Random Sample a sample that is chosen strictly
    by chance is not hand-picked
  • Probability the percentage of change that an
    event will occur

51
Parameters vs Statistics Parametric vs
Non-Parametric
  • Definitions again
  • Parameter is the true value in the population of
    interest (everyone)
  • Statistics is a number you calculate from your
    sample data in order to estimate the parameter
  • Example
  • All the Ag Teachers of the state
  • Only 25 teachers selected from the 285 that exist

52
What can make the sample different from the true
value/result of the whole?
  • Students taught by teachers using Caert will
    score higher on end of course exams than those
    who do not.
  • True difference one group actually has a higher
    capacity to learn.
  • Random Variations -- The two populations have
    identical means and the observed differences is a
    coincidence of sampling
  • Sampling error (bias) Poorly selected samples
    not representing the population.

53
Parameters or Parametric Data
  • Parameter a numerical quantity measuring some
    aspect of a population of scores.
  • Parameters are usually estimated by statistics
    computed in samples
  • Quantity Parameter Greek letters are commonly
    accepted for writing formulas
  • Statistical symbols are most common in reporting
    actual data analysis in reports or articles.

Greek letters are used to designate parameters
Quantity Parameter Statistic
Mean µ M
Standard deviation s s
Proportion p p
Correlation ? r
54
Stats tests types of data each use
  • 1 Sample t-test         1 Continuous Dependent
    Variable with normal distribution         0
    Independent Variables  
  • 1 Sample Median         1 Continuous Dependent
    Variable with non-normal distribution         0
    Independent Variables 
  • Binomial test         1 Bi-level Categorical
    Dependent Variable         0 Independent
    Variables 
  • Chi-Square Goodness of Fit      1 Categorical
    Dependent Variable         0 Independent
    Variables         
  • 2 Independent Sample t-test     1 Continuous
    Dependent Variable with normal distribution
  • 1 (2 level) Categorical Independent Variable
  •  Wilcoxon Signed Ranks Test     1 Continuous
    Dependent Variable with non-normal distribution
      
  • 1 (2 level) Categorical Independent Variable
  • Chi Square Test         1 Categorical
    Dependent Variable         1 (2-level)
    Categorical Independent Variable
  • Fisher Exact Test         1 Categorical
    Dependent Variable         1 (2 level)
    Categorical Independent Variable
  • Paired t-test     1 Continuous Dependent Variable
    with normal distribution,  1 (2 Level)
    Categorical Independent Variable
  • One-way repeated measures ANOVA 1 Continuous
    Dependent Var w/normal distribution
  • 1 (Multi-Level) Categorical Independent Variable
  • Friedman Analysis of Variance by Ranks 1
    Continuous Dependent Var w/ non-normal
    distribution
  • 1 (Multi-Level) Categorical Independent Variable
  • One-way ANOVA         1 Continuous Dependent
    Variable with normal distribution
  • 1 (Multi-level) Categorical Independent Variable
  • Kruskal Wallis        1 Continuous Dependent
    Variable with non-normal distribution
  • 1 (Multi-level) Categorical Independent Variable 

55
Results
56
Results
  • At the end of the trial experience schools using
    Caert had EOC exam scores that were 18 that
    were higher than those schools that did not use
    Caert.
  • Alpha set at plt.05
  • Observed P value of .03

57
Conclusion
  • Interpretation Given that there is no true
    (other than scores) difference between schools
    using Caert and those that dont, the probability
    of observing a 3 (.03) or more difference due to
    chance is less than .05

58
ANOVA
  • A factorial ANOVA has two or more categorical
    independent variables (either with or without the
    interactions) and a single normally distributed
    interval dependent variable. 

59
ANOVA
  • In statistics, ANOVA is short for analysis of
    variance. Analysis of variance is a collection of
    statistical models, and their associated
    procedures, in which the observed variance is
    partitioned into components due to different
    explanatory variables.
  • The initial techniques of the analysis of
    variance were developed by the statistician and
    geneticist R. A. Fisher in the 1920s and 1930s,
    and is sometimes known as Fisher's ANOVA or
    Fisher's analysis of variance, due to the use of
    Fisher's F-distribution as part of the test of
    statistical significance.

60
Z test
  • The Z-test is a statistical test used in
    inference which determines if the difference
    between a sample mean and the population mean is
    large enough to be statistically significant,
    that is, if it is unlikely to have occurred by
    chance.
  • The Z-test is used primarily with standardized
    testing to determine if the test scores of a
    particular sample of test takers are within or
    outside of the standard performance of test
    takers.

61
Pearson Correlation
  • The PEARSON Correlation is a calculation between
    the correlation coefficient between two
    measurement variables when measurements on each
    variable are observed for each of N subjects.
  • (Any missing observation for any subject causes
    that subject to be ignored in the analysis.) The
    Correlation analysis tool is particularly useful
    when there are more than two measurement
    variables for each subject. It provides an output
    table, a correlation matrix, showing the value
    applied to each possible pair of measurement
    variables.

62
Two-Sample t-Test
  • The Two-Sample t-Test analysis tools test for
    equality of the population means underlying each
    sample. The three tools employ different
    assumptions that the population variances are
    equal, that the population variances are not
    equal, and that the two samples represent before
    treatment and after treatment observations on the
    same subjects.

63
(No Transcript)
64
Research Techniques
  • Types of hypothesis testing
  • T-test comparing the mean of two groups
  • ANOVA Analysis of Variance used to compare the
    means of several variables
  • Correlation compares the relationship of two
    groups
  • Chi Square of independence explains if is a
    relationship between the attributes of two
    variables.
  • Linear regression the prediction of one variable
    based on another variable, when the relationship
    between the variables is assumed to assumed to be
    linear.

65
Normal Curve
In practice, one often assumes that data are from
an approximately normally distributed population.
If that assumption is justified, then about 68
of the values are at within 1 standard deviation
away from the mean, about 95 of the values are
within two standard deviations and about 99.7
lie within 3 standard deviations. This is known
as the "68-95-99.7 rule" or the "Empirical Rule".
66
Key points
67
(No Transcript)
68
(No Transcript)
69
(No Transcript)
70
(No Transcript)
71
(No Transcript)
72
(No Transcript)
73
(No Transcript)
74
(No Transcript)
75
(No Transcript)
76
(No Transcript)
77
(No Transcript)
78
(No Transcript)
79
(No Transcript)
80
(No Transcript)
81
(No Transcript)
82
(No Transcript)
83
(No Transcript)
84
(No Transcript)
85
(No Transcript)
86
(No Transcript)
87
(No Transcript)
88
(No Transcript)
89
(No Transcript)
90
(No Transcript)
91
(No Transcript)
92
(No Transcript)
93
(No Transcript)
94
(No Transcript)
95
(No Transcript)
96
(No Transcript)
97
Comparing groups for Sig Diff
98
Key Terms
  • Use for new terms in profession (cognitive
    processing skills)
  • Give preciseness to ambiguous term (learner)
  • General term used in special way (learning style)
  • Writing definition
  • State term
  • Give broad class to which term belongs
  • Specify how term is used that differs
  • Conclusion not always used
  • Summarizes if necessary
  • Tells reader what to expect

99
Survey Construction
  • Parts
  • Title
  • Directions introduction to survey
  • Scales
  • Items (a list of statements or questions)
  • Usually with a scale of some type
  • Rating
  • Ranking
  • Semantic differential
  • Likert type scale
  • Demographical info

100
Likert type scale
  • Ice cream is good for breakfast
  • Strongly disagree
  • Disagree
  • Neither agree nor disagree
  • Agree
  • Strongly agree

101
Rating
  • Scale of 1 to 5 or 1 to 7 , etc.
  • ? 1 Best or highest
  • ? 5 Best or highest
  • Even number of items or odd?
  • Forced choice no fence sitting
  • Middle allows a middle ground response
  • Might allow for not opinion, (NA or NO)

102
Semantic differential
103
(No Transcript)
104
(No Transcript)
105
  • In order to succeed you must know what you are
    doing, like what you are doing, and believe in
    what you are doing
  • Will
    Rogers

106
(No Transcript)
107
(No Transcript)
108
(No Transcript)
109
(No Transcript)
110
(No Transcript)
111
(No Transcript)
112
(No Transcript)
113
(No Transcript)
114
Setting Alpha Level
  • Set alpha at something like 0.05
  • Conduct a statistical test
  • Obtain a p-value

115
(No Transcript)
116
(No Transcript)
117
(No Transcript)
118
(No Transcript)
119
(No Transcript)
120
  • Parametric tests
  • Pearson Product Correlation Coefficient
  • Student t-Test
  • The z-Test
  • ANOVA
  • Nonparametric tests
  • Chi-Squared
  • Spearman Rank Coefficient
  • Mann-Whitney U Test
  • Kruskal-Wallis Test
Write a Comment
User Comments (0)
About PowerShow.com