The Theory of Sampling and Measurement - PowerPoint PPT Presentation

1 / 58
About This Presentation
Title:

The Theory of Sampling and Measurement

Description:

The Theory of Sampling and Measurement Sampling First step in implementing any research design is to create a sample. We cannot study the theoretical population of ... – PowerPoint PPT presentation

Number of Views:357
Avg rating:3.0/5.0
Slides: 59
Provided by: Davi1782
Category:

less

Transcript and Presenter's Notes

Title: The Theory of Sampling and Measurement


1
The Theory of Sampling and Measurement
2
Sampling
  • First step in implementing any research design is
    to create a sample.
  • We cannot study the theoretical population of all
    conceivable events (e.g., events that have not
    occurred), nor can we usually study all instances
    of actual events. We select some instances to
    study and not others. Those we include are our
    sample.
  • How our sample is selected is critical for
    external validity or generalizability.

3
Groups in Sampling
Who do you want to generalize to?
4
Groups in Sampling
The theoretical population
5
Groups in Sampling
The theoretical population
What population can you get access to?
6
Groups in Sampling
The Theoretical Population
The study population
7
Groups in Sampling
The theoretical population
The study population
How can you get access to them?
8
Groups in Sampling
The theoretical population
The study population
The sampling frame
9
Groups in Sampling
The theoretical population
The study population
The sampling frame
Who is in your study?
10
Groups in Sampling
The theoretical population
The study population
The sampling frame
The sample
11
Types of Samples
  • Probability Sampling
  • Simple random
  • Stratified random
  • Cluster or area random
  • Non-Probability Sampling
  • Accidental
  • Modal instance
  • Expert
  • Snowball
  • Case study (intentional selection)

12
The Sampling Distribution
Average
Average
Average
...is the distribution of a statistic across an
infinite number of samples.
The sampling distribution...
13
Population Parameter
The population has a mean of 3.75...
1
5
0
...and a standard unit of .25.
1
0
0
Frequency
5
0
0
4
.
5
4
.
0
3
.
5
3
.
0
This means
Self esteem
About 64 of cases fall between 3.5 - 4.0.
About 95 of cases fall between 3.25 - 4.25.
about 99 of cases fall between 3.0 - 4.5
14
Sampling Distribution
The population has a mean of 3.75.
1
5
0
1
0
0
Frequency
5
0
0
4
.
5
4
.
0
3
.
5
3
.
0
Self-esteem
15
Sampling Distribution
The population has a mean of 3.75...
1
5
0
...and a standard error of .25.
1
0
0
Frequency
5
0
0
4
.
5
4
.
0
3
.
5
3
.
0
Self-esteem
16
Inferring Population from Sample
The sample has a mean of 3.75...
1
5
0
...and a standard deviation of .25.
1
0
0
Frequency
5
0
0
4
.
5
4
.
0
3
.
5
3
.
0
This means
Self esteem
64 chance true population mean falls between 3.5
- 4.0.
95 chance true population mean falls between
3.25 - 4.25.
99 chance true population mean falls between 3.0
- 4.5
17
Figure 3.4 Labor Repression and Growth in the
Asian Cases, 1970-1981

18
Figure 3.5 Labor Repression and Growth in the
Full Universe of Developing Countries,1970-1981
19
Measurement
  • Operationalization is the process of translating
    theoretical constructs into observable
    indicators.
  • Construct validity and reliability are the
    criteria we use to evaluate how well you have
    operationalized your concepts.
  • Both matter regardless of the level of
    measurement and whether you are using qualitative
    or quantitative indicators.

20
The Hierarchy of Levels
Ratio
Absolute zero
Interval
Distance is meaningful
Ordinal
Attributes can be ordered
Nominal
Attributes are only named weakest
21
Nominal Measurement
  • The values name the attribute uniquely.
  • The name does not imply any ordering of the cases.

22
Ordinal Measurement
  • When attributes can be rank-ordered
  • Distances between attributes do not have any
    meaning.

23
Interval Measurement
  • When distance between attributes has meaning, for
    example, temperature (in Fahrenheit) -- distance
    from 30-40F is same as distance from 70-80F
  • Note that ratios dont make any sense -- 80F is
    not twice as hot as 40F.

24
Ratio Measurement
  • Has an absolute zero that is meaningful
  • Can construct a meaningful ratio (fraction), for
    example, number of clients in past six months

25
Construct Validity
  • Key problem is that we have abstract theoretical
    construct power, democracy, development,
    corruption, etc. that we can never observe
    directly.
  • Yet, to test propositions requires that we have
    some indicator for the construct or at least
    have proxies that we can argue are capturing some
    attributes of the construct.
  • Our indicator is an analogy (to an analogy).

26
Assessing Construct Validity
  • Translation Validity
  • Face Validity plausible on its face
  • Content Validity matches lists of attributes
  • Criterion-related Validity
  • Predictive Validity predicts accurately
  • Concurrent Validity distinguishes appropriately
    between groups
  • Convergent Validity
  • Discriminant Validity

27
The Convergent Principle
  • Alternative measures of a construct should be
    strongly correlated.

28
How It Works
Theory
You theorize that the items all reflect
self-esteem.
29
How It Works
Theory
1.00 .83 .89 .91 .83 1.00 .85 .90 .89 .85 1.00
.86 .91 .90 .86 1.00
The correlations provide evidence that the items
all converge on the same construct.
Observation
30
Convergent Validity in Measures of Democracy
  • 1985 polity2 pollib civlib
    reg
  • -------------------------------------------------
  • polity2 1.0000 -0.9148 -0.8770
    -0.8601
  • pollib -0.9148 1.0000 0.9176
    0.8440
  • civlib -0.8770 0.9176 1.0000
    0.8053
  • reg -0.8601 0.8440 0.8053
    1.0000

31
Convergent Validity in Measures of Education
  • 1985 1 2 3 4 5 6
  • -------------------------------------------------
    ------------------
  • Ed. spending 1.0000 -0.1217 0.2415
    0.3563 0.0214 0.0195
  • Illiteracy () -0.1217 1.0000 -0.5797
    -0.7306 -0.8569 -0.6196
  • Cohort to Grade 4 0.2415 -0.5797 1.0000
    0.4419 0.6553 0.3654
  • Grade School 0.3563 -0.7306 0.4419
    1.0000 0.6230 0.3612
  • Secondary School 0.0214 -0.8569 0.6553
    0.6230 1.0000 0.7576
  • College 0.0195 -0.6196 0.3654
    0.3612 0.7576 1.0000

32
The Discriminant Principle
  • Measures of different constructs should not
    correlate highly with each other.

33
How It Works
Theory
Locus-of-control construct
LOC1
LOC2
34
How It Works
You theorize that you have two distinguishable
constructs.
Theory
Locus-of-control construct
LOC1
LOC2
35
How It Works
Theory
Locus-of-control construct
LOC1
LOC2
rSE1, LOC1 .12
The correlations provide evidence that the items
on the two tests discriminate.
rSE1, LOC2 .09
rSE2, LOC1 .04
rSE2, LOC2 .11
Observation
36
We have two constructs. We want to measure
self-esteem and locus of control.
Theory
Self-esteem construct
Locus-of-control construct
SE1
SE2
SE3
LOC1
LOC2
LOC3
For each construct, we develop three scale items
our theory is that items within the construct
will converge and Items across constructs will
discriminate.
37
Theory
Self-esteem Construct
Locus-of-control construct
Green and red correlations are Convergent yellow
are Discriminant.
SE1
SE2
SE3
LOC1
LOC2
LOC3
Observation
38
Theory
Self-esteem construct
Locus-of-control construct
SE1
SE2
SE3
LOC1
LOC2
LOC3
The correlations support both convergence and
discrimination, and therefore construct validity.
Observation
39
What Is Reliability?
  • The repeatability of a measure
  • The consistency of a measure
  • The dependability of a measure

40
True Score Theory
Observed score
True ability
Random error


e
X

T
41
The Error Component
e
X

T
Two components
er
  • Random error

es
  • Systematic error

42
The Revised True Score Model
er
X
es


T
43
Random Error
Frequency
The distribution of X with no random error
X
44
Random Error
The distribution of X with random error
Frequency
The distribution of X with no random error
Notice that random error doesnt affect the
average, only the variability around the average.
X
45
Systematic Error
Frequency
The distribution of X with no systematic error
X
46
Systematic Error
The distribution of X with systematic error
Frequency
The distribution of X with no systematic error
Notice that systematic error does affect the
average we call this a bias.
X
47
If a Measure Is Reliable...
We should see that a persons score on the same
test given twice is similar (assuming the trait
being measured isnt changing).
X1
X2
48
If a Measure Is Reliable...
But, if the scores are similar, why are they
similar?
X1
X2
Recall from true score theory that...
T e1
T e2
49
If a Measure Is Reliable...
The only thing common to the two measures is the
true score, T. Therefore, the true score must
determine the reliability.
X1
X2
T e1
T e2
50
Reliability Is...
a ratio
variance of the true scores
variance of the measure
var(T)
var(X)
51
Reliability Is...
a ratio
variance of the true scores
variance of the measure
We can measure the variance of the observed
score, X. The greater the variance, the less
reliable the measure.
52
This Leads Us to...
  • We cannot calculate reliability exactly we can
    only estimate it.
  • Each estimate attempts to capture the
    consequences of the true score in different ways.

53
We want both Reliability and Validity
54
Reliability and Validity
Reliable but not valid
55
Reliability and Validity
Valid but not reliable
56
Reliability and Validity
Neither reliable nor valid
57
Reliability and Validity
Reliable and valid
58
Assignment 1
  • Assess the validity and reliability of the IRIS-3
    International Country Risk Guide.
  • Can examine a single instance, compare instances,
    analyze the full variation in the dataset,
    compare with additional measures, or use any
    other form of assessment. May use outside sources
    of data, history, or analysis (but document).
  • The only restriction is that the paper must be
    empirical and examine issues of validity and
    reliability.
  • 3-5 pages. Be concise.
  • Due Monday 10/24 at beginning of class.
Write a Comment
User Comments (0)
About PowerShow.com