Sampling%20(cont.)%20Instrumentation

1
Sampling (cont.)Instrumentation

• Measurement Plan Due 3/7

2
Sampling Demonstration
3
In Small Groups

• Check each others literature review and
  hypothesis of choice.
• Do literature review, rationale, and hypothesis
  go together?
• Offer constructive criticism-suggestions.
• Then,
• Identify your
• Target population
• Accessible population
• Sampling strategy
• Strengths
• Weaknesses

4
Instrumentation
5
Discuss Jared Diamond

• Soft Sciences are Often Harder than Hard Sciences

6
Instrumentation
Instructions Circle the choice that indicates
your opinion. 1. Teachers unions should be
abolished. Strongly Strongly agree Agree Unde
cided Disagree disagree (5) (4) (3) (2) (1) 2.
School administrators should be required by law
to teach at least one class in a public school
classroom every year. Strongly Strongly agree
Agree Undecided Disagree disagree (5) (4) (3) (2
) (1) 3. Classroom teachers should be able to
choose the administrators in their
schools. Strongly Strongly agree Agree Undeci
ded Disagree disagree (5) (4) (3) (2) (1)
7
What are Data?

• Data refers to the information researchers obtain
  on the subjects of their research.
• Demographic information or scores from a test are
  examples of data collected.
• The researcher has to determine what kind of data
  they need to collect.
• The device the researcher uses to collect data is
  called an instrument.

8
Key Questions

• The instruments and procedures used in collecting
  data is called instrumentation.
• Questions arise regarding the procedures and
  conditions under which the instruments will be
  administered
• Where will the data be collected?
• When will the data be collected?
• How often are the data to be collected?
• Who is to collect the data?
• The most highly regarded types of instruments can
  provide useless data if administered incorrectly,
  by someone disliked by respondents, under noisy,
  inhospitable conditions, or when subjects are
  exhausted.

9
Validity, Reliability, and Objectivity

• Validity is an important consideration in the
  choice of an instrument to be used in a research
  investigation
• It should measure what it is supposed to measure
• Researchers want instruments that will allow them
  to make warranted conclusions about the
  characteristics of the subjects they study
• Reliability is another important consideration,
  since researchers want consistent results from
  instrumentation
• Consistency gives researchers confidence that the
  results actually represent the achievement of the
  individuals involved
• Objectivity refers to the absence of subjective
  judgments

10
Usability

• An important consideration for any researcher in
  choosing or designing an instrument is how easy
  the instrument will actually be to use.
• Some of the questions asked which assess
  usability are
• How long will it take to administer?
• Are the directions clear?
• How easy is it to score?
• Do equivalent forms exist?
• Have any problems been reported by others who
  used it?
• Getting satisfactory answers can save a
  researcher a lot of time and energy.

11
Ways to Classify Instruments

• Who Provides the Information?
• Themselves Self-report data
• Directly or indirectly from the subjects of the
  study
• From informants (people who are knowledgeable
  about the subjects and provide this information)

12
Types of Researcher-completed Instruments

• Rating scales
• Interview schedules
• Tally sheets
• Flowcharts

• Performance checklists
• Observation forms

13
Types of Subject-completed Instruments

• Questionnaires
• Self-checklists
• Attitude scales
• Personality inventories

• Achievement/aptitude tests
• Performance tests
• Projective devices
• Sociometric devices

14
Scientific America

• Handwriting Analysis

15
Item Formats

• Questions used in a subject-completed instrument
  can take many forms but are classified as either
  selection or supply items.
• Examples of selection items are
• True-false items
• Matching items
• Multiple choice items
• Interpretive exercises
• Examples of supply items are
• Short answer items
• Essay questions

16
Unobtrusive Measures

• Many instruments require the cooperation of the
  respondent in one way or another.
• An intrusion into an ongoing activity could be
  involved which causes a form of negativity within
  the respondent.
• To eliminate this, researchers use unobtrusive
  measures, data collection procedure that involve
  no intrusion into the naturally occurring course
  of events.
• In most cases, no instrument is used, however,
  good record keeping is necessary.
• They are valuable as supplements to the use of
  interviews and questionnaires, often providing a
  useful way to corroborate what more traditional
  data sources reveal.

17
Types of Scores

• Quantitative data is reported in the form of
  scores
• Scores are reported as either raw or derived
  scores
• Raw score is the initial score obtained
• Taken by itself, a raw score is difficult to
  interpret, since it has little meaning
• Derived score are scores that have been taken
  from raw scores and standardized
• They enable researchers to say how well the
  individual performed compared to others taking
  the same test
• Examples include
• Age and Grade-level Equivalents
• Percentile Ranks
• Standard scores are mathematically derived scores
  having comparable meaning on different instruments

18
Examples of Raw Scores and Percentile Ranks
19
Norm-Referenced vs. Criterion-Referenced
Instruments

• All derived scores give meaning to individual
  scores by comparing them to the scores of a
  group.
• The group used to determine derived scores is
  called the norm group and the instruments that
  provide such scores are referred to as
  norm-referenced instruments.
• An alternative to the use of achievement or
  performance instruments is to use a
  criterion-referenced test.
• This is based on a specific goal or target
  (criterion) for each learner to achieve.
• The difference between the two tests is that the
  criterion referenced tests focus more directly on
  instruction.

20
Descriptive Statistics
21
Statistics vs. Parameters

• A parameter is a characteristic of a population.
• It is a numerical or graphic way to summarize
  data obtained from the population
• A statistic is a characteristic of a sample.
• It is a numerical or graphic way to summarize
  data obtained from a sample

22
Types of Numerical Data

• There are two fundamental types of numerical
  data
• Categorical data obtained by determining the
  frequency of occurrences in each of several
  categories
• Quantitative data obtained by determining
  placement on a scale that indicates amount or
  degree

23
Techniques for Summarizing and Presenting
Quantitative Data

• Visual
• Frequency Distributions
• Histograms
• Stem and Leaf Plots
• Distribution curves
• Numerical
• Central Tendency
• Variability

24
Summary Measures
Summary Measures
Variation
Central Tendency
Arithmetic Mean
Median
Mode
Range
Variance
Standard Deviation
25
Measures of Central Tendency
Central Tendency
Average (Mean)
Median
Mode
26
Mean

• The most common measure of central tendency
• Affected by extreme values (outliers)

0 1 2 3 4 5 6 7 8 9 10
0 1 2 3 4 5 6 7 8 9 10 12
14
Mean 5
Mean 6
27
Median

• Robust measure of central tendency
• Not affected by extreme values
• In an Ordered array, median is the middle
  number
• If n or N is odd, median is the middle number
• If n or N is even, median is the average of the
  two middle numbers

0 1 2 3 4 5 6 7 8 9 10
0 1 2 3 4 5 6 7 8 9 10 12
14
Median 5
Median 5
28
Mode

• A measure of central tendency
• Value that occurs most often
• Not affected by extreme values
• Used for either numerical or categorical data
• There may may be no mode
• There may be several modes

0 1 2 3 4 5 6
0 1 2 3 4 5 6 7 8 9 10 11
12 13 14
No Mode
Mode 9
29
Variability

• Refers to the extent to which the scores on a
  quantitative variable in a distribution are
  spread out.
• The range represents the difference between the
  highest and lowest scores in a distribution.
• A five number summary reports the lowest, the
  first quartile, the median, the third quartile,
  and highest score.
• Five number summaries are often portrayed
  graphically by the use of box plots.

30
Variance

• The Variance, s2, represents the amount of
  variability of the data relative to their mean
• As shown below, the variance is the average of
  the squared deviations of the observations about
  their mean

31
Standard Deviation

• Considered the most useful index of variability.
• It is a single number that represents the spread
  of a distribution.
• If a distribution is normal, then the mean plus
  or minus 3 SD will encompass about 99 of all
  scores in the distribution.

32
Calculation of the Variance and Standard
Deviation of a Distribution (Definitional formula)
Raw Score Mean X X (X X)2
85 54 31 961 80 54 26 676 70 54 16 256 60 54 6 36
55 54 1 1 50 54 -4 16 45 54 -9 81 40 54 -14 196 30
54 -24 576 25 54 -29 841

404.44
Standard deviation (SD)
33
Comparing Standard Deviations
Data A
Mean 15.5 S 3.338
11 12 13 14 15 16 17 18
19 20 21
Data B
Mean 15.5 S .9258
11 12 13 14 15 16 17 18
19 20 21
Data C
Mean 15.5 S 4.57
11 12 13 14 15 16 17 18
19 20 21
34
Facts about the Normal Distribution

• 50 of all the observations fall on each side of
  the mean.
• 68 of scores fall within 1 SD of the mean in a
  normal distribution.
• 27 of the observations fall between 1 and 2 SD
  from the mean.
• 99.7 of all scores fall within 3 SD of the mean.
  
• This is often referred to as the 68-95-99.7 rule

35
The Normal Curve
36
Different Distributions Compared
37
Fifty Percent of All Scores in a Normal Curve
Fall on Each Side of the Mean
38
Probabilities Under the Normal Curve
39
Standard Scores

• Standard scores use a common scale to indicate
  how an individual compares to other individuals
  in a group.
• The simplest form of a standard score is a Z
  score.
• A Z score expresses how far a raw score is from
  the mean in standard deviation units.
• Standard scores provide a better basis for
  comparing performance on different measures than
  do raw scores.
• A Probability is a percent stated in decimal form
  and refers to the likelihood of an event
  occurring.
• T scores are z scores expressed in a different
  form (z score x 10 50).