Educational Research: Descriptive Research

1
Educational Research Descriptive
Research
EDU 8603 Educational Research Richard M. Jacobs,
OSA, Ph.D.
2
Research...

• The systematic application of a family of methods
  employed to provide trustworthy information about
  problems

an ongoing process based on many accumulated
understandings and explanations that, when taken
together, lead to generalizations about problems
and the development of theories
3
Quantitative research methods...
collect and analyze numerical data obtained from
formal instruments
4
Quantitative methods...

• descriptive research (survey research)

• correlational research

• causal-comparative research (ex post facto
  research)

• experimental research

5

• descriptive research (survey research)

collects data in order to answer questions about
the current status of the subject or topic of
study
uses formal instruments to study preferences,
attitudes, practices, concerns, or interests of a
sample
6
The basic steps of descriptive research...
An orderly scientific and disciplined process,
involving
? recognizing and identifying a topic to be
studied
? selecting an appropriate sample of participants
? collecting valid and reliable data
? reporting conclusions
7
Classifications of descriptive research studies...
cross-sectional survey
involves collecting data from selected
individuals in a single time period however long
it takes to collect data from participants
8
longitudinal survey
involves collecting data at two or more
instances in order to measure changes over time
9
self-report survey
requires individuals to respond to a series of
statements or questions about themselves
10
observation study
the researcher obtains data by watching
participants in a situation
11
Types of longitudinal surveys...

• trend survey

• cohort survey

• panel survey

• follow-up survey

12

• trend survey

a study where a sample is taken from the general
population in order to collect data over time
involves different groups and different samples
over time
13

• cohort survey

a study where a specific population is examined
by sampling different groups within the
population over time
involves the same group but different samples
from that group over time
14

• panel survey

a study where the same group and the same sample
are examined over time
15

• follow-up survey

a study undertaken after a panel survey and
seeks to examine subsequent development or change
16
Conducting a questionnaire study
1. state the problem
2. select participants
3. construct the questionnaire
4. prepare cover letter
5. pretest questionnaire
6. follow up activities
7. analyze/report results
17
1. state the problem
topic must be of sufficient significance to
motivate potential respondents and justify the
research effort in the first place
define topic in terms of specific objectives
indicating the kind of information needed
18
2. select participants
use an appropriate sampling technique
participants must be able to provide the desired
information sought and willing to provide it to
the researcher
19
3. construct the instrument
should be attractive, brief, and easy to fill out
there are a variety of data collection methods
that can be used, including questionnaires,
interviews, observations, and telephone calls
20

• types of self-report items

scaled
ranked
checklist
free response
21
Designing an instrument
include a brief statement describing the study
and its purpose at the top of the instrument
provide standardized directions
organize items leaving sufficient white space on
instrument
22
place similar items together
ask general, non-threatening items first, moving
into more specific, self-revealing items
23

• some dos and donts

construct items according to a set of
predetermined guidelines
include only items relating directly to the
purpose of the study
avoid jargon or any term or concept that might
mean different things to different people
24
each item should deal with a single concept and
be worded as clearly as possible
items should indicate a point of reference
beyond the self in order to judge the self
avoid ambiguous terms like several or usually
25
avoid touchy matters in items which respondents
might not respond to honestly or at all
avoid leading questions which suggest that one
response may be more appropriate than another
each item must be able to stand on its own
26
dont jam items together
dont put the most important questions at the
end of the instrument
27
4. prepare the cover letter
a brief, neat, explanation of the significance
of the study and what is being asked of the
respondent and why
addressed to the specific, potential respondent
28
an endorsement adds credibility
guarantee anonymity or confidentiality
include a specific deadline date by which the
completed instrument (questionnaire) is to be
returned
29
individually sign each letter
include a self-addressed, stamped, return
envelope
30
5. pretest the instrument
conduct a pilot study to gather information
about deficiencies and suggestions for improving
the instrument
provides greater content validity
31
6. follow-up activities
because first mailing results tend to be low
(30 - 50 response rate), researchers need
strategies to increase the response rate
32

• initial follow-up strategies to increase response
  rate up to 20

send out reminder postcard
mail a second packet with a new, positively
worded cover letter and another self-addressed,
stamped, return envelope
33

• secondary follow-up strategies to increase
  response rate up to 10

telephone nonrespondents to encourage them to
respond
34

• dealing with nonrespondents

study whether nonrespondents differ from
respondents in some systematic way by selecting a
sample from the nonrespondents
telephone sample, aggregate data, include in
report
35

• dealing with nonresponses

study the items to determine the problem with
the item
include description of the nonresponses and the
determination of the reason in the report
36
7. analyze/report results
discriminant item analysis includes giving the
response rate for each item as well as the total
sample size and the overall percentage of
returns, since not all respondents will answer
questions
37
group items into clusters that address the same
issue and develop total scores across an item
cluster in order to avoid information overload
38
Conducting an interview study
1. state the problem
2. select participants
3. construct the interview guide
4. communicate professionally and record
responses accurately
5. pretest interview procedure
6. analyze/report results
39
1. state the problem
topic must be of sufficient significance to
motivate individuals to participate and justify
the research effort in the first place
define topic in terms of specific objectives
indicating the kind of information needed
40
2. select participants
use an appropriate sampling technique
participants must be able to provide the desired
information sought and willing to provide it to
the researcher
41
3. construct the interview instrument
(instrument guide)
indicates the question to be asked, the order,
and how much additional prompting or probing is
permitted
the goal is to obtain standardized, comparable
data from each interviewee
42
4. communicate professionally and record
responses accurately
effective interviewing requires training and
experience to avoid interviewer bias and
interviewer error
43
record responses manually on the interview guide
(requires interviewee permission) use a tape
recorder or VCR to verify accuracy of responses
44
5. pretest interview procedure
use a small group from the same population or a
similar population to the one being studied to
validate the interview instrument and the
interviewers communication and recording skills
45
6. analyze/report results
item analysis includes giving the response rate
for each item, as well as the total sample size
and the overall percentage of returns, since not
every respondent will answer all questions
46
also include in-depth data to open-ended
interview items to portray responses in more
accurate and honest terms
47
Conducting an observational study
1. state the problem
2. select participants
3. define the observational variables
4. record observations
5. assess observer reliability
6. reduce observer bias
7. analyze/report results
48
1. state the problem
topic must be of sufficient significance to
motivate potential respondents and justify the
research effort in the first place
define topic in terms of specific objectives
indicating the kind of information needed
49
2. select participants
use an appropriate sampling technique
participants must be able to provide the desired
information sought and willing to provide it to
the researcher
50
3. define the observational variables
operationalize the variables to be observed in
terms of specific behaviors that can be quantified
ensure that the observations can be quantified
in a way so that all observers will count the
behavioral activities in exactly the same way
51
simplify procedure for recording observations by
developing a coding procedure
52
4. record observations
record behavior at the time it occurs
alternate observation periods and recording
periods
53
5. assess observer reliability
use at least two observers who make independent
observations
compute interobserver reliability
54
6. reduce observer bias
train observers until a satisfactory level of
agreement is achieved (at least 80)
monitor observers
55

• types of observer bias

response set
halo effect
knowledge of participants
56

• response set

the tendency of an observer to rate the majority
of observees as above average, average, or below
average regardless of the observees actual
behavior
57

• halo effect

the tendency of an observers initial positive
or negative impressions of the observee to affect
subsequent observations
58

• self-fulfilling prophecy

the tendency of an observers knowledge of the
observees or the purposes of the study to affect
the observation(s)
59
Meta-Analysis...

• a statistical approach to summarizing the results
  of many studies that have investigated basically
  the same problem

provides a numerical coefficient expressing the
average result of the studies
60
requiring the researcher to find, describe,
classify, and code the research studies to be
included meta-analytic review, and for measuring
and analyzing the study findings
61
each studys results are translated into an
effect size (ES) which is a numerical expression
of the strength or magnitude of a reported
relationship, be it causal or not
62
Xe Xc ES ????
SDc Where Xe the mean score of the
experimental group Xc the mean
score of the control group SDc the
standard deviation of the control group
63
Strength or magnitude of an effect size (D)
-.80
.80
0.00
strong positive
strong negative
both groups performed similarly
experimental group performed better than control
group
control group performed better than experimental
group
64
Mini-Quiz

• True or false

in a descriptive research study, the researcher
predetermines what variables will be surveyed
before selecting or observing the research
participants
True
65
one of the most difficult activities on the part
of questionnaire researchers is to write or
select questions that are clear and unambiguous
True
66
longitudinal surveys are useful for studying the
dynamics of a topic or issue over time
True
67
one of the problems with longitudinal studies is
that the samples tend to shrink as time goes by
True
68
one of the problems with cross-sectional studies
is selecting samples that truly represent the
population at a particular level or ability
True
69
an external review of an instrument provides the
researcher greater assurance of it its content
validity
True
70
if the responses from respondents and
nonrespondents are essentially the same, the
researcher may assume that the response group is
representative of the whole sample and that the
survey results are generalizable
True
71
if the responses from respondents and
nonrespondents are different, the
generalizability across both the respondent and
nonrespondent groups is not present and must be
discussed in the report
True
72
analyzing clusters of instrument items related
to the same issue improves the reliability of the
scores
True
73
bias can affect the validity of the
interpretations in observational studies
True
74
although there are no hard and fast rules, it is
generally agreed than an effect size in the
twenties indicates a treatment that produces a
relatively small effect, whereas an effect size
in the eighties indicates a powerful treatment
True
75

• Fill in the blank

a study requiring individuals to respond to a
series of statements or questions about themselves
self-report study
76

• Fill in the blank

a study in which individuals are not directly
asked for information but data is obtained as the
researcher watches participants engage in a
situation
observation study
77

• Fill in the blank

items on a survey that are answered by circling
a letter, checking a list, or numbering
preferences
close-ended items
78

• Fill in the blank

the researcher guarantees that no one, including
the researcher, knows who completed each
questionnaire
anonymity
79

• Fill in the blank

the researcher guarantees that no one, except
the researcher, knows who completed each
questionnaire and promises not to divulge that
information
confidentiality
80

• Fill in the blank

the oral, in person administration of an
instrument to each member of a sample
interview
81

• Fill in the blank

determining the current status of a phenomenon
not through an instrument but as the researcher
watches the participants engage in the activity
being studied
observational study
82

• Fill in the blank

an observation in which the researcher watches
behavior as it normally unfolds
naturalistic observation
83

• Fill in the blank

an observation in which the researcher creates a
situation to be observed and tells participants
what activities to engage in
simulation observation
84

• Fill in the blank

a coefficient found by dividing the total number
of agreed observations by the total number of
agreed and disagreed observations
inter-observer reliability
85

• Fill in the blank

a means by which observers record observations
at the time it occurs by simplifying the
recording process
coding
86

• Fill in the blank

a statistical approach that summarizes the
results of many studies having investigate the
same problem
meta-analysis
87
This module has focused on...
descriptive research
which involves collecting data in order to test
hypotheses or to answer questions about the
opinions of people about some topic or issue
88
The next module will focus on...
correlational research
...which involves collecting data in order to
determine whether, and to what degree, a
relationship exists between two or more
quantifiable variables