From Concept to Hypothesis to Survey

1
From Concept to Hypothesis to Survey
2
Developing a Research Design

• Developing concepts that are derived from ideas,
  theories, or prior research
• Taking concepts and translate them into
  measurable variables.
• Select the most appropriate research method
• Develop a Sampling Strategy
• Plan How to Collect the Data
• Decide on relevant tools and tests
• How will data be analyzed?

3
Concepts

• Concepts are the building blocks of social
  theory. Concepts are simply mental images or
  perceptions. They may be impossible to directly
  observe like fairness, or sportsmanship, or they
  may be observable, like height. A dichotomous
  concept only has two possible values. A concept
  has two parts a symbol and a definition.

4
Conceptualization

• Refers to the process of taking a construct and
  refining it by giving it a conceptual or
  theoretical definition.
• A conceptual definition is a definition in
  abstract, theoretical terms. It refers to other
  ideas or constructs.
• To do this one must think about the meanings of a
  construct.

5
Constructs

• Are more complex or difficult to define. Ie.
  Happiness
• Concept clusters may be needed to measure
  constructs.

6
Concept Clusters

• Concept clusters- concepts are rarely used in
  isolation. Instead we tend to have
  interconnected groups of concepts called concept
  clusters.
• Ie. Social class is a concept. The letters that
  make up the words are the symbol. The definition
  involves another set of concepts- income,
  occupation, wealth.
• Some concepts can be quantified or take on a
  range of values- density of population, income,
  age others may not- death, disease, kindness.
• Assumptions- all concepts contain some types of
  assumptions that cannot be tested. This is
  unavoidable. If the assumptions are faulty- the
  concept is faulty.

7
Variables and Values

• A variable is a concept made measurable. It is
  variable meaning its value can change.
• A constant is a variable with only one value. It
  remains constant.

8
Hypotheses

• Untested statements that specify a relationship
  between two or more variables.

9
What Makes a Good Hypothesis?

• 1) A good hypothesis is stated as a declaration.
  It is not a question. It must be clear and
  forceful.
• 2) It should posit an expected relationship. The
  variables involved should be clear and testable.
  You are not just looking around for any
  relationship, but rather are testing a clear and
  informed hypothesis. (If it is not supported you
  have still contributed to the body of knowledge).
• 3) It should reflect the theory or literature
  upon which they are based.
• 4) It should be brief and too the point. Dont
  try to cram too much into one hypothesis. It is
  better to have 2 or 3 clear ones, than one obtuse
  one. Usually it clearly conveys the general
  purpose of the study and how things will get
  done.
• 5) It is testable.

10
Guidelines for Hypotheses

• 1) Hypotheses are linked to more abstract
  theories.
• 2) It is important that the independent and
  dependent variables be clearly specified.
• 3) It is important that the precise nature and
  direction of the relationship between variables
  be specified in the hypothesis.
• 4)Hypotheses must be stated so that they can be
  verified or refuted.
• 5) All Concepts and comparisons in hypotheses
  must be clearly stated.
• 6) A hypothesis is not written about a single
  individual. It is a relationship between
  variables.

11
Types of Hypotheses

• Positive- as the independent variable increases,
  the dependent variable increases.
• Inverse- as the independent variable increases,
  the dependent variable decreases.
• One Directional- the relationship is specified as
  positive or inverse
• Two Directional- the direction of the
  relationship is not specified.

12
Identify the hypothesis

• A) Is there a relationship between level of
  education and income?
• B) Education and income are both measurable
  variables.
• C) The higher ones level of education, the
  higher ones income.
• Bonus Is it directional? If so, is it positive
  or inverse?

13
How to Test a Hypothesis

• A hypothesis is never proved, it can only be
  disproved. But hypotheses are tested.
• Null hypothesis testing
• It is actually the null or the idea that there is
  no relationship, which is tested. If evidence
  leads the researcher to accept the null this
  implies that your alternative hypothesis is
  false.

14
The Null Hypothesis

• There is no relationship between the variables.

15
A Hypothesis and the Null

• Hypothesis As urban density increases, crime
  rates increase.
• Null Hypothesis There is no relationship
  between urban density and crime.

16
Operationalization

• The process of taking a concept and making it
  into a variable.
• Often this requires multiple items to capture the
  full range of meanings implied.
• Specifying the indicators of a concept is a key
  part of this process.

17
Operationalization

• Operationalization links a conceptual definition
  to a specific set of measurement techniques or
  procedures.
• The constructs operational definition is a
  definition in terms of the specific operations of
  actions a researcher carries out. It could be a
  survey questionnaire, a method of observing
  events in a field setting, a way to measure
  symbolic content in the mass media, or any
  process carried out by the researcher that
  reflects, documents, or represents the abstract
  construct as it is expressed in the conceptual
  definition.

18
Five Suggestions for Coming Up With a Measure

• 1) Remember the Conceptual Definition- the
  measure should match your conceptual definition
  of your construct.
• 2) Keep an Open Mind- be creative and always look
  for new measures.
• 3) Borrow from Others- borrow and cite others
  measures.
• 4) Anticipate Difficulties- logical and practical
  problems often arise when trying to measure
  variables of interest Try to anticipate and
  mitigate.
• 5) Do not forget your units of analysis- you need
  to be able to generalize your findings to the
  area of interest.

19
Dependent and Independent Variables

• Dependent- its variability depends on the
  independent variable. It is the variable being
  acted upon.
• Independent- the cause or explanation of the
  value of the dependent variable.

20
Levels of Measurement

• Nominal Variables- a nominal variable has no
  numeric value. Ie. Race, gender, political party,
  religion
• Ordinal Variables- are rank ordered, but there is
  no definable numeric distance between them. Ie.
  Highest degree earned, socioeconomic status.
• Interval variables- is numerically relevant.
  Equal spacing between categories Ie . IQ. The
  zero point, however, is arbitrary. Someone with
  a 140 IQ is not twice as intelligent as a person
  with a 70 IQ.
• Ratio variables- is numerically relevant. Equal
  spacing between categories but the zero point is
  set. Ie. Income, (0 is not arbitrary) Kelvin
  scale of temperature

21
Discrete versus Continuous Variables

• Discrete Variables- possess a finite number of
  distinct and separate values or categories. Ie.
  Sex, race, household size, number of days absent
• Continuous Variables- could theoretically be
  divided into an infinite number of categories.
  Ie. Height- could use smaller and smaller units

22
Identify the Type of Variable

• Marital Status (single (never married),
  married/cohabitating, widowed, divorced/separated)
  
• Level of Agreement ( Strongly Agree, Agree,
  Disagree, Strongly Disagree)
• Household Income measured in U.S. Dollars
• Household Income grouped into categories of
  15,000 each (lt15,000, 15,000-29,999,
  30,000-44,999 etc)
• State of Birth for those born in the U.S. ie.
  Illinois

23
Questionnaire Formats

• Self-Administered
• Computer Assisted and Web Based
• Interviews
• Telephone

24
Self-Administered Questionnaires

• Are best designed for measuring variables with
  numerous values or response categories,
  nonobservable attitudes and opinions, describing
  large populations, studying stigmatized behaviors
  that may be difficult to discuss face to face.

25
Concerns for Self-Administered Questionnaires

• Is there response bias?
• Who actually filled it out?
• Did he/she understand it?
• Response Rate
• In mailed surveys the researcher must be aware
  that it is unlikely that he/she will get a
  response rate over 80. It will be below 70
  unless procedures to maximize the response rate
  are not followed.
• A survey needs at least a 60 response rate to
  not be a disaster. 70 is minimally acceptable.

26
Maximizing Response Rates in Mailed Surveys

• Make the questionnaire attractive with plenty of
  open space.
• Use skip patterns infrequently, when necessary,
  guide the respondents visually through the
  pattern.
• Use no more than a few open-ended questions as
  respondents are likely to be put off by having to
  write too much.
• Include a professional and personalized cover
  letter. Using an altruistic appeal seems to
  produce a response rate 7 higher than indicating
  that respondents will receive some benefit.
• Have a credible research sponsor. This may
  increase the rate of response by up to 17 if the
  organization is known to the subject.
• Include a stamped self addressed envelope.

27
Tenacious Survey Collection

• Initial Mailing
• Reminder Card sent two weeks after the initial
  questionnaire.
• Replacement questionnaire with a new cover letter
  sent only to non-respondents- usually three to
  four weeks after the initial questionnaire.
• Send a final replacement questionnaire and new
  cover letter to non-respondents about 8 weeks
  after the initial mailing. If possible it may be
  a good idea to send this by certified mail.
• If these procedures are followed a researcher
  should reach at least 70.

28
Computer- Assisted Web Based Surveys

• Things to consider
• Coding and data entry may be facilitated.
• Access is limited to computer users.

29
Interviews

• Things to consider
• How will the interviewer being present affect
  ones answers.
• Advantage of having a researcher there to explain
  things if the subject doesnt understand.

30
Designing a Good Questionnaire

• Key Features
• Rules for Question Writing
• Indexes and Scales

31
Key Features to a Good QuestionnaireA Good
Survey includes Questions That

• Record views, reported actions and basic
  demographic information.
• Measure all variables relevant to hypotheses.
• What do I need to know to answer my question(s).
• Measure What Respondents Know.
• Measure Relevant Experiences of Respondents.
• Measure how consistent Respondents attitudes
  are.
• Measure the Strength of Attitudes held.

32
Rules for Question Writing

• The question must be answerable.
• -Must be answerable in the abstract .
• How many stars are there in the universe?
• -Must be answerable by the respondent.
• Do you agree with the policies of the Prime
  Minister of Zaire?
• What brand of underwear did you wear in
  pre-school?

33
Rules for Question Writing

• No Double Barreled Questions- Ask One Thing at
  a Time.
• For example Please indicate your level of
  agreement with the following statement Cal Poly
  Pomona provides a high quality education and
  offers a wide variety of classes.
• Instead have two questions
• Cal Poly Provides a high quality education.
• Cal Poly offers a wide variety of classes.

34
Rules For Question Writing

• Ambiguous Questions- Be as Specific as Possible
• Avoid questions with ambiguous terms. For
  example, On a scale of 1-10, how socially
  integrated are you?
• The meaning of terms may vary cross culturally,
  across age groups and so forth. For example, the
  meaning of gay in Western versus Latin Culture.
  Or the meaning of calling someone a dog or a dawg
  varies tremendously by context.

35
Rules for Question Writing

• Level of Wording- Use a Level of Wording most
  Respondents can Understand.
• For Example, Do you believe in lycanthrope? is
  not likely to be understood by most respondents.
  Instead ask, Do you believe in werewolves?
• A good rule of thumb is to assume an eight grade
  reading level of adults. However, depending on
  ones subject population, this may need to be
  adjusted.

36
Rules for Question Writing

• Bias in Wording
• Words have positive and negative connotations,
  your choice of wording can affect how respondents
  answer questions.
• For example, ask people if they support
  assistance for the poor and most say yes, but ask
  them if they support welfare and most say no.
• A second example, freedom fighters versus
  terrorists is simply a matter of perception.
  Benedict Arnold is a great British patriot and an
  American Traitor.
• Avoid associating questions with famous people as
  prestige bias of the individual may sway
  respondents.

37
Rules for Question Writing

• Abstract Versus Factual Questions- questions
  should refer to concrete and specific matters and
  have specific answers, if possible. Abstract
  questions are hard to answer. Instead ask
  several factual or smaller questions to construct
  an index that measures the abstract concept.
• For Example, instead of asking someone how happy
  they are, a variety of how happy one has been
  over the last week are used to construct a
  happiness index.

38
Rules for Question Writing

• Leading Questions- Questions should be carefully
  structured in order to minimize the probability
  of biasing the respondents answer by leading him
  or her and thus artificially increasing the
  probability of a particular response.
• For example, Many people get ahead by working
  hard. Others, prefer to lie, cheat and steal.
  Thus we are measuring how hard people work. Over
  the last week, how many hours did you work?
  0-10 11-20 21-30 31-40 41
• Worded this way, I would expect to discover that
  most Americans work over 40 hours a week.

39
Rules for Question Writing

• Sensitive or threatening Questions- Are prone
  to normative answers. Subjects may fear social
  sanctions, or simply disapproval if they deviate
  from the norm. Therefore ask sensitive questions
  as if there is no normative answer.
• Always place opt out answer choices last.
  Otherwise subjects may try to opt out of honest
  answers.
• For Example, How frequently do you masturbate?
• Daily Once a week Once a month One a Year Never

40
Rules for Question Writing

• Avoid Questions with Right Answers
• People Are often Deceptive- even too themselves.
  How one sees oneself affects how one answers
  questions.
• For Example, The Surgeon general recommends
  cardiovascular exercise be performed a minimum of
  3-4 times a week for at least 30 minutes. Hence,
  be careful when you ask, How often do you go to
  the gym? You are likely to find a lot of people
  who go 3-4 times a week for at least 30 minutes.

41
Rules for Question Writing

• Avoid Negative Questions- avoid asking
  individuals to respond to a negative statement in
  positive or negative ways. It can create
  confusion as to the meaning of disagreement.
• For example, Police officers should not be
  required to take psychological evaluations.
• Strongly disagree Disagree Agree Strongly agree
• Avoid double negatives- double negatives are
  grammatically incorrect and ambiguous.

42
Rules for Question Writing

• Avoid making disagreement disagreeable- sometimes
  respondents will agree with statements because
  expressing disagreement seems also to express
  opposition.
• Many people prefer not to rock the boat. Hence
  statements that ask for agreement or disagreement
  must be worded so that
• 1) It is unclear what the normative position is.
• 2) So that subjects are equally likely to chose
  any answer.

43
Rules for Question Writing

• Avoid False Premises- do not begin a question
  with a premise that not all respondents may
  agree with.
• For Example, Everyone is concerned about the
  issue of physician assisted suicide
• I would have alienated any respondents who were
  not.

44
Rules for Question Writing

• Avoid Asking About Future Intentions- what people
  say they would do is more a measure of what they
  think the ideal thing to do is, than a measure of
  personal behavior.
• For example, What would you do if you saw a
  building on fire and a baby in the window?
• a) Go about your business. Its not your
  problem.
• B) Call 911
• C) Rush in to save the baby, its worth the risk.
• D) Get out the garden hose you always carry with
  you an start dousing the fire.
• E) Stand around and watch. This could be
  interesting.

45
Rules for Question Writing

• Mutually Exclusive- all questions must have
  mutually exclusive answer sets. This means that
  each respondent must fit into only one answer
  category.
• For example, the following question has a problem
  with mutual exclusivity.
• Your pre-tax income fits into which category?
• Less than 10,000 10,000-40,000 40,000-100,000 100,
  000

46
Rules for Question Writing

• Exhaustive- All questions must have exhaustive
  answer choices. This means that everyone must
  have an answer category that he/she can
  accurately mark.
• For example, the following question has an answer
  set that is not exhaustive.
• Please indicate your current marital status?
• Single Divorced/Separated Widowed
• Oops, there is no category for Married persons.

47
Rules for Question Writing

• Dont Reinvent the Wheel- If someone already has
  a good measure of what you are interested in, use
  it.
• Be sure to cite sources.
• Make sure any necessary adjustments to the
  measure have been made.

48
Rules for Question Writing

• Be consistent about how Non-Responses will be
  coded.
• If an answer is missing or no response is given,
  you can assign a code to capture this situation.
• Will you drop non-responses from certain
  analyses?
• For indexes and scales, will missing items be
  constructed out of other responses?

49
Rules for Question Writing

• Events may temporarily sway attitudes and
  opinions.
• For example, September 12 is not a good day to do
  a survey about a variety of topics ranging from
  airline safety to fear of terrorism.
• Be cognizant of any recent events that may affect
  your survey. If possible, adjust dates of
  distribution accordingly.

50
Dont Know/Neutral Categories

• Fencesitters/Floaters- are people who are truly
  neutral.
• If you lack a neutral position, you force these
  people to either
• 1) express an opinion they dont have.
• 2) not respond to the question.

• Opting Out- On the other hand, some people are
  hesitant to express options. Having a Dont
  know/Neutral category may cause people who have
  weakly held attitudes to opt out.
• Studies reveal that the presence of a neutral
  category decreases the strength of responses.

51
Question Order

• Question Order (the order in which questions are
  asked) can significantly alter respondents
  answers to questions.
• Examples GSS Happiness Questions, GSS Abortion
  Scale

52
Questionnaire Format

• Cover Letter
• Title
• Instructions
• Survey
• Detachable Informed Consent Form

53
Cover LetterThe Cover Letter Should

• Make you and your study appear credible. Be sure
  to use official letterhead, and clearly identify
  you, your affiliations and your sponsor.
• Provide any necessary contact information so that
  respondents can contact you and your sponsor.
• Reaffirm the anonymity of the subjects and the
  confidentiality of any information provided.
• Thank the subjects for their help. Be sure to
  include an altruistic appeal.
• Make the study sound interesting, important and
  relevant.

54
The SurveyThe Survey Should

• Be attractive and easy to understand.
• All directions should be clearly stated at the
  beginning of each section.
• Answer choices must be clear.
• Leave space- do not bunch questions or answers
  together.

55
Informed Consent

• Your are required by professional codes of ethics
  to obtain informed consent from subjects.
• Each survey should include an informed consent
  form.

56
Informed Consent Form Acquisition

• Go to the Cal Poly Pomona Home Page
  www.csupomona.edu
• Go to the webpage of the office of Research and
  Sponsored Programs
• Select Forms
• Select IRB
• Download these forms and save them to your disk.
  You will be required to hand them in as part of
  your final project.