Title: Questionnaire Design
1Questionnaire Design
2It is not every question that deserves an
answer.
- Publius Syrus(Roman, 1st century B.C.)
3Ask a Sensitive Question, Get a Sensitive Answer
- Survey researchers believe that the responses
that people give are valid. - Care must be taken with sensitive questions.
- Researchers must take care in asking relevant
questions in ways that produce the most truthful
results.
4Basic Considerations in Questionnaire Design
- Questionnaire design is one of the most critical
stages in the survey research process. - A questionnaire (survey) is only as good as the
questions it asks ask a bad question, get bad
results. - The questions must meet the basic criteria of
relevance and accuracy.
5What Should Be Asked?
- Questionnaire Relevancy
- All information collected should address a
research question that helps the decision maker
in solving a current marketing problem. - Questionnaire Accuracy
- The information is valid it faithfully
represents reality. - Questionnaires should use simple, understandable,
unbiased, unambiguous, nonirritating words. - Questionnaire design should facilitate recall
motivate respondents to cooperate. - Proper question wording sequencing to avoid
confusion biased answers.
6Major Decisions in Questionnaire Design
- What should be asked?
- How should each question be phrased?
- In what sequence should the questions be
arranged? - What questionnaire layout will best serve the
research objectives? - How should the questionnaire be pretested? Does
the questionnaire need to be revised?
7Phrasing Questions
- Open-ended questions
- Aka essay questions, short-answer questions
- Fixed-alternative questions
- Aka closed or closed-ended questions
8Open-Ended Response Questions
- Pose some problem ask respondents to answer in
their own words. - Advantages
- Particularly beneficial in exploratory research,
especially when the range of responses is not
known. - Identify which words phrases people
spontaneously give. - Valuable at the beginning of an interview.
- Disadvantages
- High cost of administering open-ended response
questions. - The possibility that interviewer bias will
influence the answer. - Bias introduced by articulate individuals longer
answers.
9Example of Open-Ended Response Question
10Fixed-alternative Questions
- Questions in which respondents are given
specific, limited-alternative responses asked
to choose the one closest to their own viewpoint. - Advantages
- Require less interviewer skill
- Take less time to answer
- Are easier for the respondent to answer
- Provides comparability of answers
- Disadvantages
- Researcher may be unaware of all potential
responses - Tendency of respondents to choose more
prestigious or socially acceptable alternative
11Example of a Fixed-Alternative Question
12Types of Fixed-Alternative Questions
- Simple-dichotomy (dichotomous) Question
- Requires the respondent to choose one of two
alternatives (e.g., yes or no). - Example
- Did you make any calls with your home (landline)
phone during the 7 days? - _____ Yes _____ No
- Determinant-Choice (multiple-choice) Question
- Requires the respondent to choose one response
from among multiple alternatives (e.g., A, B, or
C). - Example
13Types of Fixed-Alternative Questions (cont.)
- Frequency-determination Question
- Asks for an answer about general frequency of
occurrence (e.g., often, occasionally, or never). - Checklist Question
- Allows the respondent to provide multiple answers
to a single question by checking off items. - Scale
- Likert, Semantic Differential, Stapel, etc.
14Phrasing Questions for Self-Administered,Telephon
e, Personal Interview Surveys
- Influences on Question Phrasing
- Means of data collection telephone interview,
personal interview, self-administered
questionnaire will influence question format
question phrasing. - Questions for mail, Internet, telephone surveys
must be less complex than those used in personal
interviews. - Questionnaires for telephone personal
interviews should be written in a conversational
style.
15Best Question Formats Vary by the Interview
Medium
16Guidelines For Avoiding Mistakes
- Simpler is better
- Avoid leading loaded questions
- Leading question directs respondents to an
answer you want them to give - Loaded question suggests a socially desirable
answer or is emotionally charged.
17Guidelines For Avoiding Mistakes (cont.)
- Avoid ambiguity Be as specific as possible
- Probably /Definitely Sometimes/Always, etc.
- Avoid double-barreled items
- Double-barreled question may induce bias
because it covers two or more issues at once. - Do you think the President is responsible for the
federal government shut-down and the currently
rising gasoline prices? Yes No - Avoid making assumptions
- Given Macys skill-level at gift wrapping, .
- All-inclusive response alternatives
- Avoid taxing respondents memory
18Avoid Common Wording Mistakes in Questionnaire
Design
19Order Bias
- Question Sequence
- Order bias
- Bias caused by the influence of earlier questions
in a questionnaire or by an answers position in
a set of answers. - Funnel technique
- Asking general questions before specific
questions in order to obtain unbiased responses. - Randomized Presentations
- Used in electronic questionnaires, but rarely
used in printed questionnaires due to coding
difficulties. - Randomized Response Techniques
- Randomly assigning respondents to answer either
the question of interest (embarrassing) or a
mundane unembarrassing question.
20Survey Flow
- Survey flow
- The ordering of questions through a survey.
- Breakoff
- A respondent who stops answering questions before
reaching the end of the survey. - Filter question
- A question that screens out respondents who are
not qualified to answer a second (or follow-up)
question. - Branching
- Directing respondents to alternative portions of
the questionnaire based on their response to a
filter question.
21Survey Flow for Eurocar Tour de France
Sponsorship
22Telephone Questionnaire with Skip Questions
23Survey Technology
- Physical Features
- Heat map question
- A graphical question that tracks the parts of an
image or advertisement that most capture a
respondents attention. - Status Bar
- A visual indicator that tells the respondent what
portion of the survey he or she has completed. - Prompting
- Informs the respondent that he or she has skipped
an item or provided implausible information. - Piping Software
- Allows question answers to be inserted into later
questions.
24Tracking Points of Interest Using a Heat Map
25Illustration of Status Bar Prompts
26Pretesting Revising Questionnaires
- Pretesting Process
- Seeks to determine whether respondents have any
difficulty understanding the questionnaire - whether there are any ambiguous or biased
questions. - Preliminary Tabulation
- A tabulation of results of a pretest to help
determine whether questionnaire will meet the
objectives of the research.
27I Give Up!
- Large portion of respondents give up before
finishing abandon the survey (break-offs). - Guidelines
- Visually appealing easy to read
- Fewer questions per page (no more than 20)
- 4 pages maximum for consumers
- 6 pages maximum for business leaders
- Question order (funnel vs. reverse-funnel)
- Sensitive questions open-ended questions
encourage break-offs /or item non-response - Sophisticated samples increase response rate
- Make information requests legitimate
- Pretesting is important (pretest EVERYTHING)
28Questionnaire Reproduction(if mail survey)
- Professional appearance
- Booklet format for long questionnaires
- Place directions as close to questions as
possible - Expect reproduction errors