Questionnaire Construction

1
Questionnaire Construction

• Assessment Team
• November 6, 2007
• PowerPoint by JD White and Joel H. Scott

2
When using a Questionnaire

• Think about the purpose of your assessment.
• Then, think about the data you want to collect.
• Next, ask yourself if a questionnaire is the
  right tool for the job.
• Remember, good questionnaires take time to
  develop. They are not just questions on a page.

3
Steps for Developing Questionnaires

• Develop the Purpose

• Create the Conceptual Framework

• Write Questions

• Design the Questionnaire

4
Developing the Purpose

• This step likely started with your assessment
  plan.

• Think about what, specifically, you want to know,
  why a questionnaire is appropriate, and what you
  want to do with the data.

• Your purpose will assist in keeping you focused
  while developing your questionnaire.

5
Creating the Conceptual Framework

• This is a multi-step process and will include a
  great deal of brain storming.

Step 1 List all the questions/areas you might
want to know
Step 2 Organize these questions into categories
and subcategories
Step 3 Define these categories
Step 4 Make sure that you are asking items that
will provide enough data to fully answer your
research assessment questions.
6
Can you name the Seven Wonders of the World?
Did you start to list the Seven Wonders of the
World or did you just say yes or no? The
most appropriate answer would be either yes or
no. Question wording matters because it will
affect the data you will collect.
7
Drafting Questions

• This step can be challenging because it asks us
  to rethink how we ask questions.

Questions have two distinct parts.
1. Question Wording
2. Response Format
Tip Think about why you are asking a question
and if you really need the information.
8
Question Wording

• Be as short and simple as possible.

• Ask one question at a time.

• Dont use jargon and the negative wording of a
  question.

• Define/clarify items as needed.

9
Question Wording cont.

• Respondents dont have the best memories, so be
  specific and try not to ask about generalities.

• Be careful when asking threatening questions.

10
Response Formats

• Open response formats dont limit/influence
  responses, but they can create data management
  issues.

• Use open-ended responses strategically.
• Before asking specific questions
• As summary for missed information

Provide labels where possible Ex. How many
days? ___days
11
Response Formats

• Closed response formats can limit/influence
  responses, but they provide some control.

Categories Use to measure factual information
and behavior. Make sure options are exhaustive
and mutually exclusive.
Ordinal and Interval Use the items to measure
the intensity of attitudes or opinions and think
about what scaling you should use (e.g., Likert,
Uni-Dimensional, Semantic Differential).
12
Issues to Consider

• To Neutral or Not to Neutral.

Think about how a respondent enters a response
scale and how many decisions the person must
make.
13
Issues to Consider cont.
Pre-coding can save time with data entry.
Pre-code
There is more than one way to ask a question. Use
a variety of question formats.
14
Types of Questions

• Yes/No

Uni-dimensional
Semantic Differential
15
Designing the Questionnaire

• Create a conversational tone in the opening text
  and throughout the questionnaire.

• Begin with easy questions first and more to
  more challenging questions.

• Limit demographic questions to only those that
  you need.

• Think about placing demographic questions at the
  end if possible.

16
Designing Questionnaires

• Ask about behaviors before attitudes or opinions.

• Create sections to break up longer
  questionnaires.

• Use page space effectively.

• Design can make a difference.

17
Formatting the Questionnaire

• Margin size, layout, and font size can make a
  big difference.

• Tables are a good tool for helping to format
  your questionnaire.

• Bullets and symbols can provide response
  circles, boxes, or other options.

• Shading can assist the respondent in completing
  the questionnaire.

18
What do you think?

• Were you satisfied with the quality of our
  services and programs?
• A. Yes
• B. No
• Double Barrel Ask only one question at a time

19
What do you think?

• What brand of television do you own?
• A. Samsung
• B. Panasonic
• Too exclusive What about other TVs?
• What about the use of Other?

20
What do you think?

• In the past two years, approximately how many
  times have you been to the gym?
• A. 1-10 times
• B. 11-20 times
• C. 21-30 times
• D. More than 30 times
• Can you recall things from 2 years ago? What
  about the past two weeks?

21
What do you think?

• Where did you grow up?
• A. country
• B. farm
• C. city
• Confusing Response Options What if I live on a
  farm?

22
What do you think?

• Are you in favor of Proposition 13?
• ____ Yes
• ____ No
• ____ Undecided
• Assumptive Question Does everyone know what
  Prop 13 is? Also, what if you dont know?

23
What do you think?

• What percent of your annual budget do you spend
  on advertising for student programs?
• ______
• Too difficult You dont want your respondents
  to pull their hair out.

24
What do you think?

• Dont you think this administration is spending
  too much money in the Iraq war?
• A. Yes
• B. No
• Leading Question Be careful of sharing your
  agenda.

25
What do you think?

• What income tax bracket do you report under to
  the IRS?
• A. under 30,000
• B. between 30,000-99,000
• C. 100,000 to 500,000
• Threatening question/Be careful of using
  abbreviated terms-spell them out!

26
What do you think?

• 1. Do you currently own a vehicle? If no, go to
  question 3.
• A. Yes
• B. No
• 2. How many miles do you drive on a daily basis?
  ______
• Be careful of BRANCHING Qs

27
Why not this

• How many miles do you drive on a daily basis?
• A. none, I do not own a vehicle
• B. 1-10 miles
• C. 11-20 miles
• D. 21 or more miles
• E. Other________________miles

28
What do you think?

• What do you think about this report?
• A. Its the worst report Ive read
• B. Its somewhere between the worst and best
• C. Its the best report Ive read
• Little variability in responsesHow are you going
  to measure this?

29
Why not this

• Please respond to the degree you agree or
  disagree with the following statements.
• I think this report is well written.
• I think this report should be shared with
  everyone in the company.
• I think this report has the potential to improve
  working conditions.

30
Questions?