Descriptive Data Analysis: Analyzing Survey Data

1
Descriptive Data AnalysisAnalyzing Survey Data

• Research Methods for Public Administrators
• Dr. Gail Johnson

2
At Long Last Analyzing Surveys

• Surveys Using Percent Distributions
• Key Topics
• Handling Exits
• Analyzing 5-point scales
• Extreme analysis
• Handling the middle categories

3
Guidelines for Exits

• Exit options on scales
• dont know
• not applicable
• no opinion
• Handling Exits no firm rules

4
Guidelines for Exits

• Key decision leave in or exclude from analysis?
• If the number of people taking an exit is very
  small, there will be little difference in the
  percent distribution so the researchers only need
  to inform the reader whether they are included or
  excluded

5
Handling Exits

• If there is a high percent of exits, does it make
  sense? If it does, exclude from the analysis
• For example, if you ask people how satisfied or
  dissatisfied they are with the zoning office, a
  high percent might say not applicable.
• That might make sense because probably only a
  small proportion of citizens contact the zoning
  office in any given year.
• The not applicable responses could be excluded,
  and the analysis should focus on the percent
  distribution who actually answered in the
  satisfieddissatisfied scale.

6
Handling Exits

• If, on the other hand, there is a high proportion
  of people taking the exits for reasons that are
  not clear, the researchers might want to spend
  some time to figure that out.
• It could be a sign that there was a problem with
  the question.
• My best adviceas a general rulewould be to
  exclude the exits from the analysis, and focus on
  only those who answered within the scale.

7
Handling Exits

• Make sure the analysis is anchored in terms of
  the number of people who answered within the
  scale.
• Of the 300 people who indicated their level of
  satisfaction, 65 reported being very or somewhat
  satisfied.

8
Analyzing Survey Scales

• Decision rules for 5-point scales
• Very Satisfied
• Somewhat satisfied
• Neither
• Somewhat dissatisfied
• Very dissatisfied

9
Analyzing Survey Scales

• Decision rules for 5-point scales.
• I first look at where a majority (50 or more)
  answered either on the positive or the negative
  side of the scales.
• Very satisfied and somewhat satisfied.
• Very dissatisfied and somewhat dissatisfied.
• It allows me summarize the set of questions
• Of the 5 questions about the city services, a
  majority reported being very satisfied or
  somewhat satisfied with library services,
  recreation department and the fire department.

10
Analyzing Survey Scales Extreme Analysis

• If a majority answered positively or negatively
  on many questions, I then do an extreme
  analysisthat is, I look for questions where a
  majority answered very satisfied or very
  dissatisfied.
• It is unusual to get a majority at the extreme
  end of scale, so that information shows the
  greatest intensity of feeling about the issue.

11
Exercise Whats The Story About Faculty
Performance?
12
Exercise Whats The Story?

• What are students most satisfied with?
• What are students least satisfied with?
• Whats the story the data tells?

13
Analyzing Survey Scales

• A majority was very satisfied or generally
  satisfied with all four aspects of faculty
  performance.
• However, a majority (54) were very satisfied
  with faculty knowledge.
• Part of analyzing the data requires identifying
  the key elements of the story.
• I would highlight faculty knowledge as being a
  strong aspect of faculty performance according to
  those completing the survey.

14
Handling the Middle Category

• As you know, I like uneven scales (scales with 5
  or 7 choices).
• For example, Likert scales have a neutral middle.
• Neither agree or disagree.
• Combining the neither category with the agree
  side or the disagree side clearly does not make
  any sense.

15
5-point Scale Without a Neutral Middle (one-way
scales)

• ___ Almost Never/Never
• ___ Seldom
• ___ Occasionally
• ___ Usually
• ___ Almost Always/Always
• Note a graduated one-way scale with soft-ends

16
Handling the Middle Category

• Another common one-way scale
• Very great extent
• Great extent
• Moderate extent
• Some extent
• Little of no extent

17
Handling the Middle Category in One-way Scales

• It is possible to combine categories when using a
  scale without a neutral middle,
• It could be honestly reported that 90 percent of
  the respondents rated the city services as
  helpful to at least some extent.
• However, the researcher is rolling 4 of the 5
  possible categories togetherand raises a
  question of whether the researcher is trying to
  hide the detail. Maybe very few respondents
  provide favorable ratings.

18
Handling the Middle Category in One-way Scales

• My advice the middle category stands alone, even
  though it is not neutral.
• It is too easy to distort the data by combining
  the middle category with either side of the scale
  to make it say what the researcher wants.

19
Handling the Middle Category in Goldilocks Scales

• Does the program offer too many, too few are just
  about the right number of electives
• Much too many
• Somewhat too many
• Just about right
• Somewhat too few
• Much too few

20
Handling the Middle Category in Goldilocks Scales

• The middle category is not neutral it contains
  important information.
• In fact, the program director would be hoping
  that most people say just about right
• It too stands alone!

21
Handling the Middle Category

• My rule the middle always stands alone.
• Let me repeat the middle stands alone.

22
Why Not Have Yes or No Scales?

• Or even scales?
• I prefer 5-point scales because it gives me the
  option of looking at the extreme ends of the
  scales.
• If I only ask people whether they are satisfied
  or dissatisfied, I do not know how many are
  really very satisfied as compared to somewhat
  satisfied.
• The difference in intensity might be important.

23
Why Not Have Yes or No Scales?

• I still have the option of collapsing the 2
  positive together and 2 negative sides of the
  scale together to simply reporting.
• The middle category gives a place for people to
  go if they really do not have an opinion one way
  or the other.
• Some people do not like to say they dont know,
  so they find a middle category.

24
Key Questions About Survey Results

• Make sure you know the decision rules did the
  researchers exclude the people answering dont
  know?
• Did they do an extreme analysis?
• How many categories did they roll together to get
  their results?
• Did they maintain the neutral middle?

25
Final Words on Analyzing Survey Data

• Be mindful of words like most peopledo they
  mean the majority (50) or do they mean a
  plurality (the greatest proportion but less than
  a majority).
• When working with survey data, round percentages
  to the nearest whole number (.5 and up, round up,
  less than .5, round down).
• Decimal points can give a false sense of
  precision and make it harder for people to
  remember the numbers.

26
Creative Commons

• This powerpoint is meant to be used and shared
  with attribution
• Please provide feedback
• If you make changes, please share freely and send
  me a copy of changes
• Johnsong62_at_gmail.com
• Visit www.creativecommons.org for more information