BS1037 Research Methods and Statistics in the Public Sector

1
BS1037 Research Methods and Statistics in the
Public Sector

• Principles of Questionnaire Construction

2
BS1037 Research Methods and Statistics in the
Public SectorPrinciples of Questionnaire
Construction

• Choice of Data Collection Methods
•  
•  

Questionnaires
Structured Interviews
High N Low
Unstructured Interviews
Observation
Participant Observation
LOW Personal
Involvement HIGH
3
BS1037 Research Methods and Statistics in the
Public Sector

• Questionnaires maximise on representativeness
  when numbers are high 
•  
• Participant observation maximises 'ecological
  validity' i.e. the data is likely to be 'richer'

4
BS1037 Research Methods and Statistics in the
Public Sector

• Questionnaire Construction 1a
•  
• Clearly identify the origin of the questionnaire
• Not too LONG or too SHORT
• Use FILTER questions
• Give CLEAR instructions on each page (and, if
  necessary, for each QUESTION)
• Keep number of categories consistent with the
  sample size

5
BS1037 Research Methods and Statistics in the
Public Sector

• Questionnaire Construction 1b
•  Think about future data 'collapse' i.e.
• 4 categories may be better than 5
• Use 'open-ended' questions sparingly
• Design the form for easy 'data transfer
• Accompany by explanatory letter /id of interviewer

6
BS1037 Research Methods and Statistics in the
Public Sector

• Questionnaire Construction - 2
•  
•   Think about what the final report will look
  like when composing the questionnaire..
• Nominal variables (e.g. sex) will generate
• frequency distributions
• contingency tables

7
BS1037 Research Methods and Statistics in the
Public Sector

• Frequency Distribution
• CLASS Social Class
  File MYSURVEY.TXT
•  
• 
  Valid Cum
• Value Label Value Frequency
  Percent Percent Percent
•  
• Professional 1 28
  14.0 14.6 14.6
• Intermediate 2 16
  8.0 8.3 22.9
• Skilled Manual 3 24
  12.0 12.5 35.4
• Semi-skilled Manual 4 32
  16.0 16.7 52.1
• Unskilled Manual 5 46
  23.0 24.0 76.0
• Pensioners 6 18
  9.0 9.4 85.4
• Not classified 7 28
  14.0 14.6 100.0
• 0 8
  4.0 MISSING
• -------
  ------- -------

8
BS1037 Research Methods and Statistics in the
Public Sector

• Professional 28
• Intermediate 16
• Skilled Manual 24
• Semi-skilled Manual 32
• Unskilled Manual 46
• Pensioners 18
• Not classified 28
•  
• Valid Cases 192 Missing Cases 8

9
BS1037 Research Methods and Statistics in the
Public Sector

• Contingency table
• Crosstabulation of SEX Sex of
  Individual File MYSURVEY.TXT
• By CLASS Social Class
•  
• CLASS gtProfes Interm Skille Semi-s
  Unskil Pensio Not cl ROW
• sional ediate d Manu killed led
  Ma ners assifiTOTAL
• SEX 1 2 3 4
  5 6 7
• ----------------------------
  --------------
• Male 1 16 12 10 16
  22 2 20 98
• ----------------------------
  --------------
• Female 2 12 4 14 16
  24 14 8 92
• ----------------------------
  --------------
• TOTAL 28 16 24 32
  46 16 28 190
• 14.7 8.4 12.6 16.8
  24.2 8.4 14.7 100.0
•  
• Valid cases 190 Missing 10
•  

10
BS1037 Research Methods and Statistics in the
Public Sector

• Questionnaire Construction - 3
•  
•  Continuous variables (e.g. income) will generate
  univariate/bivariate statistics
• ( means, standard deviations, plots,
• Correlation and regression)

11
BS1037 Research Methods and Statistics in the
Public Sector

• File MYSURVEY.TXT PAPER
  SALARY
•  
• Measures of Central Tendency
• Mean 4.299
  10.500
• Median 4.000
  10.000
• Mode 3.000
  15.000
•  
• Measures of Dispersion
• Minimum 1.000
  6.000
• Maximum 8.000
  15.000
• Range 7.000
  9.000
• First Quartile 2.000
  7.000
• Third Quartile 7.000
  13.000
• Semi-Interquartile Range 5.000
  6.000
• Variance 5.474
  9.523
• Standard deviation population 2.340
  3.086
• Standard deviation sample 2.346
  3.094

12
BS1037 Research Methods and Statistics in the
Public Sector

• Measures of Distribution Shape
• Skewness 0.164
  0.013
• Kurtosis -1.268
  -1.379
• Numbers of Cases
• N 174
  198
• Missing Values 26
  2
• N (valid pairs) 172
•  
• Bi-variate Statistics
• Correlation r -0.0597 t
  -0.780 p 1.5637
• Regression y (SALARY ) 10.787 -0.077
  x (PAPER )
•  
• T-Test (difference in means) t 21.927 D.F.
  362.46 p 0.0000  

13
BS1037 Research Methods and Statistics in the
Public Sector

• Asking Questions
• Keep simple, clear, concise
• Ask only question at a time
• Avoid 'leading' questions ('Have you
  stop beating your spouse?')
• Avoid negative questions
• Only use unambiguous words ('Tea', 'Dinner'
  mean have different meanings)
• Is the question too precise ?
• (e.g. do we need only an income BAND )

14
BS1037 Research Methods and Statistics in the
Public Sector

• Asking Questions
• Note the different types of questions
• tick a box
• supply a value
• provide a ranking (e.g. 1-5)
• provide a SCALING (e.g. 1-10)
• Multiple choice/tick
• 'Open-ended' question

15
BS1037 Research Methods and Statistics in the
Public Sector

• Maximising Response -1
•  
•  The accompaning letter should explain the survey
  to invite co-operation
• And be signed by as prestigious a person as
  possible
• Confidentiality should be stressed
• Include a stamped addressed envelope
• when necessary

16
BS1037 Research Methods and Statistics in the
Public Sector

• Maximising Response-2
•  Indicate how and by when the completed
  questionnaires should be returned
• Consider (symbolic) incentives
• Indicate the results will be made available to
  persons who complete the questionnaire
• Send 'chasers' and 'reminders' where necessary
  (including another copy of the questionnaire)
• Piloting the questionnaire may reveal
  difficulties

17
BS1037 Research Methods and Statistics in the
Public Sector

• Exercise
• Compose a journey to work
  questionnaire which incorporates
• Filters
• Categorical variables
• Continuous variables
• A 'ranking' question
• A 'scaling' question
• An 'open-ended' question
• as well as all of the other principles
  of good questionnaire design.