Methodology and Explanation XX50125

1
Methodology and ExplanationXX50125

• Lecture 3 Interviews and questionnaires
• Dr. Danaë Stanton Fraser

2
Lectures

• Introduction to Methods
• Experiments
• Interviews and questionnaires
• Usability testing
• Ethnography

3
Why Interview?

• Good for obtaining information about
• Opinions
• Feelings
• Goals (organisational, personal and system)
• Things you didnt know you needed to know
• Bad for gathering (large volumes of) data

4
Interview Preparation

• Decide objectives
• Read background material
• Decide whom to interview
• Set up the interview
• Decide interview structure
• Write down questions you will ask and pointers
  for questions you might ask

5
Open Questions

• E.g. How do you find using this technology in
  your lectures?
• Little control
• Hard to analyse
• Hard to keep focussed on interview objectives
• May unsettle interviewee
• Time consuming and can be difficult to analyse
• However
• May gain more depth, rich data
• May tell you things you had not considered
• May give you ideas for future projects

6
Closed Questions

• E.g. How many courses do you teach?
• How many times have you used this technology?
• May settle interviewee
• Easy to analyse
• Easy to maintain focus
• Observation and use of data may be more efficient
• However
• Data may not be as rich

7
Question Pitfalls

• Leading questions
• How bad is the current system?
• Double questions/poor structure
• How often does the system fail, and how do you
  fix it when it does?
• Stay focussed. If you are unlikely to be able to
  make use of the answer dont ask the question!

8
Question Structure

• Pyramid
• Start with closed questions, finish with open
• Good for settling nervous interviewees as answers
  are easy early on
• Funnel
• Start with open questions, finish with closed
• Good for aggressive interviewees as opinions are
  asked for early on
• Diamond
• Closed -gt Open -gt Closed, Good all-round
  technique
• Structured/unstructured
• All interviews may vary depending on how focused
  you intend to be on the prepared structure

9
Starting the Interview

• Dress
• Greeting first time
• Who you are
• Why you want to interview them
• What objectives you want to achieve from the
  interview
• That the information they give is confidential
  and how it might be used
• Subsequent greetings should discuss an issue from
  or summarise the previous interview
• Dont just be yourself

10
During the Interview

• Introduce each line of questioning
• Probing
• E.g. Why?
• Shows that you are listening
• Good for keeping interview focus
• Double-check information and notes
• Taking notes
• Not too many
• Maintain eye contact
• Go back over important points
• Use of sound recorders

11
Focus Groups

• Normally 3-10 people
• A representative sample of typical users
• Enables people to put forward ideas in a
  supportive environment
• Facilitator guides and prompts
• Discussion recorded
• Low-cost but need good facilitator

12
Questionnaires

• For wider coverage
• Need to know the right questions (through
  interviews and background)
• Focus on objectives
• Dont ask unnecessary questions
• Keep it short and facilitate response
• Open and Closed questions
• Scaling

13

• Likert Scale
• Short statements about the features to be
  evaluated
• Divide the items into groups with same number of
  positive and negative statements in each group
• Decide on scale even or odd, number of points

14
Questionnaire difficulties

• The halo effect
• Respondent answers everything the same by not
  differentiating between questions
• Mix questions up on different subjects
• Leniency
• Some respondents rate everything too well
• Move the average category left (or right) of
  centre and make scale descriptors stronger
• Central tendency
• Respondents rate everything as average
• Increase the number of points on the scale and
  adjust the scale descriptors to cover a smaller
  range of opinions

15
Alsoevaluation during active use

• System refinement as a result of experience or in
  response to changes in users
• interviews and focus group discussions
• continuous user-performance data logging
• look at both frequent and infrequent error
  messages
• analyse sequences of actions to suggest
  improvements or new actions
• BUT respect peoples rights and consult them
  first!
• User feedback mechanisms
• on-line forms, email and bulletin boards
• workshops and conferences