Capturing the right information: using qualitative techniques confidently

1
Capturing the right information using
qualitative techniques confidently

• Matthew Terry, York Consulting

2
Timetable

• 10.00 Welcome and introductions
• 10.10 Qualitative research what it is
• 10.30 Qualitative methods how to do it
• 11.15 Break
• 11.30 Practical use in evaluation - trying it out
• 12.30 Reflections and questions
• 13.00 Close

3

• Introductions
• Name and role
• Prior experience of qualitative research
• What do you think of this proposed new arts
  building?

4
Objectives for the day

• Develop a broad understanding of qualitative
  research, and where it may be useful -
  particularly in evaluation
• Become familiar with the main types of
  qualitative research methods, their benefits and
  limitations
• Gain confidence in using, commissioning and
  analysing qualitative research

5
Qualitative research what it is

• In this section, we will
• Consider definitions of qualitative research
• Compare and contrast qualitative and quantitative
  information
• Identify where qualitative research might be
  useful

6
What is it?

• What is qualitative information (or what isnt
  it)?
• Quantitative hard, rigorous, measurable,
  scientific
• How many people attended this seminar?
• Qualitative soft, feelings, emotions, context,
  behaviours, pictures, detailed, non-numerical
• In what ways have people found this seminar
  useful?
• The concept of researcher as instrument
• Dont get too bogged down in definitions
• The boundaries are not that clear-cut
• Gather the most appropriate evidence for your
  needs

7
Why use it?

• Advantages over quantitative information
• Richness of information
• Depth of understanding
• May be more suitable for certain topics
• The value of partnership working
• The extent of cultural change in the workplace
• Shifts in behaviour within a community
• Changes in complex policy processes
• Challenges
• Validity and credibility
• Primary and secondary information

8
Qualitative methods how to do it

• In this section, we will
• Look at the main qualitative methods
  surveys/questionnaires interviews focus groups
  and observation
• Compare the methods, their relative strengths and
  weaknesses
• Consider elements of best practice in their use

9
Surveys questionnaires

• Possible use for qualitative purposes
• Open ended questions
• But not too many time and collation
• How to survey
• Face-to-face
• Telephone
• email/online
• Who to survey
• Everyone, if time and costs allow
• Otherwise, a sample is necessary

10
Sampling a brief overview

• Questions to ask when deciding on a sampling
  method
• What resources do we have?
• How good is the data?
• How accurate do we have to be?
• How can we ensure the sample is representative?
• Types of sampling
• Random pure, systemic and stratified
• Non-random quotas, judgements and convenience
• Also relevant for other qualitative research
  methods

11
Interviews

• Unstructured, semi-structured, or structured?
• Interviews go well when
• the interviewer is prepared
• the environment is right
• a rapport is quickly established
• the interviewer listens actively
• the interviewer makes quick decisions
• Problems arise when
• leading questions are asked
• no/poor notes are taken
• it turns into a counselling session
• the interviewer becomes opinionated

12
Focus Groups

• Organised discussion
• One session, many views
• influenced by participant interaction
• Issues
• the importance of the moderator
• managing the atmosphere
• ethical considerations
• gathering and reporting the evidence

13
Observation

• Assumes what youre interested in can be observed
• Useful for researching sensitive subjects
• Issues
• Can be time-consuming
• Your presence may affect participant reactions
• Ethical considerations
• Recording information

14
Comparing methods

• Survey
• Interview
• Focus Group
• Observation
• Case study

• Representativeness
• Cost
• Time
• Replicability
• Researcher expertise

15
Good practice

• Avoid mission creep stay focused
• Allow plenty of time to draft research
  tools...and analyse results
• Pilot if time and resources allow
• Combine methods and approaches
• Case studies
• Triangulation
• Longitudinal research
• Good record keeping
• Clear questions not leading, not composite
• Encourage and facilitate high levels of
  participation
• Choose the most suitable practical method for
  your needs

16
Putting it into practice

• In this section, we will
• Place qualitative research in the context of
  evaluation
• Run through examples of how qualitative research
  might be planned into an evaluation
• Consider how to analyse qualitative evidence and
  other peoples research

17
Qualitative research in evaluation

• Evaluation different perceptions
• Audit have you spent the money properly?
• Satisfaction survey are participants happy?
• Impact/Value for money have the outcomes been
  worth it?
• Best practice what have we learned, how can we
  improve?
• External evaluations
• May be required
• May be useful
• Designing an evaluation approach
• Must be considered at the outset
• Ensure the right information is available

18

• Activity, process, etc.
• Interim/formative or final/summative?
• Establish the broad framework
• Who is the audience?
• Consider data sources
• What is already available, what will need to be
  collected?
• Identify appropriate research methods
• Bear in mind resources
• Draft research tools

What are we trying to evaluate?
What are the key research questions?
What evidence do we need?
How will we gather the evidence?
What specific questions must be asked?
19
Analysing the results

• Electronic data is easier to work with
• Specific software for qualitative analysis is
  available
• Excel can be a useful tool
• Identify common messages, words and phrases
• Use direct quotes with care
• Quantification may be possible
• Outliers can be important, but dont
  over-emphasise
• Look out for connections, formulate ideas and
  explanations

20
(No Transcript)
21
A different film?

• The film is, by common consent, not merely a
  turkey, but a squawking, gobbling prize Christmas
  clunker of a bird The Guardian
• "Ritchie tells his story so incoherently and
  indolently - and with such an alienating air of
  self-congratulation - that Revolver becomes first
  annoying, then exasperating, and eventually
  unbearable" Daily Mail
• "so incredibly, absurdly, breathtakingly awful,
  I wanted to buy a revolver so I could shoot the
  projectionist before turning the gun on myself
  Daily Mirror

22
Reflections and questions
23
A reminder of our objectives

• Develop a broad understanding of qualitative
  research, and where it may be useful,
  particularly in evaluation
• Become familiar with the main types of
  qualitative research methods, their benefits and
  limitations
• Gain confidence in using, commissioning and
  analysing qualitative research

24
Other sources of information

• The Magenta Book
• The Green Book
• Regen WM resources
• Social Research Update
• Social Research Methods Knowledge Base
• Research Observatory