Qualitative methods

1
Qualitative methods

• Examining approaches

2
Qualitative Methods

• Rely on researcher as instrument
• Introduces certain issues
• Inherent subjectivity and bias
• Consistency in data collection and interpretation
• Impact of researcher on environment (esp. when
  studying people)
• How do we control for reliability and validity?

3
Qualitative Methods
4
Qualitative Methods

• Interacting with people/environments

5
Case Study

• In-depth review of a single situation program,
  process, phenomenon
• Might be descriptive or explanatory in approach
• Might also form the basis of theory development,
  or modification of existing theories
• Case studies are usually built on one or more of
  the following methods.

6
Qualitative Study Observation

• Attempt to understand activities/ behaviors
• Role of the observer
• Complete participant
• Effects on the environment/process
• Genuine or not?
• Going native
• Present self as participant not researcher
• Ethical implications?
• Observer and participant
• Effects on process?

7
Observation

• Complete observer
• Lessen effect on environment (perhaps)
• Less likely to fully appreciate
  situations/contexts
• Other choices
• How much time/frequency on-site
• Focus- comprehensive or narrow
• Issues
• Relations to subjects

8
Qualitative Study Interviews

• Attempt to gain in-depth knowledge on a topic
• In a less structured format, interviewer may act
  more as facilitator, asking open ended questions
  and drawing the participant out.
• Structured, semi-structured, or unstructured?
• Format- in person, telephone, email?

9
Interviews

• Types of questions
• Experience/ behavior
• Feeling
• Opinion/ value
• Knowledge
• Sensory
• Demographic

10
Interviews

• Issues
• Logistics
• Building trust
• Delicate questions/ situations- ensuring honesty
• Accuracy of transcripts
• Advantages and disadvantages?
• Appropriate uses?

11
Qualitative Study Focus Groups

• Attempt to learn about the attitudes/ beliefs/
  feelings of groups and how those influence
  behavior
• Why groups? One individuals comments can
  trigger important responses from others.
• Can explore large ranges of topics

12
Focus Groups

• Issues
• How structured?
• Importance of facilitators role- how well
  trained?
• Building trust
• Finding reliable representative volunteers
• Ensuring accuracy of transcription- audio/ video
  tape? Outside observation?
• Facilities

13
Qualitative Study Think-Aloud

• Asks respondent to verbalize their thoughts while
  performing an assigned activity or task
• Attempt to gain more insight into thought
  processes
• Correct for mistakes/ assumptions of observer

14
Think Aloud

• Requires participants that are highly verbal in
  nature
• Adding this layer of feedback may affect their
  cognitive processes and behaviors- more
  self-concious.
• How to transcribe while observing? Accuracy?
• Appropriate use of clarifying questions?
• Appropriate uses?

15
Qualitative Study Ethnographic

• Tools developed in the field of cultural
  anthropology
• Attempt to better understand people/ behavior by
  observing it within natural setting
• Can study observable material items, individual
  behaviors and performances, or ideas
• Requires careful attention to detail within
  cultural context

16
Ethnographic

• Possible Tools
• Observation in context
• Key informant interviewing
• Drawing pictures
• Taking photos
• Using maps to track activities
• Videotaping

17
Ethnographic Examples

• Susan Gibbons and Nancy Fried Foster of
  University of Rochester Understanding Users to
  Develop Better Library Services (ACRL/NEC 2006)
• User-centered studies of library use
• Mapping Diaries
• Photo Elicitation Interviews

18
Photo Elicitation
19
Mapping Diaries
20
Qualitative Study Delphi Method

• A systematic interactive forecasting method.
• Involves interviewing/ surveying of experts
  within a field- generally 9 to 99
• Experts are provided with hypotheses, trigger
  statements, scenarios, etc. and asked to respond.
• First round responses are shared with the group
  anonymously, so participants can revise earlier
  statements, react to responses.

21
Delphi Method

• After several rounds, hypotheses may be refined,
  group may reach consensus
• Facilitator reviews responses to each round- can
  filter out irrelevant content, choose
  presentation of information, ask questions.
• Advantages and disadvantages?

22
Qualitative Studies

• Interacting with Artifacts

23
Types of Documents?

• Policy Manuals
• Digital reference transcripts
• Comment/ complaint cards
• Job ads
• Published Literature
• Blogs, listserv postings, etc.
• Open-ended responses

24
Evaluation Research

• Published
• Books
• Journal articles
• Conference proceedings
• Theses/dissertations
• Web
• ALA. ACRL. CIP Notes
• CLIR. www.clir.org/pubs/pubs.html
• ARL
• Other
• Peer Review

• Unpublished
• Internal reports
• Local use only reports

25
Qualitative Analysis Examining Documents

• Focus on
• Word choice
• Word frequency
• Word sequence
• Intensity of feeling/ expression (how to
  measure?)
• Key word in context
• Typology of concepts/ categories?

26
Qualitative Examining Documents

• Systematic review of text/ images content
  analysis
• Classifies textual or visual material
• Uses analytical constructs or rules to draw
  inferences about recurring aspects of text.

27
Qualitative Study Diaries

• Participants record activities, thoughts,
  reactions, etc. daily (weekly, etc.) over a set
  period of time.
• Blogs- equivalent?
• When to use?
• Issues
• How faithful are participants? How can you help
  ensure full participation?
• How much guidance to give?
• Too much/ too little detail.
• Advantages and disadvantages?

28
Examples

• Nicole Henning/ Photo Diary Study (MIT)
• 16 students tracked their information seeking
  behavior for one week.
• Used diaries and screen shots to record their
  thoughts and actions
• Hernon et. al.
• Track library directors to identify leadership
  and management behaviors
• Used diaries to record activities for two months

29
Grounded Theory

• The attempt to derive theories from an analysis
  of patterns, themes, and common categories
  discovered in observational data.
• Relies on a systematic set of procedures (i.e.
  systematic coding, reliability and validity)

30
Qualitative Study Citation Analysis

• Systematic review of bibliographies/ references
  within published literature.
• Focus on
• Authorship
• Form of publication (periodical, monograph, etc.)
• Class of material (primary, secondary, etc.)
• Language
• Subject
• Currency

31
Citation Analysis

• What can citations tell us?
• How scholarly is the cited literature?
• How current are the citations?
• How research-oriented is it?
• How interdisciplinary is it?
• How writes the literature? How collaborative?
• Where does the literature appear?

32
Citation Analysis

• Growth of literature on a subject
• Dispersion of writings on a subject across form
  and journals
• Obsolescence of literature
• Scholarly networks who cites whom?
• Publishing productivity

33
Citation Analysis

• Advantages
• Shows what is cited
• Does not involve interaction with subjects
• Profiles a literature
• Shows changes in a field over time?

• Disadvantages
• How complete is the work from which citations are
  drawn?
• How accurate are citations?
• Are all materials cited?
• Choosing easy to find/ retrieve items over better
  quality?
• How easily retrievable are works?

34
Citation Analysis Issues

• From where are citations drawn? How far can you
  generalize findings?
• Does citation reflect use?
• Self-citations and/ or gratuitous citations
• Half-life
• Impact factor (to what extent are recent articles
  in journals cited?)
• Calculated annually divide the number of current
  citations a journal receives to articles
  published in the previous two years by the number
  of articles published in those same years

35
Web Citations

• How prestigious are different online sources-
  ejournals, open access, etc.
• References to and from a Web site
• Retrievability

36
Sweeping Studies

• A type of spatial data analysis
• Useful for mapping out the physical spaces of a
  library and investigating how people use those
  spaces

37
(No Transcript)
38
Qualitative Study Concept Mapping

• Any process that represents ideas in pictures of
  maps.
• A method of organizing the ideas and thoughts of
  a group to form a common framework
• Can be used to integrate ideas from less
  structured activities such as brainstorming

39
Concept Mapping

• Typical steps
• Focus determine desired outcomes/ questions to
  be addressed
• Generating ideas Brainstorm, use trigger
  statements, ask questions
• Analysis Sort ideas into large sets
• Unstructured idea sorting ask individuals to
  sort ideas into groups and label
• Sorting by stakeholders organize ideas by group
  that generated those ideas
• Ratings assign values to ideas (importance,
  feasibility, etc.)

40
Concept Mapping

• Map analysis generate map based on idea sorting
  (could use statistical software, or do by hand)
• Interpret map Share with other groups to obtain
  understanding

41
Example IR and IL
42
Qualitative Data

• Collection and Analysis

43
Qualitative Data

• Systematic collection of data is important
• Recording observations
• Separating fact from feelings and impressions
• Empirical observations and interpretation
• Deep, thick description
• Dont rely on memory- record as much as possible
• Take notes in stages

44
Qualitative Data

• Validity
• Field research tends to be more valid than
  experimental or survey measures. Why?
• Reliability
• Researcher is the tool- biases and assumptions
  can influence analysis
• Helps to identify and acknowledge those biases
  and assumptions beforehand, be aware of them
  throughout

45
Qualitative Data Analysis

• Interplay of theory and analysis- rechecking data
  against existing theories/modes of thinking
• Discovering patterns
• Frequencies
• Magnitudes
• Structures/Types
• Processes
• Causes
• Consequences

46
Qualitative Data Analysis

• Grounded theory- generating theory solely from
  examination of data
• Usually progresses in four stages (constant
  comparative method)
• Comparing incidents applicable to each category
• Integrating categories and their properties
• Delimiting the theory
• Writing the theory

47
Qualitative Data Coding

• Coding- classifying or categorizing individual
  pieces of data
• Choosing a concepts to code
• Physically coding- by hand or with software
• Creating codes
• Open coding- naming and categorizing phenomena
  through close analysis of the data. Categories
  arise from the data

48
Qualitative Data Coding

• Memoing- writing notes to yourself as you code
  data
• Code notes- identify code meanings and labels
• Theoretical notes-reflections on meanings,
  relationships among concepts, theoretical
  propositions, etc.
• Operational notes- notes on methodological
  issues, data-collection circumstances, etc.
• Sorting memo- attempt to discover or create
  meaning in the data
• Integrating memo- ties it all together