Qualitative Methods

1
MIS 650Project Seminar

• Qualitative Methods
• Introduction to Qualitative methods and Grounded
  Theory

2
Qualitative Methods
Dept. of Information Systems Doctoral Program in
IS ND

• Differ from Quantitative methods in that the data
  used is not numerical measures but words the
  goal is often not to prove relationships but to
  derive arguments that relationships exist
• Rely on interpretation
• Such methods do not depend on amassing large
  samples, using statistics, or computing or
  counting things, but assembling arguments,
  examples, cases, etc.

3
Qualitative Methods Agenda Increasing the
Quality of the Discourse
Dept. of Information Systems Doctoral Program in
IS ND

• One of the goals is to increase our knowledge of
  the world through an indirect means by
  increasing the quality of the discourse that is
  essential to knowledge generation.
• The reasoning is thus
• Better (more, more relevant, more useful)
  knowledge requires better theories. These
  theories arise from the crucible of discussion,
  debate and discourse. Improving the raw material
  of this discourse means improving the process of
  creating theories theories arise from our
  experiences, which are socially constructed.

4
Increasing the Quality of the Discourse
Dept. of Information Systems Doctoral Program in
IS ND

Impressions, Thoughts, Felt Ideas
Debate, Discussion Dialectic Discourse
Expressed Ideas
Essential Criticism
Higher Quality Ideas
Critical Commentary
Experience
5
The Flow
Dept. of Information Systems Doctoral Program in
IS ND

Stage Objects Criteria
Idea Theory
Testing Creation Development of
Theories Concepts Arguments
Data, Methods Good Ideas Good Arguments
Correct Theories Grounding Theory Strength-
Correspondence
ening to Reality
6
The Flow
Dept. of Information Systems Doctoral Program in
IS ND

Ideas arise from researchers and others
experiences. The key is to have useful
experiences. The resultant ideas are said to be
ground- ed in these experiences. We collect
case examples of the concepts and thoughts we
want to think about in or- der to express these
thoughts better. At this stage what we want is a
wide variety of experiences to be able to
ex- press ourselves better.
Idea Creation Concepts Good Ideas Grounding

Stage Objects Criteria
7
The Flow
Dept. of Information Systems Doctoral Program in
IS ND

At this stage What we desire is to have
mean- ingful dis- cussions that produce robust
argu- ments.
We do this by (1) exposing the ideas to
the strongest non- superficial tests we can
find. (2) Structuring the debate to cover as
much as possible.
Stage Objects Criteria
Theory
Development Arguments
Good Arguments
Theory Strength-
ening
8
The Flow
Dept. of Information Systems Doctoral Program in
IS ND

Testing of Theories
Data, Methods Correct Theories
Correspondence
to Reality
Only when we have strong ideas with reasonable
argu- ments can we begin to pro- pose tests of
correspondence of those ideas to the real world.
It is only at this stage that the expensive
proof procedure is worth putting into play.
Stage Objects Criteria
9
Basis of Qualitative Methods
Imaginings about this or other external or
internal worlds
Dept. of Information Systems Doctoral Program in
IS ND

Comments or interpretations or translations of
this or other worlds
Positivism assumes the world as an objective
reality to be discovered, measured and thus
understood.
Actual Events
Interpretivism assumes the world is socially (or
otherwise) constructed and this must be
understood first and only later measured and
discovered
10
Challenges of Qualitative Methods
Dept. of Information Systems Doctoral Program in
IS ND

• Data are complex, multifaceted
• Analysis is strongly influenced by data type and,
  in fact, may be contemporaneous with data
  collection
• Not the same as easy quantitative methods
• The search is for meaning
• Because the data are words, they (the data) may
  possibly speak for themselves (but not always!)

11
Where/When to Use Qualitative Methods
Dept. of Information Systems Doctoral Program in
IS ND

• Case Studies create and expose a lot of
  qualitative data
• Content analysis uses content (i.e., words)
• Grounded theory, to create theory
• Where the words of research sources can serve as
  examples, cases, or insights
• Where the researcher must interact with the
  data to derive hypotheses or conclusions

12
Uses of Qualitative Methods
Dept. of Information Systems Doctoral Program in
IS ND

• To determine the mental processes of users of IT
  in a variety of cultural settings
• To understand the value of IT use or IT artefacts
  to members of different cultures
• To understand how people fit IT into their daily
  lives in meaningful ways
• To see what strategies people engage in to
  understand IT themselves

13
Dangers of Qualitative Data
Dept. of Information Systems Doctoral Program in
IS ND

• Thin case studies
• Treating words as content, expression or naïve
  theory as data
• Drawing conclusions of a statistical nature based
  on small samples
• Researcher-driven conclusions (i.e., interpreting
  the researcher rather than the data sources
  inner states).

14
Reliability and Validity Issues
Dept. of Information Systems Doctoral Program in
IS ND

• Source Kirk, Jerome and Marc Miller Reliability
  and Validity in Qualitative Research. Newbury
  Park Sage, 1983.
• What reality is being probed?
• What does reliability mean with samples of 1?
• What is an appropriate test of validity for
  idiosyncratic experience?
• Different sub-disciplines face these issues
  differently

15
Reliability and Validity Issues
Dept. of Information Systems Doctoral Program in
IS ND

• Validity is really a challenge making certain
  that you have the right name for what you are
  observing.
• Reliability is the serious scientific problem.
• There are three sorts of reliability
• quixotic a single method of observation yields
  an unvarying measurement
• diachronic observation stability through time
  (test-retest)
• synchronic similarity of observations in same
  time period (split halves, inter-rater)

16
Meeting Reliability Challenges
Dept. of Information Systems Doctoral Program in
IS ND

• Document your procedure
• Tell how decisions involving the research -- and
  in particular data gathering and recording --
  were made
• Describe how the researcher was prepared
• Sources of reliability
• Bias (researcher effects surface biases,
  reactivity of insiders)
• Replication (a real problem see above strive
  for disconfirmation)
• Specification (context, conditions of research)

17
Two Examples of Qualitative Methods
Dept. of Information Systems Doctoral Program in
IS ND

• Narrative Analysis understanding individuals and
  their times through stories that they tell
• The Ethnographic Method understanding people and
  their ethos through a study of their culture
• They are similar in their goals and methods but
  focus on different aspects of people. Both face
  problems with validity and reliability and meet
  the challenge in different ways.

18
Qualitative Method Example Narrative Analysis
Dept. of Information Systems Doctoral Program in
IS ND

• Source Catherine Kohler Riessman, Narrative
  Analysis, Newbury Park Sage, 1993
• The analysis of stories, first-person accounts of
  personal experience talk organised around
  consequential events. A teller in a
  conversation takes a listener into a past time or
  world and recapitulates what happened then to
  make a point, often a moral one.

19
Qualitative Method Example Narrative Analysis
Dept. of Information Systems Doctoral Program in
IS ND

How people make sense of their world

• The analysis of stories, first-person accounts of
  personal experience talk organised around
  consequential events. A teller in a conversation
  takes a listener into a past time or world and
  recapitulates what happened then to make a point,
  often a moral one.

An individuals data
The experience is real for that person
The events are important to the teller
The events behave according to a naïve physics,
sociology, etc. that tells about the teller
There is a structure to the story that is
important
The teller is aware of the audience
20
Narrative Analysis
Dept. of Information Systems Doctoral Program in
IS ND

Issues 1. Realists take issue with
correspondence with truth 2. Related to other
textual approaches such as semiotics,
hermeneutics, discourse analysis,
deconstructionism 3. Highly concerned with how
protagonists interpret things 4. Highly
subjective.
21
Narrative Analysis Research Model
Dept. of Information Systems Doctoral Program in
IS ND

Reading(5) Analyzing(4) Transcribing (3) Telling
(2) Attending(1)
Primary Experience
22
Narrative Analysis Research Model
Dept. of Information Systems Doctoral Program in
IS ND

Reading(5) Retelling as the agent of the text,
reinterpretation into other frameworks Analyzing(4
) Creation of a metastory, a false
document Transcribing (3) Representation,
fixing, culling, loss of information Telling (2)
Performance, description, organisation, taking
into account cultural context, interaction,
dialogue Attending(1) Reflect, remember,
recollect into observations, choose from primary
experience
23
Narrative Analysis Technique
Dept. of Information Systems Doctoral Program in
IS ND

What is a narrative (and then what happened?)
expectations of listeners are cultural and there
are brackets there are many different kinds of
structures (not an exhaustive list, just
exhausting!) Labovs paradigms All narratives
have six elements abstract, orientation,
complicating action, evaluation, resolution,
coda. Burkes dramatismAct (what), scene
(when/where), agent (who), agency (how done),
purpose (why). Gees poetics poetic units,
stanzas, strophes, punctuation.
24
Narrative Analysis Truth
Dept. of Information Systems Doctoral Program in
IS ND

Narrative is a root metaphor (Sarbin) it is
an achievement that brings together mundane facts
and fantastic creations time and place are
incorporated. Narrative can 1. Recreate
experience via order 2. Constitute reality
telling makes things real 3. Provide plans or
ideologies to live by Narratives dont speak for
themselves they speak about the teller. We aim
not for TRUTH but for believability, enlargement
of understanding rather than control
25
Narrative Analysis Validation
Dept. of Information Systems Doctoral Program in
IS ND

• Historical truth of narrative is NOT primary
  interest we aim for trustworthiness. There are
  four criteria
• Persuasiveness, plausibility (subject to
  rhetorical style)
• Correspondence to informants experience (expect
  problems as narratives are movable feasts)
• Coherence (globalrelative to narrators goals
  localinternal structural consistency
  themalcontent)
• Pragmatic use by others in the future (!)

26
Qualitative Method Example Ethnographic Method
Dept. of Information Systems Doctoral Program in
IS ND

• Source Stewart, Alex, The Ethnographers Method,
  Newbury Park Sage, 1998
• Characteristics of the method
• 1. Participant Observation
• 2. Holism researcher constructions society
  and the data have breadth
• 3. Context Sensitivity comprehensive data
• 4. Socio-cultural description (social relations)
• 5. Theoretical Connections to Anthropology

27
Mapping to Conventional Values
Dept. of Information Systems Doctoral Program in
IS ND

Epistemic Value Conventional
equivalent Underlying question Research
pro-cess challenge
Veracity Validity (exclu- ding
external) Verisimilitude of depiction Field
conditions, researchers constraints
Objectivity Transcendence of
perspectives Reliability (exclu- ding
consistency Context sensitivity
Reactivity,lack of resrch context
Perspicacity Generalizability, external
validity Applicability of insights
elsewhere Inability to create insights,
invalid taxonomies
28
Research Coping Tactics
Dept. of Information Systems Doctoral Program in
IS ND

Veracity Objectivity
Perspicacity
1. Prolonged fieldwork 2. Search for
discon- firming observations 3. Good
participative role relationships 4. Attentiveness
to context 5. Multiple modes of data collection
1. Trail of Ethno- graphers path 2. Respondent
validation 3. Feedback from outsiders 4.
Interrater checks on indexing, coding 5.
Comprehensive data archive
1. Intense consider- ation 2. Exploration
Very helpfulquestionable use
Dance card, network, path
29
Four Phases of Qualitative Research
Dept. of Information Systems Doctoral Program in
IS ND

Invention Getting in and Getting
Along--Finding the field Copping direction
(networking), copping a look (first
viewing), copping a taste (early episodes of
non-intrusive assessment)
Discovery Getting Data -- Collecting the
data Scoring a chance (permission), scoring the
facts (gathering data, separating data from
noise, discovering new facts)
Interpretation Getting it straight --reading
the field Checking the validity checking the
reliability (strength of the data)
Explanation Getting out and getting even --
settling accounts Splitting up (severing
relationships), splitting the take
(negotiating costs and benefits with studied
group), splitting the scene.
30
Qualitative Research Summary
Dept. of Information Systems Doctoral Program in
IS ND

Goals Methods
Results
Understanding Enlightenment Creativity Divergence
Exploration
Intense interaction Personal Exploration Involveme
nt Observation Comparison
Thick description Teaching, learning Trail of
experience Lots of data on small amounts of
experience
Qualitative research is defined by the location
of hypothesis-testing activity in the discovery,
rather than the interpretation phase. --Kirk and
Miller
31
Grounded Theory
Dept. of Information Systems Doctoral Program in
IS ND

• Why ground your research
• You might want to create a new direction for
  research
• Theory might be restricting your search
• The reality in question might be in question!
• This might be part of action research
• What methods are used in Grounded Theory
• Open coding, axial coding, selective coding
• Where are you when you finish?
• You have a prototheory

32
Grounded Theory-Philosophy
Dept. of Information Systems Doctoral Program in
IS ND

Meanings
Meanings are important (perhaps paramount or even
the entirety of the theory) Theory should fit the
reality it is meant to explain, i.e., be
grounded in that reality. Theory is meant to do
more than explain it is to guide action.
Reality
Action/ Intervention
33
Grounded Theory-Techniques Open Coding
Dept. of Information Systems Doctoral Program in
IS ND

• Disaggregate data into conceptual, labeled units
• Compare derived code labels and group them into
  larger categories
• Avoid existing sets of labels to avoid
  contamination and counter-interpretation effects
• You are looking for significant themes.

34
Grounded Theory-Techniques Open Coding
Dept. of Information Systems Doctoral Program in
IS ND

• Disaggregate data into conceptual, labeled units
• Paragraphs, sentences, phrases, fragments
• Compare derived code labels and group them into
  larger categories
• Categories are from the text itself
• Avoid existing sets of labels to avoid
  contamination and counter-interpretation effects
• You are not proving the categories validity,
  just trying to establish them.
• You are looking for significant themes.

35
Grounded Theory-Techniques Axial Coding
Dept. of Information Systems Doctoral Program in
IS ND

• Look for relationships among various categories
• Rearrange this into a hierarchy and allow
  subcategories to emerge
• Identify what is happening and why, explore the
  context for events, segregate causes and outcomes
  of campaigns, and seek co-occurrences.
• Verify derived elements as hypotheses to be
  verified with later sources seek both positive
  as well as negative cases or episodes

36
Grounded Theory-Techniques Axial Coding
Dept. of Information Systems Doctoral Program in
IS ND

• Look for relationships among various categories
• You are attempting to build a theory-generator
  here

C1 C2 C3 C4
C5 C6 C7 C8 C9
C1 C2 C3 C4 C5 C6 C7 C8 C9
Precedes, Causes, Depends upon, Is necessary for,
Is incompatible with, Is part of, etc.
37
Grounded Theory-Techniques Axial Coding
Dept. of Information Systems Doctoral Program in
IS ND

• Rearrange this into a hierarchy and allow
  subcategories to emerge (possibly networks)

38
Grounded Theory-Techniques Axial Coding
Dept. of Information Systems Doctoral Program in
IS ND

• Identify what is happening and why, explore the
  context for events, segregate causes and outcomes
  of campaigns, and seek co-occurrences.
• Verify derived elements as hypotheses to be
  verified with later sources seek both positive
  as well as negative cases or episodes

Examples, counterexamples
39
Grounded Theory-Techniques Selective Coding
Dept. of Information Systems Doctoral Program in
IS ND

• Derive the new theory based on the major
  relationships among categories
• Constantly compare incoming data with existing
  categories and data (constant comparison)
• Sampling note You are not seeking
  representative sampling of respondents or
  informants you are seeking broad representation
  of ideas, meanings, thoughts, and expressions to
  let them emerge. Thus rather than finding
  typical respondents, you are seeking to have as
  broad a range of respondents as possible.

40
Grounded Theory References
Dept. of Information Systems Doctoral Program in
IS ND

From ISWorldNet (http//www.isworld.org/isworld/is
worldtext.htm)
http//www.aukland.ac.nz/msis/isworld/grndrefs.htm
Basic/Classic Glaser and Strauss (1967), Strauss
Corbin (1990), Glaser (1992) these latter two
differ about what GT is IS Classic Orlikowski,
Wanda. Case Tools are organizational Change
Investigating Incremental and Radical Changes in
Systems Development, MIS Quarterly, 17 (3)
309-340, 1993 Pandit, Maresh R. The Creation of
Theory A Recent Application of the Grounded
Theory Method, The Qualitative Report, 2(4),
1996. (http//www.nova.edu/ssss/QR/QR2-4/pandit.h
tml)
41
Grounded Theory Example
Dept. of Information Systems Doctoral Program in
IS ND

A doctoral student is interested in what how
users of CASE (Computer Aided Software
Engineering) actually use CASE in their work.
She decides that use and meaning of use are
highly related and uses Grounded Theory to build
her theory. She begins with simple questions
asked of a few referred users questions are such
as How do you use CASE? Why do you use it
that way? How did you learn to use it like
that? Note the generality of the questions Is
she using a pre-existing theory that is being
tested?How is her initial sample located?What
does she do with the answers to the questions?
42
Grounded Theory Example (2)
Dept. of Information Systems Doctoral Program in
IS ND

After the first set of interviews, she uses open
coding to create a series of fragments such as
Took a course, Learned from my colleagues, I
just copy everyone around here, The course
wasnt very good. Then she categorized the
comments (axial coding) into a set of categories
and generated a new series of (finer) questions
about training, policy, and style. Then she
selected other users who were different from the
first set (used a different tool in a different
firm) and asked these questions of them. What is
the purpose of open coding? What is the
alternative? Why did she use fragments as
opposed to paragraphs or sentences? What else
can she do with open coded fragments?What is the
purpose of categorization?Why is she seeking
different kinds of users? What happens to
reliability?
43
Grounded Theory Example (3)
Dept. of Information Systems Doctoral Program in
IS ND

These users answers were openly coded to create
more fragments (on cards) and as the number of
cards increased, she established a table relating
the fragments (now categories) to one another in
pairs or more complicated structures. Why is
she creating a database of fragments? Why have
so many?What is the purpose of the table? What
kinds of structures is she looking for?
44
Grounded Theory Example (4)
Dept. of Information Systems Doctoral Program in
IS ND

Gradually a set of structures arose that
indicated to her the varieties of use of CASE
tools. In some cases, the use was motivated by
others in some cases the users created their own
use. But it turned out in all cases that the
uses were not the set of uses for which CASE was
intended! What kind of picture is emerging? How
can we say that is theory? Whose theory is
it?What is her major finding?Why would a more
positivist approach been more difficult?
45
Grounded Theory Example (5)
Dept. of Information Systems Doctoral Program in
IS ND

Using selective coding and constant comparison,
as the set of respondents grew (and as the number
of contrasted sources of information from the
respondents grew), she became more convinced that
the categories she derived had good reason to
exist in the causal structure she was watching
emerge. However, at no time did she try to prove
her emerging theory. Why is she constantly
comparing? What is her goal here? How does she
know how many respondents to interview?What
makes her convinced about her causal
structure?Why doesnt she use this method to
prove any theory?
46
Grounded Theory Diagnostic
Dept. of Information Systems Doctoral Program in
IS ND

Bill is attempting to discover theory about
why/how NPOs employ IS consultants. He has noted
how NPOs differ from normal businesses and has
a theory about normal business. Using this
theory, he generates a series of questions which
he asks of a few NPO managers. Based on these
answers, he creates a series of categories and
notes that the categories are very similar to
those of normal businesses. He then asks for
referrals to other NPOs and uses these categories
to see if the other NPOs are in fact similar.
When he sees they are, he starts to ask questions
of a causal nature (Do you think that X causes
you to use consultants of a certain type in a
certain way and in this way builds up a theory.
47
Action Research - Example
Dept. of Information Systems Doctoral Program in
IS ND

Results of research into collaborative
technologies (such as E-mail) have had
inconclusive or contradictory results. In this
study, 107 people in two NZ organisations
collaborated in twelve groups over 10 to 45 days
over email-based systems. Participants were
interviewed and their messages were analysed for
content. Consistent with a social influence
model, participants felt that the e-mail
intervention increased individual knowledge
learning about other departments and departmental
heterogeneity.
Koch, Ned. Sharing Interdepartmental
Knowledge using Collaboration Technologies
Action Research Study, Journal of Information
Technology Impact, 2(1) 5-10, 2000
48
Action Research - Example (2)
Dept. of Information Systems Doctoral Program in
IS ND

Note the elements of action research (1) There
was a technological intervention that was
actually employed by working groups (2) The
groups were engaged in process improvement
tasks (3) The researcher actually provided the
mediation (I.e., was part of the system).
Koch, Ned. Sharing Interdepartmental
Knowledge using Collaboration Technologies
Action Research Study, Journal of Information
Technology Impact, 2(1) 5-10, 2000