Establishing Trustworthiness

1
Qualitative Data Management

• Establishing Trustworthiness

2
We enhance reliability or rigor of our data
analysis by

• Comparing our categories to pre-existing
  frameworks.
• Having an additional person redo the analysis.
• Comparing notes from more than one source.
• Using more than one type of qualitative data in
  our analysis (observation, interviews, document
  analysis).
• Supplementing the qualitative analysis with
  information from another quantitative source (for
  example, a survey).
• Keeping a record (audit) of how you established
  data categories and identified themes.
• Establishing a feedback loop so that participants
  can verify whether or not the analysis is
  accurate enough to reflect their views (member
  checking).
• We call this process in qualitative research as
  trustworthiness.

3
Padgett (1998) defines trustworthiness as

• A trustworthy study is one that is carried out
  fairly and ethically and whose findings represent
  as closely as possible the experiences of the
  respondents (p. 92)
• Lack of trustworthiness comes from
• a) reactivity
• b) researcher biases
• c) respondent biases/ such as withholding
  information or the halo effect.

4
Qualitative Data Analysis

• Looks for common themes and patterns.
• Sample quotations are used.
• Categories of responses are identified and the
  number of responses that fall within these
  categories are identified.
• No statistical analysis is required but
  demographic information may be expressed in
  percentages or placed in tables.

5
Example of using quotes (from gang study in Sin
City)

• One fourth of the youths surveyed said that they
  didnt like gangs or didnt want to join a gang.
  Almost 40 said that being in a gang was bad or
  stupid. One youth described being in a gang as a
  dead end choice. Nine percent felt that being
  in a gang was dangerous or scary. One respondent
  said being in a gang was scary because they
  always talk about killing or beating up other
  people. Only 6 of the respondents thought being
  in a gang was fun or was necessary for
  protection. However, one youth said that being in
  a gang made him feel safe and good, cool to be
  around.

6
In the previous quote there were categories

• Didnt like gangs
• Gangs are bad or stupid
• Being in a gang is fun

7
One approach to writing narratives can also
include

• The writers own thoughts, values, and beliefs.
• An interpretation of the research participants
  behavior or thoughts.

8
For example, this quotation from Fadiman, A.
(1997). The spirit catches you and you fall down.
New York Farrar, Straus, Giroux.

• While Foua was telling me about the dozens of
  tasks that constituted her easy work in Laos, I
  was thinking that when she said she was stupid,
  what she really meant was that none of her former
  skills were transferable to the U.S. none, that
  is, except for being an excellent mother to her
  nine surviving children. It then occurred to me
  that this last skill had been officially
  contradicted by the American government, which
  had legally declared her a child abuser.

9
One approach to writing a narrative is thick
description creation of a picture of observed
events, people involved, rules associated with
certain activities, and social context or
environment. Thick description can also
incorporate the researchers perspectives.
10
Clients articulate their belief that the welfare
system is not designed to help them succeed or
care for their families.Often it feels as if the
information they received from workers is
blatantly wrong. In one focus group, participants
talked assuredly of the misinformation they had
received .As one women saidThe policy is a
lie. This what happens in the welfare
system.Such a lack of trust raises serious
questions about whether or not clients will heed
front-line staff. From Sandfort, Kalil,
Gottschalk (1999). The mirror has two faces.
Journal of Poverty, 3 (3), 71-91.
11
A narrative or text-based summary should include

• Identification of common themes in responses.
• Patterns of behavior
• Cultural or other symbols found in the setting or
  described by respondents.
• Identification/description of cultural norms
• Common words or phrases used by many respondents
  with sample quotations
• Minority responses with sample quotations

12
According to Jorgensen (1989 as cited in Seidel
(1998)

• Analysis is a breaking up, separating, or
  dissassembling of research materials into pieces,
  parts, elements, or units. With facts broken down
  into manageable pieces, the researcher sorts and
  sifts them, searching for types, classes,
  sequences, processes, patterns, or wholes. The
  aim of this process is to assemble or reconstruct
  the data in a meaningful or comprehensible
  fashion (p. 107).

13
Steps in Data Analysis

• Choose a unit of analysis (word, phrase,
  sentence, paragraph, entire transcript).
• Identify possible categories to classify these
  data elements.
• Classification can take place using one or both
  of the following methods
• - Categories are developed for each of the
  questions to which respondents gave answers.
• - Categories are developed for the responses as
  a whole. These categories are also called themes.
  

14
The process for doing this is often called
constant comparative method

• You make comparisons among data elements.
• You may need to take three or four attempts to
  analyze and interpret the data.
• Your categories or themes may change as you
  examine additional transcripts.
• You can also use a process called negative case
  analysis. Are there responses that dont fit the
  other categories. How does this change your
  analysis?

15
Additional Steps

• Make sure that you have been able to classify all
  your data.
• May develop alternative categories and then
  choose one that best indicate a pattern or theme.
• You probably will need to make a count of all the
  possible data elements in order to ensure that
  youve been able to classify all of them.

16
Data Management Techniques

• Cut and paste transcripts on note cards and then
  put all elements that fit into one category in a
  pile. This allows you to reshuffle the cars.
• Use magic marker or crayon to color code your
  categories.
• Just use a scissor to cut and paste data
  elements.
• Cut and paste on the computer some
  qualitative software packages allow you to do
  these or allow you to search for similar words
  and phrases.
• Develop a numerical code and place these codes
  within a copy of the transcript.

17
First stage of Coding is Open-Coding,
identifying what you see in a transcript
Text Code
I always wanted to be the thinnest, the prettiest. I wanted to look like the girls in the magazines. Im going to have so many boyfriends, and boys are going to be so in love with me. I wont have to work and Ill be taken care of for the rest of my life. Thinnest Prettiest Look like girls in magazines Boys will love me Positive body image Provides economic resources Thin Rationale Thin as a means of security Media creates standards
18
Important to Keep Notes

• Rationale for Coding Category (could be on
  transcript)
• Need notes on your reasoning for classification.
• Keep track of themes in data.
• Try to tie categories together. Are there cause
  and effect relationships in the data?
• Can keep analytic memos
• Can keep memos on your own reaction to the data.
• Note all these elements can be incorporated
  into write up

19
For example, if we were to use the following
interview guide, we would transcribe all
responses underneath each question in a WORD
processing program. (Sample interview guide)

• 1. Can you describe how you first became aware of
  your deafness?
• Respondent 1
• Respondent 2
• Respondent 3
• 2. How do you see yourself today, in terms of
  your deafness?
• Respondent 1
• Respondent 2
• Respondent 3
• From Janesick, V. (1998). "Stretching" exercises
  for qualitative researchers. Thousand Oaks, CA
  Sage, p. 75.