Ethnography in Designing Socio-Technical Environments

1
Ethnography in Designing Socio-Technical
Environments

• Wednesday, March 15, 2006
• Stefan Carmien?
• Melissa Dawe?
• Anja Kintsch

2
Computer Scientists and People
?
?

• Our formal training as technologists and
  designers often leaves gaps in how to do research
  in the socio side of socio-technical
  environments..

3
What we want to share

• Our experiences in using ethnography as a tool
  to understand the context and the stakeholders in
  our work designing systems for persons with
  cognitive disabilities and support communities.

4
Ethnographic Methods

• What are they?
• Who uses them why
• What are their attributes
• What are the skills of an ethnographer?
• What artifacts do they produce?

5
Qualitative Ethnography

• We are only focusing on Qualitative aspects of
  ethnography, why?
• Participant observation
• Fieldnotes
• Coding
• Hypothesis generation
• Semi-structured interviews
• Inspired by Grounded theory
• Allow topics to emerge from raw data

6
Ethnography and L3D

• How we can use it
• Why we must use it

7
Three Projects

• Stefan - MAPS
• Melissa Mobile Communication Study
• Anja - Smart Care

8
Stefan

• MAPS system research focus
• How person with cognitive disabilities live
  learn
• How new AT is used
• Roles
• Person with cognitive disabilities
• Caregivers
• Parent
• Job coach
• Transition team
• Group home staff
• Study in pairs

9
Stefan - Methods

• Semi- structured interviews
• At end of usability studies
• Exit interviews at end of MAPS introduction
• Participant observation lttypically 20 hoursgt
• Several days just hanging out
• Activities of daily life (ADL)
• Introduce MAPS
• Caregivers making scripts
• Three scripts
• Controlled environment (home)
• Less controlled environment
• Open environment (out of home)

10
Stefan Methods (contd)

• Each participant observation session was
  documented
• Fieldnotes (little note pad)
• Digital recorder
• Post Observation write-up
• Transfer Fieldnotes to log ltannotated with
  observationsgt
• Roughly transcribe recording to log ltdittogt
• One hour observation 1.7 hours entry in log
• Each Dyads log was between 20 and 30 pages
• Coding pattern extraction
• Illustrative quotations from recordings
• Each finished dyad informed the approach of the
  study of the remaining ones

11
Stefan -How I have used the data

• Experiencing the life of my population (both
  parts of the dyad)
• Daily variations in ability
• Details of what works and doesn't work in scripts
  (Voice, size of steps, internal scripts)
• Battle of time of caregivers
• Design with Scenarios (usability editor
  features)
• Caregivers implicit requests
• Caregivers hopes, expectations, fears
• Left to do
• Code dyad studies
• Re-code Melissa's interviews
• Extract abstractions from coding
• Use quotes in dissertation

12
Melissa Mobile Communication StudyResearch Focus

• 1) How can technology support the mobile
  communication needs of young adults with moderate
  cognitive disabilities?
• 2) How can caregivers and individuals with
  cognitive disabilities authentically participate
  in the design process?

13
Melissa Mobile Communication StudyMethods

• Phase 1 Semi-structured interviews with families
• Recruiting participants
• Developing an interview guide
• Conducting interviews in peoples homes
• Coding
• Qualitative analysis
• Maintaining a relationship with participants

14
Melissa Mobile Communication StudyMethods
(contd)

• Phase 2 Interviews and observations focused on
  mobile communication
• Phase 3 Technology probe study
• Work with families (participatory design) to
  design UI for their child
• 8 week field trial to study co-evolution of users
  and technology
• Technology-as-ethnographer through detailed
  logging
• Design-in-use requirements emerge through doing

15
Melissa Mobile Communication StudyParticipatory
Design Activity
16
Melissa Mobile Communication Studyworking with
AbleLink

• AbleLink Pocket Ace software is a cell phone
  application designed for individuals with
  cognitive disabilities
• In exchange for use of their software, I will
  provide usability feedback and feature
  requirements
• I have full access to source code, dev support

17
Anja Observations on observing and interviewing
Imagine!

• Fact-finding investigation of factors related to
  caregiver activities, quality of life, and use of
  personal technologies among individuals with
  cognitive disabilities who are residents in group
  homes and community settings.
• Inform the development of context-aware
  technologies.
• Face-to-face semi-structured interviews and in
  situ observations
• Part of RERC

18
Goals of the Questionnaire

• Made sure our questions reflected our research
  questions.
• Interview procedures and questions intended to
  collect quantitative as well as qualitative data
• Made questions similar for all respondents
  managers, group home caregivers, parents, and
  clients.

19
Experiences with the Questionnaire

• Caregivers interpreted the questions differently
  than we intended.
• We were looking for specific answers so we could
  say x out of y say z. - that didnt happen.
• Please describe a situation where a client who
  you know well can almost do something thats
  important to them? what would be needed to allow
  them to do the task independently? - no answer
  involved A.T.

20
Timing challenges with the Questionnaire

• Once we settled on the questionnaire we were
  bound to it because we had to get 58 interviews
  completed and we were behind schedule.

21
Interviewing Caregivers

• Different group home caregivers responded to my
  presence differently
• Guardedness - are you a spy from management?
• Defensiveness - I am good at my job!
• You are here to fix things - let me tell you the
  problems
• Open and confident
• For some it was enjoyable for others it felt like
  a requirement
• Was not capture in any formal method

22
Observing when people with cognitive disabilities
are present

• We planned on passive observation, but I ended up
  being participatory observer
• Observer effect compounded Dont respect the
  boundaries of observer and observed - would come
  up and interact with me.

23
How to do research with people with cognitive
disabilities - Potential Problems?

• Tangential thinking and lack of familiarity with
  the context
• Caregivers answering or interpreting too much
• When working with clients with limited speech.
  How do you offer choices without influencing the
  answer?
• Foils?

24
What are your most favorite things to do each
week?
Potentially correct choices of daily activities
but are not Imagine! classes or necessarily
considered selectable events
Potentially correct choices as offered by Imagine!
Incorrect and absurd foils
Foils are offered by Imagine! but client not
participating in these classes.
25
What We Got Using Ethnographic Methods That We
Could Not Have Gotten Otherwise

• Stefan
• Understanding caregivers relationships
• Getting young adults ability needs
• Vary over time
• Typical teenage issues
• uneven map of abilities (unexpected faculties)
• Scenario based design

26
What We Got Using Ethnographic Methods That We
Could Not Have Gotten Otherwise

• Melissa
• Understand that for families, assistive
  technology includes kitchen timers, memo
  recorders, Yahoo! games, etc.
• Learn the importance of mobile communication in
  supporting independence, social-connectedness,
  sense of safety for families
• A design process where families guide the design
  of their own technology

27
What We Got Using Ethnographic Methods That We
Could Not Have Gotten Otherwise

• Anja
• Compare what management sees as important issues
  verses what caregivers on the ground see as
  important
• See the extreme limits of many clients in the
  group home environment - maybe it isnt the best
  place to work on independence
• See that group homes are like large families

28
Our goal

• We hope to spark a discussion with the larger L3D
  community that have used or are interested in
  using these methods.

29
Resources- Classes

• Classes
• ANTH 7300 -- Research methods in cultural
  anthropology
• COMM 6030/001 -- Interpretive Research Methods
• SOCY 5031/001 -- Research Design

30
Resources- Tools

• Digital recorders (Olympus)
• Transcribing software - Olympus DSS player
  (adaptive speed control)
• Coding tools
• HyperResearch
• NVIVO

31
Resources- Texts

• Texts
• Analyzing and Interpreting Ethnographic Data,
  Lecompte
• Analyzing Social Settings, Lofland Lofland
• Basics of Qualitative Research Techniques and
  Procedures for Developing Grounded Theory,
  Strauss Corbin
• Contemporary Field Research, Emerson
• Qualitative Research Methods, Berg
• Research Methods in Anthropology, Bernard
• Writing Ethnographic Fieldnotes, Emerson, Fretz,
  Shaw

32
Resources- People

• Local Expertise
• Leysia Palen
• Tammy Sumner
• Sophia Liu
• Heinrich Schwartz (Comm Dept.)
• Kathleen Tierney (Sociology Dept.)

33
Answers to the questions on slide 4

• What are they?
• Ethnography is "the study of people in their own
  environment". So it's a collection of methods
  that are intended to understand something about
  people, and involve some sort of fieldwork in
  their environment. Different methods are
  appropriate based on the setting, and the
  research questions.
• Who uses them and why?
• Historically, first were anthropologists. The
  classic method was participant observation in an
  extended study, often studying foreign cultures
  (think Margaret Mead living in Samoa). Now
  sociologists,communication studies researchers,
  HCI, even marketing departments, etc.,a wide
  array of researchers interested in some aspect of
  human behavior in a natural setting.
• What are the attributes?-
• Studying people, some aspect of human behavior
  (including social behavior)
• In the field -- it involves studying people in
  their environment-
• Often involve data collection that is analyzed
  qualitatively, but also conducive to quantitative
  analysis (or some hybrid)-
• Often used for exploratory research or early
  phases, when research questions are being
  defined-
• Can be used in complement with other research
  methods (e.g. formal surveys, etc.) in "data
  triangulation" (gaining a more complete view by
  approaching the problem from different
  perspectives)-
• Traditionally assumed the researcher did not
  alter the people or environment under study, but
  now there are types of ethnography where this is
  definitely a desired outcome (action research,
  design research like we do!)
• What are the skills of an ethnographer?-
• Depends on techniques used (interviews,
  observations, extended fieldwork, etc.), but in
  general
• study design-
• doing the fieldwork-
• qualitative analysis (usually coding and
  analyzing large amounts of data)-