Reporting qualitative data

1
Reporting qualitative data
2
The problem how to convey to the rest of the
design team what was learned in qualitative needs
assessment?
3
Representation

• No representation is an objective, unbiased
  report of whats real
• Representations of work are interpretations in
  the service of particular interests and purposes,
  created by actors specifically positioned with
  respect to the work represented. Suchman,
  Making Work Visible, p. 58
• A representation consists of
• A point of view
• Selections and deletions
• Foreground and background
• Representations embody interpretations
• About what is being represented
• About how the representation will be used
• The process of representation creates situated
  artifacts intended to be used in situated ways,
  in interaction with other people and the world.
  
• Representations create as much as they reflect
  the world.

4
Representation, cont.

• For representa tions to be seen as faithful to
  reality, the work of representation gets
  deleted, which makes invisible the choices
  made and by whom, and what gets left out.
• A key question Who represents whom or what?
• Power
• Self-representation is a form of empowerment.
  Suchman

5
Representation, cont.

• From introduction to a volume on representation
  in science
• If the studies in this volume agree on anything,
  it is that scientists compose and use particular
  representations in a contextually organized and
  contextually sensitive way....The studies in this
  volume endeavor...to show that the particular
  representations they discuss have little
  determinate meaning or logical force aside from
  the complex activities in which they are
  situated. Lynch Woolgar 1990 p. vii-viii.

6
Representation and seeing

• Prior to the work of representation is that of
  seeing.
• We learn what to pay attention to and what to
  see what is significant and what can be
  overlooked.
• Goodwin shows how archaeology students learn to
  see the color of dirt by specific, hands-on
  practices of wetting the dirt and comparing it
  with a standard color chart,
• And how an attorney could change how the jury
  saw the videotape of the Rodney King beating
  through the eyes of police officers.
• Representations help teach people how to see.
• Law Lynch 1990 show how the naturalistic
  drawings and photos in field guides to birds
  differ in how they choose to present the birds,
  what factors they emphasize, and how they diverge
  from pure naturalism in order to make apparent
  the details needed for identification.

7
Invisible work

• Work that is take for granted
• Work that is only visible when it fails or breaks
  down
• Undervalued
• Often left out of representations and therefore
  out of design
• Includes work that is not talked about doesnt
  fit normative view of how work should proceed
• Process vs. practice
• Failures, problems, difficulties
• Includes articulation work
• The work required when things go off-track to
  bring them back on-track

8
Ethnographic/qualitative investigation, design
and usability, and representation
9
Locus of translation from empirical observation
to design

• Observers develop requirements, formally or
  informally
• The translation from empirical world to
  requirements is done by the observers, invisible
  to the system developers
• Observers often participate in design work as
  proxies for users
• Observers describe work design team translates
  to requirements
• Ethnographic narratives no fixed format
• Formalizations
• Contextual inquiry elaborate set of
  models/diagrams requires team to be familiar with
  modeling conventions
• Other kinds of flow charts, descriptive reports
• Observers may participate or may hand off to
  development team

10
Scenarios and Personas

• Currently very popular fads
• Can be used to summarize and report qualitative
  findings
• Can also be used to perpetuate stereotypes,
  disempowering views of users

11
Scenarios - Carroll

• Stories about people carrying on an activity.
• Problem scenario problem as it exists prior to
  technology
• Design scenario describes new vision
• Consist of (Rossen Carroll, p. 18)
• Setting, context of use
• actors with high level goals, task goals, and
  plans
• actions (observable behavior)
• events (external actions produced by computer or
  other features of a setting)
• Evaluation (mental activity interpreting
  situation)
• Plot sequence of actions events
• Peoples goals, plans, and understandings can
  change.
• Suchman, Plans and Situated Actions rafting
  example.
• Promote a work orientation.

12
Scenarios Cooper

• Daily use scenario frequent actions
• Necessary use scenario ALL actions that MUST be
  performed
• Edge case scenario unusual events

13
Benefits

• Concrete and flexible can keep open exactly how
  ends are achieved.
• Tentative working representation that can be
  elaborated, revised.
• Focus on consequences of specific design
  decisions.
• Balance action with reflection what if.
• Describe problem situation in a way
  understandable to all stakeholders.
• Focus discussion on ends to be achieved before
  design details.
• Useful for thought experiments concerning
  various design use possibilities.

14
Personas

• Fictional people with life stories, goals, tasks.
• Personas come before scenarios.
• Instead of focusing on the task in a work
  context, focus on a whole person (performing a
  task)
• Powerful, engaging more interesting than
  scenarios? (Grudin and Pruitt)
• A fad that may or may not endure.

15
Where do scenarios personas come from?

• Designers ideas, assumptions about how
  technology could be used and by whom.
• Engagement with users user and task analysis,
  contextual inquiry, ethnography.
• Designers and/or users hypotheses about
  possible future activities, users.

16
Uses of scenarios personas

• Problem scenarios personas
• Develop and illustrate assumptions about users
  and uses or targeted users and uses
• Summarize empirical findings about users, uses
• Used
• For design, communication within design group
• Configuring the user
• For communication with users (e.g., is this
  realistic?)
• For evaluation of designs, prototypes
• E.g., walk-throughs
• Design scenarios
• Present possible solutions, identify possible
  problems

17
Benefits of scenarios and personas

• Easy way to summarize and make memorable complex
  info
• Easy shorthand for communication
• More realistic than abstractions more closely
  related to real world (if designed appropriately)
• The power of narrative complex personalities and
  story lines in an easily-understood format
• A way to translate empirical findings into a form
  that is communicable and usable

18
Risk substituting fiction for reality

• Too much reliance on something not sufficiently
  grounded empirically.
• May seem more empirical, more true than they
  are.
• Too few, too simple of scenarios.
• Tendency to reduce complexity to one or a few
  actors, single story line.
• Caricatures, stereotypes.
• Confirming rather than challenging assumptions.
• The real world is messy.
• Unrepresentative scenarios, actors.
• Good stories rather than representative ones.
• Stories that designers think are representative
  (but arent).
• Used in place of empirical info on actual tasks,
  conditions, actors, work practice.

19
Other risks

• Use of stereotypes reinforcing stereotypes
• Inessential information that can be misleading or
  limiting
• Jane is a computer engineer who likes to cook and
  has a 5-year-old daughter Jim is a computer
  engineer who spends his spare time re-building
  cars.
• (Unintentionally) developing personas that fit
  the planned/proposed technology rather than
  technology that fits the personas