The Analyst

1
The Analysts Tool

• and its use in the construction of Groupware
  systems
• Alex Feinman
• afeinman_at_cs.brandeis.edu

2
Developing Groupware

• Developing Groupware systems is HARD
• Users dont know what they need
• Developers dont know what the users need
• Solutions are of mixed usage
• Forcing the users to use what you come up with
• Design Guidelines
• E.g., GUI designs Patterns
• Iterative Design
• Design -gt Implement -gt Evaluate -gt Redesign

3
Iterative Design

• Slow, Expensive
• Requires many cycles for useful software
• Initial design may be very far from acceptable
  product
• Often users are not patient enough for a polished
  product to be produced
• Same problems are solved over and over again by
  different developers
• Exponentially more difficult for multi-user
  system ( subgroups 2n)

4
Iterative Design Evaluation

• Much work has been done on Design (e.g., GOMS)
  and Implementation (e.g., ADAPTIVE)
• Less work on Evaluation
• More precise evaluation should allow for fewer
  cycles, leading to faster development
• Multi-user evaluation has its own extra
  complications

5
Analysts Tool

• Part of a larger system for implementing and
  analyzing Groupware

Analysts Tool
More Groupware Systems
VesselWorld
SAGE (replay tool)
Basic Groupware Tools (chat room, shared
whiteboard, etc.)
ADAPTIVE
6
Methodology

• Implement a base groupware system, providing only
  basic tools such as a chat room and shared
  whiteboard
• Collect significant amount of data of real
  workgroups doing their task using the base system
• Analyze data using analysts tool to isolate and
  evaluate coordination issues
• Implement Coordination Representations to solve
  recurring problems of coordination

7
Guidelines for Building Coordinating
Representations

• Library of patterns in coordinating
  representations is under construction
• list, checklist, set, table, flowchart
• media attributes Clarks list reviewable,
  editable, synchronous, etc. verification
  needed, approval needed, vetoable voluntary vs.
  automatic updates etc.
• interface design issues screen real estate,
  effort to create/modify/remove items,
• Guidelines for matching coordination problems to
  attributes also under construction
• e.g., if a known of actions needs to be performed
  in order, a checklist may be indicated
• First, need gather and analyze data to determine
  which coordinating representations are needed

8
Gathering Data

• ADAPTIVE automatically collects data
• Data collected consists of
• Chat transcripts
• GUI actions performed by users
• mouse clicks, window movement, keystrokes, etc.
• Domain-specific actions of users
• in VesselWorld, lifting and moving waste, moving
  barges, navigating the harbor, etc.

crane2 lets get swaste1 crane1 no way dude. We
need to wait for teh tug crane2 tug, isnt
swaste1 no eq? crane2 no, wait, its
unknown tug1 havent idd it yet crane2 so
wait crane1 we could do the xl int he corner
9
Analyzing the Data

• Primarily analyzing the chat transcript,
  supported by investigation of domain actions
• Chat can be analyzed using augmentation of
  Conversation Analysis techniques
• Four basic areas of interest
• Segmenting
• Turn construction
• Turn-taking
• Group dynamics

10
Segmenting

• Segment tool allows analyst to easily delineate
  interesting sections of data
• Allows alignment of corresponding sections of
  similar segments and extract the shared structure
• Allows construction of segment hierarchy, from
  which a hierarchy of actions can be inferred
• Once a case-base of possible user actions has
  been constructed, user choices in
  turn-construction are made more visible

11
Segmenting Tool
Segment Comparison View
Segment A jerry segment
Segment B are you sure thats the net
crane2 so I says to him, jerry, I says, I dont
want to go crane1 quite and submit
something tug1 waste at 300/125 needs
net crane1 what size? crane2 and he says,
shut up and be quick about it. crane1 tug1
What size is the waste at 300, 125? tug1 waste
at 320/98 needs nothing tug1 net crane1
tug1 not equipment, what size is it? crane2
doesnt anyone want to hear my storiy? tug1
small waste at 300/125
crane1 whatst hew aste at 300 190? crane1 oops
190 300 crane2 ln_at_300,190!l crane1
thanks tug1 Are you sure thats net? crane2 I
thought so tug1 I had no equipment. Let me
check before you get it. crane1 ok.
FRAME WASTE INFORMATION QUERY -------------------
----------------- SLOT 1 NAME Request for
information COMPLICATIONS no response (busy,
ignoring?) (A1b) self-correction
(B1b) correction after next (no
examples) STRUCTURES none recorded
yet ------------------------------------ SLOT
2 NAME Information returned COMPLICATIONS
mismatch of CG between users (B2c-2e)
wrong information returned (A2a-2c) STRUCTUR
ES jargon (B2a)

1a

1b
2a

2b
1a
2c

2d
1b
2e


2a
2b

2c
12
Turn construction

• Analyst can tag individual utterances as desired
  to manually extract patterns
• Two sets of tags
• Domain-independent list of tags, generally
  dealing with coordination issues (e.g.,
  Commitment, Reference Error)
• Domain-specific list of tags (e.g., Waste
  Dropped, Joint Lift)
• Tag lists are customizable by analyst

13
Turn-taking

• Quantitative analysis tools allow examination of
  trends in turn-taking
• Turn-taking clusters may point to subgroup
  formation
• Patterns in turn-taking may indicate negotiation
  of roles

14
Group Dynamics

• Structured notepad allows analyst to record
  impressions of group dynamics
• Overview/timeline tool presents impressions in
  situ, allowing comparison of dynamics across
  problem-solving sessions and groups.

15
Statistical Analysis Tool

• Allows rapid investigation into statistical
  trends in the data.
• Currently under development by Michael Alex

16
Integration of Tools

• Tools not meant to be used in isolation
• e.g., segmenting and comparing segments allows
  easier extraction of case-base of
  (conversational) actions
• Tool data is persistent and shareable, allowing
  collaboration by multiple analysts either
  concurrently or in sequence
• Tool could perform automatic comparison of
  analysis by multiple reviewers, strengthening
  confidence in results