Comparative Investigation of Collaboratories: CrossCutting Themes

1
Comparative Investigation of Collaboratories
Cross-Cutting Themes

• June 20, 2003
• University of Michigan
• Ann Arbor

2
Reminder Where Weve Been

• UM group 15 years of experience with
  distributed collaboration
• SOC project
• 40 Collaboratories at a Glance (C_at_G)
• 10 in-depth studies
• Sept. 02 SPARC/UARC, CFAR, Bugscope, EMSL
• June 03 NEESgrid, InterMed, GriPhyN, iVDGL,
  AfCS, BIRN
• The Literature
• Your input

3
What We Learned Here

• Review Cross-cutting Themes
• Modify
• Refine
• Eliminate
• Add
• Framework for generalizations
• What leads to success, failure?
• Source of design prescriptions
• How to do the next one?

4
Cross-cutting themes From Prior Work

• Collaboration readiness
• Collaboration vs. competition in science
• Bottom-up vs. top-down origins
• Technology readiness
• Experience with collaboration tools
• Infrastructure readiness
• Both technical and social
• Common ground
• Extent of shared knowledge critical in
  interdisciplinary work
• Coupling of work
• The interdependencies among individuals

5
Collaboration Readiness

• Can a collaboratory be mandated by an external
  agency (e.g., funding source)?
• NEESgrid collaboratory capability as a
  condition of funding
• High risk details in presentation discussion
• History of collaboration
• High energy physics vs. earthquake engineering
• Science driven
• AfCS
• BIRN

6
Common Ground

• NEESgrid
• Differences in terminology between CS EE
  communities
• InterMed
• Importance of establishing shared vocabulary
• Boundary objects, pidgins
• GryPhyN, iVDGL
• Too much common ground ? Boundary objects as key
  concept G. Bowker
• BIRN
• Attention to metadata, ontology

7
Cross-cutting Themes from SOC Analyses

• What is success?
• Detailed discussion in June 2001 workshop
• What are the incentives for participation?
• Survey study in progress
• What kinds of collaboratories are there?
• Taxonomy presented later
• How do collaboratories evolve?
• Some ideas based on our taxonomy presented
  later

8
What is Success?

• Use of the collaboratory tools
• Software technology
• Direct effects on the science
• Science careers
• Effects on learning, science education
• Inspiration for other collaboratories
• Learning about collaboratories in general
• Effects on funding, public perception

9
Measures of Success

• GriPhyN, iVDGL
• Persist beyond ITR funding
• Spending less time on tools, more on science
• BIRN
• Cover story in Nature
• Lots of publications
• Multiple audiences
• Beyond the scientists
• Students, government, industry, general public
• Collaboratory ? NSF STC

10
Incentives

• AfCS
• Alliance with Nature
• BIRN
• Guidance re publications
• LHC
• Shift in time scale of experiments
• Implications for careers

11
Evolution

• Ecology of collaborations
• Movement from limited to full collaboration
• Data wisdom hierarchy G. Furnas
• Movement up and down over time and space
• Relates to social vs. technical processes
• Where did the field come from, where is it going?
• Historical context as critical
• Multi-tasking of individuals (G. Mark)
• Time scale issues
• AfCS bioinformatics earlier?
• BIRN savings across successive BIRNs

12
The relationships
Practice and Expertise

• Wisdom
• Knowledge
• Information
• Data
• The world

Distributed Research Centers
Community Data Systems
Shared Instruments
13
Cross-cutting Themes from SOC Analyses

• Do collaborations have an ideal size?
• Collaboratories allow for larger ones
• How do they scale?
• What are various organizational models for how to
  structure collaboratories?
• How does the control and flow of resources affect
  collaboratory success?
• The money flow the relation to the sponsor(s)
• How much flexibility should be designed in?
• What kinds of early commitments?
• How much flexibility will funders allow?

14
Ideal size

• ATLAS
• Collaboration of 2000
• But very organized
• Beyond ATLAS?
• Manhattan
• Apollo
• How many working groups can be supported?
• Organizational science as source of clues
• What does technology enable?
• How to scale from literature on teams (G. Mark)

15
Flexibility

• Retrenchment, redefining of goals
• G. Bowker may be key to success
• Funding models
• AfCS enough flexibility? (A. Prakash)
• Adapting to new developments
• InterMed 1995 shift to focus on guidelines
• AfCS 2003 changing cells

16
Cross-cutting Themes from SOC Analyses

• How important are data issues in collaboratories?
• Data seems to be a central component of all
  collaboratories
• For what kind of work do you need real-time vs.
  asynchronous interactions?
• How important is security?
• Whats the mix of tailor-made vs. off-the-shelf
  tools?

17
Data Issues

• Metadata
• Provenance
• Persistence, archiving
• Rationale for transformations
• NEESgrid, GriPhyN, iVDGL, AfCS, BIRN
• Details of size, usage different software
  needs?
• What level of processing? Different disciplines
  may vary D. Sonnenwald
• Data sharing across jurisdictional boundaries
  BIRN
• IRB data from humans
• International

18
Cross-cutting Themes from SOC Analyses

• How crucial are platform issues?
• What is the emerging role of middleware?
• What is the role of emerging infrastructure such
  as the Grid?
• How does one move from early prototypes to
  production versions of collaboratories?
• Why isnt there more reuse of collaboratory
  tools?
• To what extent are the issues specific to science
  domain or are general?

19
Moving to Production Versions

• Tensions between CS and domain users
• NEESgrid innovation vs. extrapolation
• GriPhyN iVDGL
• Moving beyond initial demo stages
• Slow adoption
• InterMed
• Sustaining the investment
• NEESgrid NEES consortium infrastructure set up
  in advance
• GriPhyN, iVDGL seeking a sustaining support
  process
• BIRN
• Incentives
• build hardware J. Leigh
• Diffusion of Innovation literature

20
Domain specificity

• The unusual character of HEP
• Long history since Manhattan
• Scale LHC
• Common knowledge, self-esteem, etc.

21
New Issues

• Human subjects issues
• IRBs across jurisdictional boundaries
• Need for new approach?
• Management
• NEESgrid management lags implementation
• InterMed need for tight management
• GriPhyN iVDGL hiring project managers
• AfCS charismatic management
• BIRN governance manual adding steering
  committee
• Vision
• Whos vision
• Acephalous projects (G. Bowker)
• Leadership issues charisma

22
New Issues

• What kind of technology?
• Specific applications vs. APIs
• Generic collab vs collab in specialized tools (S.
  Poltrock)
• Economics of the Grid (M. Cohen)
• Standards as a unifying process
• Politics of standards setting
• BIRN in a box
• If you build it, they will come
• Highly flawed model
• NEESgrid
• InterMed
• GriPhyN, iVDGL
• Tied to incentives
• Expectation management

23
New Issues

• Intellectual property
• Who negotiates?
• What are the arrangements?
• Evaluation
• Who does it?
• Within the project formative
• Outside the project summative
• What is it?
• Cross sectional
• Longitudinal
• Over what time period?
• Lag effects, long term indirect effects
• Be sophisticated
• science talk vs. informal talk (G. Bowker)

24
Biggest issues my candidates

• What is success?
• Evolution ecology
• Transition to production versions, sustaining the
  vision
• Data issues
• How to manage collaboratories?

25
Some SOC Issues

• Are we asking the right questions?
• Are we doing the right kinds of analyses?
• Measures
• Control groups
• Are our representations useful?
• Resource diagrams
• Mix of science and engineering