Scaled Agility

1
Scaled Agility Distributed Collaboration Improvi
sing a Grid for Particle Physics Dr Yingqin
Zheng Dr Will Venters Dr Tony Cornford
This research was undertaken as part of
Pegasus EPSRC Grant No EP/D049954/1 www.pegasus.
lse.ac.uk
2
http//www.youtube.com/watch?vceGuAUVAZ30feature
related
3
The Scale

• Currently constructing the worlds most powerful
  particle accelerator the Large Hadron Collider
  (LHC)
• Searching for Higgs Boson 1 person in 1000
  worlds, or a needle in 20 million haystacks
• 12-14 million gigabytes per year.
• 100,000 CPUs.
• 40PB disk, 40PB tape.
• Worlds biggest Grid

CD stack with 1 year LHC data ( 20 km)
4
(No Transcript)
5
Grids Technology

• Emerging platform for coordinated resource
  sharing and problem solving on a global scale for
  data-intensive and compute-intensive applications
  (Foster, 2001)
• As Internet protocols enable the sharing and
  integration of information on the Web, so Grid
  protocols aim to allow the integration of
  sensors, applications, data-storage, computer
  processors and most other IT resources
  (Wladawsky-Berger, 2004)
• Centred around standard protocols and middleware.
• 1 No central control.
• 2 Standard open protocols.
• 3 Non-trivial level of service.

Experiment layer
Application Middleware
Grid Middleware
Facilities and Fabrics
6
What Defines a (computing) Grid ?
7
The LHC Computing Grid

• Building the LHC Computing Grid (LCG)
• Highly distributed, complex and poorly defined
  systems development task.
• Cutting edge hardware and software used.
• New software standards being negotiated.
• Middleware and support software being developed
  in a range of languages.
• Grid must be distributed and proceed at different
  paces because of funding.
• Particle physics has a long tradition of such
  large scale global collaborations (Traweek 1988).

8
GridPP A Distributed Collaboration

• Collaboration of 230 people in 19 UK
  universities, RAL and CERN.
• Decisions are made democratically and
  consensually, and implemented by influence and
  persuasion.
• Network rather than hierarchy
• Virtual, federated, overlapping and
  inter-connected.
• Virtual meetings, wikis, blogs, mailinglists

9
How Often Do You Travel For Work?
10
Frequency and effectiveness of communication
methods
Frequency 1 - Never 2 - Rarely 3 -
Occasionally 4 - Regularly 5 - Very
often Effectiveness 1 - Very ineffective 2
- Ineffective 3 - Average 4 - Effective 5 -
Very effective
11
Distributed Management
12
How do you know what needs to be done in your
job?
Frequency 1 - Never 2 - Rarely 3 -
Occasionally 4 - Regularly 5 - Very
often Comment using meetings to update others
13
Bricolage
14
Essential skills for individual roles
15
Organizational Improvisation

• Metaphors
• Jazz (Weick 1992, 1999 Barrett 1998, Hatch
  1999)
• Improvisational Theatre (Crossan, 1998)
• Cunha (1999) the conception of action as it
  unfolds, by an organisation and/or its members
  drawing on available material, cognitive,
  affective and social resources
• Convergence in time of conception and execution
• Bricolage finding solutions from available
  rather than optimal resources

16
Analytical Framework
Improvisation-Paradox Related theoretical constructs Sources (organizational improvisation)
Paradoxes of Learning (Lewis 2000) Pragmatic Creativity environmental turbulence task uncertainty unplanned-for occurrences task complexity drop your tools visions (Moorman and Miner, 1998, Ciborra, 1996) (Dahlbom and Mathiassen, 1993) (Miner et al., 2001) (Hutchins, 1995, Weick and Roberts, 1993) (Weick, 1993a) (Hatch, 1999, Mintzberg and McHugh, 1985, Hutchins, 1991, Weick, 1993b)
Paradoxes of Learning (Lewis 2000) Retrospective Order retrospective sense-making ex post interpretation transient constructs Persistent structure (Weick, 1993b) (Lanzara, 1999) (Lanzara, 1999)
Paradoxes of Organizing (Lewis 2000) Oriented Drifting convergence of planning and execution mixing the pre-composed and the spontaneous magnetic fields minimal structure plan to improvise artful planning (Moorman and Miner, 1998) (Weick, 1998) (Weick, 1993a) (Cunha et al., 1999) (Miner et al., 2001) (Baskerville, 2006)
Paradoxes of Organizing (Lewis 2000) Managed Serendipity organized anarchy collateral structure experimental culture the aesthetic of imperfection a sense of urgency. (Cohen et al., 1972) (Cunha et al., 1999) (Cunha et al., 1999) (Weick, 1999) (Crossan, 1998, Hutchins, 1991, Mirvis, 1998)
Paradoxes of Belonging (Lewis 2000) Collective Individuality (Mirvis, 1998) facilitative leadership trust and kinship emotional communication hanging out fluid communication. (Crossan, 1998) (Crossan, 1998, Weick, 1993a) (Hatch, 1999) (Barrett, 1998) (Orlikowski, 1996, Miner et al., 2001)
Paradoxes of Belonging (Lewis 2000) Anxious Confidence (Mirvis, 1998) Individual skills and creativity formative context organizational memory moods (Hutchins, 1991, Moorman and Miner, 1998, Orlikowski, 1996) (Ciborra and Lanzara, 1994) (Moorman and Miner, 1998) (Ciborra, 2002)
17
Pragmatic Creativity

• GridPP faces many unplanned for occurrences and
  environmental turbulence in funding, human
  resources, external and internal technological
  changes, hardware and software configurations,
  user requirements from the experiments, computer
  market conditions, and other institutional and
  political factors.
• The project is committed to something that it
  isnt quite funded (PMB member).
• we have somehow learned how to organize
  things, at project management level and how to
  get things, to take the pragmatic view and to,
  faced with a problem, how to get from here to the
  solution... not just in GridPP but in building
  hardware and building detectors... Theres this
  background in problem solving and project
  management and the sort of pragmatic approach.

18
Retrospective Ordering

• A significant part of GridPPs activity, achieved
  by various means both formal and less formal,
  lies in monitoring, accounting for, and making
  sense of the behavior and performance of the
  system so far.
• With a range of different service challenges
  undertaken regularly statements such as we have
  to understand what is causing this phenomenon or
  find out what is behind the data are commonly
  heard during meetings.
• There is then Knorr-Cetinas (1999) humming of
  collaboration with itself, about itself, which
  maintains a constant collective reflexivity,
  exemplifying Giddens (1984) monitored character
  of the ongoing flow of social life and which
  makes retrospective sense-making an inherent and
  natural component in their process of system
  development.

19
Oriented Drifting

• I think you need to keep enough of an idea of
  the general direction which represents progress,
  and the very specific goals which advance you
  You need your head in the clouds to see the big
  picture, but you very much need your feet on the
  ground because you have to put one foot in front
  of the other, and day to day we keep putting one
  foot in front of the other and different
  people, depending on their role in the project
  are more oriented towards the ultimate goal or
  more oriented towards the little concrete
  footsteps that need to be taken...
• We wanted to establish the fact that we had the
  right to change our deliverables. So we set up
  this project map and we set up the formality of
  change forms. So this was to formalise our
  freedom to change the project and at the next
  Oversight Committee we managed to get this sort
  of structure through to them that yes, we had a
  set of milestones but you know, we had a
  mechanism to change them because we have to be
  responsive. (PMB member).

20
Managed Serendipity

• physicists are happier with an ad hoc solution
  just to get the job done and push them through.
  A physicist also highlighted this saying that
  while computer scientists will put together the
  most elegant thing in the universe, but it will
  never work Physicists will come up with the most
  hacked solution in the world but it will work.
• Management in GridPP does not rely on vertical
  lines of command, and while there is an extensive
  structure of management boards, committees, and
  technical groups, they serve more as
  communication channels than authority
  hierarchies. Managerial roles in the
  collaboration serve most of the time as
  representatives, spokesperson, or coordinating
  facilitators.
• Different solutions often compete with each other
  within the collaboration for a while until one of
  them wins by forming more alliances or others die
  in a natural course e.g. due to technical
  failures, low up-take, lack of funding or other
  circumstances. The technical systems then emerge
  from contests of unfolding (Knorr-Cetina 1999)

21
Collective Individuality

• This environment is based on, if you want,
  charismatic leadership and people doing things
  relatively independent but also having the
  freedom to do them, and not having to report
  every two minutes on what they are doing.
• Everyone trusts each other to be doing the best
  they can That fundamental trust drives our
  particle physics group. You have to trust that
  people will step up and do the dirty work as
  well as doing the glamorous work.
• Going to the pub together when they meet, for
  example, is one aspect of it. It fosters a bond
  between people many aspects of working in this
  project are frustrating because it's so large.
  And so if you can go out together you can
  identify the problems and let out steam about
  them

22
Anxious Confidence

• The project is constantly fire-fighting,
  discovering problems, managing crises, and
  negotiating solutions.
• Yet almost everybody in the collaboration that we
  have asked have a firm belief that the Grid will
  work it may not work perfectly, but it will
  work. There is a high level of confidence despite
  the sense of urgency and chaos on the surface.
• Source of confidence
• Individual competence
• HEP history of front-edged computing
• HEP tradition of distributed collaboration
• Aesthetics of imperfection

23
Scaled Agility

• Scaled agility as an organization capability
  distributed, innovative, flexible, under
  constraint in time and resources, and more or
  less decentralized.
• GridPP as a case of Scaled Agility
• which on the surface seems chaotic, haphazard,
  unplanned, and full of tension.
• Underlined by leadership, planning, strong
  commitment, emergent order, and the collective
  dynamic capability
• The framework of paradoxes of improvisation
  allows us to examine system development practices
  in the project regarding aspects that are often
  pushed to the background in discussions of system
  development methodologies, such as environmental
  conditions, individual skills, organizational
  structure, communication pattern, interpersonal
  relationship
• Most studies of improvisation have stated that it
  is easier performed in a small group, such as a
  jazz band. Our case shows that it is possible in
  a large group, when the ambience is right.