Agile Project Management and Performance Measurement

1
Agile Project Management and Performance
Measurement
Dr. Brian M. Barry Chief Technical Officer

• info_at_xiasystems.com

2
Outline

• Motivation
• Example Just-In-Time-Software Method
• Underlying Principles
• Basics of Agile PM
• Performance Measurement

3
In the Beginning Was the Waterfall Model
A process designed by the managers to keep the
customers from understanding what the programmers
are really doing Anonymous
Analysis
Design

• You start with Analysis, and after that things
  just keep going down hill Stu Feldman

Coding
Testing
Maintenance
4
The Waterfall Revisited

• First articulated (although not named) by Winston
  Royce
• Managing the Development of Large Software
  Systems, IEEE WESCON, 1970
• It may come as a surprise but Royces point was
  that the model was a useful but naïve
  idealization he certainly wasnt advocating that
  software should be built this way!
• But it was simple and easy to understand..
• Subsequently there were many variants
• Spiral Model, DOD-STD-2167A ,
• Despite various improvements, all were embedded
  with the basic Waterfall assumptions about how
  software is conceived, implemented and deployed
• A recent survey showed that the Waterfall Model
  is still the dominant software process model
  ACM Queue Oct. 28 2005

5
So Whats the Problem?

• System Development is a change management process
• The normal case is a change from something to
  something else
• The first development cycle is special, a change
  from nothing to something
• Waterfall and its successors are preoccupied with
  the first cycle
• Getting the requirements right with todays
  complex systems is hard and requires iteration
• Worse yet, in the typical case the requirements
  keep changing
• The mainstream software industry has a horrible
  track record for failing to delivering working
  software on time and within budget
• Widely report that 80 or more of all projects
  fail
• Is there a way to work with more confidence and
  realistic visibility into the process?

6
And Now for Something Completely Different
Agile Technology
Leap of Faith
Traditional Technology
7
Agile Practices Offer An Alternative

• We are uncovering better ways of developing
  software by doing it and helping others to do it.
• Through this work, we have come to value
• Individuals and interactions over processes and
  tools.
• Working software over comprehensive
  documentation.
• Customer collaboration over contract negotiation.
• Responding to change over following a plan.
• That is, while there is value in the items on the
  right, we value the items on the left more.
• From The Manifesto for Agile Software
  Development, 2001

8
Why Agile Evolution?

• Respond to software grid lock concurrent
  releases with increased defects and fewer
  features.
• Respond to the business/customer needs.
• Make software delivery predictable.
• Achieve predictable and reproducible quality.
• Retain key domain and technical staff and support
  them with best practices.
• Cope with complexity of legacy and current
  technology.
• Successfully transition people and products as
  time and budget permit.

9
Just In Time Software

• A proven, customer-driven approach used over 20
  years to build major products and applications
  ranging from embedded systems to mainframes
• Applied to software tools, banking and HR
  applications, instrumentation, defence
  electronics, pervasive devices..
• Taught by mentoring and coaching
• Uses a development centric model-driven approach
• Assumes talented motivated people, managers who
  understand peopleware, team leads who coach as
  well as code
• Leverages modern incremental tool environment
• Time box-based method that stresses timely
  results prioritized by customers over features
  defined by developers or customers
• Component-oriented with strong code ownership and
  developer commitment
• Software teams include everyone involved with the
  product customer, developer, tester, technical
  writer...

10
Used to Build both VisualAge and Eclipse

• Team of 150 OTI engineers in 10 locations, 300
  IBM engineers in 5 locations
• Development challenges
• Multiple programming languages (assembler, C,
  C, Smalltalk, Java)
• Multiple platforms (Windows, Linux, Unix, Apple,
  OS/390, )
• Constantly evolving class libraries, language
  specifications, other requirements
• Business challenges
• Always starting a year or more later than
  competitors
• International language support required for IBM
  products
• Leverage substantial IBM application and systems
  software assets
• Results
• VA Java was a modern IDE competitive with MS, put
  IBM on the map for tools
• Eclipse is the most popular tool platform on the
  planet, with 50 million downloads (Computerworld,
  August 2005)

11
Underlying Agile Principles

• Highest priority is to satisfy the customer
  through early and continuous delivery of valuable
  software
• Welcome changing requirements, even late in
  development. Agile processes harness change for
  the customers competitive advantage
• Deliver working software frequently, from a
  couple of weeks to a couple of months, with a
  preference to the shorter timescale
• Business people and developers must work together
  daily throughout the project
• Build projects around motivated individuals.
  Give them the environment and support that they
  need, and trust them to get the job done.
• The most efficient and effective way of conveying
  information to and within a development team is
  face-to-face conversation.

12
Underlying Agile Principles (continued)

• Working software is the primary measure of
  progress.
• Agile processes promote sustainable development.
  The sponsors, developers, and users should be
  able to maintain a constant pace indefinitely.
• Continuous attention to technical excellence and
  good design enhances agility.
• Simplicity - the art of maximizing the amount of
  work NOT done - is essential
• The best architectures, requirements, and designs
  emerge from self-organizing teams
• At regular intervals, the team reflects on how to
  become more effective, then tunes and adjusts its
  behavior accordingly

13
Which Method is Right?

• Every Drivers Ed instructor (and every
  methodologist) has his/her own particular version
  of how to drive
• When youre a novice its best to pick one and
  stick with the program
• A lot of your time and attention is still on the
  mechanics
• Every instructor is teaching the same basic
  principles
• Once youre proficient you develop a deeper model
• Pay more attention to the big picture
• Skip or merge steps when it makes sense
• Learn from other drivers
• The same strategy works in software
• Pick one Agile Method and learn it well
• Explore the other processes and customize later

14
Agile Approach to Project Planning

• We plan
• To make sure we are working on the most important
  things at all times
• Not to predict the future
• No plan survives first contact with the real
  world
• This does not mean that planning is unimportant
• A high level plan or vision is universally
  important, but a detailed and inflexible plan
  will become the enemy of success
• Dont live the lie of a PERT or GANTT Chart
• Your development efforts will improve your
  understanding of the problem and the solution
• As your understanding improves, you will need to
  change your plan and priorities accordingly
• Engage in Planning, dont be a slave to a Plan
• Plan in detail for a day or week by keeping to do
  lists
• Plan specific deliverables for 30-60 days in
  timeboxes
• Plan roughly or crudely beyond 60 days

15
Decomposing a Project

• A set of many User Stories will comprise a
  Project
• We expect the stories will change a lot
• The Project is planned into Releases
• Releases typically last 2-3 months
• Releases are planned into Iterations
• Iterations typically last 2-3 weeks
• Iterations are planned into Tasks
• Tasks typically last 1-2 days
• Tasks are planned into Test Cases
• Test Cases typically take 5-20 minutes to develop

16
Iteration Planning

• The team, and the customer, negotiate the next
  iteration
• Developers review story estimates
• Customers shop for the user stories they want in
  this iteration
• The team cannot accept more units of work than
  they completed in the previous iteration
• The first release is divided into a series of
  iterations
• Each iteration will implement several stories
• The customer decides which stories go in the
  first iteration
• The developers decide which order to develop the
  stories within the iteration
• Stories chosen for an iteration must not exceed
  teams velocity
• When the plan is wrong
• If the team finishes early, they ask the customer
  for more stories
• If the team gets into trouble and cannot finish
  all the stories in time, they talk to the
  customer early and get the customer to remove
  some stories or tasks
• This affects what the team can commit to in the
  next iteration
• Never, ever, slip the date - reduce scope!

17
Task Planning

• The team breaks each story into a series of
  Engineering Tasks
• The tasks are estimated by the developers
• May require splitting of stories
• These estimates are a commitment
• Developers sign up for tasks
• Cant sign up for more than their current
  individual velocity
• The team (with the customer present) breaks the
  stories for the iteration into tasks
• The customer reads a story out loud
• The team discusses what needs to be done to build
  the story. Write these tasks on a whiteboard
• The team may notice commonality between stories
• Make these common things tasks
• May need some technical tasks
• Update vendors DB
• Try to make business cases for these

18
The Timebox Concept

• Fixed (short) time duration
• Generally a time box is scheduled for 30, 45, 60
  or 90 days
• Once set, it is never extended
• Variable, prioritized set of objectives
• Specific objectives are listed and prioritized
• Completing the greatest proportion of what can be
  done is goal
• Throw out tasks for which prerequisites are not
  ready
• Certain deliverables, with elastic content
• A working system must be delivered on time
• It is better to do 80 flawlessly, than 100
  unreliably
• Build Success into the project plan
• Delivering on a certain date builds a sense of
  accomplishment amongst the team
• As more timeboxes are completed, the planning
  improves and the hit rate increases, resulting
  in less overflow

19
Timebox Governing Principles

• Small teams are essential to success
• Deliverables should be assigned to small teams of
  2-5 people, let them work out the schedule
  details within the timebox
• Engage each team in the planning of their timebox
  contents, such as lining up resources and setting
  scope
• Delegation of authority and ownership of timebox
  tasks builds competence and reliability over time
• Deliverables must be on time, all the time
• It is better to deliver something incomplete that
  is well tested and working, than something that
  is late and unreliable
• Reassess the time box contents a the midpoint and
  defer the unattainable to the next timebox
• Never start a timebox if the checkpoints are not
  validated
• Without the checkpoints satisfied, you can
  guarantee that the tasks within the timebox will
  fail to be accomplished
• A willingness to say no, we are not ready and
  to re-evaluate can save a timebox from disaster
  or embarrassment

20
Agile Software Practices
Customer Collaboration
Continuous Integration
CollectiveOwnership
Test Driven Development
Acceptance Tests
Scrum
Pairing
Refactoring
Simple Design
Metaphor
Sustainable Pace
Small Releases
21
Test Driven Development
Start
Write a test for new capability
Compile
Refactor as needed
Fix compile errors
Run the test And see it pass
Run the test And see it fail
Write the code
22
Continuous Integration and Testing

• Continuous Integration with Automated Testing
  Environment
• Dedicated compile, build and test machines to
  perform hourly, daily runs
• Reports status of all compiles, builds, tests
• Avoids integration surprises
• Ensures that builds actually work
• Essential for Refactoring, TDD
• Executes Unit and Acceptance Tests
• Executes tests in realistic environment

23
Performance Measures Fundamentals

• What do we want to measure?
• Efficiency are resources being optimally
  deployed?
• Progress is the project on track for time and
  budget?
• Productivity how much code per unit of labor?
• Activity heartbeat, how active is the project
  day to day?
• Quality how good is the software being produced?
• Heisenberg Uncertainty Principle of Software
  Physics
• Once management starts collecting and publishing
  a particular metric it will affect the value, and
  change team behavior
• It will also change the implicit values of other
  (unpublished) metrics because effort will be
  redeployed
• Example Clean Room experiments at IBM Federal
  Systems

24
Scalable Metrics

• Metrics for Agile Development is an active area
  of research
• No broad consensus
• The following represents work in progress
• Not sufficient to measure the performance of a
  single team
• Additive roll up metrics across projects,
  divisions, enterprise
• Comparative compare teams across projects,
  divisions, enterprise
• Scalable Metric Additive Comparative
• In practice it is difficult to find metrics with
  both properties
• Use an Equivalence Pair of related metrics,
  one additive, the other comparative
• Usually related by a formula or relation

25
Scalable Metrics for Agile Development

• Agile Metrics Terminology
• user stories requirements
• product backlog the complete set of stories
• iteration backlog stories addressed in a
  particular iteration
• story point estimate of effort needed to
  implement a story
• velocity estimated sum of story points for
  completed stories
• burndown effort required to complete sprint
  backlog
• Velocity and burndown are dynamic measures, which
  are updated as the sprint progresses
• There is a lot of variation in how various Agile
  methods (and methodologists) calculate these
  measures
• If you get the spirit right then in practice it
  tends to work out

26
Velocity and Burndown
Ideal Burndown
Effort
Actual Burndown
Velocity
Time
27
Efficiency Metrics

• Velocity is the most commonly used measure of
  efficiency
• Additive as long as teams use the same measure of
  effort, e.g. ideal man-hours
• But not comparative
• Depends on team size, number of stories, length
  of sprint, etc.
• Calculate Normalized Velocity to obtain a
  comparative measure
• Define velocity estimate on day 1 Initial
  Velocity

Normalized velocity velocity / initial velocity

• Normalized velocity is not additive
• E.g. imagine we added the normalized velocities
  of several teams with good results to that of a
  very large bad team
• Velocity and normalized velocity are an
  equivalence pair

28
Progress Metrics

• Burndown is the most commonly used measure of
  progress
• As before, additive as long as teams use the same
  measure of effort, e.g. ideal man-hours, but not
  comparative
• Same reasons team size, number of stories,
  length of sprint, etc.
• Calculate Normalized Burndown to obtain a
  comparative measure
• Define burndown estimate on day 1 as Initial
  Burndown
• Represents the estimated total effort to complete
  the project backlog

Normalized burndown burndown / initial burndown

• Burndown and normalized burndown are an
  equivalence pair
• Again, normalized burndown is not additive

29

• Questions?