Better Software Conference

1
Process Improvement Based on Models,
Implemented in Reality Better Software
Conference and EXPO W7 Wednesday, September 29,
2004, 1130 AM

• By Barbara Ainsworth CSQA, CSTE
• Process Plus International, LLC
• Phone [phone] Email ainsworth_at_papadocs.c
  om

2
Introduction
Experience demonstrates that no matter the end
product financial services, insurance, beer,
construction, etc. the foundation for and focus
on implementing process improvement remains
consistent. Using models as the basis for
improving processes makes success more likely
however, no single model has all the right
answers. Overview and comparison of popular
models Determine aspects of the available
models that are right for you Examples of
starting and reinvigorating process improvement
initiatives
Acknowledgements Thank you to Peer Review
participants Tom Ainsworth, Brad Carroll, Inger
Dooley , Quince Hedley, Kimi Ziemski.
Additional thanks to Brad Carroll for his UML
expertise and willingness to share for the
benefit of the community.
Disclaimer and Limitation of Liability The
author has used best efforts in designing and
developing this presentation. There are no
representations or warranties with respect to
accuracy or completeness of the contents of this
publication and specifically disclaim any implied
warranties or merchantability or fitness for any
particular purpose and shall in no event be
liable for any loss of profit or any other
commercial damage, including but not limited to
special, incidental,consequential, or other
damages
3
Overview Terms Premise
Process Improvement An effort to improve the
processes used in an organization to accomplish
its business Model Framework Structured
collection of elements that describe
characteristics of effective processes Model-Based
Process Improvement A process improvement
effort that uses a model to appraise the quality
of its current processes, identify and prioritize
needed changes, and guide its improvement
activities Process Management Premise The
quality of a system is highly influenced by the
quality of the process used to acquire, develop
and maintain it System Boundaries The focus of
process improvement must have defined boundaries
that surround and contain the problem to be
addressed by the PI initiative. May be entire
organization, department, product, project, or
function.
4
Overview Why Use Models?

• IMPROVE THE BOTTOM LINE !
• Business objectives are traceable to deliverables
• Internal operational efficiency lower costs
  less rework
• Metrics indicate bottom line impact
• Greater customer retention and satisfaction,
  increased market share, and improved
  profitability
• Evidence suggests a long-term link between the
  use of models and improved business performance,
  growth, and prosperity in the world marketplace
• Some models are a means to earn certifications
  or awards
• Business contracts may require certifications
  based on models
• Some organizations use resultant
  certifications/awards as marketing tools

5
Overview Why Use Models?

• Provide answers to important questions related to
  organizations current maturity
• Assess maturity of entire or specific parts of
  the organization identify strengths areas for
  improvement
• Promote organizational maturity awareness among
  senior management
• Attribute organizational success to process
  management
• Better employee relations, higher productivity
• Manage development, acquisition, and
  contractors/outsourcing processes
• Cohesive, comprehensive approach to guiding
  individuals, managing projects and achieving
  organizational strategies

PROCESS IMPROVEMENT REQUIRES FRAMEWORKS Models !
6
Overview About Models

• Numerous models from various organizations
• Membership and/or Public Information
• Some models are related some content matches,
  overlaps, and/or links
• Models provide starting place, benefit of
  experiences, common language/shared vision,
  framework for prioritizing actions, guide to
  define improvement
• Support measurement framework for assessment
• Accepted widely across the US and around the
  world
• Models improve over time

7
Overview About Models

• Risks
• There is no silver bullet
• All models are wrong some are useful - George
  Box
• Need to expand the depth and breadth in order to
  successfully implement
• Some provide high level guidelines - the what
  others provide more details - the how
• Still need to address crucial project success
  issues
• professional judgement appropriate model
  interpretation
• expertise in particular application domains
• determination of specific software technologies
• selection/hiring/motivating/and retaining
  competent people
• Some provide cross functional focus others
  maintain stovepipes
• Need STRONG implementation management

8
Overview Models In Review

• Software Engineering Institute (SEI)
• IDEAL? Model
• Software Capability Maturity Model (SW_CMM?)
• Capability Maturity Model Integrated (CMMI?)
• Personal Software Process (PSPSM)
• Team Software Process (TSPSM)
• International Standards Organization (ISO)
  Family of Standards
• Quality Assurance Institutes (QAI)
  Implementation Approach Model and Bodies of
  Knowledge
• Project Management Institutes (PMI) Models and
  Body of Knowledge
• Malcolm Baldrige National Quality Award (MBNQA)
• Six Sigma
• Agile
• Unified Modeling language (UML)

9
Overview SEI IDEAL? Model
Learn from experience improve ability to adopt
new technologies
Software Engineering Institutes
IDEAL? Model
Learning
Analyze
Propose
and
Validate
Future
Implement
Actions
Solution
Refine
Solution
Stimulus for
Set
Build
Charter
Change
Acting
Context
Sponsorship
Infrastructure
Do work accordingto plan
Pilot/Test
Solution
Initiating
Characterize
Lay groundwork for successful improvement
Current
Desired States
Create
Solution
Develop
Diagnosing
Recommendations
Determine where you are relative to where you
want to be
Plan
Actions
Set
Develop
Priorities
Approach
Plan specifics how to reach destination
sei.cmu.edu
Establishing
10
Overview SEI SW_CMM?
SEI Software Capability Maturity Model v 1.1
Level
Focus
Result
Key Process Areas
5 Optimizing
Defect prevention Technology innovation Process
change management
Productivity Quality
Continuous process improvement
4 Managed
Product and process quality
Process measurement and analysis Quality
management
Organization process focus Organization process
defn. Peer reviews Training program Inter-group
coordination Software product engineering Integrat
ed software mgt.
Engineering processes organization processes
defined Performance more predictable
3 Defined
Requirements Mgt. Software Project
planning Software Project Tracking
Oversight Software Quality Assurance Software
Configuration Mgt. Software Subcontract
Mgt. Software Test Mgt. DRAFT
Project management in place, individual
performance repeatable
2 Repeatable
1 Initial
Process informal and ad-hoc, unpredictable
performance
Risk
sei.cmu.edu
11
Overview SEI CMMI?
SEI Capability Maturity Model Integrated v1.1
Staged
Process Areas
Focus
Level
5 Optimizing
Organizational Innovation and Deployment Causal
Analysis and Resolution
Continuous Process Improvement
Quality Productivity
4 Quantitatively Managed
Organizational Process Performance Quantitative
Project Management
Quantitative Management
Requirements Development Technical
Solution Product Integration Verification Validati
onOrganization Process Focus Organization
Process Definition Organizational
Training Integrated Project Management Risk
ManagementIntegrated Teaming Integrated Supplier
Management Decision Analysis and
Resolution Organizational Environment for
Integration
Process Standardization
3 Defined
Requirements Management Project Planning Project
Monitoring Control Supplier Agreement
Management Measurement Analysis Process and
Product Quality Assurance Configuration
Management
BasicProjectManagement
2 Managed
Risk / Rework
1 Initial
2002-2003 by Carnegie Mellon University
Introduction to CMMI - Staged V 1.1 Module
9-090103 Pg. 12
12
Overview SEI TSPSM/PSPSM
Team Software ProcessSM Personal Software
ProcessSM
PSPSM Skill-building
TSPSM Team-building
TSPSM Team-working
Personal measures Process discipline Estimating
planning Quality management
Project goals Team roles Team process Project
plan Balanced plan
Risk analysis Team communication Team
coordination Status tracking Project reporting
Team Members
Team Management
Team Disciplines
Integrated Product Teams
http//www.sei.cmu.edu/tsp/mapping-tsp-cmmi.pdf
Nov 2003
13
Overview ISO

• International Standards Organization
• Quality Management System (QMS) has essential
  features where relevant ISO 9000/ISO 14000
  families of standards provide requirements
• THE ISO 9000 2000_12_15 FAMILY OF STANDARDS -
  ISO 9000 Quality Mgmt. Systems - Fundamentals
  and Vocabulary ISO 9001 Quality Mgmt.
  Systems - Requirements ISO 9004 Mgmt.
  Systems - Guidelines for Performance
  Improvements ISO19011 is applicable to both
  quality and environmental mgmt system audits -
  The 20 elements in ISO 9001 contain five main
  clauses titled Quality Mgmt.System
  Mgmt.Responsibility Resource Mgmt. Product
  Realization Measurement, Analysis Improvement
  - ISO 9001 Quality Management System (QMS)
  Contents (free samples available)
  Overview Manual Preparation QMS Quality Manual
  QMS Quality Procedures EMS Quality Procedures
  Reports and Forms Index - The new
  standards have been aligned with the existing
  standards
• ISO/IEC for Software software search 71
  standards 1994 -2001 - ISO/IES 900032004
  Guidelines for applying ISO 90012000 to
  software - ISO/IEC 12207 IT SW Life Cycle
  Processes - ISO/IEC 15504 IT SW Process
  Assessment

www.iso.orghttp//www.iso.org/iso/en/CatalogueDet
ailPage.CatalogueDetail?CSNUMBER35867ICS135ICS
280ICS3
Requires 3rd Party Certification
14
Overview QAI

• Quality Assurance Institutes Implementation
  Approach Model
• Bodies of Knowledge containing Skill Categories
  for
• Quality Assurance, Testing, and Project Management

www.qaiusa.com
15
Overview PMI

• Project Management Institute
• PROJECTS - Project Management Body of Knowledge
  (PMBOK)-
• Initiation, Scope, Time, Cost, Quality, Human
  Resources, Communication, Risk, Procurement
• PEOPLE - Project Manager Competency Framework
  (PMCF)-2002
• Improving performance of project personnel
• ORGANIZATIONS - Organizational Project Management
  (OPM3)-03
• a standard for organizational project management
  andorganization project management maturity
• describes an organization's overall ability to
  select and manageprojects in a way that supports
  its strategic goals
• Elements Knowledge Foundation, Assessment
  Improvement
• use in concert with PMBOK and PMCDF

www.pmi.org
16
Overview MBNQA

• Malcolm Baldrige National Quality Award
• The Baldrige performance excellence criteria
  categories focus
• Leadership-Senior executives guidance
  organizations responsibilities to the public
  practices good citizenship
• Strategic planning-Setting strategic directions
  key action plans
• Customer and market focus-Customer and market
  requirements expectations customer
  relationships
• Measurement, analysis, and knowledge
  management-Data management key processes
  performance mgmt. system support
• Human resource focus-Workforce development
  alignment with business objectives
• Process management- Design, management, and
  improvement of key production/delivery and
  support processes
• Business results-Key business areas performance
  improvement performance relative to competitors

17
Overview Six Sigma

• Highly disciplined process to focus on developing
  and delivering near-perfect products and services
• There are three key elements of quality
  Customers Delighting Customers Process
  Outside-In Thinking Employee Leadership
  Commitment
• All GE employees are trained in the strategy,
  statistical tools and techniques of Six Sigma
  quality. Training courses are offered at various
  levels
• Six Sigma Quality Approaches and Models
• Design for Six Sigma (DFSS) a systematic
  methodology utilizing tools, training
  measurements to enable products and processes
  design that meets customer expectations be
  produced at Six Sigma quality levels.
• Define, Measure, Analyze, Improve and Control
  (DMAIC) a systematic, scientific, fact based,
  closed-loop continuous improvement process
• Six Sigma A vision of quality which equates
  with only 3.4 defects per million opportunities
  for each product or service transaction. Strives
  for perfection.

18
Overview Agile

• Agile Software Development
• A Philosophy - 4 value statements and 12
  principles (Agile Manifesto)
• A Set of methods - over 10 methods method
  families
• A collection of practices - e.g., 19 engineering
  practices in eXtreme Programming (XP) alone
  http//www.sei.cmu.edu/tsp/tug-2003-presentations/
  smiley/
• Be Agile Slash the budget If it doesn't work,
  kill it Keep requirements to a minimum Build
  on success, not hope Keep your development teams
  small Assign non-IT executives to software
  projects http//www.cio.com/research/current/agile
  

www.agilemanifesto.org/
Note cautions at comparison slides
19
Overview UML

• Unified Modeling Language
• Object Management Group - Independent
  non-profit organization that researches,
  maintains and publishes UML, Common Warehouse
  Metamodel (CWM), MOF 2.0 XMI ( XML Metadata
  Interchange) and other standards
  http//www.omg.org
• Nine UML Models Use Case, Class Diagram,
  Sequence Diagram, Activity Diagram, State
  Diagram, Collaboration Diagram, Component and
  Deployment
• Universality Commonly accepted and used
  throughout software industry, domestically and
  internationally, to model and communicate
  software functional specifications, architecture
  and designs

20
Comparison Commonality

• Terminology can be industry common or model
  unique
• Process Focus
• Required processes and process content differ by
  model
• Processes,procedures,practices,documentation,gates
  /status indicators, etc.
• Tools,Training,Support,Assessments,Metrics,
  Continuous Improvement
• Improved over time
• Numerous references, tools, and other guidance
  interpreting, documenting, training, FAQs and
  answers, survival guides, business mapping
  techniques, support links, sample documentation,
  style guidelines, workbooks, etc.
• Certified companies
• Compared to other models
• Can be successfully used or disastrous

Input
Tasks, Tools Techniques
Output
21
Comparison SEI CMM/CMMI
Level
CMMI Process Areas
SW_CMM v1.1 Key Process Areas
5 Optimizing
Defect Prevention Technology Change
Management Process Change Management
Causal Analysis and Resolution Organizational
Innovation Deployment
4 Managed
Quantitative Process ManagementSoftware Quality
Management
Organizational Process Performance Quantitative
Project Management
Organization Process Focus Organization Process
Definition Training Program Integrated Software
Management Software Product Engineering Interg
roup Coordination Peer reviews
Organization Process Focus Organization Process
Definition Organizational Training Integrated
Project Management Risk ManagementRequirements
Development Technical Solution Product
Integration Verification ValidationDecision
Analysis and Resolution
3 Defined
Requirements Management Software Project
Planning Software Project Tracking
Oversight Software Subcontract Management Software
Quality Assurance Software Configuration
Management Software Test Management DRAFT
Requirements Management Project Planning Project
Monitoring Control Supplier Agreement
Management Product and Process Quality
Assurance Configuration Management Measurement
Analysis
2 Repeatable
2003 Carnegie Mellon University Software
Engineering Institute
22
Comparison CMMI /TSPSM/PSPSM
CMMI Builds organizational capability
TSPSM Builds quality products on cost and schedule
PSPSM Builds Individual skill and discipline
http//www.sei.cmu.edu/tsp/tsp.html
23
Comparison CMM/ISO9000
SEI SW_CMM compared to ISO-9001 Compliant
Organization
The Capability Maturity Model Guidelines for
Improving the Software Process CMU SEI
pg.417 Software Engineering Institutes
Technical Report CMU/SEI-94-TR-12 See
appendices re detailed listing - what practices
are addressed where.

• SEI CMMI compared to ISO-90012000
• Similarities, differences, synergies
• CMMI missing some ISO items

http//www.sei.cmu.edu/cmmi/presentations/sepg03.p
resentations/cmmi-iso.pdf
24
Comparison PMI/SEI/QAI
Sample Model Comparison -Not intended to be
comprehensive
25
Comparison

• MBNQA to ISO 9000 http//www.baldrige.nist.gov
  - ISO 9000 registration covers less than 10 of
  Baldrige Award criteria - Responsible for
  making quality a national priority
  disseminating best practices across USA
  - For organizations utilizing Six Sigma Quality,
  TQM, ISO9000 and process improvement -
  MB feedback report is arguably the best bargain
  in consulting in USA improve organization
  or business overall health
• CMMI To OTHER MODELS http//www.sei.cmu.edu/cmmi/
  adoption/comparisons.html - CMMI to Six Sigma
  to Achieve Levels 4 5 DMAIC process aligns
  with CMMI practices
  http//www.sei.cmu.edu/cmmi/presentations/sepg04.p
  resentations/sixsigma.pdf - Six sigma process
  is applied to CMMI process improvement
  optimizing processes at Level 5
  http//www.dtic.mil/ndia/2003CMMI/Kovar.ppt
  - Using CMMI to Balance Agile and
  Plan-Driven Methods
  http//www.dtic.mil/ndia/2003CMMI/turner.ppt

26
Comparison

• Agile to ISO and CMM http//www.computer.org/compu
  ter/homepage/0603/GEI/ - Some common CMM
  ISO practices at odds with Agile value set
  - CMM relies on processes, plans, and
  delivering to original contract ISO
  requirements not met - If need CMM
  certification less interest in Agile value
  proposition - If need agility for
  business reasons less interest in CMM or ISO
  9000 certification
• TSPSM Compared to Agile http//www.sei.cmu.edu/tsp
  /tug-2003-presentations/smiley/
• Part compatible with TSPSM/PSPSM not all
  TSPSM/PSPSM met
• In practice, the two approaches tend to align
  with different- Philosophies on design-
  Degrees of formalism in design verification-
  Levels of team data gathering and analysis

Caution - Agile Development has hit a nerve.
Some argue for it, some strongly aginst it, and
others mix agile plan-driven approaches.
http//www.computer.org/computer/homepage/0603/GEI
/- Considered Harmful for Reliable Software
Development - 6/2/02 http//www.agileallianc
e.org/articles/articles/XPConsideredHarmful-Gerald
Keefer.pdf
27
Comparison UML / SW_CMM
Sample Model Comparison -Not intended to be
comprehensive
28
Comparison Chart

• Mappings are subjective
• Data shows that some models/frameworks
• measure for Awards, Certifications,
  Classifications
• meet and exceed requirements of others
• partially meet/provide high level guidance for
  others
• support/can be used to help achieve requirements
  of others
• have some conflict with others http//www.sei.cmu.
  edu/tsp/tug-2003-presentations/smiley/
• lack sufficient data for comparison

Which fit your organizations criteria?
29
Determine Right Model Factors

• Symptoms Indicating Need for Process Improvement
• Slow to market product development
• Difficulty translating strategic vision to
  operational reality
• High failure rate of software projects
• No defined process owners
• Depend upon heroics of one or more people to get
  things done
• Organizational silos inhibit communication
• Overly dependent upon outside experts for product
  or project knowledge

30
Determine Right Model Factors

• Categories of factors in your decision
• Business - How well known are the business
  objectives? How mature are the current processes?
  How strongly are processes mapped to the
  business objectives? Will improvements be made
  across the organization or within a product line?
  Which business objectives will be supported by
  the Process Improvement Program (PIP)?
• Culture - What is the organizations ability to
  deploy a PIP? Is the organization process based
  and experienced in PI? Does a specific process
  need improvement quickly?
• Legacy - Is the organization experienced with a
  model? Has the organization invested resources
  and deployed processes across the organization
  using a model?

CMMI Guidelines for Process Integration and
Product Improvement
31
Determine Right Model Tips

• Review empirical data of organizations using the
  model
• SEI data sei.cmu.edu SEIR is repository of data
  must register to use no cost can re-register
  and de-register not as much data on CMMI as
  SW_CMM
• MBNQA organizations are required to
  communicate/share
• ISO certified companies are listed
• Conference materials are on the internet
• Match business needs criteria to model
• What is expected ROI?
• Define the process improvement system boundaries
• Criteria considerations
• Standard yet scalable, flexible and customizable
• culture, staff size, personnel, system critically
  consequences of error, application domain,
  size, optimized quality, dynamism,
  innovativeness http//www.sei.cmu.edu/tsp/tug-2
  003-presentations/smiley/smiley.pdf

32
Start/Re-Energize PI Current Status?
Where are you?
Whats in place?
Whats Missing?
Elements of Change Effects of Missing
LinksSEI Visitors Day, June 20, 1996 -
Transition Enabling, Mike Phillips from Ambrose,
1987
33
Start/Re-Energize PI Tips

• Organizational Commitment
• Choose your approach to process improvement
• Models are the foundation need to expand depth
  and breadth in order to successfully implement
• Ensure Trainers, Consultants are in-the-trench
  experienced
• Deploy Project Management practices on PI
  Initiative
• Team Approach
• Address Whats in it for me
• Implementation is Key - Institutionalization is
  Critical !
• Only Some Model Experts are needed when the
  processes are compliant to model requirements
• Drivers influencers access to all training
  compliance small wins measure improve Manage
  Resistance

34
Start/Re-Energize PI Example

• Example Use Adaptive or Predictive Process?
  Consider the following...
• 1.) (a) Factors suggest adaptive process
• Uncertain or volatile requirements
• Responsible and motivated developers
• Customer who understands will get involved
• (b) Factors suggest predictive process
• A team of over a hundred
• Fixed price, or more correctly a fixed scope,
  contract http//www.martinfowler.com/articles/newM
  ethodology.html
• 2.) need developers and business execs
  alikewho can follow ...principles of
  flexibility, teamwork, trust, reflection.
  ...these environments...few far between.
  http//comment.cio.com/soundoff/081601.html
• 3.)(a) Combine TSPSM/Agile advantages based on
  orgs project characteristics(b) Criteria
  considerations culture, staff size, personnel,
  system criticallyconsequences of error,
  application domain, size, optimized quality,
  dynamism, innovativeness http//www.sei.cmu.edu/ts
  p/tug-2003-presentations/smiley/smiley.pdf

If adaptive then...? If predictive then?
Not intended as a comprehensive list of
considerations.
35
Start/Re-Energize PI Examples

• Implementation Examples
• Malcolm Baldrige National Quality Award
• 1988-2003 Recipients Contacts Profile
  http//www.baldrige.nist.gov/Award_Recipients.htm
• Motorola
• First MBNQA winner Implemented "Motorola
  University, Six Sigma, Total Quality Management
  (TQM), in early 1990's.
• Includes rear-end software to track measurements
• Six Sigma
• General Electric http//www.ge.com/sixsigma
• Originated Six Sigma the way they do
  business everyone responsible trained
• Raython http//www.dtic.mil/ndia/2003CMMI/Kovar.pp
  t
• Six Sigma Process and Its Application for CMMI
• ISO
• Certified Companies search Yahoo.com by ISO
  CERTIFIED COMPANIES provides specific listings,
  directories of all listings, etc.

36
Start/Re-Energize PI Examples

• Implementation Examples
• IDEAL, SW_CMM, CMMI,QAI, PMI by Barbara
  Ainsworth ainsworth_at_papadocs.com
• Focus on Process Improvement using the IDEAL in
  concert with
• Level 2 and Level 3 KPAs transition to CMMI
  Level 2 3
• Project Management, Quality Assurance, and
  Testingprinciples and practices
• IDEAL ISO CMMI by Hughes Bearing Point
  2003 http//www.sei.cmu.edu/cmmi/presentations/se
  pg03.presentations/cmmi-iso.pdf
• Focus on Process Improvement using the IDEAL
  problem solving process and ISO 90012000 and
  CMMI synergy
• CMMI Level 2 to Level 5 by David F. Rico
• Implementation Plan http//davidfrico.com/s-cmmi-i
  mp-plan.pdf
• Work Breakdown Structure http//davidfrico.com/s-c
  mmi-wbs.pdf
• Policies and Procedures (gt500 pgs.)
  http//davidfrico.com/s-cmmi-procs.pdf
• Cost Model http//davidfrico.com/s-cmmi-cost-mode
  l.pdf

37
Summary

• Base process improvement on business objectives
  trace through implementation measure ROI
• Models/frameworks are the foundation need to
  expand depth and breadth in order to successfully
  implement
• Choose and use models/frameworks wisely
• Implementation is key implement compliant
  processes and measure compliance
• Change takes time, commitment, resources

Questions? Need help? Contact info is on title
page...
38
References
39
References
40
References
41
References