PROCESS ASSESSMENT

1
PART III

• PROCESS ASSESSMENT

2
Chapter 9 ISO/IEC 15504 Draft Std for Software
Process Assessment zahran

• Objective
• To introduce the ISO/IEC 15504 draft Standard for
  software process assessment
• To understand the structure of ISO/IEC 15504

3
Chapter 9 ISO/IEC 15504 Draft Std for Software
Process Assessment zahran

• Introduction
• Benefits
• Structure of ISO/IEC 15504
• ISO/IEC 15504 Capability Measurements
• Guide to Conduct Assessment

4
Introduction

• Humphrey points out ? the first step in any
  program to improve sw capability is to understand
  the current status of the development process
• Thus ? process assessment is the means of
  achieving this
• A number of assessment methods and models are
  available

5
Introduction

• The SPICE (Software Process Improvement and
  Capability dEtermination) project is an
  international collaborative effort to support the
  development of the international standards
• The project was established by the international
  committee on SE stds, ISO/IEC JTC1/SC7, through
  its Working Group 10 on software process
  assessment, in Jan 1993

6
Introduction

• Evolution of SPICE to ISO/IEC 15504 (refer to
  handout given in class)

7
Benefits of a standard for process assessment

• Organizations can use the ISO/IEC 15504 draft std
  in one of the following modes
• Capability determination mode to help determine
  the capability of a potential sw supplier
• Process improvement mode to help improve their
  own sw dev. Process
• Self-assessment mode to help determine their
  ability to undertake a new project

8
Benefits of a standard for process assessment

• Assists the following potential users to achieve
  the following objectives
• Purchasers to determine the capability of sw
  suppliers and assess the risk involved in
  selecting one supplier over another
• Acquirers ability to determine the current and
  potential capability of a suppliers sw processes
• SW suppliers ability to use one process
  assessment scheme instead of having to go through
  numerous schemes

9
Benefits of a standard for process assessment

• SW development organizations to have a tool to
  initiate and sustain a continuous process
  improvement programme
• Managers to ensure that their sw development
  process is aligned and supports the business
  needs of the organization
• Assessors a framework that defines all aspects
  of conducting assessments

10
Architecture of the ISO/IEC 15504 Draft Std

• The set of documents initially developed by the
  SPICE project and currently published by the ISO
  as the draft ISO/IEC 15504 (refer to handout
  given in class)
• Components of the ISO/IEC 15504 Std

11
Structure of ISO/IEC 15504 Reference Model

• ISO/IEC 15504 reference model architecture for
  processes and process capability is made up of 2
  dimensions
• Process dimension ?characterized by process
  purposes which are the essential measurable
  objectives of a process, and the expected outcome
  of the process that indicates its successful
  completion

12
Structure of ISO/IEC 15504 Reference Model

• Process capability dimension ? characterized by a
  series of process attributes, applicable to any
  process, and represents measurable
  characteristics necessary to manage a process and
  improve its capability to perform

13
Process Dimensionrefer to handout

• The model defines 40 processes that comprise and
  characterize the sw life cycle. These processes
  are classified into 5 process categories
• The customer-supplier process category (CUS)
  ?consists of processes that directly impact the
  customer, support development and transition of
  the sw to the customer, and provide for its
  correct operation and use

14
Process Dimension

• The engineering process category(ENG)? consists
  of processes that directly specify, implement, or
  maintain a system and sw product and its user
  documentation
• The support process category (SUP)? consists of
  processes that may be employed by any of the
  other processes(includes other supporting
  processes) at various points in the sw life cycle

15
Process Dimension

• The management process category (MAN) ? consists
  of processes that contain practices of a generic
  nature, which may be used by anyone who manages
  any sort of project within a sw life cycle
• The organization process category (ORG)? consists
  of processes that establish the business goals of
  the organization and develop process, product and
  resource assets that, when used by the projects
  in the organization, will help the organization
  achieve its business goals

16
Process Dimension

• Each process is defined through a detailed
  statement of the purpose of the process, together
  with a list of the expected outcomes of executing
  the process

17
Capability Dimension refer to handout

• Defines an ordinal scale for expressing the rated
  capability of a process. Six points are defined
  on the scale, and described as capability levels
  
• Level 0 Incomplete there is general failure
  to attain the purpose of the process.
• There are no easily identifiable work products or
  outputs of the process

18
Capability Dimension refer to handout

• Level 1 Performed the purpose of the process
  is generally achieved.
• The achievement may not be rigorously planned and
  tracked.
• There are identifiable work products for the
  process, and these testify to the achievement of
  the purpose

19
Capability Dimension refer to handout

• Level 2- Managed the process delivers work
  products according to specified procedures and is
  planned and tracked
• Work products conform to specified standards and
  requirements
• The primary distinction from the level 1, is that
  the performance of the process now delivers work
  products that fulfill expressed quality
  requirements within defined timescales and
  resource needs

20
Capability Dimension refer to handout

• Level 3 Established the process is performed
  and managed using a defined process based on good
  SE principles
• Individual implementations of the process use
  approved, tailored versions of std, documented
  processes to achieve the process outcomes
• The resources necessary to establish the process
  definition are also in place
• The primary distinction from the level 2, is that
  the process of the established level is using a
  defined process that is capable of achieving its
  process outcomes

21
Capability Dimension refer to handout

• Level 4 Predictable the defined process is
  performed consistently in practice within defined
  control limits to achieve its defined process
  goals
• Detailed measures of performance are collected
  and analyzed
• Leads to a quantitative understanding of process
  capability and an improved ability to predict and
  manage performance

22
Capability Dimension refer to handout

• Performance is quantitatively managed
• The quality of work products is quantitatively
  known
• The primary distinction from the level 3, is that
  the defined process is now performed consistently
  within defined limits to achieve its process
  outcomes

23
Capability Dimension refer to handout

• Level 5 Optimizing performance of the process
  is optimized to meet current and future business
  needs, and the process achieves repeatability in
  meeting its defined business goals
• Quantitative process effectiveness and efficiency
  goals (targets) fro performance are established

24
Capability Dimension refer to handout

• Continuous process monitoring against these goals
  is enabled by obtaining quantitative feedback,
  and improvement is achieved by analysis of the
  results
• Optimizing a process involves piloting innovative
  ideas and technologies and changing ineffective
  processes to meet defined goals or objectives
• The primary distinction from level 4, is that the
  defined and std processes now dynamically change
  and adapt to effectively meet current and future
  business goals

25
ISO/IEC 15504 Capability Measurements

• The ISO/IEC 15504 measurement model defines how
  to rate the process capability of any the
  processes described in previous sections
• The achievement of a capability level is
  evaluated by rating the extent of achievement of
  a set of defined process attributes (refer to
  handout)

26
ISO/IEC 15504 Capability Measurements

• The rating of processes capability is performed
  by evaluating the extent of achievement of the
  nine defined process attributes for each process
  within the scope of the assessment
• The rating scale is a percentage scale from 0 to
  100 percent that represents the extent of
  achievement of the attribute

27
ISO/IEC 15504 Capability Measurements

• A four-point ordinal rating scale is used to
  calibrate the levels of achievement of the
  defined capability of the process attributes
• Not achieved no evidence of achievement of the
  defined attribute

28
ISO/IEC 15504 Capability Measurements

• Partially achieved there is some achievement of
  the defined attribute
• Largely achieved there is significant
  achievement of the defined attribute
• Fully achieved there is full achievement of the
  defined attribute
• refer to handout given

29
ISO/IEC 15504 Capability Measurements

• Minimal output from an assessment of capability
  is thus a series of profiles of ratings of
  process attributes, for each process within the
  scope of the assessment

30
ISO/IEC 15504 Draft Guide To Conducting
Assessment zahran

• ISO/IEC 15504 guidelines on conducting assessment
  are primarily aimed at the following
• The assessment team use the document to prepare
  for the assessment
• The assessment participants use the document to
  help understand the assessment and interpret the
  results
• All staff within organizations need to
  understand the details and benefits of performing
  process assessment
• Tool and method developers wish to develop
  tools or methods supporting the process
  assessment model

31
ISO/IEC 15504 Draft Guide To Conducting Assessment

• ISO/IEC 15504 assessment stages
• Reviewing the assessment input
• Selecting the process instances
• Preparing for assessment
• Collecting and verifying information on practices
• Determining the actual ratings for process
  instances
• Determining derived ratings
• Validation the ratings
• Presenting the assessment output

32
Stage 1reviewing inputs to the assessment
STAGE 2
33
Stage 2 selecting the process instances
34
Stage 3 preparing for a team-based assessment
35
Stage 4 collecting and verifying information
STAGE 3
36
Stage 5determining the actual ratings for
process instances
STAGE 4
37
Stage 6 determining the derived ratings
Aggregation between process instances
STAGE 5
STAGE 7
Aggregation across process instances
38
Stage 7 validating the ratings
STAGE 6
39
Stage 8 presenting the assessment output
STAGE 7
Assessment report
40
ISO/IEC 15504 Guidance on Assessment Instruments

• An assessment instrument
• is a tool, or set of tools, used during the
  performance of an assessment to assist the
  assessor in obtaining reliable, consistent and
  repeatable results
  
  
  
  
  
  
  
  
  
  
  
  
  
  
• instrument provides a mechanism for the assessor
  to record notes and results, and may provide a
  means of capturing other types of information for
  use in process improvement or process capability
  determination
• Provide assistance to the assessor in analyzing
  ratings and compiling process profiles

41
ISO/IEC 15504 Guidance on Assessment Instruments

• Format for an assessment instrument ?could be
  implemented as a manual, paper-based tool a
  questionnaire an automated, on-line tool or
  even as an expert system
• Requirements for an assessment instrument are
  independent of a particular design, instrument
  style or mode of use

42
ISO/IEC 15504 Guidance for Conducting Assessments

• Field of application

A SW Process Capability Determination
A SW Process Improvement
43
ISO/IEC 15504 Guidance for Conducting Assessments
Software Process Assessment
44
ISO/IEC 15504 Guidance for Conducting Assessments

• Within a context of SPI
• process assessment provides the means of
  characterizing the current practice within an
  organizational unit in terms of the capability of
  the selected processes (identifies strengths,
  weakness and risks inherent in the process)
• Leads to the ability to determine whether
  processes are effective in achieving their goals,
  to identify causes of poor quality, or overruns
  in time or cost
• Provide basis for prioritizing SPI actions

45
Process Improvement
46
ISO/IEC 15504 Guidance for Conducting Assessments

• Within a context of SW process capability
  determination
• Assessment is concerned with analyzing the
  proposed capability of selected processes against
  a target process capability profile in order to
  identify the risks involved in undertaking a
  project using the selected processes

47
Process capability determination
Specified requirements
Capability report
PROCESS IMPROVEMENT

• Assessment Report
• Generic practice adequacy ratings
• Process capability level ratings
• Assessment record

• Assessment Input
• Purpose
• Scope
• Constraints
• Responsibilities
• Extended process definitions
• Additional information to
• be collected

• Model-Part2
• Process purpose
• Practices
• Part 5
• Process indicators
• Process mgt indicators

48
ISO/IEC 15504 Guidance for Conducting Assessments

• ISO/IEC 15504 Conformance
• ISO/IEC 15504 is a set of requirements and
  guidelines for assessment methods
• An assessment method based on and consistent with
  these requirements will be considered as
  ISO/IEC15504-conformant

49
Summary

• The International Std provides a framework for
  the assessment of software processes
• This framework can be used by organizations
  involved in planning, managing, controlling, and
  improving the acquisition, supply, development,
  operation, evolution and support of software