Cleanroom Method

1
Cleanroom Method

• CS 415, Software Engineering II
• Mark Ardis, Rose-Hulman Institute
• March 20, 2003

2
Outline

1. Harlan Mills
2. Cleanroom method
3. Industrial use of cleanroom

3
Harlan Mills
1919 - 1996
4
Mathematics and Programming

• Roman accounting
• "to go from programming as an instinctive,
  intuitive process to a more systematic,
  constructive process"

5
Cleanroom Method

• Incremental (spiral)
• Box structure specification and design
• Design verification
• No debugging
• Statistical testing

6
Box Structures

• Black boxes behavior only
• State Boxes behavior state
• Clear boxes procedures

7
Black Boxes
8
State Boxes
State Data
S
R
stimulus, old state ? response, new state
9
Clear Boxes
State Data
S
R
Procedures
stimulus, old state ? response, new state
10
Box Description Language (BDL)

• Invocation use lttypegt ltnamegt ltargsgt
• Sequence do B1 B2 od
• Alternation if ltcondgt then B1
  else B2 fi
• Iteration while ltcondgt do B od

11
Box Structure Hierarchy

• BB
• SB
• CB
• BB BB BB
• SB SB SB
• CB CB CB

12
Cartoon of the Day (1/3)
13
Cartoon of the Day (2/3)
14
Cartoon of the Day (3/3)
15
Design Verification

• Procedures in BDL are checked for correctness
  with their higher-level descriptions
• All boxes (and all procedures) describe functions
• Formal proofs of correctness can be performed
  (but often informal proofs are done, instead)

16
Verification of Sequence

• Given a high-level function f for statement
  do g h od
• Does g followed by h compute the same
  function as f?
• Example
• f(x) 2 x 7
• g(x) 2 x
• h(x) x 7

17
Verification of Selection

• Given a high-level function f for statement
• if ltcondgt thengelsehfi
• Whenever ltcondgt is true, does g compute the
  same function as f?
• Whenever ltcondgt is false, does h compute the
  same function as f?

18
Verification of Iteration

• Given a high-level function f for statement
• while ltcondgt dogod
• Whenever ltcondgt is true, does g followed by f
  compute the same function as f?
• Does the loop always terminate?
• Whenever ltcondgt is false, does the empty function
  compute the same function as f?

19
Usage Testing

• Develop an operational profile of use
• Generate random tests that fit the probabilities

20
Example
Function Usage Probability Distribution Interval
Update 32 0-31
Delete 14 32-45
Query 46 46-91
Print 8 92-99
21
Test Generation
Test Random Numbers Test Cases
1 29, 11, 47, 52, 26, 94 U, U, Q,Q, U, P
2 62, 98, 39, 78, 82, 65 Q, P, D,Q, Q, Q
3 83, 32, 58, 41, 36, 17 Q, D, Q,D, D, U
4 36, 49, 96, 82, 20, 77 D, Q, P,Q, U, Q
22
Industrial Use

• Used in a few areas of IBM
• Used by some military contractors
• Tried at NASA

23
Software Engineering Laboratory (SEL)

• Joint program of NASA Goddard Space Center,
  Computer Sciences Corporation, and the University
  of Maryland
• Conduct experiments and case studies on new
  software technology

24
SEL Experience

• First trial at University of Maryland
• controlled experiment (10 experiment teams,5
  control teams
• FORTRAN
• 1.5 KLOC
• 3 case studies at Goddard
• flight-dynamics ground support systems
• FORTRAN
• 40 KLOC, 22 KLOC, 160 KLOC

25
SEL Results University Experiment

• Cleanroom teams
• use fewer computer resources
• satisfy requirements more successfully
• make higher percentage of scheduled deliveries

26
SEL Results Goddard

• More effort spent in design
• Better reliability of final product
• Smaller projects achieve higher productivity, but
  large project just average

27
Summary

• Cleanroom may be an effective method for
  achieving higher reliability
• Requires some culture change (no debugging)
• Still being investigated by researchers and
  practitioners

28
References

• Victor Basili and Scott Green, "Software process
  evolution at the SEL", IEEE Software 11(4),
  58-66, July 1994.