Structured Programming: Retrospect and Prospect

1
Structured ProgrammingRetrospect and Prospect

• By Harlan D. Mills, IBM Corp.
• Presented by Maulesh Jhala

2
Background

• Edsger Dijkstras 1969 article Structured
  Programming

3
Impact of Structured Programming

• Dijkstras argument centered on shortening
  correctness proofs by simplifying control logic
• Proposed restricting program logic control to
  sequence, selection and iteration.
• Greeted with controversy you must be kidding
• Proposal was theoretically sound by results from
  Corrado Boehm and Giuseppe Jacopini

4
Early Industry Experience

• The New York Times Project
• Online storage and retrieval system
• Carried out by IBM over two year period
• 85k LOC
• Primarily used chief-programmer team, top-down
  development, hierarchical modularity and
  functional verification
• Productivity increased, compared to other
  projects of same size and complexity, by a factor
  of 5
• System crashed only once that year
• Total of changes required 25
• Most of the changes related to data editing
  subsystem
• 1/3rd changes were external specs
• 1/3rd changes were definite error rate 0.1/1000
  LOC
• 1/3rd changes were interpretable either way

5
Early Industry Experience Cont

• Structured theorem
• Boehm and Jacopinis research results were
  valuable to management when recast in structured
  theorem
• Structured theorem permits management by
  exception in program design standards
• By the time program is produced, its well
  understood and good solution has been found
• Order in which lines are written may not matter
  but due to human fallibility, it can greatly
  affect correctness and completeness of program
• Top-down corollary
• Described Key management benefit from top-down
  programming
• Chronologically written lines can be verified by
  reference only to lines already written, not to
  lines yet to be written

6
Early Industry Experience Cont

• Using top-down corollary
• counterintuitive in 70s because programming was
  thought out to be synthesis process of assembling
  instruction into programs instead of analytic
  process of restructuring specs into program
• Easy to misunderstand
• Doesnt claim that thinking should be done
  top-down
• Benefit in the later phases of program design

7
Early Industry Experience Cont

• NY Times Project
• 2 years Team of 4, expanded to 11
• Used structured theorem and top-down corollary
• Structured theorem difficult to understand
• Top-down corollary challenging in program design
• NASAs Skylab and Apollo Project

8
Early Industry Experience Cont

• Language Problem
• Unlike Algol, which permitted direct structured
  programming, Assembly language Fortran, and Cobol
  were conspicuous problem for structured
  programming
• Two approaches with these languages
• Design in structured form, hand translate to
  source language and then to code
• Create language processor to permit final coding
  in extended language to be mechanically
  translated to source language
• Hence such languages has been slow to benefit
  from structured programming

9
Current Theory Practice

• Mathematical correctness of structured program
• Early discussions of structured programming
  focused on absence of gotos and theoretical power
  of program with control logic and syntactic and
  typographic aspect
• Syntactic and typographic aspects allowed for
• Readable code by other programmers
• Structured walk-through and program inspections
• Managers understood the development progress
  better
• Dijkstras 1st article didnt mention syntax,
  typography, readability, stepwise refinement, or
  top-down development.
• Primary focus was to shorten the mathematical
  proof of correctness of programs

10
Current Theory Practice Cont

• 2 reason that Dijkstras argument about the size
  of proofs of correctness for structured programs
  seems to be inspired prophecy are
• Proof of programs correctness is a singularly
  appropriate definition for its necessary and
  sufficient documentation
• The size of a correctness proof seems at least a
  partial measure of the complexity of program
• With out proper understanding of proof of
  correctness, itll be difficult to realize the
  insight
• Dijkstras article How to write Correct Programs
  and know it explains that which can, in turn,
  reduce complexity and permit better documentation

11
Current Theory Practice Cont

• Program Function Correctness
• Terminating program rule for mathematical
  function that converts an initial state of data
  into final state
• Non-terminating programs non-terminating loop
  that executes terminating subprograms endlessly
• Function defined by such terminating program is
  simply set of ordered pairs the initial and
  final state of data
• Since details of execution are abstracted out by
  mathematical functions, correctness of program
  can be discussed with respect to its specs
• Program will be correct with respect to specs
  iff, for every initial value permissible by the
  specs, program will produce a final value
  corresponding to initial value.

12
Current Theory Practice Cont

• In principle, direct way to prove mathematical
  correctness of a program is clear.
• In practice, due to complexity of spaghetti
  program such a proof may not be possible
• For such programs, can think of program as a rule
  for a function which is being executed with
  actual data.
• Structured program has more powerful way to think
  about it a function rule that uses simpler
  function.
• Algebra of part function.
• Functions built from nested parts of structured
  program
• Rules are very simple and regular
• Axiomatic and functional verification
• Axiomatic verification Proves correctness by
  reasoning about effects of program data
• Functional verification based on functional
  theory from the outset
• Biggest difference is treatment of loops

13
Looking to the future

• Data-structured programming
• Structured program proofs of the algebraic op on
  involved functions are same at every level but
  functions become more complex upper in hierarchy
• 2 features in the data of the program have effect
  on formal proofs
• of program variables that define program data
• Array assignment
• Array represent arbitrary data access like gotos
  which increases the complexity of proofs
• Possible eliminate use of array in structured
  program, and use instead the data abstraction
  without arbitrary access
• Stack, sets and queue can replace array
• Fewer variables used
• Shorter proof

14
Looking to the future Cont

• Functional verification instead of unit debugging
• Functional verification permits production of
  high-quality software without unit debugging.
• showed that sw can be developed without debugging
  by the developers with some beneficial results
• Has surprising synergy with system level
  statistical testing
• Testing sw against user-representative,
  statistically-generated input.
• Has not been used much as the development
  technique since sw requires considerable defect
  removal to make it work
• However, statistical testing of functionally
  verified structured program can be effective.

15
Looking to the future Cont

• Cleanroom software development
• Combined discipline of no unit debugging and
  statistical testing
• Emphasis on defect prevention instead of removal
• Permits development of software under statistical
  quality control by iterative and incremental
  development and testing
• At first look, no unit debugging seem strange
  since its easy to remove most of the defect.
• At a deeper level it eliminates tunnel vision
  that can undermine concentration and discipline

16
Looking to the future Cont

• Eliminating unit test and debugging leads to
• Serious attention to design and verification as a
  integrated personal activity by each programmer
• Serious attention to design and verification to
  inspection by programming teams
• Preserver design hypothesis for statistical
  testing and control
• High moral of quality personal
• User-representative, statistical testing of SW
  before debugging leads to
• Valid scientific estimates of the reliability and
  the reliability growth rate when bugs are
  discovered and fixed
• Forced recognition by programmers of the spec
  input space and program design by decomposition
• Most effective way to increase reliability of sw
  thru testing and fixing
• Evidence showed bugs found after verification to
  be qualitatively different than from debugging.
• Functional verification bugs are due to
  mathematical fallibility which can be easily
  discovered by statistical tests

17
Looking to the future Cont

• Limits of humans performance
• Easy to overestimate the future abilities of
  machines and underestimate the human capabilities
• From early days, its been axiomatic that errors
  necessary in programs because people are fallible
• Such errors compensate according to program
  complexity
• More complex programs have more error per
  thousand lines of code
• NY Times released error rate was about one error
  per person-year of effort
• Paul Friday, in 80 for census software system,
  received gold medal for running 25k lines of
  program error free for a year

18
Conclusion

• Structured programming reduced much of the
  complexity of programming and increased human
  expectations and achievements
• Better sw development tools are needed to reduce
  human fallibility, not just better debuggers
• Not enough to count lines of code to measure
  productivity. Instead better understanding of
  management is needed
• Principal challenge for management is to organize
  and focus well-educated software engineers in
  effective teams

19
References

• Harlan D. Mills, IBM Corporation, Structured
  Programming Retrospect and Prospect, IEEE
  Software, Vol. 3 No. 4, Nov 1986, pp. 58-66

20
Questions?