Title: People, Poetry, and Culture: Reinventing Software Development
1People, Poetry, and CultureReinventing Software
Development
- Dr. David West
- OTUG Distinguished Lecture Series
- July 2001
2What shall we talk about?
- Past
- Some history and context
- 1968
- Present
- Contemporary dialogs
- Future
- Presuppositions
- Reinventing Software Development
3History Plato on Calculation
If you cannot calculate you cannot speculate on
future pleasure and your life will not be that of
a human, but that of an oyster or a
jellyfish. Philebus, 21
4History - Calculation
- Algebra
- India (terminological / abbrev.), Arabia
(symbolic) - Europe (particular reasoning to global reasoning)
- Calculus
- Rome (pebbles, basic operations of arithmetic)
- Newton / Leibniz (differential, integral)
- Vector, tensor, predicate
5History Binary Arithmetic
- Leibniz 1679
- W.H. Eccles F. W. Jordan flip-flop
- C.E. Wynn-Williams, 1932 thyatron tube binary
electronic counting machine - John von Neumann 1945 binary representation
for computers - (George Boole Logic)
6History - Calculators
- Abacus
- 1617 John Napier (Napiers Bones)
- 1623 Wilhelm Shickard (calculating clock)
- 1642 - Blaise Pascal (the Pascaline)
- 1673 Leibniz (four function digital)
- 1885 Burroughs (adding and listing machine)
7History Aristotles Dream
Suppose every instrument could by command or by
anticipation of need execute its function on its
own suppose (like the carvings of Daedelus or
the figurines of Hephaestus which, the poet says,
could take on a life of their own) that spindles
could weave of their own accord, and plectra
strike the strings of zithers by themselves then
craftsmen would have no need of hand-work, and
masters have no need of slaves.
8History - Automata
- Hero of Alexandria
- Al Jazzari (sequential automata)
- Jacques de Vaucanson, 1738, (programmable
sequential automata the Digesting Duck) - Henri-Louis Jaquet-Droz, 1770, (android musician)
9History - computer
- Jacquards Loom
- Babbage -- Lady Lovelace
- Hollerith
- Watson and IBM (Harvard Mark I)
- SSEC (conditional branching, subroutines, stored
program) - Eniac
- Vannevar Bush, Mauchly, Eckert, von Neuman
10 Recap Essential Historical Ideas
- One Zero and place-value math
- Two algebraic thought
- Three logical thought
- Fourth algebraic and binary logic
- Fifth set theory
- Sixth mathematical logic (predicate calculus)
- Seventh Turing Machine
11The End of History
- Amazing body of knowledge and understanding
- Of computation
- Of computers
- Of Automata
- Formally defined deterministic mechanical and
electro-mechanical systems - A bad case of hubris
- Rationalists thought they had found the answer to
life, the universe, and everything. Ask the
Dolphins.
12Context - Metaphor
- Quine and the fringes of science
- MacCormac
- Diaphor poetry
- Epiphor prose
- Lexical usage
- West
- Paraphor (melded concepts of Kuhns paradigm
and metaphor) between epiphor and lexical
13Life Cycle of Metaphors (MacCormac)
- Diaphor poetry
- what if was like
- Epiphor prose
- an atom can be visualized as a small planetary
system with electrons (planets) orbiting a
central nucleus (sun), the orbits corresponding
to - Lexical usage
- Both the human brain and the electronic computer
are instances of physical symbol systems, hence
the brain IS a kind of computer.
14Off the Beaten Track (West)
- Paraphor
- When a metaphor becomes a paradigm
- Kuhns notion of paradigm
- Lexical use of metaphor
- Metaphor becomes part of cultural perspective
- Values associated with paraphor become the values
of the culture - Rational is good emotional is bad
- Control is essential chaos is evil
- Efficiency is a virtue waste is a vice
15Context Paraphors We Knew and Loved
- Spinning our fates (drop spindle / potters wheel)
- God The Watchmaker (clock)
- Wind and Water (dynamics)
- Blowing off steam (heat engine)
- I compute, therefore I am (digital computer)
- J. David Bolter, Turings Man, Chapel-Hill UNC
Press, 1984
16We Hold These Truths
- Philosophers Stone
- Everything it touched turned to gold
- Single idea powerful enough to explain everything
- Rational Formalism Paraphor for modernity
17Descartes Dream
- November 10, 1619 crawled into a wall stove,
warmed up, had a vision. - Unification and illumination of the whole of
science, even the whole of knowledge, by one and
the same method the method of reason. - Mirabilis scientiae fundamenta
- Accept only what is clear in ones own mind
- Split large difficulties into smaller ones
- Argue from simple to complex
- Check, when done
18Context Leibnizs Dream
- Characteristica universalis - universal method
whereby all human problems, whether of science,
law or politics, could be worked our rationally,
systematically, by logical computation. - De arte combinatoria a general method in which
all truths of the reason would be reduced to a
kind of calculation. At the same time this would
be a sort of universal language or script, but
infinitely different from all those projected
hitherto for the symbols and even the words in
it would direct the reason and errors, except
those of fact, would be mere mistakes in
calculation.
19Context Leibniz on Binary Math
Omnibus ex hihil ducendis sufficit unum.
(With one, everything can be drawn out of
nothing.)
Saw binary arithmetic in the 64 Hexagrams of the
I Ching (Yin / Yang). Tried to convert the
Emperor of China (an avid mathematician) to
Christianity via arguments that the Unity stood
for God and the Zero for the Void, that the
Supreme Being had drawn every existing thing from
out of the void just as unity and zero suffice to
represent every possible number.
20Context Bertrand Russel
The most astonishing feature of modern science
is its return to Pythagorism.
the Universe is ruled by the laws of a
physical world whose properties could be
expressed as abstract concepts, the numbers,
themselves developed by the efforts of human
thought which once it had attained the capacity
of extreme abstraction could act on the objects
of this same universe.
21Context Maurice dOcagne on Calculation
Calculation is important in the domain of theory
as it is in matters of practice. All the
material progress of our civilisation (sic)
derives directly or indirectly from Science, and
the progress of Science itself constantly depends
on calculation. This is not only the case for
the so-called exact sciences but also for
those which were considered experimental or
observational
22On the Eve of a Conference
- Rational Machines, Rational Theory
- Rational Universe
- Rule governed (classical binary logic coupled
with mathematics) - Top-down centralized control (God as the master
control module) - Deterministic cause and effect
- Objective reality not dependent on observer(s)
- Rational Software
- Rational Process
231968 We have seen the enemy and he is us.
(Pogo)
- NATO Conference on Software Engineering
- Inevitable application of prevailing paraphor to
software development - Software artifacts are machines
- Deterministic, cause and effect, formally
describable - People are potential machines
- After installation of proper formalisms
(education/training) - With appropriate management discipline
- Systems are Cartesian
- Model machine
- Possible to define a formal syntax and grammar
capable of unambiguous description of the
implemented machine.
24Inspiration from an Architect Christopher
Alexander - I
"Today more and more design problems are reaching
insoluble levels of complexity. At the same
time that the problems increase in quantity,
complexity, and difficulty, they also change
faster than before.
25Inspiration from an Architect Christopher
Alexander - II
It is not hard to see why the introduction of
mathematics into design is likely to make
designers nervous. What they do not realize,
however, is that modern mathematics deals at
least as much with questions of order and
relations as with questions of magnitude
becoming a very powerful tool indeed if it is
used to explore the conceptual order and pattern
which a problem presents to its designer.
26Inspiration from an Architect Christopher
Alexander - II
logic in the widest sense, refers to something
far more general. It is concerned with the form
of the abstract structures, and is involved the
moment we make pictures of reality and then seek
to manipulate these pictures so that we may look
further into the reality itself. It is the
business of logic to invent purely artificial
structures of elements and relations and then,
because the logic is so tightly drawn, we gain
insight into the reality which was previously
withheld from us.
27Plato on Decomposition
First, perceiving and bringing together under
one Idea the scattered particulars, so that one
makes clear the thing which he wishes to do...
Second, the separation of the Idea into
classes, by dividing it where the natural joints
are, and not trying to break any part, after the
manner of a bad carver... I love these processes
of division and bringing together ... and if I
think any other man is able to see things that
can naturally be collected into one and divided
into many, him I will follow as if he were a
god. Plato
28Diderot on Analysis and Synthesis
bring the discoveries and organize them, so
that men may be enlightened, While it is
important to disassemble, to resolve the whole
into its parts, it is also indispensable to
combine, to situate, and to re-assemble these
diverse, separated fragments that are collected.
This creates a new whole, now intelligible to
the inquisitive mind, before putting them in
the mind of others according to its functional
individuality and its characteristic law of
action and development.
29Engineering - I
" software development today is in a
pre-engineering phase analogous in many respects
to the pre-engineering phases of the now
traditional engineering disciplines." "We do not
seem to be even trying to learn from the history
of the classical engineering fields."
30Engineering - II
"The essential factor which accounts for the very
substantial difference between design correctness
yesterday and today in classical engineering
fields it seems clear to me, is a scientific and
mathematical foundation and the fact that
practitioners apply that foundation regularly and
as a matter of course in their work."
31Engineering - III
"The scientific and mathematical foundations
referred to above are provided by Newton's laws
(for the civil and mechanical engineer) and
Maxwell's laws (for electrical engineers). The
comparable foundations for software engineering
in the truest sense of the word engineering is
provided by the work of Floyd, Hoare, Djikstra
and others, but has not yet come into widespread
use in practice. We are still living and working
in the pre-engineering days of software
development."
32Engineering - IV
- Robert L. Barber - IEEE Annals of the History of
Computing 194, Oct.-Dec. 1997. - 29 years since advent of SE and it still is not
being done? - Why
- Doesnt work
- Isnt worth the effort to try
33Engineering V (Winograd)
- There is a given reality out there which we
come across during software development. By
analyzing facts of this reality, we obtain
requirements for the software - The essential task of the software developer is
starting from the problem defined in that reality
to find a correct solution in the form of a
program system. - It is possible to separate the production of
software from its use. Software engineering is
concerned with the production of software on the
basis of fixed requirements.
34Engineering V - (Winograd, cont.)
- Software production is based on models
representing reality. Models should map reality
correctly. - The whole process is largely independent of
individuals. For one and the same problem,
different developers should arrive at the same
results. The developers should be
interchangeable.
35Engineering V (Winograd, cont.)
- Communication should be restricted and regulated
via fixed interfaces. The division of labor can
be worked out on an ad hoc basis. Subject to
technical feasibility, any desired parts of the
production process can be automated. - The developers responsibility covers only
proper construction of the product in accordance
with the requirements specification. Any ethical
considerations that go beyond this are quite
separate from the technical aspects of the work.
36Contemporary Dialogs
- The artificial intelligence of computer
scientists - A political pamphlet from the middle ages.
- Greece vs. Rome
- CASE and automated development
- Object Orientation
- eXtremely Agile methodologies
- Information Ecologies / Communities of Practice
- Complexity Theory
- Humanity
37The AI of Computer Scientists
- Over the holidays, Alan (Newell) and I created
an intelligent program. Herbert Simon - Perceptrons cant compute I can mathematically
prove it. Marvin Minsky (And while I am at it I
will mathematically prove that hummingbirds cant
fly. - Dreyfus and Dreyfus What Computers Cant Do
38A political pamphlet from the middle ages.
- 1979 - DeMillo, Lipton, Perlis
- Social construction of proofs
- Slide title Djikstras response
- 1989 Fetzer again challenges the efficacy of
proofs and formal methods, evoking similar
firestorm - Heuristics versus Formal Methods
- David Parnas Top Down Design how and why to
fake it.
39Greece vs. Rome (Robert L. Glass)
- Greece
- Informal
- Codes
- Uses structured-stuff
- Minimum documentation
- Uses things as tools
- Democratic
- Empirical/inductive
- Intuitive
- Substance
- Rome
- Formal
- Manages project
- Uses Structured Stuff
- Maximum documentation
- Uses people as tools
- Imperial
- Analytical/deductive
- Logical
- Form
40Process
- Better process better products!
- SEI / CMM, Capers Jones, TQM
- Formal process assumes formal systems
- Empirical studies fail to show either increased
quality or increased efficiency - Robert A. Frosch (1969)
- Absence of detailed specification
- Hard part first
- Satisficing solutions
- Rapid iterations
41Object Orientation
- Language Wars
- Smalltalk is the only true OO language and it
fails to fully implement object concepts - Method Wars
- CRC versus RUP
- Patterns or Pattern Languages
- Inspiration from an architect
- Timeless Way of Building versus A Pattern Language
42Inspiration from an Architect Christopher
Alexander
A building or town will be alive to the extent
that it is governed by the Timeless Way. To seek
the Timeless Way we must first know the Quality
Without a Name. To reach the Quality Without a
Name we must then build a living Pattern Language
as a gate. Once we have the gate, we can pass
through it to the practice of the Timeless
Way. And yet the Timeless Way is not complete,
and will not fully generate the Quality Without a
Name, until we leave the gate behind.
43XP versus the Agile Alliance
Agile says we value individuals and interactions
over processes and tools. XP says put the
individuals together and have them work this
way. Agile says we value working software over
comprehensive documentation. XP says write
software the first day and every day, test the
first day and every day, refactor the first day
and every day. Agile says We value customer
collaboration over contract negotiation. XP
says Sit the customer down with the programmers
and have them steer the project every single
day. Agile says We value responding to change
over following a plan. XP says Set up intense
and rapid feedback and govern yourselves by
it. The fact is, XP's real viewpoint is so
radical that it can't even be properly expressed
by comparing with those wimpy weightless Agile
preferences.
44Data is Dead Information is Comatose
- J Arsac on Information (ns)
- Data is what can be stored in a DBMS
- Information is the assembly and presentation of
data according to formally defined rules - Information Ecologies (Nardi and ODay)
- The Social Life of Information (John Seeley Brown)
45J. Arsac on Information
A piece of information is a written formula
capable of imparting knowledge I distinguish
amongst the various sources and reserve the term
information solely for those which I can write
down. Anything that can be measured can be
written down I refuse to give the name
information to ways of acquiring knowledge that I
am unable to describe explicitly.
46Information Ecology
We define an information ecology to be a system
of people, practices, values, and technologies in
a particular local environment.
System dense interrelationships Diversity
essential to health evolution Coevolution all
members adapt and create in response to
change Keystone species essential role for
smooth function Locality local habitation and
a name.
47Social Life of Information
- Process
- Discrete
- Well-defined
- Isolable
- Black box
- Longitudinal (sequential)
- Information centric
- Practice
- Diffuse, distributed
- Ill-defined
- Holistic
- Complex
- Lateral and longitudinal
- Experience / knowledge centric
48Software Development and Reality Construction
- Dreyfus brothers ( Merleau Ponty and Dilthey)
- Winograd (Heidegger, Maturana and Flores)
- Coyne (Derrida)
- Floyd
49Christiane Floyd, et. al.
- Challenge and refutation of rationalist
assumptions listed by Winograd. - No objective reality all reality is socially
constructed - Software development is a moral and social
process, an exploration of a constructed reality - Artifact and use are inextricably intertwined and
mutually defining. - The map is not the territory.
- Communication is essential multidirectional,
multi-vocal, and context specific. - Ethics and morality are essential aspects of the
development tasks not add-ons.
50Not Complicated. Complex!
- Formal systems are merely complicated
- A complex adaptive system is
- An ensemble of independent agents
- Who interact to create an ecosystem
- Whose interaction is defined by the exchange of
information. - Whose individual actions are based on some system
of internal rules - Who self-organize in nonlinear ways to produce
emergent results - Who exhibit characteristics of both order and
chaos - Who evolve over time.
51Adaptive Development Model LifeCycle - Highsmith
- Does not Manage (bounds, directs, nudges,
confines) - Relies on emergent order not imposed order
- Component (outcomes / products) not task based
- Highly iterative
- Unpredictable (deviations guide us to the
correct solution)
52Highsmith ADMLC Graphic
53Recap
- Software Engineering was the last hurrah of the
Age of Reason. - Some good results general dissatisfaction
complete failure in achieving objectives - Nothing intrinsically wrong with the foundation
ideas they are just applied to the wrong
domain. - Long tradition of loyal opposition
- Time to reinvent software development
54Reinvention - Scope
- Philosophy
- Values
- Products
- Systems architecture / infrastructure
- Components
- Applications
- Systems
- Process
- Community
55Philosophy
- Metaphysics
- Objective reality is overrated, and irrelevant at
the level of applications and systems - Software development IS reality construction
- Complexity Science (emergence will win out)
- Epistemology
- Context is everything, syntax is convenience
- Ineffable knowledge exists and is essential
56Values
Open source
Formality, NO!! Discipline, yes.
Man, the measure of all things.
People First!
57Products
- Systems architecture / infrastructure
- Espalier (David Gelernter)
- Components
- Objects, primitive objects, atomic objects
- Applications
- Affordances
- Systems
- Information ecologies
58Infrastructure
Node
Virtual Node
Just Add Web.
59Objects
node
Objectionary
60Process
- Explore to improve
- Hybrid of eXtreme Programming ADMLC
61Community
- Software Craft
- Raw materials (objects, infrastructure)
- Hand tools
- Ineffable knowledge
- Supporting Guild (The Order)
- Enculturation
- Apprenticeship
- Community of Practice
- Extended family
- Users, managers, ecology members
62Becoming
- Pilgrimage
- The Ashram
- Apprenticeship
- Lore and wisdom (reuse)
- Rites-of-Passage
- Novice, enabler, coordinator, listener
- In the World but not of It
63The Business Case
- Not going to change the world
- Recognized group
- Dorsai (Gordon Dickson)
- Amish lt The Order lt Mormons
- Value
- Order of magnitude (10x) better outcomes
- 2x 5x typical fees
- Assets
- Open source everyone can have access to
tangible assets - Ineffable knowledge proprietary assets