EEE493.13 Open Source Process Methods

1
EEE493.13Open Source Process Methods
Royal Military College of Canada Electrical and
Computer Engineering

• Dr. Terry Shepard
• shepard_at_rmc.ca
• 1-613-541-6000 ext. 6031

Major JW Paul Jeff.Paul_at_rmc.ca 1-613-541-6000
ext. 6091
2
outline

• definition of hacker
• some elements of open source process models
• netscape, mozilla and open source
• the cathedral and the bazaar
• lessons from the bazaar
• noosphere, cyberspace and ergosphere
• homesteading the noosphere
• a higher level process
• the halloween documents

http//www.catb.org/esr/writings/cathedral-bazaar
/ http//www.catb.org/esr/writings/cathedral-baza
ar/homesteading/ http//www.catb.org/esr/faqs/hac
ker-revenge.html http//www.opensource.org/hallowe
en/ first three available in .doc format on
course website
3
What is a Hacker ?

• General definition (from wiki)
• a programmer who hacks or reaches a goal by
  employing a series of modifications to exploit or
  extend existing code or resources
• note black hat hacker cracker
• hacks can be elegant or kludges
• Open source definition
• a more than competent programmer who contributes
  to open source software development and evolution

4
What is Open Source ?

• a philosophy or otherwise a pragmatic methodology
  related to production practices that promote
  access to the origins of the products
• debate and confusion over the extent of this
• it does not have to be open to everyone
• it does not mean free

5
Aside Open Source Conditions (from wiki)

• Free Redistribution the software can be freely
  given away or sold.
• Source Code the source code must either be
  included or freely obtainable.
• Derived Works redistribution of modifications
  must be allowed.
• Integrity of The Author's Source Code licenses
  may require that modifications are redistributed
  only as patches.
• No Discrimination Against Persons or Groups
  no-one can be locked out.
• No Discrimination Against Fields of Endeavor
  commercial users cannot be excluded.
• Distribution of License The rights attached to
  the program must apply to all to whom the program
  is redistributed without the need for execution
  of an additional license by those parties.
• License Must Not Be Specific to a Product the
  program cannot be licensed only as part of a
  larger distribution.
• License Must Not Restrict Other Software the
  license cannot insist that any other software it
  is distributed with must also be open source.
• License Must Be Technology-Neutral no click-wrap
  licenses or other medium-specific ways of
  accepting the license must be required.

6
Why is open source successful?

• largely (nearly all??) infrastructure software
• developers can write the requirements
• widespread freedom to question and drive changes
• many eyeballs make all bugs shallow
• Linuss law
• widely shared understanding of expected
  evolution paths
• evolution not privately controlled

Keep these concepts in mind as we go through the
following slides...
7
What is an open source processEric S. Raymond
- Cathedral and the Bazaar

• Traditional software (and process) Cathedral
• crafted by individual wizards or small bands of
  mages working in splendid isolation
• architectural integrity (MMM chap 4 / Reims
  Cathedral)
• guiding hand planning/controlling project over
  years (though the plan may change)
• Open Source software (and process) Bazaar
• great babbling bazaar of differing agendas and
  approaches
• guiding hand controls what is accepted - not the
  direction of the project (gives innovation in
  features and solutions)

Not mutually exclusive
8
How does this work

• planned releases, driven more by quality and
  features than by deadline
• sometimes daily releases
• core developers, surrounded by a halo
• only top 5 are in the core? code gods
• 1001 difference in productivity
• many local specific processes
• eg. Mozilla super-review (more later)
• a set of globally respected taboos/behaviours

9
global taboos/behaviours

• We have access to the source code, but
• strong social pressure against forking
  (developing independently of the rest)
• distributing changes without cooperation of
  moderators is frowned on, except for essentially
  trivial changes
• removing a persons name from a project history,
  credits or maintainer list is absolutely not done
  without the persons explicit consent
• gift culture

10
the hacker milieu as gift culture

• three kinds of cultures/ways of organizing
• command hierarchy
• scales poorly, becomes brutal
• exchange economy
• decentralized, competitive
• gift culture
• culture arising from abundance
• social status determined by what you give

11
Ownership and Open Source

• three ways to acquire ownership of an open-source
  project
• found the project
• have ownership of the project handed to you by
  the previous owner
• observe that the project needs work and the owner
  has disappeared or lost interest
• Analogy with property
• three ways to acquire ownership of land
• homesteading
• transfer of title
• adverse possession

12
Noosphere, Cyberspace Ergosphere

• Noosphere
• Cyberspace
• Ergosphere

• sphere of human thought all the interacting
  minds on earth, and the transcendent
  consciousness that may arise
• information resources and activities accessible
  through computer networks, the internet in
  particular.
• totality of energy applied every day in every
  kind of human activity.
• spinning area surrounding the event horizon of a
  black hole

13
Cathedrals in the Ergosphere

• the cathedral parts of the ergosphere are driven
  by management
• define goals, and lead everyone to them
• monitor, to ensure coverage of crucial details
• motivate people to do boring things
• organize for best productivity
• marshal resources as needed
• almost sounds like a military model!
• Note the proactive control

14
Cathedrals without managers

• what happens to these management functions in
  cyberspace open source?
• marshalling resources in large organizations
• defensive you have to fight for your turf
• creation of empires managers compete with each
  other
• marshalling resources in cyberspace
• open source developers self-organize
• motivation arises from interest, less from money
• monitoring by peers is key to open source
• shared goals work better than imposed goals
• stills sounds like a military model???

15
Bazaars in the Ergosphere

• the bazaar parts of the ergosphere are driven by
• taboos/behaviours
• ownership homesteading the noosphere
• a gift culture
• the bazaar meets the Fortune 500
• a higher level process
• the bazaar goes mainstream http//www.catb.org/es
  r/faqs/hacker-revenge.html
• Forget bottom-up Work on top-down
• Linux is our best demonstration case
• Capture the Fortune 500
• Coopt the prestige media that serve the Fortune
  500
• Educate Hackers in Guerilla marketing techniques
• Use the Open Source certification mark to keep
  things pure

16
Lessons from the Bazaar

• Why (how?) open source works
• from Cathedral and the Bazaar

17
Lessons from the Bazaar

• Every good work of software starts by scratching
  a developer's personal itch.
• Good programmers know what to write. Great ones
  know what to rewrite (and reuse).
• Plan to throw one away you will, anyhow.
  (Fred Brooks, The Mythical Man-Month, Chapter 11)
• If you have the right attitude, interesting
  problems will find you.
• When you lose interest in a program, your last
  duty to it is to hand it off to a competent
  successor.

18
Lessons from the Bazaar

• Treating your users as co-developers is your
  least-hassle route to rapid code improvement and
  effective debugging.
• Release early. Release often. And listen to your
  customers.
• Given a large enough beta-tester and co-developer
  base, almost every problem will be characterized
  quickly and the fix obvious to someone.
• Smart data structures and dumb code works a lot
  better than the other way around.

19
Lessons from the Bazaar

• If you treat your beta-testers as if they're your
  most valuable resource, they will respond by
  becoming your most valuable resource.
• The next best thing to having good ideas is
  recognizing good ideas from your users. Sometimes
  the latter is better.
• Often, the most striking and innovative solutions
  come from realizing that your concept of the
  problem was wrong.
• Perfection (in design) is achieved not when
  there is nothing more to add, but rather when
  there is nothing more to take away. (Antoine de
  Saint-Exupéry)

20
Lessons from the Bazaar

• Any tool should be useful in the expected way,
  but a truly great tool lends itself to uses you
  never expected.
• When writing gateway software of any kind, take
  pains to disturb the data stream as little as
  possibleand never throw away information unless
  the recipient forces you to!
• To solve an interesting problem, start by finding
  a problem that is interesting to you.
• If the coordinator has a communications medium,
  (at least as good as the Internet, and knows how
  to lead without coercion) many heads are
  inevitably better than one.

21
Netscape and the Bazaar
22
Netscape Embraces the Bazaar

• Netscape announced plans to give away the source
  for Netscape Communicator (renamed Mozilla -
  there are other versions)
• development of Mozilla was only a qualified
  success - problem needed a Motif License
• mozilla.org is a thriving open source community

• January 1998
• mid-2000
• now - 2005

going open will not necessarily save an existing
project that suffers from ill-defined goals or
spaghetti code or any of the software
engineerings other chronic ills. Mozilla has
managed to provide an example simultaneously of
how open source can succeed and how it could
fail.
23
Mozilla code check-in procedure
http//www.mozilla.org/hacking/code-review-faq.htm
l
An example of an open source process element

• module owner and peer review is required before
  code is checked into the mozilla.org CVS
  repository
• In addition, mozilla requires a super-review
• there is a set of rules about what code needs
  super-review, and what the exceptions are
• extended by a set of informal rules and tips
• "Super-review" is a name we don't really like.
  Something like "Infrastructure Review" or
  "Integration Review" is probably a more accurate
  name, and doesn't add a false glamour that
  "super" seems to imply
• published list of super-reviewers

24
Mozilla super-review
The goals of super-review are to increase code
quality, promote best practices, and reduce
regressions

• super review looks at
• quality of the code itself,
• potential effects on other areas of the tree,
• its use of interfaces
• otherwise its adherence to Mozilla coding
  guidelines
• super-reviewers have discretion to raise design
  issues as well as code red flags, and stipulate
  remedies, as necessary.
• necessary because mozilla.org does not require a
  separate design review prior to code review
• expect hackers to talk about their designs, post
  specs or less formal blurbs about their
  intentions, and take feedback throughout the
  (usually non-linear) design/implementation
  process.

25
The Cathedral Notices the Bazaar

• Microsoft Halloween Documents
• http//www.opensource.org/halloween/

26
The Halloween Documents

• internal Microsoft memos, leaked in Oct 1998
  (thus the name Halloween Documents)
• an internal strategy memorandum on Microsoft's
  possible responses to the Linux/Open Source
  phenomenon
• annotated by Eric Raymond, and released to the
  National Press as part of a campaign to give OSS
  more credibility (now up to Halloween XI)
• Microsoft has publicly acknowledged that the
  memorandums are authentic, but dismissed them as
  engineering studies that do not define policy

27
The Halloween Documents Quotes

• OSS poses a direct, short-term revenue and
  platform threat to Microsoft, particularly in
  server space. Additionally,the intrinsic
  parallelism and free idea exchange in OSS has
  benefits that are not replicable with our current
  licensing model and therefore present a long term
  developer mindshare threat.
• Recent case studies (the Internet) provide very
  dramatic evidence ... that commercial quality can
  be achieved /exceeded by OSS projects.

28
The Halloween Documents Quotes

• ...to understand how to compete against OSS, we
  must target a process rather than a company.
• OSS is long-term credibleFUD (fear, uncertainty,
  and doubt) tactics can not be used to combat it.
• Linux and other OSS advocates are making a
  progressively more credible argument that OSS
  software is at least as robust -- if not more --
  than commercial alternatives. The Internet
  provides an ideal, high-visibility showcase for
  the OSS world.

29
The Halloween Documents Quotes

• Linux has been deployed in mission critical,
  commercial environments with an excellent pool of
  public testimonials.
• Linux outperforms many other UNIXes ... Linux is
  on track to eventually own the x86 UNIX market
  ...
• Linux can win as long as services / protocols are
  commodities.
• OSS projects have been able to gain a foothold in
  many server applications because of the wide
  utility of highly commoditized, simple protocols.
  By extending these protocols and developing new
  protocols, we can deny OSS projects entry into
  the market.

30
The Halloween Documents Quotes

• The ability of the OSS process to collect and
  harness the collective IQ of thousands of
  individuals across the Internet is simply
  amazing. More importantly, OSS evangelization
  scales with the size of the Internet much faster
  than our own evangelization efforts appear to
  scale.

31
Questions?
32

• Every good work of software starts by scratching
  a developer's personal itch.
• Good programmers know what to write. Great ones
  know what to rewrite (and reuse).
• Plan to throw one away you will, anyhow.
  (Fred Brooks, The Mythical Man-Month, Chapter 11)
• If you have the right attitude, interesting
  problems will find you.
• When you lose interest in a program, your last
  duty to it is to hand it off to a competent
  successor.
• Treating your users as co-developers is your
  least-hassle route to rapid code improvement and
  effective debugging.
• Release early. Release often. And listen to your
  customers.
• Given a large enough beta-tester and co-developer
  base, almost every problem will be characterized
  quickly and the fix obvious to someone.
• Smart data structures and dumb code works a lot
  better than the other way around.
• If you treat your beta-testers as if they're your
  most valuable resource, they will respond by
  becoming your most valuable resource.
• The next best thing to having good ideas is
  recognizing good ideas from your users. Sometimes
  the latter is better.
• Often, the most striking and innovative solutions
  come from realizing that your concept of the
  problem was wrong.
• Perfection (in design) is achieved not when
  there is nothing more to add, but rather when
  there is nothing more to take away. (Antoine de
  Saint-Exupéry)
• Any tool should be useful in the expected way,
  but a truly great tool lends itself to uses you
  never expected.
• When writing gateway software of any kind, take
  pains to disturb the data stream as little as
  possibleand never throw away information unless
  the recipient forces you to!
• To solve an interesting problem, start by finding
  a problem that is interesting to you.
• Provided the development coordinator has a
  communications medium at least as good as the
  Internet, and knows how to lead without coercion,
  many heads are inevitably better than one.