Can HEP benefit from Open Source ?

1
Can HEP benefit from Open Source ?

• Manuel DelfinoEuropean Organization for Nuclear
  Research (CERN)
• Permanent address Physics Dept., Universitat
  Autonoma de Barcelona, Spain

2
What is Open Source ?

• Direct descendent of Internet toolkit and
  Berkeley Unix hackers ?
• A technologically related social movement ?
• An emerging business model ?
• Make up your own mind by
• Reading The Cathedral and the Bazaar by Eric S.
  Raymond, ISBN 1-56592-724-9
• Browsing http//www.opensource.org

3
Why a 20 year gap ?

• In my opinion, part of it is technological
• Commoditization of computing hardware
• Huge increases in Internet speed
• but also part of it is socio-political
• Global familiarity brought by CNN, inexpensive
  travel, cultural exchanges
• Disappearance of many trade barriers
• Arrival of a world that WANTED to be
  interconnected and was READY to work together

4
But there are more ingredients

• Perhaps it has taken 20 years to realize that
• Sharing your ideas and listening to others
  opinions is more valuable than carefully guarding
  your ideas
• Many people have similar problems
• The value is in the ideas, but to fully
  collaborate you have to share code
• Vision and architecture are more effective than
  management to create a following
• Humility is a great quality

5
Bazaar-style development

• Some of Raymonds 19 principles
• 3. Plan to throw one away you will, anyhow
• 5. When you loose interest in a program, your
  last duty is to hand it off to a competent
  successor
• 6. Treating your users as co-developers is
  route to rapid code improvement and debugging
• 7. Release Early. Release Often. And listen to
  your customers
• 11. The next best thing to having good ideas is
  recognizing good ideas
• 12. Often, the most striking and innovative
  solutions come from realizing that your concept
  of the problem was wrong

6
Role of Project Leader

• the cutting edge of the open-source software
  will belong to people who start from individual
  vision and brilliance, then amplify it through
  the effective construction of voluntary
  communities
• Other characteristics of open-source
  contributors
• self-promotion mercilessly criticized
• qualitymust be left to speak for itself
• attacking the author rather than the code is
  not done

7
So, can HEP benefit from Open Source ?

• (Actually, it already has
• the real question is should HEP do some of its
  development in open-source)

8
Almost 2/3 of CERN computer center is in Linux PCs
9
Opportunities

• Why does each major lab write its own management
  software for tape/disk ?(see E176, C223, C68,
  C308)
• Did you know that Cisco wrote a print server
  system whose goals are the same as E369, and
  place it in open-source ?
• Is our need for event-oriented statistical
  analysis similar enough to Migros ?

10
but no magic wand

• Warnings to the bandwagon jumpers
• Going Open-source implies a commitment to deliver
  and maintain a high quality product.
• The product must be of sufficient interest to
  others and must be general enough.
• This can often result in the need to increase the
  resources in the original development team for
  quite some time before and after open-source
  release.

11
Is HEP really ready for open source ?

• It is said that open source projects are tapping
  the top 5 in quality developers worldwide.
• Certainly the project leaders are impressive.
• Just as computing technologies build on the
  commoditization and banalization of previous
  ones, the open-source developers count on people
  being fluent in everything from architecture and
  design to multiple languages and details of
  CORBA
• Does HEP have such people ?
• Are they allowed to contribute to community ?

12
The importance of vision and architecture

• Linux is by far the biggest successful
  open-source project. However, it inherited its
  architecture from Unix.
• Emacs is another great success, but it is a
  relatively simple toolkit. Its success seems to
  be due to its flexibility.
• Mozilla (eg. Open-Netscape) is not doing so well.
  Perhaps it lacks vision and architecture ?
• The closer we get to experiments software, the
  more we seem to depend on our own HEP architects.
  But how much does Torvalds know about chips ?
  Could we still benefit from others?

13
How about HEP pseudo-open bazaar?

• Within an experiment ?
• It has been tried and it sort of worked
• Sadly, degenerates to personal battles
• Seems hard to convince experiments management
  fond of librarians and official code
• Between experiments ?
• Perhaps there is an opportunity ?
• How much can be moved from experiment-specific to
  HEP Foundation?

14
What is the bottleneck for HEP ?

• My personal opinion is that the HEP community as
  whole has not really absorbed the Internet way
  to do things
• Over-emphasis of face-to-face meetings
• Tribes come together at meetings, show their
  stuff, then go back and continue to do their
  own thing -gt Institutionalized forking
• Very little open e-discussion
• No productization Production on Prototypes
• Management very zealous of people not working
  directly for the experiment

15
So, can HEP benefit from Open Source?

• My conclusion Yes, it probably can
• Thanks to the large effort made in the last 5
  years to align software techniques used in HEP
  with the rest of the world, at least now we can
  probably explain ourselves to others (though
  maybe less so to our own collaborators !!!)
• But
• Carefully evaluate on a case-by-case basis the
  cost and the benefits
• Need project leaders with vision and humility
• Continuous training aligned with the world
• Build trust using e-communication
• Face investment without clear direct benefit
• Convince management of virtues of open-source