HyperNews for LHCb

1
HyperNews for LHCb

• Tim Adye, Rutherford Appleton Laboratory
• presented by
• Jim Libby, Oxford University
• Physics - HLT Computing Plenary
• LHCb Collaboration Week, Heidelberg
• 13th September 2006

2
Outline

• What is HyperNews?
• Why switch?
• HyperNews in LHCb
• How do we get started?
• User registration and subscription
• Migrating existing lists
• Concluding thoughts
• References
• Backup slides

3
What is HyperNews?

• We could use it as a replacement for most of our
  Simba2 mailing lists
• Also has features of
• bulletin board systems (like RootTalk) and
• news systems (like Usenet)
• Not intended to replace TWiki or web pages
• Widely used in HEP
• BaBar, H1, STAR, Geant4, CMS, ATLAS,
• Heavily used BaBar 400,000 messages since
  1997CMS 32,000 (December 2005)Atlas 8,000
  (January 2006) LHCb test 200
• Continuing development
• HyperNews server managed by LCG
• runs independent instances for all four LHC
  experiments

4
Why Change from Simba E-mail Lists?

• Web archive is central to HyperNews
• Web access to Simba archives very cumbersome
• Not so bad via Outlook, but only really for CERN
  local users
• single, integrated, system instead of disjoint
  mailing lists
• can search all fora at once
• see all recent posts (all or a subset of fora)
• displays threads
• Threads, sub-threads, individual postings have a
  static URL for easy reference
• Eg. in other postings, TWiki pages, private
  e-mail
• easy to browse, very fast to search
• I searched for database in 400,000 BaBar posts.
  It found 18,000 in under a second
• Can post via the web or e-mail

5
(No Transcript)
6
(No Transcript)
7
Keeping up to Date

• Can subscribe to receive posts as e-mails
• Simple and powerful control of what you receive
• Whole forum or individual threads
• Reply to post with e-mail, or follow link to web
  display
• ... or keep up-to-date / catch up via the web
• recent posts page

8
Give it a try in the HyperNews Test Forum
9
(No Transcript)
10
Lots of other nice features/uses
)
(

• on holiday switch
• automatic thread subscription
• full thread view for printing
• refer to old thread with a simple URL
• Forum ideas
• new forum announcements, QA, chat, paper
  reviews, recipes, logbook
• anyone can ask for a forum, even if of minority
  interest
• Just ask any former BaBarian, and theyll tell
  you how much they miss HyperNews

11
HyperNews for LHCb

• LHCb HyperNews running since July
• Contains a few initial fora
• Tested by migrating lhcb-core-soft e-mail list
• No big problems
• Some concerns about how best to manage migration
• Can always catch up with the recent postings
  web page
• Once we get the go-ahead, new fora can be created
  on request
• Migrate existing lists adiabatically
• Let each group manage their own transition in
  their own time
• A similar migration has been relatively painless
  in CMS and Atlas

12
How do I start using HyperNews?

• From https//hypernews.cern.ch/ select LHCb
• The first time (only) you use HyperNews, follow
  thenew member instructions
• If you have an LHCb AFS (lxplus) account, you can
  use this to register just takes a minute
• If you dont, send an e-mail to the admin address
  listed there(hn-lhcb-admin_at_cern.ch)
• LHCb members without AFS accounts get a
  manually-created account
• Non-LHCb people can be subscribed to specific
  fora
• Subscribe to the fora you are interested in
• Just scan the list and check the boxes
• If you subscribe to HyperNews Announcements
  then you will learn about new fora as they are
  created.
• ... or just wait for the announcement and follow
  instructions

13
How do I Migrate my E-mail list

• Post a 2-week warning to the old list, so people
  know when the list will be migrated
• Add hn-lhcb-admin_at_cern.ch as an owner of the old
  list
• Allows me to manage the transition for you
• Select Request a New Forum and fill in the
  details
• Include the old list name in the description
• I will take care of the rest...
• Create the HyperNews forum
• post final notice on the old list and in
  HyperNews Announcements
• Will include reminders of how to register and
  subscribe
• block posts and subscriptions on the old list
• This is important to prevent people accidentally
  posting to the old list, not realising that the
  message only goes to old members
• I will keep an eye out for people accidentally
  using the old list

14
Final Thoughts

• Effective communication is crucial in a large
  geographically-distributed collaboration like
  LHCb
• Meetings, documentation, TWiki, and e-mail all
  useful forms of communication
• But e-mail has problems when collaborating with
  many people
• private e-mail restricted to those in the loop
• mailing lists OK for announcements, not so good
  for discussion
• mailing lists most useful if also easily used
  after the fact
• HyperNews addresses these issues
• makes collaboration e-mail more Wiki-like
• encourages open communication
• opens up archive documentation for free

15
References

• HyperNews development
• D.A.Smith, P.Elmer, T.Hung, HyperNews - managing
  discussions in HEP, CHEP06
• http//hypernews.slac.stanford.edu/
• LHCb instance
• https//hypernews.cern.ch/HyperNews/LHCb/top.pl
• which is linked from
• https//hypernews.cern.ch/

16
Backup Slides
17
Design choices

• Why not automatically register all LHCb
  members?Why not automatically transfer
  subscriptions from the old lists?
• Start as we mean to go on
• Simba based on e-mail address, HyperNews based on
  account name (usually tied to AFS account)
• No 1-to-1 correspondence for many people, leading
  to confusion and lots of extra work
• Besides, self-subscription is trivial
• just check the box
• can review what other lists to join on the same
  page
• Why tie HyperNews account to AFS account?
• Allows quick, reliable, and secure initial
  authentication
• Allows easy HN password setting, if forgotten
• Though many people let their web browser remember
  for them
• Compatible with CMS and Atlas simplifies
  management

18
Why not use ltinsert-cool-technologygt instead?

• I havent done a full survey but, for
  collaboration, I find HyperNews the most
  convenient
• cf. Usenet, BBS (like RootTalk), discussion
  blogs
• and of course mailing lists and private mails
• It is widely used in HEP, so familiar to some
• Ask any former BaBarian, and theyll tell you how
  much they miss HyperNews
• LCG run the server for us
• We know the developers
• at SLAC and CERN (Pete Elmer, CMS)

19
Limitations of HyperNews

• All fora are open to all collaboration members
• private lists should remain in Simba2
• All e-mails to/from HyperNews are converted to
  plain text
• attachments are stripped off give a URL instead
• good practice anyway dont fill up peoples
  inboxes
• URLs are automatically turned into links in web
  view
• Support for HTML and attachments will be added
  soon
• Used to be a problem with threading posts from
  Microsoft Exchange
• MS hotfix in January (applied at CERN, RAL,...)
• After only 29 years, Microsoft support SMTP RFC!
• Still relies on Simba spam filtering
• Next version will have SpamAssassin built-in

20
CMS Migration Experience

• 30 CMS lists migrated since December 2005
• All of the computing and software lists are in
  HyperNews
• Most of the big "analysis" and "analysis/reco"
  lists migrated
• Only some smattering of detector groups, so far.
• Following a similar adiabatic procedure as we
  plan
• LHCb have 65 Simba lists in total, though I don't
  know how many of those are active.
• Also created 60 entirely new HyperNews fora

21
Discussion fora (1)

• Everyone should subscribe to HyperNews System
  Announcements
• New fora announced here, allowing people to
  subscribe
• Eventually could migrate nearly everything that
  currently has a mailing list
• Eg. one forum for each detector and software
  subsystem
• In some cases, it is useful to have separate
  development discussions and user questions
  fora
• Experts subscribe to, or watch, both
• Users post to the latter, receive e-mail
  responses without having to subscribe
• Can also browse/search this forum to see if it
  has already been answered
• Soon users start to help other ?

22
Discussion fora (2)

• Task forces and system/paper reviews have their
  own fora
• Separates out short-term discussion, while
  maintaining a record in the archive
• Can also be used as a shared record of important
  e-mails
• Eg. communications with journals, approvals,
  minutes of external meetings, etc.
• Just forward them to the appropriate e-mail
  gateway address
• I already created some generic fora
• Linux, programming, and (La)TeX questions
• HyperNews announcements, feedback, and test
• Use HyperNews Test Forum to play with the
  system
• CERN Computing Announcements
• Automatic feed from info-experiments_at_cern.ch
• Chat and discussion that doesnt fit elsewhere

23
Other Use Cases

• If a question or discussion has come up before,
  just post a link to it
• No need to copy text
• The questioner can see the full discussion /
  thread
• Sometimes a discussion strays to another forums
  topic
• Post replies in new forum with link to old
  quick link old?new
• Short-term documentation, recipes, and kicks can
  be posted as a HyperNews announcement with a link
  from the documentation
• When it becomes obsolete, remove the link
• a record remains of what people recommended
• Sort-term only not a substitute for proper
  documentation!
• Can easily migrate to Wiki

24
Forum categories

• Fora requested by each group, but categories
  defined in advance
• can be changed of course, but not from the web
  form

• Perhaps add
• Online Computing
• Offline Computing
• Physics Analysis
• Trigger
• Detector Subsystems
• or one category per subsystem?
• National Discussions
• Collaboration Matters
• Detector Commissioning / Operations

• Currently have
• Computing
• Computing Operations
• General Discussions
• HyperNews
• Could add now or later
• Journal Paper Reviews