Core Application Software Planning and Preparation

1
Core Application Software Planning and
Preparation

• Ian Fisk
• DOE/NSF Breakout Session
• Jan. 17, 2002

2
Outline

• Specific Changes in Planning since the last
  review
• Near-term plans for project sub-tasks
• Preparation for Data Challenges and Physics TDR
• Time for Discussion

3
WBS 2.1 AFT and Reconstruction
Planning Changes

• 2.1.1 Addition of item for database and data
  model development and deployment
• US-CMS Contribution to LCG effort
• Zhen Xie of Princeton
• 2.1.2 COBRA Development
• Refined to concentrate on COBRA development to
  change the persistency scheme
• Bill Tanenbaum of FNAL
• 2.1.3 Sub-system Development
• 2.1.3.1 Calorimetry Reconstruction Framework
  Coordination
• Vladimir Litvin Caltech

• 2.1.3.2 Physics Reconstruction, Calibration, and
  Testbed Implementation
• New task designed to focus on aiding US detector
  groups on implementation within the common
  framework
• 2.1.3.3 Detector Description Database
• This effort is maturing and there is interest in
  common effort
• CMS needs to decide how much can be done within
  the context of the LCG
• Michael Case of UC Davis
• OSCAR Development put on hold for US-CMS
• CMS is reorganizing OSCAR and currently
  concentrating effort at CERN

4
Near Term Plans for 2.1

• Continue to develop ROOT COBRA Prototype
• Release of COBRA Version for validation out
• We have Closed-out the investigational prototype
  phase
• Currently working on a production quality
  application to run small GEANT4 validation tasks
  during the first six months of the year
• Prepare to deploy POOL for testing as functional
  prototypes are available
• Work on POOL Prototype
• First external release occurred in Dec.
• Next external release expected by the end of Feb.
• DDD deployment in CMS software clients is
  proceeding. Transition in the reconstruction
  software expected by February
• Refine and improve DDD
• Assess future participation in common projects

5
2.2 Visualization and Analysis

• Planning Changes small, mainly name changes to
  clarify tasks
• IGUANA planning kept up to date by Lucas Taylor
  and the IGUANA team
• 2.2.1 Graphical User Interfaces
• This has entered primarily an MO phase
• Still conduct reviews periodically to re-assess
  technology choices
• 2.2.2 Detector and Event Visualization
• Largely unchanged from previous WBS
• ORCA
• OSCAR
• The addition of one task to support detector
  specific visualization packages
• 2.2.3 Analysis Software Implementation,
  Integration, Commissioning and Testing
• Change from Interactive Analysis and
  Presentation to more clearly indicate that the
  CMS responsibility is not development of analysis
  tools.

6
2.2 Near Term Plans

• Schedule is to have a baseline visualization and
  data browsing tools synchronized with the
  development of the new persistency baseline.
• Working nicely so far
• Feature improvements
• Time to revisit the physics communities
• Assessment of integrating analysis software

7
2.3 Distributed Computing

• Section with the largest changes
• This section became something
• of a hodge-podge
• Mixture of on and off project
• activities
• A number of items that could
• be more easily classified in other
• areas of the CMS effort
• Monitoring and Simulation
• go to UF
• Leave items coupled to CMS Core software
• Grid Interfaces
• Production Tools
• Data and Production Challenge Software
  Preparations
• More effectively track the grid efforts by
  activity
• Dedicated section on Distributed Production and
  CMS Grid Testbed activities

8
Near Term Plans for 2.3

• 2.3.1.1. Establish Grid Requirements with LCG for
  CMS software
• 2.3.1.2 Create plan of satisfying mutual
  requirements within the CMS software framework
• 2.3.3.9 Complete Post DAQ TDR production and
  concentrate on development of CMS tools needed to
  complete the pre-DC04 production
• Requires good integration with grid activities.
• 2.3.2.2 Maintenance and Development of MC_RunJob
  for local and grid distributed production

9
Preparation for DC04

• CMS has completed the DAQ TDR Milestone and now
  must concentrate on the next major milestone the
  5 data challenge. Now referred to as DC04
• Software Requirements
• Validated GEANT4 Based Simulation
• Reliable Persistent Storage Solution
• Full Detector Description Database
• Reconstruction Framework
• Ideally calibration and alignment applications
• Tools for analysis
• Not clear in DC04 how detailed an analysis will
  be attempted
• Grid Requirements
• Authentication and Authorization Services
• Information Services
• Software Distributions
• Data and Job Management
• Monitoring at the Job and Resource Level
• Production Requirements of 50 million events

10
Software Preparation

• Progress is being made mainly at CERN on the
  GEANT4 based simulation replacement
• Need to get framework in shape for small
  validation samples soon.
• Talked about Framework, Persistency, and the DDD
  on Thursday
• Lots left to do, but the development is
  progressing
• Calibration and Alignment Applications
• Need to help PRS groups perform basic analysis
  tasks from within the framework
• Currently most groups use the framework to create
  a decoupled summary data file (Root file or
  Ntuple)
• Breaking the improved data accessibility offered
  by the database
• Proposed replacement position at FNAL would work
  in this area

11
Software Preparation

• Analysis
• In DC04 full real-time analysis of the
  reconstructed data is not a priority. Simply
  reconstructing and calibrating in real-time is
  sufficiently ambitious
• For the completion of the Physics TDR analysis
  for establishing requirements in the Computing
  TDR we need to start creating distributed
  analysis strategies and tools now
• US-CMS has been funding this work with ITR
  proposals
• Caltech and UCSD received a small ITR to work on
  development of distributed analysis prototypes
• See Julians Talk
• We are clearly in the RD period
• We have not attempted to perform analysis like
  this in the past
• Not clear what final products will come out of
  the development effort
• US-CMS recently submitted a large ITR
  pre-proposal to finish the development and
  empower the US community to take full advantage
  of LHC physics
• See Ruths Talk

12
Grid Preparations

• There are aspects of Grid preparations that span
  the two sides of the project.
• Elements that are clearly CAS, like the interface
  of CMS software to the grid
• Elements like authentication and resource
  monitoring which are facilities
• Elements like data management and process
  monitoring that have elements in both
• Facilities is handling issues of authentication
  and authorization
• Thorny issue that requires buy-in from a lot of
  different places
• Software Distributions
• CAS engineer Natalia Ratnikova developed DAR
  (Distribution After Release) which was used very
  successfully during the Spring production run
• Currently working on interfacing it to Grid
  Software Distribution tools like PacMan.

13
Data and Job Management

• Tony Wildish has been organizing evaluations of
  tools needed for the pre-challenge production and
  data challenge itself
• Currently looking at 4 externally developed data
  management packages
• dCache
• Interesting tool developed at DESY for managing
  data locally and across the WAN
• Has some functionality in common with the AMS and
  the RRP
• GDMP
• EDG has made interesting improvements in GDMP
  functionality
• SRB (Storage Resource Broker)
• Developed by NPACI, has large development team
  and reasonable user base
• Good replica catalog and data management features
• Enhanced Tonys Scripts
• Tonys scripts using Grid tools for
  authentication and underlying transfer protocol.
  Should improve the ease of use and the
  performance.

14
Grid Preparations

• Job Management
• This fall there was a very successful test of
  grid prototypes for production
• Using Condor-G and Globus for job management
• 5 US sites were able to produce 1.5M events for
  CMS official production on the IGT.
• Centrally managed from FNAL
• Still requires people to watch it and needs
  automation, but very real progress
• Job and Resource Monitoring
• Resource Monitoring and the development of
  monitoring systems is within the Facilities
  project
• Job Monitoring Requires hooks into the software
• This will be done in combination with the grid
  interface development
• New FNAL person will work in this area.