CSEP Information Technology IT: People, Programs, Processes

1
CSEP Information Technology (IT)People,
Programs, Processes
Philip Maechling SCEC IT Architect17 October
2005
2
CSEP IT Capabilities

• Required capabilities of the CSEP IT system
  include
• Provide sufficient computing infrastructure to
  run a wide variety of predictive algorithms for
  evaluation.
• Allow users to contribute and validate algorithms
  on CSEP systems.
• Run predictive algorithms over extended periods
  of time in a stable computing environment.
• Provide information to contributors about health
  and status of algorithms.
• Provide access to algorithms results.
• Maintain sufficient information about computers,
  programs, and input data to reproduce algorithmic
  results.

3
CSEP IT Challenges

• CSEP IT challenges are primarily software
  integration and system operations challenge.
• Majority of software algorithms are developed by
  geoscientists and contributed to CSEP for
  evaluation.
• Software integration, software validation, and
  software configuration management, and data
  management, are key to reproducibility of
  results.
• Carefully managed system administration is
  required for long term execution of predictive
  algorithms.

4
CSEP Solutions to IT Challenges

• Our proposed system design involves the following
  aspects
• Small collection of computers, networks, and
  storage.
• Role-based organization to integrate and run
  algorithms.
• Execution of predictive programs on CSEP
  computers.
• Process-based program integration, software
  validation, source code management, and system
  administration to ensure repeatable results.

5
CSEP Solutions to IT Challenges Computers

• Integration System A flexible, accessible,
  collection of computers, network, and storage
  devices
• Scientists with predictive algorithms will
  install their software in the collaboratory and
  validate results.
• Operational System A stable, secure, highly
  available, restricted access system
• Once validated, algorithms are transferred to
  operational system and run for long periods of
  time.
• Distribution System Remotely accessible, highly
  available system for distribution of status and
  results.
• CSEP operational system will be transparent in
  operation to allow external review

6
Overview of CSEP IT System
7
CSEP Solutions to IT Challenges

• People
• Role-based CSEP organizational structure assures
  all key roles have a responsible individuals
  assigned to them.
• Programs
• CSEP programs and data are installed and
  validated on integration system. Once validated,
  programs are transferred to operational system
  and run for evaluation.
• Processes
• Repeatable, process-based approach to algorithm
  integration, algorithm validation, software
  configuration management, and system operation.

8
CSEP Solutions to IT Challenges People

• Role-based organization structure will be used so
  that significant Project responsibilities are
  always clearly identified with a Project
  participant
• Types of roles in CSEP organization
• Scientific Roles
• Algorithm Scientific Lead
• Software Integration and Development Roles
• Software Configuration Lead
• Validation Test Lead
• Algorithm Results Distribution Lead
• Algorithm Technical Contact
• System Administration and Operation Roles
• Operational System Administrator
• Operational System Monitor

9
CSEP Solutions to IT Challenges Programs

• Predictive Programs integrated into CSEP system
  through collaborative efforts of scientists and
  CSEP personnel.
• Programs and data will be moved into CSEP
  Integration System.
• Correct operation of Predictive Program will be
  established on Integration System. Predictive
  Program will be run until all are satisfied it is
  operating correctly.
• Predictive Program will be transferred to
  Operational System. Access to operational system
  is restricted to CSEP personnel only.

10
CSEP Solutions to IT Challenges Processes

• Standardized, repeatable processes will be
  established to ensure efficient and accurate
  integration of program and reproducibility of
  results. Candidate Processes include
• Evaluation and qualification of predictive
  algorithms
• Integration and validation of algorithms on CSEP
  systems
• Transfer of Algorithms from integration system to
  operational system
• Software configuration management
• Data management
• System administration
• Software and system maintenance

11
CSEP Solutions to IT Challenges Processes

• Example of Evaluation and Qualification Process
• Collect standardized information about program
• Computer Hardware
• Software packages used
• Input Data
• Output format
• Review program computational and data
  requirements with geoscientists and computer
  scientists.
• Identify Diagnostic information available

12
CSEP Solutions to IT Challenges Processes

• Integration and validation of algorithms on CSEP
  Systems Process
• Build software on CSEP computers.
• Check source code into Configuration Management
  system and version the software.
• Define and save known input data set.
• Generate and save known good output data set.
• Create automated testing procedure.

13
Conclusions

• The capabilities and skills required to build the
  CSEP IT system match the capabilities skills and
  of SCEC.
• Experience developing distributed collaborative
  computing system.
• (e.g. SCEC/CME)
• Experience establishing collaborative and
  software development processes.
• (e.g. SCEC/CME Standards Process)
• Experience coordinating scientific validation
  exercise
• (e.g. SCEC/PEER wave propagation studies)
• Domain expertise (geo-scientific expertise)
  available to support IT development.
• (e.g. Through SCEC Scientific Working Groups)
• Access to experienced, large-scale, operational
  computer organization.
• (e.g. Collaboration with USC High Performance
  Computing)