Application Group Overview

1
Where the physics gets done
2
HEP Challenges Frontiers of Information
Technology

• Rapid access to PetaByte/ExaByte data stores
• Secure, efficient, transparent access to
  heterogeneous worldwide distributed computing and
  data
• A collaborative scalable distributed environment
  for thousands of physicists to enable physics
  analysis
• Tracking the state and usage patterns of
  computing and data resources, to make possible
  rapid turnaround and efficient utilization of
  resources

3
Grid Analysis Environment (GAE)

• The Acid Test for Grids crucial for LHC
  experiments
• Large, diverse, distributed community of users
• Support for 100s to 1000s of analysis tasks,
  shared among dozen of sites
• Widely varying task requirements and priorities
• Need for priority schemes, robust authentication
  and security
• Operates in a severely resource limited and
  policy constrained global system
• Dominated by collaboration policy and strategy
• Requires real-time monitoring task and workflow
  tracking decisions often based on a global
  system view
• Where physicists learn to collaborate on analysis
  across the country, and across world regions
• Focus is on the LHC CMS experiment but
  architecture and services can potentially be used
  in other (physics) analysis environments

4
Build services on web service frameworks such as
Clarens and provide end-2-end monitoring using
systems such as MonALISA
Emerging Vision A distributed set of rich and
complex services to support analysis in a
distributed environment

• HTTP,
• SOAP,

• XML-RPC,
• JSON, RMI

Grid Services Web Server
Tier2 Site
Workflow Execution
Workflow Definitions
Catalogs
Compute Site
Scheduler
Metadata
Applications
Fully- Abstract Planner
Storage
Virtual Data
Partially- Abstract Planner
Data Management
Replica
Monitoring
Fully- Concrete Planner
Network
GAE Architecture
Global Command Control
Reservation
Planning
Monitoring
Execution Priority Manager
Grid Wide Execution Service
5
Other Clients Web browser ROOT (analysis
tool,caves) Python (codesh) Cojac (detector
viz.)/ IGUANA (cms viz tool)
Analysis Flight Deck JobMon Client JobStatus
Client MCPS Client
CLARENS Performance (1400 Calls/Second)

• HTTP,
• SOAP,

• XML-RPC,
• JSON, RMI

Grid Services Web Server
Monitoring Clients MonALISA Clients
Clarens
Tier2 Site
MCPS
Workflow Execution
Workflow Definitions
Discovery
JobStatus
Runjob
Catalogs
Compute Site
CLARENS Web Service Framework Dynamic Service
Discovery, Access Control Management, Certificate
based access
Scheduler
Metadata
Applications
DCache
Fully- Abstract Planner
Storage
ROOT
Sphinx
Virtual Data
FAMOS
Partially- Abstract Planner
JobMon
Data Management
ORCA
Replica
BOSS
estimators
Monitoring
MonALISA
Fully- Concrete Planner
MonALISA
Network
MonALISA
Global Command Control
Reservation
Planning
Monitoring
Execution Priority Manager
Grid Wide Execution Service
BOSS
6
NETWORK All Resources, including networks will be
sparse, and need to be actively managed

• HTTP,
• SOAP,

• XML-RPC,
• JSON, RMI

Grid Services Web Server
Tier2 Site
Workflow Execution
Workflow Definitions
Catalogs
Compute Site
Scheduler
Metadata
Applications
Fully- Abstract Planner
Storage
Virtual Data
Partially- Abstract Planner
Data Management
Replica
Monitoring
Fully- Concrete Planner
Network
Global Command Control
Reservation
Planning
Monitoring
Execution Priority Manager
Grid Wide Execution Service
7
Other Clients Web browser ROOT (analysis
tool,caves) Python (codesh) Cojac (detector
viz.)/ IGUANA (cms viz tool)
Analysis Flight Deck JobMon Client JobStatus
Client MCPS Client
MONALISA Global view of the system through
monitoring

• HTTP,
• SOAP,

• XML-RPC,
• JSON, RMI

Monitoring Clients MonALISA Clients
Grid Services Web Server
Tier2 Site
Clarens
MCPS
Workflow Execution
Workflow Definitions
Discovery
JobStatus
Runjob
Catalogs
Compute Site
Scheduler
Metadata
Applications
DCache
Fully- Abstract Planner
Storage
ROOT
Sphinx
Virtual Data
FAMOS
Partially- Abstract Planner
JobMon
Data Management
ORCA
Replica
BOSS
estimators
MonALISA
Monitoring
Fully- Concrete Planner
MonALISA
MonALISA
Network
MONALISA Proactive in minimizing Grid traffic
jams Network Services
Global Command Control
Reservation
Planning
Monitoring
Execution Priority Manager
Grid Wide Execution Service
BOSS
8
Rich set of Clarens Web Service Clients Web
browser ROOT (analysis tool,caves) Python
(codesh) Cojac (detector viz.)/ IGUANA (cms viz
tool)
Analysis Flight Deck JobMon Client JobStatus
Client MCPS Client

• HTTP,
• SOAP,

• XML-RPC,
• JSON, RMI

Monitoring Clients MonALISA Clients
Grid Services Web Server
Tier2 Site
Clarens
MCPS
Workflow Execution
Workflow Definitions
Discovery
JobStatus
Runjob
Catalogs
Compute Site
Scheduler
Metadata
Applications
DCache
Fully- Abstract Planner
Storage
ROOT
Sphinx
Virtual Data
FAMOS
Partially- Abstract Planner
JobMon
Data Management
ORCA
Replica
BOSS
estimators
Monitoring
MonALISA
Fully- Concrete Planner
MonALISA
Network
MonALISA
Global Command Control
Reservation
Planning
Monitoring
Execution Priority Manager
Grid Wide Execution Service
BOSS
9

• NUST collaboration http//ultralight.caltech.edu/
  web-site/nust/html/index.html
• GAE http//ultralight.caltech.edu/web-site/gae/ht
  ml/index.html

10
Publications
11
Support new Discoveries in the LHC Physics Era