(Some) OSG Users and Applications

1
(Some) OSG Users and Applications

• Torre Wenaus (BNL)
• Open Science Grid Advisory Group Meeting
• Chicago
• June 12, 2007

2
LIGO

• Provides two production sites to OSG (Penn State,
  UW Milwaukee)
• Developing OSG aware workflows for
  (initially)Compact Binary Coalescence (CBC)
  analysis
• Pegasus/Condor workflow, O(10k) jobs, TBs of data
• OSG middleware VDT, Globus, Condor
• Sustained running gt200 jobs for gt1 week
• gt100 job target scheduled for June 07 achieved in
  Feb
• Peak utilization 331 jobs (19k CPU hrs) in 1 week
• Fall target1000 jobs peak
• Developing data management to enable utilization
  of small sites with low disk storage

3
LIGO Integrated OSG CPU
4
CMS
5
CMS Data Transfer Analysis
6
ATLAS

• OSG production stable, efficient, well exceeding
  US share (19 share 29 delivered), up to 2800
  CPUs/day
• Simu, reco at T1, T2s. 3-4x scale-up by year end
• 400 TB data at BNL (largest data resource in
  ATLAS), 100 TB at T2s
• Biggest challenge is data management tightly
  integrated in production workflow
• 10 US production sites, have now expanded beyond
  ATLAS resources to opportunistic sites
• US-developed Panda system for production and
  analysis, .1 FTE operations load, fault
  tolerant, good scaling, Condor based
• 100 analysis users insulated from grid
  complexities
• Expanded to WLCG and also supporting non-ATLAS
  OSG use

7
US ATLAS Apr/May Error Rates
8
Observations

• OSG well focused on requirements of primary
  stakeholders
• No significant pull away to costly new
  requirements from other domains (eg fine-grained,
  data security)
• With some exceptions, our requirements are often
  a superset of those of others
• At the same time, OSG is proving itself serious
  about drawing in and supporting a broader user
  community
• Both these are greatly aided by OSG being
  operated as a cohesive project, controlling its
  resources and asserting its program priorities
  with participants

9
Comments of a Grid User

• Do we (HENP) still have a grid problem?
• Over-promised, under-delivered
• But very well resourced
• And therefore, poor return for the investment
• Yes
• Is OSG part of the problem?

10
Comments (2)

• Is OSG part of the problem?
• No, in my opinion
• OSG is trying to do its job right, and so far
  with promising results, in the ways that LCG
  tried initially to do it right, and failed
• Where its job is to turn the grid into an
  effective, efficient science producer

11
Comments (3)

• LCGs mantra in early years was LCG is not a
  middleware development project
• Rather, LCG would focus on middleware
  evaluation/feedback/acceptance, and
  integration/deployment
• And, LCG would seek to establish collaboration
  among the experiments on the higher levels of the
  infrastructure -- distribution production/data
  management/analysis
• Worthy, well-directed ambitions which didnt
  happen
• OSG is following this path, though, and so long
  as it does, stands an excellent chance of
  succeeding at keeping a science customer focus,
  mediating middleware project interactions with
  this focus, undertaking collaborative
  science-focused higher-level infrastructure
  projects, and delivering -- at a relatively
  modest cost -- an effective infrastructure for
  producing science.

12
Conclusions

• OSG as a funded project is young
• But not so young that we cant see a direction
• One thats absorbed lessons from other projects
• For the large customers, and I think evidence
  suggests for the small as well, the direction
  looks good so far
• US ATLAS reported in its last agency review that
  were on a trajectory to meeting all distributed
  computing needs of the physics program when
  datataking arrives
• This anticipates OSG continuing on its present
  trajectory of strong alignment with real
  experiment needs