An OGSABased Accounting System for Allocation Enforcement across HPC Centers

1
An OGSA-Based Accounting System for Allocation
Enforcement across HPC Centers

• TS10 Service Applications
• Thomas Sandholm sandholm_at_pdc.kth.se
• Olle Mulmo
• Peter Gardfjäll
• Erik Elmroth
• Lennart Johnsson

2
Key Question
?

• How do we share national Grid compute resources
  in a fair, secure, open, and scalable way

3
Outline

• Requirements on Software Qualities
• Open Grid Services Architecture
• SweGrid National Grid Testbed
• SweGrid Accounting System
• Results
• Lessons Learned
• Q A

4
Fairness vs. Utilization

• Trade-off Fair resource distribution and optimal
  resource utilization
• Soft real-time quota enforcement
• User preferences
• Resource policies
• Allocation authority policies

5
Security vs. Scalability

• Integrity Privacy
• Single Sign-on/Impersonation
• DoS/Replay Attack prevention
• Privilege Delegation
• Message Level vs. Transport Level
• Policy Driven Authorization PDP, PAP, PIP, PEP
• Scale
• National Grid
• No single point of failure but coordinated
  allocationenforcement

6
Openness Interoperability

• Systems Integration Platform
• Scheduler/Workload Manager Agnostic
• Programming Language/Model Agnostic
• Portable (100 pure Java)
• XML Based Standards XPath, XQuery, XSLT, GGF-UR,
  XML-Signature, XML-Encryption, XACML
• Web/Grid Services Standards SOAP, WSDL,
  WS-Security, OGSA, GGF-UR, GSI, GSSAPI, OGSI/WSRF

7
Outline

• Requirements on Software Qualities
• Open Grid Services Architecture
• SweGrid National Grid Testbed
• SweGrid Accounting System
• Results
• Lessons Learned
• Q A

8
Open Grid Services Architecture

• Global Grid Forum Standardization Initiative
• Architecture extending SOA and WSA to dynamically
  share stateful resources across organizational
  boundaries (realizing the Grid vision)
• defining, within a service-oriented
  architecture, a set of core capabilities and
  behaviors that address key concerns in Grid
  systems. OGSAv1
• Assumes state modeling according to OGSI/WSRF
• Core Infrastructure offering Inspection,
  Discovery, Lifetime Management, Notifications,
  Fault Handling
• WS-Resource stateful resource and associated
  Web service.
• Provide context for message exchange
• Addresses Grid security requirements such as
  Delegation and Single Sign-On

9
Role of Accounting in OGSA

• Accounting foundational service to
• Job Execution
• Make sure that only jobs with sufficient quota
  can be executed on the compute resource
• Decide queue priority based on available funds
  and usage history
• QoS/SLA Management
• Negotiate pricing based on resource usage
• Optimizing Utilization
• SLA Attainment/Policing
• Security
• Auditing
• Access Control
• PEP/PIP

10
Outline

• Requirements on Software Qualities
• Open Grid Services Architecture
• SweGrid National Grid Testbed
• SweGrid Accounting System
• Results
• Lessons Learned
• Q A

11
SweGrid

• SweGrid connects 600 compute nodes (Intel P4)
  across 6 Swedish HPC centers interconnected by
  10Gbs GigaSunet network
• 400 HPC users at all centers (some overlapping)
• Inaugurated March 2004
• 50 currently active researchers
• Up to 10k jobs per month per site

12
SweGrid Continued

• Resource quotas allocated by Swedish National
  Allocations Committee (SNAC) after peer-review of
  promising research projects with high
  computational demands (c.f. NRAC)
• Initially homogeneous hardware but heterogeneous
  scheduling, security, and accounting environment
  (policies, tools, data, processes, etc)
• Wanted Uniform resource quota use allocation

13
Outline

• Requirements on Software Qualities
• Open Grid Services Architecture
• SweGrid National Grid Testbed
• SweGrid Accounting System
• Results
• Lessons Learned
• Q A

14
SweGrid Accounting System (SGAS) Key Design Points

• Decentralized accounting solution based on
  standard, open protocols in compliance with the
  proposed OGSA
• 3-party (user, resource, allocation authority)
  policy customization
• Non-intrusive to local site accounting systems
• All components governed by a scalable
  cross-organizational authorization framework

15
SGAS Component Overview
PAT
AddUser
WSDL
WSDL
Query
SubmitJob
Bank
LUTS
Reserve/Release
PublishUR
WSDL
WSDL
JARM
Resource Manager
Broker
User
Resource
Scheduler
Policy Administration ToolLogging and Usage
Tracking ServiceJob Account Reservation Manager
16
SGAS Security Design
XML-Signature
XML-Encryption
PAT
External Authorization Service
PDP
Membership/CommunityService
PIP
Bank
PAP
PDP
LUTS
PIP
PAP
WS-SecureConversation
JARM
PEP
PDP
Site Policy Manager
PIP
Broker
User
Resource Manager
PKI
Resource
Scheduler
Kerberos
Policy Administration PointPolicy Decision
Point Policy Information Point Policy Enforcement
Point
Credential Delegation
17
Outline

• Requirements on Software Qualities
• Open Grid Services Architecture
• SweGrid National Grid Testbed
• SweGrid Accounting System
• Results
• Lessons Learned
• Q A

18
Overdraft XACML Policy
ltCondition FunctionId "urnoasisnamestcxacml
1.0functioninteger-less-than-or-equal"gt
ltApply FunctionId "urnoasisnamestcxacml1.0
functioninteger-one-and-only"gt
ltEnvironmentAttributeDesignator
AttributeId "sgasoverdrawpercentrequeste
d" DataType "http//www.w3.org/2001/
XMLSchemainteger"/gt lt/Applygt ltAttributeValue
DataType "http//www.w3.org/2001/XMLSchemain
teger"gt 175 lt/AttributeValuegt lt/Conditiongt
19
Overdraft Fuzzy Logic Policy
R1 overdraft is low ? allocation left is much ?
allow reservation R2 overdraft is high ?
allocation left is little ? disallow
reservation R3 allocation proximity is soon ?
overdraft is high ? allocation left is much ?
allow reservation R4 allocation proximity is
soon ? overdraft is low ? allocation left is
little ? allow reservation
20
Super Computing 2004 Demonstration
21
Outline

• Requirements on Software Qualities
• Open Grid Services Architecture
• SweGrid National Grid Testbed
• SweGrid Accounting System
• Results
• Lessons Learned
• Q A

22
Conclusions

• Document centric communication in conjunction
  with semi-structured native XML databases is a
  very flexible combination
• Batch charging and eager prepare reservation
  needed for scalability
• Timestamp based allocations distributed in a
  staggered monthly flow result in the best
  trade-off between fairness and utilization
• Generic PEP/PDP/PIP/PAP model useful for
  encapsulating and evolving authorization code
• OGSI/WSRF state management ideal for controlling
  fine grained service state such as account
  quotas, reservations and policies in a standard
  way

23
Future Work

• With large-scale flexibility and configurability
  comes complexity and it becomes hard to optimize
  high-level goals and to realize detailed user QoS
  requirements development of an SLA Management
  framework and user/resource goal driven
  optimizing agents (WS-Agreement, ContractNet)
• Initial focus has been on scientific community
  resource sharing - support economic brokering and
  for-profit banks
• Multi jobs may overload the bank - SAML
  assertions (c.f. cheques) as a multi-allocation
  payment and reservation method

24
Learn more

• http//www.sgas.se
• http//www.swegrid.se

25
Outline

• Requirements on Software Qualities
• Open Grid Services Architecture
• SweGrid National Grid Testbed
• SweGrid Accounting System
• Results
• Lessons Learned
• Q A