Towards Self-Organizing Distributed Computing Frameworks

1
Towards Self-Organizing Distributed Computing
Frameworks

• Vaidy Sunderam
• Dawid Kurzyniec
• Tomasz Wrosek
• Dept. of Math Computer ScienceEmory
  University, Atlanta, GAvss,dawidk,yrd_at_mathcs.em
  ory.edu
• Research supported by NSF and US DoE

2
Outline

• Background and context
• Goals and motivations
• H2O resource sharing model
• Heterogeneous distributed computing
• Multiple administrative domains
• Support for multiple programming paradigms
• Unified low level fabric
• Summary and conclusions

3
Background and Context

• The Harness project
• Joint project with ORNL and UTK follow on to PVM
• Heterogeneous metacomputing platform
• Facilitating dynamic, extensible, reconfigurable
  virtual machines
• Component oriented permits rapid replacement of
  technologies, models, etc
• Status
• Java-based framework for heterogeneous
  distributed computing
• Support for multiple programming models PVM,
  Javaspaces, FT-MPI
• Performance compared to native platforms within
  5 to 15
• Exploit usual advantages of component
  architectures
• Next stage
• Cooperative resource sharing across multiple
  administrative domains
• The H2O project Foundational substrate for
  distributed metacomputing

4
Goals and Motivations

• Goals enable efficient resource sharing
• Lightweight
• Minimal state
• Decentralized and/or localized control
• Self-organizing global status implicitly defined
  by local conditions
• Motivations
• Middleware is cumbersome and expensive
• Steep learning curve
• Rigid and fragile due to tight coupling
• Need for modularity, upgradeability, adaptability
• Proposal
• H2O client-provider-tpr architecture

5
H2O Abstraction

• Providers own resources
• CPU cycles, storage, information, applications,
• They independently make resourcesavailable over
  the network
• specify access policies
• Clients discover, locate, and utilizeresources
• Resource sharing occurs between single provider
  and single client
• Relationships may be tailoredas appropriate
• Including identity formats, resource allocation,
  compensation agreements
• Aggregation and reselling
• Resource aggregation is purely a client
  abstraction
• Clients can be providers (tprs) gt cascading
  pairwise relationships

Providers
Clients
Network
6
Major challenges

• Clients have different needs
• Hide specifics from a provider
• May want to just provide raw resources (CPU
  power, data storage, ...)
• Clients (or third-party resellers) may want to
  cook resources before using/serving
• Provider independence
• Resource sharing is an individual choice what,
  how much, when, to whom
• No coordination/synchronization between providers
• Security and resource management
• Providers wish to keep control over shared
  resources
• Clients require privacy, integrity and quality of
  service
• Shallow learning curve
• Interoperability
• Exchange of data and connectivity with existing
  software
• Web Services, CORBA, .NET, ...

7
Proposed Solution

• Resources provided as services
• Service active software component exposing
  functionality of the resource
• May represent added value
• They run within a providers container (execution
  context)
• They may be deployed by any authorized party
• Provider, client, or third-party reseller
• Different from traditional model
• Decoupling
• Providers from clients (to a large extent)
• Providers from each other

8
Example usage scenarios

• Resource computational service
• Reseller deploys software component into
  providers container
• Reseller notifies the client about the offered
  computational service
• Client utilizes the service

• Resource raw CPU power
• Client gathers application components
• Client deploys components into providers
  containers
• Client executes distributed application utilizing
  providers CPU power

• Resource legacy application
• Provider deploys the service
• Provider stores the information about the service
  in a registry
• Client discovers the service
• Client accesses legacy application through the
  service

9
Model and Implementation
Interface StockQuote double
getStockQuote()

• H2O nomenclature
• container kernel
• component pluglet
• Object-oriented model, Java-based prototype
  implementation
• Pluglet remotely accessible object
• Must implement Pluglet interface, may implement
  Suspendible interface
• Used by kernel to signal/trigger pluglet state
  changes
• In order to do something useful, components must
  be able to handle remote requests
• Remote Method Invocation (RMI) model adapted
• Pluglets export functional remote interfaces
• Remote interface set of remotely invokable
  methods
• Multiple inheritance of remote interfaces

Clients
Functionalinterfaces
(e.g. StockQuote)
Pluglet
Suspendible
Interface Pluglet void init(ExecutionContext
cxt) void start() void stop()
void destroy()
Interface Suspendible void suspend()
void resume()
10
Pluglet Interoperability

• Issues of remote method invocations
• Serialization of sophisticated object graphs
• Lack of shared address space (pass-by-copy
  semantics)
• Dealing with remote references, distributed
  garbage collection
• Handling remote failures
• Which invocation protocol to use?...
• JRMP, SOAP, IIOP, ...
• Expressiveness vs Interoperability vs Performance
• No ultimate choice important to support multiple
  protocols
• Sub-project RMIX
• Extended RMI framework for Java
• Supports multiple, pluggable invocation protocols
• Client can negotiate a protocol with a kernel
• Allows for multiple, customizable object
  endpoints
• Provides asynchronous invocation modes

11
Deployment of core functionality
12
Security and Resource Management

• Providers and resellers must have fine-grained
  control over resources
• Who can do what and when?...
• Authentication
• JAAS, Pluggable Authentication Modules (PAM)
• Facilitating standards (X.509 certificates,
  Kerberos, NT, Unix authentication)
• Allowing for customized or mixed solutions
• Authorization resource control
• H2O exploits Java platform safety and security
  features
• control over local filesystem, network adapters,
  thread usage
• Policies for temporal resource control
• Extensible XML based specification (by provider)
  of usage rules
• Remote invocations on pluglets are guarded by
  security proxies
• possible to restrict available interfaces and
  access rights based on invoker identity
• Privacy and integrity
• Leverage TSL/SSL layer

13
Login and Pluglet loading
14
Multiple programming paradigms

• H2O simple and lightweight, low-level resource
  sharing framework
• Can serve as an engine for diverse programming
  paradigms
• Metacomputing
• PVM, MPI, RMI, Other (e.g. OGSA H2O kernel
  OGSA factory)
• Client-server
• Enterprise applications, Web Services
• Task-Farm Computing
• SETI_at_home, UD Cancer Research, ...
• Peer-to-Peer
• File sharing, messaging, opportunistic
  compute/storage/service sharing

15
Metacomputing

• Clients aggregate resources
• that may belong to them
• or may come from independent providers
• Clients upload components
• DVM-enabling components communicate with each
  other and provide an illusion of Distributed
  Virtual Machine
• Specific pluglets provide concreteprogramming
  environments
• PVM and Fault Tolerant MPI components currently
  being adapted to H2O
• Distributed state managed by DVM-enabling
  components alone
• statelessness at the provider level
• resources decoupled and independent

Collaboratingresearchers(clients)
App 1
App 2
Applications
FT-MPI
PVM
Java RMI
Activeobjects
...
Programming model
Virtual layer
DVM-enabling components
Provider B
Provider A
Provider C
16
Client-Server

• Provider uploads pluglets into own kernel
• Important case Web Services
• Enterprise apps, or high-performance
  metacomputing, or join multiple DVMs/clusters
• Client communicates with pluglet using SOAP
• RMI over SOAP supported as a plugin toa
  multiprotocol RMIX suite
• Published information about the servicehas a
  form of a WSDL document
• H2O toolkit pluglet2wsdl tool

17
H2O and Web Services

• Web Service Description Language (WSDL)
• Popular, XML-based, extensible, standardized
• May be used for pure interface specification

ltdefinitions nameweatherservice xmlnsgt
ltservice nameWeatherservicegt ltmessage
nameWeather.GetTemprgt ltpart
namezipcode typexsdstringgt lt/messagegt
ltmessage nameWeather.GetTemprResponsegt
ltpart nameResult typexsdfloatgt
lt/messagegt ltportType nameWeatherSoapPortgt
ltoperation nameGetTemprgt ltinput
messagewsdlnsWeather.GetTemprgt
ltoutput messagewsdlnsWeather.GetTemprResponsegt
lt/operationgt lt/portTypegt
lt/servicegtlt/definitionsgt
18
H2O and Web Services

• Web Service Description Language (WSDL)
• Popular, XML-based, extensible, standardized
• May be used for pure interface specification
• or, to define specific remote binding

ltdefinitions nameweatherservice xmlnsgt
ltservice nameWeatherservicegt ltmessage
nameWeather.GetTemprgt ltpart
namezipcode typexsdstringgt lt/messagegt
ltmessage nameWeather.GetTemprResponsegt
ltpart nameResult typexsdfloatgt
lt/messagegt ltportType nameWeatherSoapPortgt
ltoperation nameGetTemprgt ltinput
messagewsdlnsWeather.GetTemprgt
ltoutput messagewsdlnsWeather.GetTemprResponsegt
lt/operationgt lt/portTypegt
ltbinding nameWeatherSoapBinding
typeWeatherSoapPortgt ltsoapbinding
stylerpc transport/soap/http/gt
ltoperation nameGetTemprgt
ltsoapoperation soapAction/Weather.GetTempr/gt
ltinputgt ltsoapbody
useencoded /gt lt/inputgt
ltoutputgt ltsoapbody useencoded /gt
lt/outputgt lt/operationgt
lt/bindinggt lt/servicegtlt/definitionsgt
19
H2O and Web Services

• Web Service Description Language (WSDL)
• Popular, XML-based, extensible, standardized
• May be used for pure interface specification
• or, to define specific remote binding
• or, to describe a specific service instance
• But
• WS not well matched to HPDC
• Deployment issue dynamism, lifetime mgmt not
  well supported
• Localization issue local bindings and deployment
  techniques
• Data encoding issue currently WS are text
  protocol oriented

ltdefinitions nameweatherservice xmlnsgt
ltservice nameWeatherservicegt ltmessage
nameWeather.GetTemprgt ltpart
namezipcode typexsdstringgt lt/messagegt
ltmessage nameWeather.GetTemprResponsegt
ltpart nameResult typexsdfloatgt
lt/messagegt ltportType nameWeatherSoapPortgt
ltoperation nameGetTemprgt ltinput
messagewsdlnsWeather.GetTemprgt
ltoutput messagewsdlnsWeather.GetTemprResponsegt
lt/operationgt lt/portTypegt
ltbinding nameWeatherSoapBinding
typeWeatherSoapPortgt ltsoapbinding
stylerpc transport/soap/http/gt
ltoperation nameGetTemprgt
ltsoapoperation soapAction/Weather.GetTempr/gt
ltinputgt ltsoapbody
useencoded /gt lt/inputgt
ltoutputgt ltsoapbody useencoded /gt
lt/outputgt lt/operationgt
lt/bindinggt ltport nameWeatherSoapPort
bindingWeatherSoapBindinggt ltsoapaddress
locationhttp//www.emory.edu/ /gt
lt/portgt lt/servicegtlt/definitionsgt
20
H2O Extensions to WSDL
ltdefinitions nameweatherservice xmlnsgt
ltservice nameWeatherservicegt ltmessage
nameWeather.GetTemprgt ltpart
namezipcode typexsdstringgt lt/messagegt
ltmessage nameWeather.GetTemprResponsegt
ltpart nameResult typexsdfloatgt
lt/messagegt ltportType nameWeatherSoapPortgt
ltoperation nameGetTemprgt ltinput
messagewsdlnsWeather.GetTemprgt
ltoutput messagewsdlnsWeather.GetTemprResponsegt
lt/operationgt lt/portTypegt
ltbinding nameWeatherJavaBinding
typeWeatherJavaPortgt . . .
ltoperation nameGetTemprgt . . .
ltinputgt (zipcodeint) lt/inputgt
ltoutputgt (temperaturefloat)
lt/outputgt lt/operationgt lt/bindinggt
ltport nameWeatherJavaPort
bindingWeatherJavaBindinggt
ltjavainstance locationjavaobject//MyWeatherSvc
/ /gt lt/portgt lt/servicegtlt/definitionsgt

• Deployment issue
• H2O kernel is consistent runtime hosting
  environment
• Lifetime management defined at pluglet level and
  implemented by kernel
• Localization issue
• Propose local dispatch e.g. JavaBinding
• javaobject/// protocol similar to rmi//
• Local registry within component container
• Data encoding issue
• Non-XML XDR-based encoding scheme for structured
  data
• Exploit SOAP extensibility XDR-enc binary
  attachments

21
Task-Farm Computing

• Conventional model
• Farmer (e.g. SETI) publishes worker pluglets in a
  well-known registry
• Workers (e.g. volunteers) acquire them and load
  into their kernels
• Worker pluglets register themselves with Farmers
  application
• Farmers application breaks the computation into
  tasks and assigns them to workers
• Alternative model
• Providers setup kernel allowing
• Pluglet-wrapped tasks to run
• Dot-sah files to be created
• Client discovers and utilizes resources

Farmer
Application
Worker
Worker
22
Peer-to-Peer

• File sharing
• Users load file-sharing pluglets into their
  kernels
• Pluglets of the same kind communicate with each
  other and establish network of peers
• Pluglet-specific discovery, security, accounting,
  etc.
• GUI-based file-sharing client talks to local
  pluglets via one of multiple protocols
  supported by RMIX
• Pluggable P2P logic abstracted from the client
• Opportunistic resource sharing
• Other P2P scenarios, e.g. transport routing,
  distributed storage

23
Summary

• H2O Project resource sharing infrastructure
• Lightweight
• Decoupled
• Reconfigurable
• Self-organizing
• Project Status
• Early alpha version available
• Preliminary performance experiments performed
• (Near) Future plans
• Develop library of useful pluglets
• MPI, PVM
• File sharing, communication
• Web services hosting
• Implement preliminary security and resource
  control facilities
• Initial public release November 2002