Jini Meets the Grid

1
Jini Meets the Grid

• Mark Baker

University of Portsmouth, UK October 29th
2001 http//www.dcs.port.ac.uk/mab/talks/
2
Contents

• Why Jini and What is Jini.
• Jini technologies and architecture.
• Jini services
• Discovery,
• Lookup,
• Joining.
• Leases, Transactions and Events.
• Object activation and security.
• Jini summary

3
Contents (contd.)

• The Grid
• Application types,
• Grid vision and requirements,
• Globus.
• Jini and the Grid
• Fundamental interactions,
• Usage Scenarios,
• Interactions,
• Use Cases,
• Design Issues.
• Summary.

4
Why Jini?

• Distributed computing is more difficult than
  local computing because of
• Network latency,
• Concurrency issues,
• Memory management,
• Partial failure.

5
Why Jini?

• Provides Network plug and work.
• Enables a service-based architecture.
• Spontaneous networking.
• Erase the distinction between hardware and
  software.
• A distributed computing infrastructure to makes
  writing distributed programs easier.

6
What is Jini

• Jini addresses the problems of distributed
  computing using a set of simple interfaces and
  protocols.
• Jini enables spontaneous networks of software
  services (and devices) to assemble into working
  groups of objects, or Federations.
• Jini enables self-healing when one or more
  devices are removed from the Federation
• Jini builds on the Java 2 Platform.

7
What is Jini

• Jini can be used with both hardware as well as
  software services
• Everything is represented by Java Objects.
• Everything is located and accessed through Java
  Interfaces.

8
Jini Technology Builds on Java Technology

• Feature
• Java virtual machine,
• Portable object code,
• Downloadable code,
• Unified type system
• Benefit
• Homogeneous network,
• Architecture independence,
• Dynamic environment.

9
Simple, Reliable, and Flexible

• Based on the Java programming language.
• Interfaces are designed for robustness.
• Services come and go without administration.
• Federation, not central control the community
  view of computing!

10
Jini Technologies
11
Jini Architecture
12
Jini Extends the Java Platform
13
Lookup service

• Repository of available services a database!
• Stores each service as an extensible set of Java
  objects.
• Service objects may be downloaded to clients as
  required.
• May be federated with other lookup services a
  distributed database!!
• Lookup service interface provides the usual
• Registration, Access, Update, Search, Removal.

14
Jini Lookup Service

• Repository of available services.
• Stores each service as Java objects.
• Clients download services on demand.

15
Registering a Service
16
Finding a Service
17
Discovery and Join Protocols

• Used to find and join a group of services.
• Based on UDP multicast Unicast discovery also
  supported also.
• Services advertise capabilities.
• Provide required software drivers.
• Establish reference with lookup service.

18
Discovery Protocol
19
Join Protocol
20
Steps in Discovery and Join

• A Jini enabled Object (representing a hardware
  and/or software service) multicasts a packet with
  a reference to itself.
• The service receives an RMI reference to one or
  more Lookup service.
• The service joins a Federation by placing an
  object representing its capabilities into the
  Lookup service for other clients and services to
  use.

21
Finding a Service

• To find a service, a Jini client locates a
  service by querying a Lookup Service by type
  (Java Interface).
• Code moves from service to client via Lookup
  Service.
• Code needed to use service is dynamically loaded
  on demand.

22
Finding a Service
23
Using a Service

• Communication is between service and its proxy.
• Independent of wire protocol due to the use of
  the service API.
• Protocol can change without affecting client.
• RMI semantics core to functionality.

24
Using a Service
25
Jini Lookup
26
Why Distributed Leasing?

• Problem partial failure in distributed systems
  can lead to unchecked resource consumption.
• Traditional solution system administration
• Error-prone,
• Costly,
• Only happens when it is too late.

27
Distributed Leasing in Jini

• Protocol for managing resources using a
  renewable, duration-based model.
• Contract between objects just like a contract
  that would be made if you rented something like a
  apartment.
• Resources can be shared or private
• Provides a method of managing resources in an
  environment where network failures can, and do,
  occur.

28
Leases Summary

• Time-based grants of resources or services.
• Loose contracts between grantor and holder.
• Negotiated for a set period of time.
• Can be shared or exclusive.
• Provides a method of managing resources in an
  environment where failures can, and do, occur.

29
Distributed events in Jini

• Enables Java event model to work in a distributed
  network.
• Register interest, receive notification.
• Allows for use of event managers.
• Can use numerous distributed delivery models
• Push, pull, filter...
• Uses leasing protocol.

30
Transaction model in Jini

• Designed for distributed object coordination
  light weight, object-oriented.
• Supports
• Nested transactions
• Various levels of ACID properties (Atomicity,
  Consistency, Isolation, Durability).
• Uses leasing protocol.
• Implemented in Transaction Manager service
  another Jini service (mahalo).

31
Object Activation

• The RMI Activation Daemon (rmid) allows us to
  register servers (objects).
• The daemon starts servers on an as-needed basis.
• The servers terminate themselves when they are
  idle.
• This reduces the resources required since it only
  runs active objects.

32
Security in Jini

• Security for Jini is based on the JDK 1.2
  security model.
• In Java a Security Manager is used to grant or
  deny access to resources.
• The security manager makes use of a security
  policy.
• This is typically given in a policy file, which
  is stored in some default location or specified
  at runtime.

33
Summary

• Discussed Jini and how it tries to bridge the
  perceived problems with distributed systems.
• Looked at
• Lookup
• Registration and Discovery
• Leasing
• Events and Transactions

34

• The Grid

35
Why Grids?

• A biochemist exploits 10,000 computers to screen
  100,000 compounds in an hour.
• 1,000 physicists worldwide pool resources for
  Petaop analyses of Petabytes of data.
• Civil engineers collaborate to design, execute,
  analyze shake table experiments.
• Climate scientists visualize, annotate, analyze
  Terabyte simulation datasets.
• An emergency response team couples real time
  data, weather model, population data.

36
Why Grids? (contd.)

• A multidisciplinary analysis in aerospace couples
  code and data in four companies
• A home user invokes architectural design
  functions at an application service provider
• An application service provider purchases cycles
  from compute cycle providers
• Scientists working for a multinational soap
  company design a new product
• A community group pools members PCs to analyze
  alternative designs for a local road

37
Grid computing

• Analogous to electrical power grid
• Electricity is available seamlessly (user can
  only see the wall socket)
• Characteristics of Grid computing systems
• Geographically diverse resources.
• Consisting of a potentially large numbers of
  commodity resources (computers/networks/devices).
• Aims
• To provide more computing capacity.
• To provide better solutions for complex problems.
• Practically unlimited computing and storage
  capacity.

38
The Globus Project

• Close collaboration with real Grid projects in
  science and industry.
• Development and promotion of standard Grid
  protocols to enable interoperability and shared
  infrastructure.
• Development and promotion of standard Grid
  software APIs and SDKs to enable portability and
  code sharing.
• The Globus Toolkit Open source, reference
  software base for building Grid infrastructure
  and applications.
• Global Grid Forum (GGF) Development of standard
  protocols and APIs for Grid computing.

39
Layered Grid Architecture
40
Fabric Protocols Services

• Just what you would expect the diverse mix of
  resources that may be shared
• Individual computers, Condor pools, file systems,
  archives, metadata catalogs, networks, sensors,
  etc.
• Few constraints on low-level technology.
• Defined by interfaces not physical characteristics

41
Connectivity Protocols Services

• Communication
• Internet protocols IP, DNS, routing, etc.
• Security Grid Security Infrastructure (GSI)
• Uniform authentication, authorization, and
  message protection mechanisms in
  multi-institutional setting
• Single sign-on, delegation, identity mapping
• Public key technology, SSL, X.509, GSS-API
• Supporting infrastructure Certificate
  Authorities, certificate key management,

GSI www.gridforum.org/security
42
Resource Protocols Services

• Grid Resource Allocation Mgmt (GRAM)
• Remote allocation, reservation, monitoring,
  control of compute resources.
• GridFTP protocol (FTP extensions)
• High-performance data access transport.
• Grid Resource Information Service (GRIS)
• Access to structure state information.
• Network reservation, monitoring, control.
• All built on connectivity layer GSI IP.

GridFTP www.gridforum.org GRAM, GRIS
www.globus.org
43
Collective Protocols Services

• Index servers aka metadirectory services (GIIS)
• Custom views on dynamic resource collections
  assembled by a community .
• Resource brokers (GRAM)
• Resource discovery and allocation.
• Replication services.
• Co-reservation and co-allocation services (GARA).
• Workflow management services.
• etc.

44
(No Transcript)
45
Resource Management Architecture
46
Online Access to Scientific Instruments
Advanced Photon Source
wide-area dissemination
desktop VR clients with shared controls
real-time collection
archival storage
tomographic reconstruction
DOE X-ray grand challenge ANL, USC/ISI, NIST,
U.Chicago
47
Data Grids forHigh Energy Physics
Image courtesy Harvey Newman, Caltech
48
Mathematicians Solve NUG30

• Looking for the solution to the NUG30 quadratic
  assignment problem.
• An informal collaboration of mathematicians and
  computer scientists.
• Condor-G delivered 3.46E8 CPU seconds in 7 days
  (peak 1009 processors) in U.S. and Italy (8
  sites).

14,5,28,24,1,3,16,15, 10,9,21,2,4,29,25,22, 13,26,
17,30,6,20,19, 8,18,7,27,12,11,23
MetaNEOS Argonne, Iowa, Northwestern, Wisconsin
49
Network for EarthquakeEngineering Simulation

• NEESgrid national infrastructure to couple
  earthquake engineers with experimental
  facilities, databases, computers, each other.
• On-demand access to experiments, data streams,
  computing, archives, collaboration.

NEESgrid Argonne, Michigan, NCSA, UIUC, USC
50
Home ComputersEvaluate AIDS Drugs

• Community
• 1000s of home computer users.
• Philanthropic computing vendor (Entropia).
• Research group (Scripps).
• Common goal advance AIDS research.

51
For More Information

• Globus Project
• www.globus.org
• Grid Forum
• www.gridforum.org
• Book (Morgan Kaufman)
• www.mkp.com/grids
• Grid Support Centre
• www.grid-support.ac.uk

52
Jini and the Grid

• Put all services from Grid community into Jini
  community

CoGDesk
MPJ
Lapack/Blas
Printer Service
Globus Toolkits
Visualisation
53
Fundamental Interactions

• Two fundamental types of interaction
• Use Jini services from the Grid,
• Use Grid services from Jini.
• Obviously there could be mixed interactions, but
  if the two fundamental issues are solved, then
  this area is not a problem.

54
Recap on the use of Jini Services

• Service
• Registration of service, and the existence of a
  service interface.
• Client
• Discover LUS,
• Search through LUS for service.
• Access service via its proxy service interface.
• Reliability ensure leasing paradigm OK.
• Cancellation and abort (events/trans)

55
Usage Scenarios

• Want to permit serial and parallel jobs.
• Native (Java) and Non-Native (Fortran/C) access
  to Jini service (s).
• Want to use
• Jini service (s) from the Grid e.g.
  visualisation system or MPJ (Java SPMD
  environment).
• Grid service (s) from Jini batch system like
  PBS or a Cave.
• Likely that Jini-side will be predominantly Java
  and Grid-side non-Java (C/Fortran).

56
Usage Scenarios

• Two interaction scenarios
• Direct heavyweight client
• Intelligent Client!
• Up-front knowledge of usage needed.
• Direct Jini/Grid usage.
• No intermediate broker.
• Indirect lightweight client
• Simple Client.
• Up-front knowledge NOT needed.
• Indirect Jini/Grid usage.
• Intermediate resource broker.

57
Use Case (Direct ) Grid Job

• Native access from the Grid (C/Fortran Job)
• Discover find LUS, need to ensure evidence of
  LUS is stored in GIS and LUS can be found.
• Search need to download the LUS interface and
  search for services needed.
• Access after service is found, need to access
  it via its interface using RMI then use sockets
  directly.
• Problems (some)
• RSL Parsing maybe Jini/Grid Markup Language.
• Find LUS (hardwired) and its attributes.
• Wrapper and transfer for non-Java jobs.
• Security Certs, Digital Signatures, SSL.

58
Direct Interaction Grid-Jini
Jini Enabled
Jini
Grid
GIS
LUS
GIS/LUS Interface
Service Proxy
Grid Client
Cert
Cert
Service
59
Direct Interaction Grid-Jini
Jini Enabled
Jini
Grid
GIS
LUS
GIS/LUS Interface
Service Proxy
Grid Client
Cert
Cert
60
Indirect Interaction Grid-Jini
Jini
Grid
LUS
GIS
Jini Enabled
LUS Interface
Resource Broker
Service Proxy
Grid Service
Service
Cert
Cert
61
Use Case (II) Jini Job

• Non-Native access from the Jini (Java Job)
• Discover find LUS, and locate a Grid
  Information Service (LDAP).
• Search - need to download the LUS interface to
  GIS and search for services needed.
• Access after service is found, need to access
  it via its interface using a well defined
  protocol.
• Problems
• Find GIS and its attributes.
• Java access to non-Java service go via service
  proxy, so need to deal with non-Java access.

62
Direct Interaction Jini - Grid
Jini
Grid
GIS
LUS
LUS/CoG Interface
Jini/CoG Enabled
Grid Service
Cert
Client
Cert
63
Indirect Interaction Jini - Grid
Jini
Grid
LUS
GIS
LUS Interface
Jini/Cog Enabled
Resource Broker
Grid Service
Cert
Cert
Client
64
Design Issues Scalability

• Jini designed to be used in a LAN setting
  dependant on multi-cast.
• No infrastructure for supporting Jini communities
  distributed over a wide area.
• An interesting research topic and critical for
  widespread take up of Jini and the Grid.
• Could use
• Over lapping LUSs.
• P2P environments JXTA.
• Well known Jini gateways.
• New protocols.

65
Design Issues Search Flexibility

• Can search LUSs
• Exact matches,
• Service attributes,
• Jini ID,
• Vague print, or ?rint.
• Probably want a fuller search capabilities to be
  more flexible and adaptable.

66
Design Issues Markup Language

• Need to match Jini terminology with that of the
  Grid.
• Probably scope to develop matalanguage JiniGrid
  Markup Language (JGML).
• Parsing of RSL to JGML.

67
Summary

• Described the basic Infrastructure of Jini and
  Grid architectures.
• Briefly outlined Grid and Jini usage of each
  others services.
• Looked at various different integration
  scenarios
• Direct - using Jini dynamic ability.
• Indirect introducing an intelligent broker.
• Discussed some of the integration design issues
  discovery, parsing, wrapping, security, etc.

68
Conclusions

• Integrating Jini and the Grid together is useful
  for a number of reasons
• Proof of concept integrating two distributed
  environments with similar goals but radically
  different implementation details.
• Increasing numbers of Grid and Jini-based
  environments and so the need for interaction is
  more and more likely.
• Issue raised are ongoing research efforts.