Motivation

1
(No Transcript)
2
Agenda

• Grid computation.
• Motivation, a very simple example.
• Architecture and implementation problems.

• Grid computation.
• Motivation, a very simple example.
• Architecture and implementation problems.

• Web Services
• What are they?, why do we need them?
• Some technologies involved XML, SOAP, WSDL, UDDI.

• Putting it all together

3
Motivation (some history)

• 95 of CPU power in the world is idle.
• Multiple scientific tasks like require
  supercomputing power (weather forecast, weapons
  simulations, etc).
• The cost of this supercomputers is millions of
  dollars.
• Why not using existing resources to tackle this
  tasks.

4
The Idea

• Use the CPU power available with the help of
  volunteers ready to contribute to your project.
• Connect the computers throw existing networks
  (internet).
• Get a de facto supercomputer.
• Save millions.

5
SETI_at_home

• SETI (Search for Extra-Terrestrial Intelligence).
• SETI_at_Home connects more than 4.000.000 computers
  throughout the world having a work-power bigger
  than any existing super-computer.
• SETI Statistics.
• Users 4.763.661 (the original target was
  300.000).
• Total CPU time 1706429.454 years.
• FPO 3.979678e21.
• 24 hours FPO 61.86 TFLOPS/sec.
• The earth simulator (350 million US) has 40
  TFLOPS/sec.

6
CRASS (1)

• The problem.
• More than 7000 observed orbiting objects larger
  than 10 cm.
• Between 70.000 to 120.000 objects larger than 1
  cm.
• High speeds can make collisions fatal, making the
  debris problem worse.
• 1 on 10 shuttle missions have performed
  collisions avoidance maneuvers.

7
CRASS (2)

• USS space com DB tracks the position of the
  debris.
• CRASS uses the DB resources and a model of the
  forces acting to predict possible collisions.
• CRASS master assigns to each node the operational
  spacecraft and some debris.

8
CRASS (3)

• The nodes calculate the debris orbit and issue
  collisions warnings in a specific time scope.
• Master receives the results and assign another
  period of time till the simulation is completed.

9
CRASS (4) Architecture
DB
MASTER
- Test speed of the nodes - Distribute the
debris accordingly
Scheduler
node
node
node
node
10
Conclusions for CRASS

• CRASS works because the debris does not interact
  between them.
• No need of communication between nodes.
• Overhead of data is small (the number of
  operational spacecraft is small in relation to
  the debris).
• The problem is easily distributed.
• But there are other not so simple cases.
• Grids are more than CRASS.

11
A Grid is more than that

• A grid is a seamless computing information
  environment composed of
• Storage systems.
• Networks.
• Data servers.
• On-line instruments (telescopes, particle
  accelerators, etc).
• Computing processors.

12
Implementing a Grid

• Putting together so different elements is a
  complex task.
• Need of a common language between them.
• Need of communications protocols.
• Security issues.
• Grid applications are usually distributed and
  thus hard to program.

13
Implementing a Grid (2)

• Component (object) oriented programming has
  helped.
• Reusability of code.
• OO design subdivides problems in small ones.
  Suitable approach to deploy big applications.
• Making objects work together is still a hard task
  involving many issues.
• CORBA was supposed to be the solution

14
More difficulties

• Slow communications between computers.
• Wide geographical area.
• Different users platforms (WIN, UNIX, MAC).
• Some problems require strong interaction between
  the different CPUs.
• For these problems supercomputers cannot be
  replaced (for example chain reactions, weather
  simulations...).
• So we still need supercomputers.

15
Dividing the problems

• Its very important to divide developers and
  users.
• Scientific programmers (no knowledge of grid
  technologies).
• Grid developers (no knowledge of the scientific
  problem).
• End users (no knowledge at all).
• Each group needs appropriate tools to solve their
  problems.
• We need tools to connect the elements.

16
Dividing the problems (2)

• Throughout the presentation we will focus in the
  second group problems (grid architecture).
• Some new arising tools called hosting
  environments help also to deploy highly
  distributed applications.
• The programs (elements, objects) do not run over
  the OS but over a hosting environment.

17
Grid Portals

• Grid portals are a solution for end users.
• They provide an easy way to use then resources of
  a grid with no special knowledge of computing.
• The user will be able to use the portal for
• Locate grid resources.
• Set in the portal the task he wants to perform.
• Define execution parameters (number of
  processors, memory needs, etc).

18
Nimrod Portal Applet
19
Grid Architecture elements

• Security Protocols
• Authentication and privacy.
• Information Services
• users need to see which resources are available
  for use.
• Schedulers
• multiple resources can be scheduled concurrently.

20
An Example of a Grid Portal Architecture
Web Browser
(1) The user browser connects to the Portal
Server.
(2) The Server checks if the user is a
registered grid user
(3) The Grid Application Manager is notified of
the user presence
(4) The Grid Application Manager connects to the
user through a messages channel
21
Towards Grid Services

• The grid model is developing.
• Not only scientific cooperation.
• Business want to link their different departments
  maybe in a very wide geographical area using grid
  technologies.
• Outsourcing companies are using specialized
  companies and reducing their IT departments.
• Economies of scale.
• Price/performance ratio.

22
Towards Grid Services (2)

• We would like to make grids more open. Other
  people to use the grid capabilities.
• Redefine the grid elements as services.
• service network-enabled entity that provides
  some capability.
• The key issue is interoperability between
  services.

23
Towards Grid Services (3)

• Redefine the architecture elements of a grid as
  services
• Security services authentication and privacy.
• Information services what resources do we have
  in the grid (data, processors, instruments).
• Job-submission resources.
• Co-scheduling service multiple resources.
• Caching in large grids caching data can improve
  performance dramatically.

24
Towards Grid Services (4)

• The interoperability problem basically consists
  in
• Defining what the service does.
• Giving instructions for its use (invocation
  APIs)
• We would also like a mechanism to find
  appropriate services.
• We need semantic information about the services.

25
Its Not Easy

• We need specific protocols (hopefully universally
  accepted) to define our services.
• XML helps with its semantic approach but its
  not enough.
• If we use the Internet as a transport channel how
  do we avoid firewalls?
• RPCs are not firewall-friendly.

26
Agenda

• Grid computation.
• Basic ideas.
• Implementation problems.

• Web Services
• What are they?, why do we need them?
• Some technologies involved XML, SOAP, WSDL, UDDI.

• Putting it all together

27
What the Web Can Offer

• Web sites offer a myriad of services. Buy books,
  check stocks, bank transactions
• We would like to be able to do that
  automatically.
• Amazon could buy book from providers when its DB
  shows stock is low.
• Business would like to talk to each other.
• We need a ways to define and provide services.

28
Some History

• Earlier attempts to B2B failed because HTML was
  not able to give enough semantic information
  about the services provided.
• In 2001 several companies got together with W3C
  and came up a precise set of XML based standards
  in order to help B2B.

29
Some Standards

• WSDL Web Services Description Language.
• An XML Schema that defined the service
  capabilities and how to invoke it.
• UDDI Universal Description, Discovery and
  Integration.
• Registry specification on how to publish WSDL
  documents.
• WSIL Web Services Inspection Language.
• How to find WSDL documents on a site.
• The idea is that these tools will help a user to
  find a WS and use it (automatically).

30
A Possible Scenario
Registry sends to user the matching services
USER wants to buy books by ISBN, searches the
registry using WSIL
USER
WS Registry
Registry (written in UDDI) contains WSDL
documents describing the services
31
Using a WS

• There are two types of web services.
• RPC (remote procedure call) the user calls a
  function provided by the service and gets a
  value.
• DOC the user invokes the service and gets a
  documents as a response.
• In order to use the service we need to know how
  to call it, i.e under which protocol is it
  working (TCP, SMTP, HTTP or other).

32
Understanding a WS

• A WS is a collection of endpoints.
• An endpoint is a combination of a binding and an
  address (URI).
• A binding is a concrete protocol and data format
  for a port type
• The set of message exchanges is called an
  operation in WSDL terms
• error messages, encoding, parameters type, etc.

33
Understanding a WS (2)

• Port types are interfaces (in wsdl 1.2 they are
  changing the term).
• Related concrete end points can be grouped in
  interfaces and thus provide abstract endpoints,
  also called services

34
Messages
TCP
HTTP
SMTP
endpoints
interface operation operation
interface operation operation
SERVICE
Application (resource)
35
SOAP

• Communication with the WS is done through
  messages, however we do not want to write the XML
  messages ourselves.
• SOAP simple object access protocol solves that.
• SOAP is simply a standard to send messages. There
  are others, but Microsoft, W3C and Sun use SOAP
  so its a de facto standard.

36
SOAP (2)

• We send a message in a SOAP envelope. Using
  whatever protocol we like (FTP, HTTP, JMS,...)
• The SOAP processor is the one in charge of
  understanding the SOAP message and call the WS.
• There are Apache packages for java and Microsoft
  packages designed to use SOAP well see some
  examples.

37
SOAP (3)
38
Using SOAP to Invoke a WS

• Barnes and Noble provides a service to get a
  books price with its ISBN
• method name getPrice(String isbn).
• The service is located in the Web Server of
  Xmethods.
• Send though HTTP a SOAP message requesting the
  price of isbn 0439139597 (Harry Potter 5).

39
SOAP request
The SOAP envelope header information about the
SOAP specification

• ltSOAP-ENVEnvelope
• xmlnsSOAP-ENV "http//schemas.xmlsoap.org/soap/
  envelope/" xmlnsxsi"http//www.w3.org/1999/XML
  Schema-instance" xmlnsxsd"http//www.w3.org/1999
  /XMLSchema"gt
• ltSOAP-ENVBodygt
• ltns1getPrice xmlnsns1"urnxmethods- BNPrice
  Check"
• SOAP-ENVencodingStyle "http//schemas.xml
  soap.org/soap/encoding/"gt
• ltisbn xsitype"xsdstring"gt0439139597lt/isbngt
• lt/ns1getPricegt
• lt/SOAP-ENVBodygt
• lt/SOAP-ENVEnvelopegt

ltSOAP-ENVEnvelope xmlnsSOAP-ENV
"http//schemas.xmlsoap.org/soap/envelope/"
xmlnsxsi"http//www.w3.org/1999/XMLSchema-insta
nce" xmlnsxsd"http//www.w3.org/1999/XMLSchema"gt
ltSOAP-ENVBodygt ltns1getPrice
xmlnsns1"urnxmethods- BNPriceCheck" SOAP-EN
VencodingStyle "http//schemas.xmlsoap.org/
soap/encoding/"gt ltisbn xsitype"xsdstring"gt043
9139597lt/isbngt lt/ns1getPricegt lt/SOAP-ENVBodygt
lt/SOAP-ENVEnvelopegt
ltSOAP-ENVEnvelope xmlnsSOAP-ENV
"http//schemas.xmlsoap.org/soap/envelope/"
xmlnsxsi"http//www.w3.org/1999/XMLSchema-insta
nce" xmlnsxsd"http//www.w3.org/1999/XMLSchema"gt
ltSOAP-ENVBodygt ltns1getPrice
xmlnsns1"urnxmethods- BNPriceCheck" SOAP-EN
VencodingStyle "http//schemas.xmlsoap.org/
soap/encoding/"gt ltisbn xsitype"xsdstring"gt043
9139597lt/isbngt lt/ns1getPricegt lt/SOAP-ENVBodygt
lt/SOAP-ENVEnvelopegt
ltSOAP-ENVEnvelope xmlnsSOAP-ENV
"http//schemas.xmlsoap.org/soap/envelope/"
xmlnsxsi"http//www.w3.org/1999/XMLSchema-insta
nce" xmlnsxsd"http//www.w3.org/1999/XMLSchema"gt
ltSOAP-ENVBodygt ltns1getPrice
xmlnsns1"urnxmethods- BNPriceCheck" SOAP-EN
VencodingStyle "http//schemas.xmlsoap.org/
soap/encoding/"gt ltisbn xsitype"xsdstring"gt043
9139597lt/isbngt lt/ns1getPricegt lt/SOAP-ENVBodygt
lt/SOAP-ENVEnvelopegt
The information about the method invoked and the
encoding type
The parameter needed (isbn)
40
SOAP answer

• ltSOAP-ENVEnvelope
• xmlnsSOAP-ENV "http//schemas.xmlsoap.org/soap/
  envelope/" xmlnsxsi"http//www.w3.org/1999/XML
  Schema-instance" xmlnsxsd"http//www.w3.org/1999
  /XMLSchema"gt
• ltSOAP-ENVBodygt
• ltns1getPriceResponse xmlnsns1"urnxmethods-
  BNPriceCheck"
• SOAP-ENVencodingStyle "http//schemas.xml
  soap.org/soap/encoding/"gt
• ltreturn xsitype"xsdfloat"gt15.57lt/isbngt
• lt/ns1getPriceResponsegt
• lt/SOAP-ENVBodygt
• lt/SOAP-ENVEnvelopegt

ltSOAP-ENVEnvelope xmlnsSOAP-ENV
"http//schemas.xmlsoap.org/soap/envelope/"
xmlnsxsi"http//www.w3.org/1999/XMLSchema-insta
nce" xmlnsxsd"http//www.w3.org/1999/XMLSchema"gt
ltSOAP-ENVBodygt ltns1getPriceResponse
xmlnsns1"urnxmethods- BNPriceCheck" SOAP-EN
VencodingStyle "http//schemas.xmlsoap.org/
soap/encoding/"gt ltreturn xsitype"xsdfloat"gt15
.57lt/isbngt lt/ns1getPriceResponsegt lt/SOAP-ENVB
odygt lt/SOAP-ENVEnvelopegt
The return value of the method
41
Using JAVA with SOAP

• SOAP packages for JAVA give the possibility to
• Create SOAP envelopes.
• Set the envelope data (URL, URI, method name,
  parameters,
• SEND the envelope.
• GET the answer (also a SOAP envelope).
• PARSE the envelope and get the relevant data.

42
Java SOAP
import java.net.URL import java.util.Vector  imp
ort org.apache.soap. import org.apache.soap.rpc.
public class Client public static void
main(String args) throws Exception        URL
url new URL("http//xmethods.vet/sd/2001/"
BNQuoteService") // Build the
call.    Call call new Call()    call.setTarget
ObjectURI("urnxmethods-BNPriceCheck")    call.se
tMethodName(getPrice")
import java.net.URL import java.util.Vector  imp
ort org.apache.soap. import org.apache.soap.rpc.
public class Client public static void
main(String args) throws Exception        URL
url new URL("http//xmethods.vet/sd/2001/"
BNQuoteService") // Build the
call.    Call call new Call()    call.setTarget
ObjectURI("urnxmethods-BNPriceCheck")    call.se
tMethodName(getPrice")
The URN of the method and the invoked method name
43
Java SOAP (2)
call.setEncodingStyleURI(Constants.NS_URI_SOAP_ENC
) Vector params new Vector()        
params.addElement(new Parameter("name",
String.class, args0,
null)) call.setParams(params) // Invoke the
call. Response resp null try resp
call.invoke(url, "")   catch(
SOAPException e ) System.err.println("Caught
SOAPException (" e.getFaultCode() ") "
e.getMessage())         System.exit(-1)     
call.setEncodingStyleURI(Constants.NS_URI_SOAP_ENC
) Vector params new Vector()        
params.addElement(new Parameter("name",
String.class, args0,
null)) call.setParams(params) // Invoke the
call. Response resp null try resp
call.invoke(url, "")   catch(
SOAPException e ) System.err.println("Caught
SOAPException (" e.getFaultCode() ") "
e.getMessage())         System.exit(-1)     
Setting args0 as the isbn to look
44
Java SOAP (3)
// Check the response. if( !resp.generatedFault()
) Parameter ret resp.getReturnValue() Objec
t value ret.getValue()            System.out.p
rintln(value) else Fault fault
resp.getFault()             System.err.println("
Generated fault ") System.out.println
("  Fault Code   " fault.getFaultCode())
   System.out.println ("  Fault String "
fault.getFaultString())    
// Check the response. if( !resp.generatedFault()
) Parameter ret resp.getReturnValue() Objec
t value ret.getValue()            System.out.p
rintln(value) else Fault fault
resp.getFault()             System.err.println("
Generated fault ") System.out.println
("  Fault Code   " fault.getFaultCode())
   System.out.println ("  Fault String "
fault.getFaultString())    
Getting the returned value
45
Agenda

• Grid computation.
• Basic ideas.
• Implementation problems.

• Web Services
• What are they?, why do we need them?
• Some technologies involved XML, SOAP, WSDL, UDDI.

• Putting it all together

46
The Idea

• Use the WS standards to solve the implementation
  problems of a grid.
• WS are platform-independent and
  programming-language independent.
• Communication protocols are widely accepted
  (firewalls-friendly).
• WSDL offers a way to publish the services.
• UDDI offers a way to find the wanted service.

47
Grid Services As WS (2)

• However there are grid needs that are not
  provided by the WS specifications and tools. We
  need new interfaces.
• Grids need dynamic creation of application
  instances. (The application instance is created
  when the user needs it).
• Grids need lifetime management. operations.
• WS are usually stateless.

48
Grid Factories

• The factory creates an instance of a new WS in
  some resource.
• Dynamic creation of WS is supported is hosting
  environments as J2EE Server, .NET and AXIS.
• An application manager instance is created and
  publishes a WSDL document.
• The client can contact the new instance through
  the WSDL directly (we do not need the factory any
  more).
• A messages channel is opened.
• Keep alive messages, destroy...

49
Grid Factories (2)
User
50
Lifetime Management

• Normally a transient service will be created and
  will run till its task termination. However
  errors can (and do) happen.
• Need a ways to kill idle WS that consume
  resources. A possible solution will be
• The service is created with a specific lifetime.
• The user can request to extend the lifetime.
• The user needs to send keep-alive messages, or
  explicitly kill it.

51
- The user invokes create service from the 2
factories. Storage room is allocated.
- Keep Alive messages and status messages are
sent
- As no more keep-alive messages are received the
services kill themselves freeing all their
resources.
52
The role of hosting environments

• A grid service architecture should be independent
  of the implementations issues.
• Nowadays most grid application rely in native OS
  (for example the creation of a new service means
  the creation of a process).
• This fact means for example that the factory
  service has to be platform-aware.

53
The role of hosting environments (2)

• WS can be created with more sophisticated tools
  called hosting environments or containers as J2EE
  (Sun), WebSphere (IBM) or .NET (Microsoft)
• These containers work in a component-based basis.
• Hosting environments build complex applications
  by using components that adhere to specific
  interfaces (i.e. EJB).

54
The role of hosting environments (3)

• Building applications which can have components
  distributed in different machines and places.
• Communications are handled by the hosting
  environment.
• Easy security.
• Is not the component that handles communication
  or creation of instances but the hosting
  environment.
• Messaging between components is also handled by
  the host no matter where the are.

55
The role of hosting environments (4)
Clients DB
An accountancy- bean wants to get some data from
the sales department DB.
The bean asks the host for a handle to the
sales-bean.
The container looks in the directory where the
bean is and returns a handle
The beans communicate as if they where running in
the same machine. Communications and security
issues are managed from the host
Sales bean
Accountancy bean
56
The role of hosting environments (5)

• This approach affects also the third level of
  users (scientific programmers).
• The programmer needs to follow specific
  interfaces for his application.
• In other words if he is a java programmer he
  needs to deliver beans.

57
Conclusion

• WS provide a powerful tool to implement grid
  services.
• New interfaces (hopefully widely accepted) should
  be created to support specific grid needs
• Factories.
• Lifetime management.
• Security...

58
References

• Articles.
• The anatomy of the grid.
• www.globus.org/research/papers/anatomy.pdf.
• The physiology of the grid.
• www.globus.org/research/papers/physiology.pdf.
• Programming the grid distributed software
  components, P2P and grid web services for
  scientific applications.
• http//citeseer.nj.nec.com/gannon02programming.htm
  l.
• Grid services for distributed system integration.
• www.globus.org/research/papers/ieee-cs-2.pdf.

59
References (2)

• Some links
• SOAP www.w3.org/TR/SOAP/
• UDDI www.uddi.org
• WDSL www.w3.org/TR/wsdl
• To see some WS in action www.xmethods.net
• Microsofts WS page http//msdn.microsoft.com/web
  services/default.aspx