State of Service Oriented Science Tools

1
State of Service Oriented Science Tools

• Open Source Grid Cluster Conference
• Oakland

2
Agenda

• Presentation
• Demo
• Questions

3
Service-Oriented Science

• People create services (data or functions)
• which I discover ( decide whether to use)
• compose to create a new function ...
• then publish as a new service.
• ? I find someone else to host services, so I
  dont have to become an expert in operating
  services computers!
• ? I hope that this someone else can manage
  security, reliability, scalability,

!
!
Service-Oriented Science, Science, 2005
4
Creating ServicesIntroduce gRAVI
Shannon Hastings Scott Oster David Ervin Stephen
Langella
Kyle Chard Ravi Madduri
5
Introduce Overview

• A framework which enables fast and easy creation
  of Globus based grid services
• Provide easy to use graphical service authoring
  tool.
• Hide all grid-ness from the developer
• Utilize best practice layered grid service
  architecture
• Integration with other core grid services and
  architecture components
• GAARDS Security Infrastructure (Dorian,
  GridGrouper, CSM)
• Globus Index Service
• Global Model Exchange (GME)
• Cancer Data Standards Repository
• Extension Framework for integrating with other
  architecture components

6
Inside the Introduce created service

• Services have many moving and configurable parts
  which support features such as
• Advertisement
• Discovery
• Invocation
• Security (Authentication/Authorization)
• Stateful Resources
• The Introduce Toolkit can keep all these features
  in sync as the developer creates and modifies the
  grid service

7
Introduce Features

• Supports modification of operations
• Adding operations
• Removing Operations
• Updating Operations
• Importing Operations
• Graphical Configuration
• Advertisement
• Security
• Service Metadata Specification
• Service Metadata Editing
• Service Configuration Properties
• Auto Generates Code for Service
• Auto generates a client API for service.
• Graphical Deployment of Service
• Globus
• Tomcat
• JBoss

8
Created Skeleton Layout
introduce managed
globus/axis managed
developer implements
9
gRAVI

• Grid Remote Application Virtualization Interface
• Builds on Introduce
• Define service
• Create skeleton
• Discover types
• Add operations
• Configure security
• Wrap arbitrary executables

Introduce
Repository Service

Index service
Container

10
Discovering Services

• People create services (data or functions)
• which I discover ( decide whether to use)
• compose to create a new function ...
• then publish as a new service.
• ? I find someone else to host services, so I
  dont have to become an expert in operating
  services computers!
• ? I hope that this someone else can manage
  security, reliability, scalability,

!
!
Service-Oriented Science, Science, 2005
11
Discovering ServicesWS-MDS Taverna
Laura Pearlman Mike Darcy
myGrid Team
12
Discovering Services

• ? The ultimate arbiter?

• ? Types, ontologies

• ? Can I use it?

• ? Billions of services

• Assume success
• Semantics
• Permissions
• Reputation

13
Discovery (1)Registries
14
Discovery (2)Standardized Vocabularies
Core Services
IndexService
Queries Service
Enterprise Vocabulary Services
Cancer DataStandardsRepository
Metadata Aggregated In
Registers To
Uses Terminology
Described In
References
Subscribes to
ObjectsDefined in
and Aggregates
Grid
Discovery
Service
Client API
Publishes
Service
Metadata
15
Composing Services

• People create services (data or functions)
• which I discover ( decide whether to use)
• compose to create a new function ...
• then publish as a new service.
• ? I find someone else to host services, so I
  dont have to become an expert in operating
  services computers!
• ? I hope that this someone else can manage
  security, reliability, scalability,

!
!
Service-Oriented Science, Science, 2005
16
Composing ServicesTaverna GT4
Taverna team Wei Tan Ravi Madduri
17
Composing Services
18
Hosting Services

• People create services (data or functions)
• which I discover ( decide whether to use)
• compose to create a new function ...
• then publish as a new service.
• ? I find someone else to host services, so I
  dont have to become an expert in operating
  services computers!
• ? I hope that this someone else can manage
  security, reliability, scalability,

!
!
Service-Oriented Science, Science, 2005
19
Provisioning ServicesWS-GRAM VWS
Martin Feller Stuart Martin
Kate Keahey Tim Freeman Joshua Boverhof
20
Provisioning using WS-GRAM

• gRAVI uses JSDL for Application Description
• Generates a method on the generated service
  called ltappNamegtGRAM
• Generates implementation that creates a GramJob
  from Application Description
• Uses the bootstrapped community credential to run
  the application as a grid job
• Used widely in realizing the usecases from caBIG
  community

21
gRAVI WS-GRAM

Container With Application Service
Invoke get results
22
Using VWS/Cloud Computing

• gRAVI service
• Wrap the application as Service
• Create the GAR and put in repository
• Provision resources
• Use clouds (VMM)
• start service
• Transfer gar, deploy
• Index Service
• Grid service registers itself

provision
Create VM
Register service
Transfer, deploy
Repository Service

Index service
portal
GAR
discovery
23
gravi portlets
Portal
register
VM
WAR
Invoke executable
Repository Service
service

GAR
24
Cloud Computing
25
User Communities and Early Adopters
26
Advanced Photon Source

• APS Requirements align well with goals of SOS
• They have data, analytics and workflows
• We wrapped one of the 2 applications as a service
• Goal is to create and run a workflow with
  services generated using gRAVI and provision the
  workflow using Nimbus

27
Cancer Bio-informatics Grid

• Primary usecase for Service Oriented Science
• Demo at the Architecture F2F meeting
• geWorkbench from Columbia University
• Hierarchical Clustering Service
• gRAVI ? WS-GRAM on TeraGrid
• Integrating into caGrid Tools
• Taverna Integration for composition
• Use gRAVI to wrap more popular services into Grid
  Services

28
caGrid Service

29
TeraGrid Aware caGrid Service
30
Creating the Gateway Service

• Manually stage the binary (jar file) on TeraGrid
• Takes in .ser files as input
• Produces results also in a .ser file
• Used the gRAVI plugin for Introduce to create the
  gateway service
• http//www-unix.mcs.anl.gov/neillm/ravi/
• Gateway gridFTPs input data and parameters from
  geWorkbench to TeraGrid
• geWorkbench passes input to the gateway in
  geWorkbenchs native format (caDSR compliant)
• Gateway serializes the input before gridFTPing to
  TeraGrid
• Gateway invokes the staged binary
• Gateway gridFTPs results back to geWorkbench
• Gateway deserializes the result file
• Gateway returns results to geWorkbench in its
  native format
• Gateway service is a secured caGrid service which
  in turn invokes TeraGrid with a caBIG community
  account

31
Steps to establishing geWorkbench/caGrid/TeraGrid
Interface
32
caGrid Security (GTS, Grid Grouper, Dorian, CDS)
http//www.cagrid.org/mwiki/index.php?titleGAARDS
Main
33
Future Work (Near Term)

• Plan for integrating Virtual Workspace services
  as a provisioning mechanism
• Details of the Plan in the following slides
• caBIG demo in June 2008 using both WS-GRAM and
  VWS as backends
• Google Summer of Code Projects AJAX front-end
  to gRAVI and Service Composition
• Two summer students

34
Future Work

• gRAVI portlets
• Plugin also creates a grid portlet
• interface to the generated client
• Or simple portal exposing the WSDL operations
• Publish Services Portlets
• Add window for publishing via Introduce
• Publish gRAVI services to a Repository
• Entry contain service (GAR) and portlet (WAR)
• Metadata describing resource requirements, etc
• Use a Service from Portal
• User selects the service(s) from Repository
• Identify or Create resources (VM) based on
  metadata
• Deploy service(s)
• Discover service(s)
• User access the service through portlet or
  client.

35
Success stories from Early Adopters

• From Brian Tieman at APS I was able to muddle
  through and get it working.  Total time spent 2
  hours.  I'm happy with that, and I'm sure future
  applications will go even faster!  I've only run
  the sample client and I'm not trying to pass
  arguments yet so there's still a bit more to do
  before I can try integrating this into my
  application, but so far so good!
• From Paul Burkhardt at NCI Inc We used the
  gRAVI extension to the Introduce Grid Service
  Authoring Toolkit to demonstrate the ease of
  deploying arbitrary executables as Grid Services
  and GRAM jobs. Migrating legacy applications to a
  Web Services model presents an imposing hurdle to
  adoption. Many important legacy applications
  would require a significant amount of man-hours
  to rewrite as Grid Services. An alternative is to
  create JNI interfaces to wrap the executables but
  this approach still requires effort and expertise
  to create the necessary WSDL and corresponding
  Java implementation. This development cost can be
  prohibitive. The gRAVI tool has provided us the
  solution to low-cost and rapid deployment of
  legacy applications for Grid infrastructures.