Interoperability Challenges in Grid for Industrial Applications

1
Interoperability Challenges in Grid for
Industrial Applications

• David De Roure, IAM
• Mike Surridge, IT Innovation
• Electronics and Computer Science
• University of Southampton, UK
• Semantic Grid Workshop at GGF9

2
Grid Resources for Industrial Applications

• Objectives
• make the Grid usable for business and industry!
• Participants
• University of Southampton
• IT Innovation
• Intelligence, Agents and Multimedia
• NTUA
• Dolphin
• CESI
• Kino

www.gria.org
3
GRIA Objectives and Drivers

• To create a Grid for business
• addressing business processes and issues
• not another academic grid infrastructure
• To address critical issues for business users
• security
• service levels
• To create an interoperable Grid system
• open at the standards level
• making use of third-party components
• being used by other Grids

4
GRIA A Grid for Business

• Industrial grid
• computational workflow
• business processes
• For each computation
• find resources
• negotiate access
• agree service levels
• perform service
• determine outcome
• bill the customer
• A market, not a commons

5
GRIA Applications

• Virtual Digital Studio
• KINO case study
• Post-production tasks
• scene rendering
• image enhancement
• Outsourcing/Collaborative scenarios

• Structural Identification
• ENEL case study
• Structural analysis, e.g.
• finite elements
• model updating
• Outsourcing/Insourcing scenarios

6
Business Workflows
7
GRIA Features

• Message security
• encryption/decryption at the boundary
• authentication and integrity checking
• Dynamic authorisation service
• enforcing business workflows
• excluding unwanted clients
• Capacity management
• connects to but doesnt manage resource pool
• expected behaviour represented via pseudo
  reservation model
• resource availability and run time prediction wrt
  model
• Services for negotiation as well as applications

8
Semantic Grid Challenges

• GRIA is very conservative yet very adventurous
• uses off-the-shelf e-Commerce technology, and
• semantic web/semantic grid to do business!
• Main challenges
• representing quality of service
• for collections of related jobs
• embedded in complex business processes
• is this getting scary yet?

9
Example querying 3rd parties

• I want to run a very large finite-element
  analysis, 100000 DOFs, fully dynamic treatment,
  and I need the result in one hour.
• My GRIA client can figure out 10 workload
  parameters and establish which GRIA service
  providers can deliver the necessary computation.
  But none are able to finish the work in the time
  I need!
• I know EPCC carry FEM codes running under GT3,
  and Stuttgart supplies services over UNICORE to
  several automotive customers.
• My needs are small by their standards but do
  they have enough spare capacity to meet my
  deadline, and can I use them?

10
Querying third party providers

• GRIA QoS is based on
• client-side workload estimation
• server-side estimation of capacity to deliver the
  work
• Need
• application-independent workload representation
• related to resource capacity
• Use 10 load parameters
• CPU, memory, disk, data transfer, network
  transfer
• predicted from application parameters
• Exploring ontological mapping to GT3 terminology
• ref WWW 2003 demonstration
• various related GGF and Semantic Web activities

11
Example 2 multiple related workloads

• I work for KINO, and I am making a commercial. I
  need a 60-second sequence showing athletes
  preparing for the final day of competition, and
  my client wants several complete scenarios
  featuring different athletic events from which to
  edit the final cut.
• I need to render 4500 high-definition frames
  based on an architects model. I need the
  results first thing tomorrow morning, and my
  budget is only 15 for this job.
• Three service providers can meet my needs. One
  will charge 0.10 per frame, but can handle 25000
  frames for overnight delivery. There are cheaper
  suppliers, and my in-house systems can do it for
  free (but only if nothing else is running).

12
Multiple Related Workloads

• Addressing end-user problems
• rendering sequences of video frames
• parametric identification of structural defects
• Sequences of jobs
• some common inputs
• some input-output dependencies
• some scope for concurrent execution
• Exploring workflow/process representation
• easy to create and enact (e.g. WSFL)
  representations
• but we cant reason about them and negotiate QoS
• Ref Jim Hendler Neither OGSA nor DAML-S offers
  a powerful enough process model

13
Example 3 embedding in processes

• Our friend from KINO is desperate. The usual
  supplier cant take on the excess work after all,
  and he needs to find other service providers.
  There are a couple of new guys on the block, but
  they dont use the normal tender/order/deliver/inv
  oice process
• The National Technical University of Athens
  provides access to rendering codes. They post
  Condor class ads describing resources and assign
  them to the highest bidder using an auction
  protocol.
• The local cable company provides rendering
  services on their digital video on demand servers
  during off-peak periods. However, they allow
  access only to signed-up clients, who can then
  submit jobs with no QoS negotiation. Delivery is
  on a best efforts basis, and tariffs depend on
  when each computation is submitted and when the
  result is needed.

14
Embedding in Business Processes

• GRIA takes a conservative approach
• implementing established procurement processes
• accounts, invitations to tender, proposals,
  orders, invoices, etc
• Must allow various negotiation models
• lightweight, best-efforts, no negotiation
• highly specified SLA, subject of iterative
  negotiation
• Want GRIA to handle arbitrary business processes
• need to represent processes and conversations
• cant assume globally agreed namespaces
• Currently investigating FIPA agents communicating
  via OWL
• no equivalent of our ref, your ref
• no way to represent multi-party negotiations

15
U1
BH1
SP1
A1
open-account(client_idBH1, client-ref75,
sp-idSP1, sp-refNONE)
create-new-conversation(client-idBH1,
parent-conv-idNONE, open-account-conversastion)
conv-id conv-1
open-authorisation(client-idBH1, conv-idconv-1,
check-account-status)
OK
open-account-ack(client-idBH1, client-ref75,
sp-idSP1, sp-refconv-1)
get-account-status(client-idBH1, client-ref75,
sp-idSP1, sp-refconv-1)
check-authorisation(client-idBH1,
conv-idconv-1, check-account-status)
OK
account-status(client-idBH1, client-ref75,
sp-idSP1, sp-refconv-1, statuspending)
credit checks, etc
16
Conclusions

• GRIA is ultra-conservative
• applying basic, very well established business
  processes
• using off-the-shelf technology
• Even this leads to profound questions for the
  semantic web/semantic grid
• standard ways to represent and compute QoS?
• how to represent and reason about multiple
  related jobs?
• how to interoperate even with simple negotiation
  mechanisms?
• We suspect a combination of methods are needed
• semantic web representations
• agents and autonomous reasoning
• adaptive behaviour
• Discuss!

17
Fulfillment
Semantic Grid
Concept
Business
reasoning
Problems
Knowledge-
based services
Business
Interop, Ontologies, Reasoning, Discovery
Workflow
Value Chain
Processes

Application
services
Jobs and
Grid
Data
computing
Raw
Resources
18
Credits

• National Technical University of Athens for
  representations of load and capacity vectors
• CESI, Kino and NTUA
• Terry Payne for discussions about OWL-S
• IST Project Number 33240
• See www.gria.org
• Contact
• ms_at_it-innovation.soton.ac.uk
• dder_at_ecs.soton.ac.uk