DL Template

1
Requirements and Expectations from Workflows Asif
Akram e-Science Grid Technology Group
2
WOSE Project

• Workflow Optimisation for Services in e-Science
• EPSRC funded project
• In collaboration with Imperial College and
  Cardiff University
• CCLRC is investigating the user requirements
• Developing Use Cases based on existing e-Science
  Application
• e-HTPX An e-Science Resources for High
  Throughput Protein Crystallography

3
Best Practices for Workflows

• Modular Design
• Exception Handling
• Compensation Mechanism
• Adaptive Workflow
• Flexible Workflows
• Management of Workflow

4
Modular Design

• Thorough investigation of different activities
• Group similar or related activities
• Group activities with minimum side effects
• Module components should be scalable
• Module components can be replaceable
• Reliable to serve as repeatable units
• Modules operate as black boxes in the overall
  workflow, with their own variables, computational
  logic, dependency constraints, and event handlers.

5
Exception Handling

• Exceptions can be due to
• inconsistent data,
• divergence of tasks from the workflow model,
• unexpected contingencies and
• un-modeled changes in the environment.
• Type of Exceptions expected exceptions
  (variations) and unexpected exceptions
• Exception Handlers Global Exception Handlers,
  Scoped Exception Handlers and Inline Exception
  Handlers.
• Exception handling should be localized

6
Compensation Mechanism

• undoing steps in the business process that have
  already completed successfully
• Reverse the effects of successful activities in
  the abandoned workflow
• Un-handled Exceptions requires compensation
• Compensation behavior must be defined by the
  services to ensure reversal of a operation.
• Synchronous operations (i.e. a single connection
  request/reply) have short lifetimes and do not
  require compensation to release resources.
• Asynchronous communications requires some sort of
  compensation (unluckily we have normally RPC
  style Web Services)

7
Adaptive Workflow

• Adaptive nature of a workflow can be static or
  dynamic depending on whether changes are
  accommodated at design time or runtime
  respectively.
• Redesign and optimization of a process to gain
  greater efficiency and effectiveness requires
  adjustment in the workflows in fact the final
  goal is to constantly improve the workflow/
  process by evolutionary redefinition.
• Modifications break a pre-defined process in an
  attempt to solve the problem in better way.
• Dynamically adaptive workflows adjust at runtime
  i.e. due to unexpected results

8
Flexible Workflows

• Flexibility is required during the early stages
  of a workflow design when services are un-stable.
  
• Flexibility in the data types and service
  endpoint.
• Flexibility in data, is achieved by applying
  generic base or parent data types
• Flexibility in service endpoint, requires
  integration of additional and re-located services
  through WS-Addressing.
• Potential callouts and logic for selecting
  partner service has to be built into the business
  process as external configuration file .
• Assuming all the services have similar
  interfaces which may not always be the case.

9
Management of Workflow

• Workflow management involves the addition of
• Breakpoints arbitrary location crucial to the
  process
• Steering capabilities generalizes breakpoints and
  involves reset, re-schedule, restart, undo,
  abort, complete, recover, ignore or jump
  operations.
• Persistence of state for fault tolerance, to
  allow re-execution with different experimental
  data sets, and to facilitate inspection of
  intermediate data values
• Monitoring for optimization and analysis of
  internal state, data flow, inspection of values,
  re-scheduling of tasks and re-ordering of tasks

10
Portals Web Services
Service End-points
Workflow
Internet
Repository for Authorisation and Policies
Clients
11
Business Process Execution Language

• Builds on top of XML and Web Services
• Abstract, which allow specification of the public
  message exchange between participating services
  and doesnt include the internal details of
  process flows.
• Executable, which allows specification of the
  exact details of a workflow. Normally BPEL is
  used for executable processes.
• Can utilize other Web Services specification for
  reliable messages, transactions etc

12
BPEL

• Provides different types of activities
• Primitive activities can be structured in Modules
• BPEL support different types of structured
  activities i.e. sequence, flows and scope
• Extensive support for Exception handling
• Can trigger Events from Exception Handler
• Compensation Handlers can be called from
  Exception Handler
• Limited monitoring is possible through different
  primitive activities like onMessage, onAlarm
  and pick

13
(No Transcript)
14
(No Transcript)
15
BPEL Extension

• Different implementation has varying level of
  extensions
• Oracle BPEL engine supports notification
• Oracle supports NFlow for parallel execution of
  any module n times
• Oracle has proprietary management capabilities
  and user interaction functionality
• Engines provides API to query and interact with
  process at runtime
• Proprietary API to populate SOAP Headers
• Limited support for WS-Security
• Support for WSIF and EJB

16
Limitations of BPEL

• Limited reusability of primitive and structured
  activities
• Cant re-execute activities defined earlier
• ltonMessagegt or ltonAlarmgt events are not to modify
  the workflow at runtime
• BPEL specification does not explicitly support
  user interactions and notifications
• Difficult to integrate security without support
  from Vendors
• BPEL is fairly complex specification

17
Future Work

• Moving to Document Style Web Services
• Evaluating WSRF for e-HTPX
• e-HTPX mainly passes URL of images in different
  services (candidate for Resource Properties)
• Moving EPR across services is more efficient
• Built in support for Notifications
• Support for persistence (flat files but can be
  extended for DB) e-HTPX is using Hibernate
• WS-Core supports XML Database Xindice
• Support for different level and type of security