DL Template

1
Best Practices in Distributed Workflows and Web
Services Rob Allan, Asif Akram, David Meredith
CCLRC Daresbury Laboratory
2
WOSE Project

• Workflow Optimisation for Services in e-Science
• EPSRC funded project
• In collaboration with Imperial College and
  Cardiff University
• CCLRC is investigating the user requirements
• Developing Use Cases based on existing e-Science
  Application
• e-HTPX An e-Science Resources for High
  Throughput Protein Crystallography

3
Best Practices for Workflows

• Best practices for interoperable Web Services
• Modular workflow design
• Hierarchical workflow exception handling
• Compensation mechanisms provided by partner
  services
• Build adaptivity and flexibility into workflow
• Account for workflow management

4
Best Practices for Workflows

• Best practices for interoperable Web Services
• Selection of WS style
• Abstraction of data model
• When to use loosely or strongly typed Web
  services
• Approach to writing Web services (code first /
  WSDL first)
• Modular workflow design
• Hierarchical workflow exception handling
• Compensation mechanisms implemented by partner
  services
• Build adaptivity and flexibility into workflow
• Account for workflow management

5
a) Selection of WS style

• RPC/encoded (depreciated only use is for
  legacy purposes)
• RPC/literal WS-I compliant but has limitations
  for validation of complex data (due to namespace
  issues)
• Document/literal (wrapped) style is best (100
  WS-I compliant). Supports modelling and
  validation of complex data from different
  namespaces in plain XML instance documents.

6
b) Abstraction of data model
Separating the XML Schema elements and complex
types defined within the ltwsdltypesgt section of
the WSDL file into separate files. Recommended
as data can be modelled in different documents
according to different namespace and domain /
data requirements
ltwsdldefinitions ltwsdltypesgt ltxsdschema
gt ltxsdimport namespace
schemaLocation"BulkMRFinal.xsd"/gt
lt/xsdschemagt lt/wsdltypesgt .. lt/wsdldefinit
ionsgt
ltschemagt ltxsdinclude
schemaLocationschema1.xsd/gt ltxsdimport
schemaLocationschema2.xsd/gt lt/schemagt
Schema1.xsd
Schema2.xsd
7
b) Abstraction of Data Model (Benefits)
Separation of roles Data model can be developed
in isolation from WSDL. Schema / data model
re-usability Share and re-use schema rather
than re-designing schema for each new
WS. Isolation of changing components Data model
can be subject to change. Its isolation limits
the impact on other WS components such as
concrete WSDL file. Data model avoids
dependencies on WSDL SOAP namespaces Good
data model design. Full Schema functionality
XML Schema is powerful (e.g. xsd
patterns/regular-expressions, optional elements,
enumerations, type restrictions etc) Extensible
Modelling Schema can be extended without
breaking software that uses the original schema
through the use of xsdany / xsdanyType
(wildcard / placeholder for extending schema
where necessary).
8
b) Abstraction of Data Model

• Abstract the XML Schema(s) used in the
  ltwsdltypesgt section of the WSDL file into
  separate files.
• Model data logically in separate schema files
  with different namespaces as required by the data
  model.
• Combine separate Schema files as required using
  ltxsdincludegt and ltxsdimportgt
• Import the schema file(s) into the WSDL file.
• Reduces the size complexity of the WSDL file.

9
c) When to use loose or strong typing

• A strongly typed WS
• Defines a WSDL with a complete definition of a WS
  operations input and output messages in XML
  schema with tight constraints on the allowed
  values.
• xsdpatterns, xsdrestrictions,
  xsdenumerations, xsdsequences etc

• Advantages
• Well defined service interface all info
  necessary to invoke the service is encapsulated
  within the WSDL (client and automation friendly)
• Strong control on the data that enters the
  business logic of the service
• Disadvantages
• Requires a working knowledge of XML Schema
• Resistive to change in data model

10
c) When to use loose or strong typing

• A loosely typed WS
• Uses generic data types in the WSDL which are
  used to wrap data of other formats
• String can wrap markup fragments (e.g. xml,
  name-value pairs, SQL)
• xsdany / xsdanyType - placeholders for
  embedding arbitrary XML
• SOAP attachments (used to send data of any
  format, e.g binary)

• Advantages
• Flexible single WS can handle multiple types
  of message message data
• Easy to develop
• Disadvantages
• Incomplete WSDL interface - Requires manual
  negotiation between client and service to
  establish format of data wrapped by loose type.
• Prone to message exceptions (requires WS to be
  v.tolerant in what it accepts)

11
d) Approach to writing WS
Two Choices a) Code first (bottom up) b)
WSDL first (contract driven)
a) Code first (generating WSDL from WS
implantation)

• Advantages
• Simple
• Disadvantages
• WSDL files created from src code are always
  loosely typed (e.g. a src code String is just a
  String cant impose xsdpatterns or
  xsdrestrictions if starting with a String).
• WSDL auto-generation tools can introduce
  technical dependencies upon implementation
  language !
• Not all language specific data types can be
  mapped into interoperable XML (the Object XML
  mismatch).

12
d) Approach to writing WS
b) WSDL first or contract driven approach
(generate the WS operation from the WSDL file)

• Advantages
• Authored WSDL can be strongly and loosely typed
  where necessary
• Interoperability issues between different
  languages are avoided if starting with plain XML
  
• Can auto-generate service implementation classes
  from WSDL.
• Disadvantages
• Developer requires a reasonable knowledge of
  XML Schema WSDL

13
Best Practices for Workflows

• Best practices for interoperable Web Services
• Modular workflow design
• Hierarchical workflow exception handling
• Compensation mechanisms implemented by partner
  services
• Build adaptivity and flexibility into workflow
• Account for workflow management

14
Modular Design

• Modules operate as object oriented black boxes
  in the overall workflow, with their
• own variables,
• computational logic,
• dependency constraints,
• event handlers.
• Group similar or related activities
• Module components should be scalable and
  replaceable (to serve as repeatable units)

15
Best Practices for Workflows

• Best practices for interoperable Web Services
• Modular workflow design
• Hierarchical workflow exception handling
• Compensation mechanisms implemented by partner
  services
• Build adaptivity and flexibility into workflow
• Account for workflow management

16
Exception Handling

• Types of Exception
• expected exceptions (variations)
• unexpected exceptions
• Exception handlers can be defined at different
  hierarchical levels.
• Global (unexpected)
• Scoped (expected)
• Inline (expected)
• Exceptions should be handled at the lowest
  available level and unrecognized exceptions are
  passed to a higher level.

17
Exception Handling
18
Best Practices for Workflows

• Best practices for interoperable Web Services
• Modular workflow design
• Hierarchical workflow exception handling
• Compensation mechanisms provided by partner
  services
• Build adaptivity and flexibility into workflow
• Account for workflow management

19
Compensation Mechanism

• undoing steps that have already completed
  successfully
• Reversing the effects of successful activities in
  the abandoned workflow
• Provision of compensation is responsibility of
  the service.
• Long running processes require some sort of
  compensation.
• TDO

20
Best Practices for Workflows

• Best practices for interoperable Web Services
• Modular workflow design
• Hierarchical workflow exception handling
• Compensation mechanisms provided by partner
  services
• Build adaptivity and flexibility into workflow
• Account for workflow management

21
Adaptive Workflow

• Workflow can be static or dynamic depending on
  whether changes are accommodated at design time
  or runtime.
• Adaptivity and flexibility cater for
  adjustments/modifications without breaking a
  workflow.
• Provide place holders for future extensions
  e.g. replaceable empty activities
• Externalizing changeable data e.g. partner
  service endpoints specified in configuration
  files.
• Use generic/loose data types (rather than
  strongly typed data).

22
Best Practices for Workflows

• Best practices for interoperable Web Services
• Modular workflow design
• Hierarchical workflow exception handling
• Compensation mechanisms implemented by partner
  services
• Build adaptivity and flexibility into workflow
• Account for workflow management

23
Management of Workflow

• Workflow management involves
• Breakpoints arbitrary location crucial to the
  process
• Steering points e.g. reset, re-schedule,
  restart, undo, abort, complete, recover, ignore
  or jump operations.
• Persistence state recorded at breaksteering
  points (allows re-execution of workflow from
  prior state, recording/inspection of prior state
  data values)