Get SOAphisticated with Web Services

1
Get SOAphisticated with Web Services

• Jim Webber
• SOA Practice Lead
• ThoughtWorks

2
ThoughtWorks Overview

• Enterprise systems consultancy serving Global
  1000 companies
• 700 employees
• Growing at 50 per year
• Seven countries
• USA, UK, India, Canada, Australia, China
• Specialists in Agile software methods, enterprise
  systems
• .Net
• J2EE
• esoteric stuff

3
Roadmap

• Historical Perspective
• What is a Web Service?
• Evolution of Web Services
• Message-Orientation The MEST Architectural Style
• Architecture
• Protocol
• Implementation
• Enterprise Issues
• Scalability
• Dependability
• Consistency
• Moving to SOA
• Summary
• QA

4
What is a Web Service?

• A service is piece of application logic exposing
  message-oriented interface.
• Any scale from inter-process to Internet
• A Web Service is a system which exposes a message
  oriented-interface whose messages are in SOAP
  format.
• SOAP is the critical differentiator.
• It is what differentiates a Web Service from
  other kinds of services (e.g. OS services).
• SOAP is the lowest point in the WS stack.

5
In the Beginning...

• In the old days we used SOAP-RPC
• RPC style SOAP plus encoding rules for
  serialising and deserialising application-level
  objects into messages.
• A kind of XML RPC
• Similar to RMI etc but without all the good stuff
  like garbage collection, security, and
  performance.

This is not SOA
6
RPC with Documents

• The community learned that SOAP RPC was pretty
  awful for interoperability
• Different implementations interpreted the
  encoding rules differently.
• Document-oriented SOAP (doc/literal) solved (or
  at least moved) interoperability problems.
• But left the inappropriate object-like model in
  place.
• RPC with documents is the mainstream of Web
  Services today.
• It is not the future of Web Services, just a
  point on the learning curve.

7
The RESTifarian Jihad

• Seeing that the Web was good and that RPC was
  bad, the RESTifarians waged war on the Web
  Services community.
• REST-ful services have a uniform API and exchange
  messages using that API.
• The API is HTTP-like and allows messages to be
  sent (POST) and responses to be solicited (GET)
• Along with PUT and DELETE etc.
• Unfortunately RESTful semantics arent always
  mappable onto protocols other than HTTP.
• Whats SMTP GET?
• Imposing an API even a uniform API is too
  constraining.
• But the notion of exchanging messages is correct.

8
Web Service Architecture

• Todays Web Services architecture is based on the
  notion of
• Services, which exchange
• SOAP Messages, which are constrained in form and
  exchange pattern by
• WSDL, which may also advertise
• Or SSDL!
• Quality of Service Protocols, defined by the
• WS- specifications

9
Message-Orientation

• The action taken in response to receiving a
  message is up to the recipient
• At the highest architectural level, a message
  exchange is simply a transfer of information from
  sender to receiver.
• The business semantics of a message exchange are
  service-specific.
• Transfer of messages is transport protocol
  independent
• The means of transporting a message carries no
  semantics
• I.e. POSTing or GETting does not change the
  semantics of a message.

10
A Web Services Application
Example ServiceA sends ServiceB a MessageX.
ServiceB responds with a MessageY or a MessageZ
depending on the content of the MessageX it
received.
11
Architectural Tenets

• Boundaries are explicit
• The boundaries of a service are well-defined when
  they are incorporated into a distributed
  application. Other services do not see the
  internal workings, implementation details, or
  resource representations of a service.
• Services are autonomous
• Service implementations are developed and evolve
  independently from one another.
• Services share schema and contract, not classes
• In service-oriented architectures, no single set
  of abstractions (classes) spans an entire
  application. Services share schemas (contracts)
  that define the structure of the information that
  they exchange, not information about their
  underlying type systems.

12
More Architectural Tenets

• Policies determine service compatibility
• Services interact with one another only after it
  has been determined based on policy assertions
  that they can meaningfully exchange
  information.
• See D. Box, Service-Oriented Architecture and
  Programming (SOAP)
• http//msdn.microsoft.com/msdntv/episode.aspx?xml
  episodes/en/20030827SOAPDB/manifest.xml
• http//msdn.microsoft.com/msdntv/episode.aspx?xml
  episodes/en/20030902SOAPDB/manifest.xml

13
SOA and Web Services Aims

• Integration
• Reuse
• Agility
• Lower cost of development/ownership for
  applications
• No such thing as a free lunch
• Have to architect and implement to achieve these
  features

14
High Level Architecture
15
High Level Architecture Tasks

• Standard enterprise architecture
• Tool support Whiteboard, PowerPoint, and coffee
• Identify functional and non-functional
  requirements
• Identify existing Web Services as candidates for
  inclusion in the system
• Those that satisfy the functional and
  non-functional requirements
• Identify what functionality you need to deploy
  yourself.
• Identify the interactions between Web Services

16
Architecture Key WS- Specs

• SOAP
• The lowest part of the Web Services stack.
• Consider messages in terms of
• Content for functional requirements
• Content for non-functional requirements.
• SOAP provides a transfer mechanism that combines
  both
• SOAP body for transferring information pertinent
  to the functional requirements
• SOAP headers for transferring out-of-band
  information to address non-functional
  requirements.
• WS-Addressing
• Remember - SOAP is the lowest layer in the WS
  stack!
• It is the transfer mechanism for Web Services
• Everything else is transport.
• WS-Addressing allows SOAP messages to be routed
  round a system irrespective of the underlying
  transport.

17
SOAP WS-Addressing
18
Protocol

• Each Web Service we develop has to deal with
  message exchanges
• Those that it requires to expose its own
  functionality
• Those that the Web Services it uses to interact
  with other Web Services
• Message exchanges pertaining to the QoS protocols
  it supports.
• Protocol demolishes the idea of Web Services as
  entities which are invoked.
• If you insist on having an operation to invoke,
  then invoke the ProcessMessage operation.
• ProcessMessage is abstract, semantics
• A transfer of a message from sender to receiver
  and
• A request to process the received message.
• The MEST architectural style

19
Deriving Contracts

• Protocol is supported by contracts
• Form and pattern of messages exchanged by a Web
  Service
• Note contracts rather than interfaces
• Interface is an overloaded term that implies type
  information
• No type information in Web Services, only
  schematised messages
• Derive contract by examining SOAP message
  exchanges
• WSDL contract describes messages and message
  exchanges that the service will participate in.
• SSDL a better alternative?

Protocol
Contract
20
Contract and Policy

• Contracts are binding and immutable
• The associated service cannot violate the terms
  it sets out in the contract.
• Consumers only information about the service is
  its contract.
• Serves two purposes
• Facilitates interoperability with other services
• Advertises message exchanges, and possibly QoS
  support/requirements
• Decouples service implementation from consumers.
• Hides the implementation of a service behind a
  message-oriented façade.
• Policy declarations determine the QoS protocols
  used for an interaction.
• Policies can be determined statically or
  dynamically
• Static policies can be attached to the service
  contract
• Dynamically created policies communicated to
  service consumers during interactions with the
  service

21
WSDL is not Object IDL

• WSDL contracts should not leak information
  about their services.

Anything in declared a WSDL contract must be
preserved until the associated Web Service is
retired.

• Remember WSDL is immutable
• Service implementations are not
• They will be subject to maintenance, bug fixes
  etc
• Use WSDL to shield your service implementation
  from its consumers.
• Expose only messages and message exchanges.
• WSDL isnt Object IDL.
• A WSDL contract betrays no type information and
  must not expose the type system of the underlying
  implementation.

22
Protocol Key WS- Specs

• In addition to the architecture specs
• Metadata
• WSDL, WS-Policy, WS-MetadataExchange
• Security
• WS-Security, WS-Trust, WS-SecureConversation
• Reliable Messaging
• WS-ReliableMessaging
• Transactions
• WS-AT/BA or WS-TransactionManagement

23
Implementation

• The internal architecture of a service is
  relatively mundane.
• The layering of messaging, logic, and state is
  similar to the classic N-tiered architecture.
• The message processing layer is interesting
  though...

24
Message Processing Layer

• Deals with the conversion of messages on the wire
  (in SOAP format) to programmatic abstractions.
• Does not try to abstract message exchanges into
  RPC
• Presents a truthful picture of the underlying
  message exchanges to the service logic
• Allows the service logic to be tolerant of the
  underlying messaging infrastructure
• Latent, lossy, asynchronous etc
• Messages are the abstraction that the service
  logic binds to, not other services
• Loose coupling!

25
Todays SOAP API

• public class GameServiceStub
• public void StartGame()...
• public GridPos Move(GridPos gp)
• ...
• // Other remote methods omitted for brevity

26
A Message-Oriented API

• public interface IMessage
• ...
• public class StartGameMessage IMessage
• ...
• // Other messages omitted for brevity
• public class GameMessagingLayer
• / Outbound messages /
• public void SendMessage(IMessage msg)...
• / Inbound messages /
• public event MoveMessageReceivedDelegate
  MoveMessageArrived
• public event StartGameMessageReceivedDelegate
  
  StartGameMessageArrived
• // Other messaging events omitted for brevity

27
Service Logic

• The arrival of a message at a service causes some
  processing to occur
• From the MEST architectural style
• Service implementation is stateless
• All the information it needs to perform a
  specific task is contained with the message that
  initiates that task, is computed, or is contained
  in the persistent data storage tier.
• c.f. architecture of the web.
• Non-functional requirements (e.g. security,
  transactions) set context for message processing
• But are orthogonal to the business logic

28
Implementation Some Key WS- Specs

• Some WS- specs are designed to be driven by
  service implementations directly
• WS-Eventing
• multicast
• WS-Enumeration
• Streaming/data cursor
• WS-Transfer
• CRUD semantics
• WS-BPEL is an alternative means of implementing
  services
• Workflow expressed in terms of Web Services and
  messages
• If these make Web Service development easier, use
  them
• If not, then ignore them

29
Enterprise Issues

• Web Services are not an enterprise platform in
  the traditional sense
• Though an individual Web Service will usually be
  deployed onto such a platform (J2EE app server,
  .Net)
• Just a collection of specs and implementations
• messages, message exchange patterns, and
  semantics
• Cannot rely on the infrastructure to manage
  dependability characteristics
• Security, reliable message transfer,
  transactionality, scalability etc
• Need to understand how the WS- specs can help
• And how then cannot help.

30
Security

• WS-Security does not make your Web Services
  secure!
• It can help with message-level security
• Privacy
• Integrity
• Non-repudiation
• HTTPS is not sufficient
• Messages traverse arbitrary networks
• Point to point mechanisms are difficult to scale
• WS-Security elements are embedded within the
  message
• And easily traverse arbitrary networks

31
Secure Message Transfer
32
Reliable Messaging

• In theory, reliable messaging is transparent to
  the architecture
• Reliable messaging gives us at most once/exactly
  once notification of delivery of messages
• No indication whatsoever of whether a message was
  processed
• Messages can go missing or be duplicated, the
  protocol will detect that
• Not in a catastrophic failure case
• Can rely on the underlying protocol to smooth out
  the lumps in a large-scale Web Service networks
• No such thing as a free lunch
• The lumps will always be there, RM can only
  reduce their frequency and size

33
Reliable Messaging

• The underlying protocols still have to be exposed
  to our Web Service implementations
• Implementations have to be architected to deal
  with failures in messaging
• Should all processes should be idempotent?
• Dependability characteristics of the Web Service
• Mission critical every operation has to be
  idempotent.
• At some point RM will fail. Without idempotent
  operations you are in a world of pain
• Somewhat critical take a chance?
• Make common or particularly important routines
  idempotent
• Uncritical
• Leave it to the underlying RM protocol!

Complex Big investment
Somewhat complexLower
Easy (if nothing goes wrong) Cheap!
34
Scalability
Data Tier

• Stateless services scale easily.
• Add more hardware, deploy services.
• Leave state management to underlying
  enterprise-grade Database.
• DB guys know how to manage state in a dependable
  manner

Server Farm
35
Dependability

• Statelessness makes failover fault tolerance
  trivial
• Well mostly...
• If it can be detected that a Web Service or a
  host server has crashed, simply route messages to
  live Web Services on live hosts
• Standard systems management stuff for the
  hardware
• WS-Management may help for Web Services
• Stateful services will cause you pain
• Do you really want to have to build state
  migration capabilities into your WS code?
• Avoid WS-RF and dependent toolkits/specs.

36
State Sucks
Server Farm
Unknown Conversation

Multi-Protocol Session-Affinity Aware Router
37
State is the DBAs Problem!
Data Tier
Server Farm
Inexpensive Round-Robin Router
38
Consistency

• Never expose lockable resources outside your
  administrative domain
• Denial of service attacks possible otherwise
• Cannot have consistent data all the time.
• Have to be satisfied with snapshots of data
• Only the administrative domain that owns the data
  has the most up-to-date information
• Web Services transactions dont help
• They just keep logical processing of messages in
  sync.
• Design applications with the understanding of the
  age of the data in mind
• See Pat Hellands Data on the Inside vs. Data
  on the Outside for in-depth analysis
• http//msdn.microsoft.com/architecture/overview/de
  fault.aspx?pull/library/en-us/dnbda/html/dataouts
  ideinside.asp

39
SOA Today

• Applications, objects, frameworks
• Best practice encapsulate and expose as service
• Non-invasive

40
Tomorrow

• Individual applications are internally
  service-oriented
• Indigo
• Design individual applications to expose useful
  services
• Factor applications to facilitate replacement or
  outsourcing of functional units

41
The Future

• Networks of services are the norm
• Applications are software entities which
  orchestrate service interactions
• With complex dependability requirements...
• Applications are services too!

42
Summary

• Web Services are about document exchanges, not
  APIs or interfaces
• Messages are the source of truth, use them to
  decouple your service from its consumers
• Be careful what you advertise in WSDL, youll
  have to carry it forever
• Three architectural views
• Global architecture (Service-Oriented)
• Protocol (Message-Oriented)
• Implementation (Event driven)
• Web Service interactions are stateless
• WS- can only get you so far

43
Questions?