Messaging Systems and Technology

1
Messaging Systems and Technology
2
Introduction

• Synchronous middleware doesnt suit all
  applications
• Message Oriented Middleware (MOM) provides
  features like
• asynchronous communications between processes
• store-and-forward capabilities
• transactional messaging

3
Basic Asynchronous Operations

• Send (dest, message)
• Receive ( target, message)

queue
receive
send
4
MOM usage

• Applications often need
• deferred processing of some slow transaction
  (eg printing an invoice)
• support efficient 1-to-many and many-to-many
  communications
• to send messages whether the server is available
  or not
• to provide event notification to a dynanic user
  community
• loosely coupled client-server systems

5
Example - Transactions
6
MOM Technologies

• Broadly two (somewhat intersecting) categories
• Message brokers (IBM, BEA, Microsoft, etc)
• Broadcast-based publish-subscribe (TIBCO, IONA)
• Both categories differ greatly in their features
  and capabilities

7
Some Application Examples

• Stock price update notification
• General workflow systems
• Application integration
• System management

8
Some Example Products

• TIBCO/Rendezvous
• IONAs OrbixTalk
• IBMs MQ Series (Publish-Subscribe)

9
TIB/Rendezvous

• Pioneers of broadcast/multicast publish-subscribe
  technology
• Publishers and subscribers communicate using
  subjects

Register/ Subscribe
Create/ Publish
Subject
Sub
Pub
Sub
Sub
10
Subjects

• Hierarchical names identify a subject of interest
• /CSIRO
• /CSIRO/gossip
• /CSIRO/gossip/ADSaT
• /CSIRO/work/ADSaT
• Wildcards can be used
• /CSIRO/
• /CSIRO//ADSaT

11
Multicast/Broadcast

• A published message is sent across the network
  only once
• All subscribers receive the same message
• IP Multicast or broadcast is used on LANs
• Important scalability issue as bandwidth/processor
  usage is low

12
LAN Architecture
13
WAN Architecture
Filtered, Point-to-Point
14
Quality-of-Service (QOS)

• Reliable
• publisher informed after time-out period if
  message cannot be delivered to a subscriber
• Certified
• guaranteed delivery or both parties informed of
  failure
• to survive process failure messages can be logged
  to disk until specified time-out period expires

15
Programming Options

• C/C/Java
• Self-describing message format
• TIB/ObjectBus layers a standard CORBA-compliant
  ORB upon the TIBCO protocols.
• TIBIOP provides broadcast messages in a CORBA
  environment.

16
Security

• Security is a problem in publish-subscribe
  multicast technology
• TIBCO Data Security (DS) product
• publisher controls secure key distribution to
  subscribers using digital signatures
• subscribers use key to encrypt messages on a
  subject
• choice of encryption algorithm available

17
OrbixTalk

• Uses IP multicast in CORBA environment

18
Topics

• Communication based on topics (read subjects,
  wildcards, etc)
• 3 QOS levels
• otmcp UDP IP multicast. Limits packet size to
  1280 bytes and no guarantee of receipt.
• otrmp Augments IP multicast to ensure message
  delivery to all subscribers until configurable
  time-out expires
• otsfp OrbixTalk MessageStore daemon uses a
  store-and-forward protocol to provide guaranteed
  message delivery.

19
OrbixTalk Architecture
20
Architecture Issues

• Daemons can have hot backups to support failure
  of primary daemon
• Several MessageStore daemons can be used to
  facilitate load balancing
• each OrbixTalk process only use one MessageStore
• Compaction utility must be scheduled to remove
  old messages

21
Security/WANs

• No specific security or WAN support
• security can be programmed using Orbix
  Transformers
• WANs traffic needs appropriately configured
  routers, or an IIOP bridge between LANs to be
  written

22
MQSeries PubSub

• MQSeries is probably most widely deployed MOM
  product
• Basic technology provides a message queue
  architecture
• communication via shared queues managed by Queue
  Managers
• persistent/non-persistent messages
• transactional queue access with XA
• broad platform support

23
MQSeries PubSub
1 PubSub Broker per QM
LAN/WAN support
Topics interchanged by Brokers if needed
24
Architecture

• Brokers can be organized to communicate
  hierarchically

HQ
USA
Asia
West
East
HK
25
Features

• Typical hierachical topic names, wildcards, etc
• Streams can be used as higher level topic
  partitioning scheme
• DEFAULT stream per broker
• optional additional streams for specified topics
• broker allocates a thread to handle each stream

26
Quality-of-Service

• Persistent
• messages written to log file
• survive broker/QM failure
• Non-persistent
• messages lost if broker fails
• fast
• Message priority supported in queues

27
Other features

• MQ can act as an XA transaction manager
• Basic security provided, integrated with native
  operating system
• Security exits support integration with 3rd party
  products
• Other products, eg MQ Integrator

28
Performance Test

• Publishers and subscribers run on different
  machines
• Dual pentium 500Mhz NT 4.0 boxes, 0.5GB memory
• MQ QM/broker and Orbix daemons run on same
  machine as publishers
• Reliable, non-presistent protocols used

29
256 Byte Message
30
512 Byte Message
31
Interpretation

• Multicast technologies are fast!
• TIBCO rvd becomes bottleneck
• Publisher too fast exceptions
• OrbixTalk generates large number of interrupts
• one per message per listener
• MQ scales well, but queue/broker architecture is
  inherently slower

32
Summary

• MOM provides excellent solution to many business
  problems
• Range of technologies available, with very
  different
• architectures/technologies
• features
• performance
• Important to understand issues and select right
  one for your business