2003

1
MQ Series with jBASE

• Overview of MQ Series and how to use it from jBASE

Jim Idle Temporal Wave
2
What is MQ Series

• Software package owned by IBM
• Provides many operating systems and programming
  languages with Queues upon which to exchange
  information.
• Works as client/server configuration
• Manages Queues across one or more computers.
• Now called WebSphere MQ

3
What is a Queue

• Like a named pipe except
• More than one program can open it to put messages
  in it
• More than one program can open it to get messages
  from it
• Does not require the message Provider and the
  message Consumer to know anything about each
  other, or be running at the same time

4
What is a Queue

• Queues accept any form of binary data as a
  Message
• Consumers can retrieve messages in any order,
  not just FIFO and can browse a queue for specific
  types of message
• Queues can be assigned permissions so that only
  certain users can read or write to them

5
What is a Queue

• A queue can be defined as Persistent
• Means that the messages are stored on disk and do
  not go away if the machine is turned off or
  crashes
• A queue can hold many thousands of messages at
  once
• Messages can be grouped together in a
  transaction, so that they dont appear on a queue
  until the transaction is complete.

6
What is a Message

• A message consists of two things
• A message header constructed either automatically
  or with user supplied information
• Message ID
• Message Group
• Expiry time (time to live)
• The user supplied data (any type of binary data
  at all, including dynamic arrays from jBC (BASIC))

7
What is a Message

• A message can be be very large (Gigabytes if
  required), but is generally something like a file
  record or transaction details
• From jBC, a message is sent and received as a jBC
  variable (dynamic array) and is very easy to use

8
Things to Send in Messages

• Stock quotes
• Transaction requests
• Handle this incoming call
• Transaction results
• Complete, Aborted, etc
• Triggers
• Someone wake up and do this!

9
Message Providers

• Data streams
• Stock ticker
• Alerts / Notifications
• Transaction requests
• Trade this stock for me with these parameters
• Please look up the value of this stock
• Please process this order
• Trigger messages
• Cause an application to be started

10
Message Consumers

• Passive consumption
• Wait for messages and print them
• Stock ticker
• Leave message on the queue for others to see
• Active consumption
• Takes possession of the message to do something
  with it
• Reduces stock numbers in a database
• Records billing information
• Interactive Consumption
• Takes possession of the message to do something
  with it and replies to the sender. Many messages
  can be exchanged
• Stock trader trades stock and replies with
  execution details
• Client Server Web server and application
  processing

11
Passive Example

• Message/Alert program
• Many small clients wait for a message then
  displays a popup
• Informational message to users (text)
• Notification that an interface, program, etc is
  unavailable
• Indicates this with a graphical indication rather
  than the contents of the message

12
Active Example

• A system exists to record telephone call history
• When the main application process a call, it
  places a message on a queue when the call has
  completed.
• Main application then forgets about it
• Consumer gets the message and updates a database
  with the call details. When it gets the message,
  it removes it from the queue so that it is only
  processed once

13
Interactive Example

• A trading system executes trades by passing the
  details to an external system.
• It must send the information and receive
  confirmation of the trade to inform the user of
  the results.
• The application creates a message with the trade
  details. It tags the message header (ID) with a
  transaction ID and places it on the queue. Then
  it listens on a response queue for a reply with
  the same message ID.
• The trading system sees the message and performs
  the trade. It sends the trade details back on the
  required queue and tags the header with the
  transaction ID.
• The application sees the response and informs the
  user of the results.

14
Web Server

• The web server uses standard tools to create
  pages, serve the user GUI, accept forms, etc
• Sends an MQ Message to do some part of the app or
  return results
• Varying number of jBC programs serve the queue(s)
  and perform the tasks in ordinary jBC code
• jBC Programs can reply with raw data, entire
  pre-formatted web page, or a URL that points at
  the pages the produced
• Much simpler and faster than CGI, Java, .Net etc

15
Call/Task Distribution

• Standard Pick Techniques
• Write records in a file
• Have many programs continually SELECT or read an
  item
• Burn lots of CPU and system resources in the
  process
• Difficult to schedule, prioritize calls/tasks,
  increase capacity etc

16
Call/Task Distribution

• MQ Techniques
• Event trigger (phone rings, modem rings, user
  presses Any key, nuclear war), causes a message
  to go on a single queue with a specified priority
• One to N jBC programs wait on that queue, using
  no CPU or system resource
• MQ Series distributes message according to who is
  waiting and available and the priority of the
  message wake up time is virtually instant
• Thats it.

17
EXCEL Auto Update

• MQ Series has an ActiveX component
• Using this in EXCEL, a spreadsheet awaits the
  delivery of MQ messages
• Triggers on jBASE files cause a message to be
  sent to the queue with interesting data such as
  sales, lawsuits, etc.
• The Excel VB macro updates the spreadsheet data
  and a graph updates in real time to show lawsuits
  per minute, sales per hour, calls per hour etc.

18
MQ Series Configurations
Replication Channel(s)
Q Manager X
Q1
Q2
Q Manager Y
System A
Client Channel
19
Using MQ Series from jBC (JBCMQ)

• Simple interface with functions that appear to be
  part of the language
• hides complexities of C
• Manages connection handles, queue handles, queue
  persistence across SUBROUTINEs
• Simple queues
• Just stick the message on the queue
• Just get anything back
• Finer control
• Set expiry times, message ID, message group ID etc

20
Using MQ Series from jBC (JBCMQ)

• Consumers react instantly to message receipt
• But consume 0 CPU cycles while waiting
• Extremely efficient!
• Can have more than one instance of a jBC program
  consuming the queue
• Allows doubling, tripling, quadrupling, etc.
  system throughput
• Can address messages to specific
  users/clients/programs etc

21
Using MQ Series from jBC (JBCMQ)

• Triggers
• We can define triggers such as Start another
  copy of TRADE.RCV if there are more than 500
  messages waiting
• Can define the trigger to be the thing that
  starts the program in the first place no more
  start of day process to start phantoms.
• Allow the system to increase its throughput
  capacity automatically
• Programs waiting for messages can timeout and
  exit gracefully once the rush is over. The
  trigger will start another copy if the queue
  backs up again.

22
jBC Functions

• See www.temporal-wave.com (products link) for
  full documentation on-line
• MQConnect()
• Connect to a queue manager on a system
• MQOpenQueue()
• Open a named queue for sending and receiving
• MQSend()
• Send a message
• MQGet()
• Wait and retrieve a message

23
Example

• Fax-me distribution system

SQL Server
Fax Out Q
Fax Out Q
jBCDC Program
Raw Details Q
Fax Senders
Fax Formatters