Chapter 18 Message Brokers The Preferred EAI Engine

1
Chapter 18 Message Brokers The Preferred EAI
Engine

• Message brokers bridge many different platforms
  and development solutions than until now have
  been more isolated than open.
• The services provided by message brokers can be
  put into several distinct categories message
  transformation, rules processing, intelligent
  routing, message warehousing, flow control,
  repository services, directory services,
  management, APIs, and adapters.
• An advantage of brokers is that they mirror the
  business, allowing for non-invasive integration,
  and automating functions currently performed
  manually.
• Brokers build on top of existing middleware
  technology, and could be called the middleware
  of middleware (See Fig 18.1).

2

• Fig. 18.1 Message brokers are able to
  integrate many different types of middleware, as
  well as applications and databases.

3

• Message Broker is based on asynchronous,
  store-and-forward messaging. While its standards
  are not yet well defined, major vendors
  implementations include at least these three
  essential components (Fig 18.2)
• message transformation layer
• rules engine
• and intelligent routing mechanism
• An application publishes a message to the broker,
  while another application subscribes to the
  message. Applications need NOT be session
  connected, which alleviates scalability problems
  of other integration solutions.
• Moreover, brokers also able to transform and
  convert the payload, reformat the message, and
  route to any number of targets (which determined
  by the rules engine).
• All brokers must offer any-to-any and
  many-to-many messaging capabilities
  publish/subscribe is an effective model to
  achieve both goals.
• They provide value-added services from the
  uppermost layer in the ISO model the
  application layer.

4

• Fig. 18.2 Message Brokers have three
  primary components the message transformation
  layer, the rules engine, and the intelligent
  routing mechanism.

5

• Broker could interface with source or target at
  the data level, application interface level,
  method level, and the user interface level.
• The strength of message brokers is that they are
  able to link any number of different types of
  systems, adjusting to the differences between all
  source and target systems by exposing an API, and
  sometimes an adapter. (Fig 18.3).

Fig. 18.3 Message Brokers provide services to
applications through an application programming
interface, or an adapter.
6

• Message Transformation Layer
• The Layer understands the format of all messages
  being passed among the applications and changes
  them on the fly data from source is
  restructured to target(s) format (Fig 18.4). It
  is done through parsing and pattern-matching
  methods that are constructed to describe any
  message format.
• Information required for transformation is
  generally stored in a repository that keeps track
  of the source system, the format of the message,
  the target system, and the desired format of the
  target system.
• Format incompatibilities resolved through schema
  conversions most data conversion are automatic,
  except for conversions between numeric and
  alphanumeric formats. All conversions must take
  place dynamically.
• In rear cases, conversion cannot be done (e.g.
  length/value of the message is unknown)
  rules-processing capabilities of brokers allow
  resolution of such situations.

7

• Fig. 18.4 The Message
  Transformation Layer

8

• Rules Processing
• Rules Processing is an application development
  environment supported by the message broker to
  address the special requirements of integrating
  applications. In effect, it is an application
  between applications, one that does nothing more
  than provide the logic for sharing information.
• Routing of message to more than one target,
  performing computations or look-ups on data as
  its being transformed, etc. are examples of Rules
  Processing.
• Rules are created programmatically, primarily
  with script languages, and with virtually no
  bound to their flexibility.
• Intelligent Routing
• Also known as flow control or content-based
  routing builds on capabilities of both rules
  layer and the message transformation.
• Broker can identify message source and route it
  to proper target(s) transforming it if required
  (Fig 18.5).

9
Fig. 18.5 Intelligent Routing means identifying
the message and sending it on to the proper
destination.
10

• Message Warehousing
• Message Warehouse is a database that is able to
  store messages that flow through the message
  broker (Fig 18.6). Provision of message
  persistence is facilitated to meet several
  requirements
• Message Mining
• Capability for extraction of business data to
  support decisions
• Message Integrity
• Warehouse provides persistent state for message
  traffic
• Message archiving
• Capability to store messages for archiving/other
  purposes
• Auditing
• Enabling to diagnose the health of the EAI
  solution through analysis of message traffic

11

• Fig. 18.6 Message Warehousing

12

• Repository Services
• Repository is a database of information about
  source and target applications, which may include
  data elements, inputs, processes, outputs, and
  the inter-relationships among applications. The
  goal is to keep track of sophisticated
  information about source/target applications
  (e.g. metadata, schemas, security and ownership
  perms).
• In essence, repository is a master directory for
  the entire EAI domain, and will ultimately become
  the enterprise metadata repository, able to track
  all systems and databases connected to the
  Message Broker.
• Directory Services
• Directory Services provide a single point of
  entry for applications and middleware, utilizing
  naming service standards. They are nothing more
  than a way to classify resources on the network
  in a way consistent with any other method of
  classification (e.g. DNS, X.500, etc.).

13

• Adapters
• Adapters are layers between the message broker
  interface and the source or target application
  for example, a set of libraries that map the
  differences between two distinct interfaces the
  message broker interface and the native interface
  of the source or target application and that
  hide the complexities of that interface from the
  end user or even from the EAI developer using the
  message broker.
• For example, broker may have adapters for
  applications (SAP, Baan, PeopleSoft), databases
  (Oracle, DB2, Sybase) and even to specific brands
  of middleware. They are also getting smarter
  with intelligence placed in adapters running on
  source/target systems for better event capture.
• There are two types of adapters in the world of
  message brokers thin adapters and thick
  adapters and adapters may behave in two
  different ways dynamic and static.

14

• Thin Adapters
• They are simple API wrappers, or binders, that
  map the interface of the source or target system
  to a common interface supported by the message
  broker (Fig. 18.7). Their advantage is simplicity
  of implementation and greater granular control.
• Disadvantage is impact of performance without
  increase of functionality, and considerable
  programming effort is required thin adapters
  are nothing more then another interface software.

Fig. 18.7 Thin Adapter
15

• Thick Adapters
• In contract, these provide a lot of software and
  functionality between the message broker
  infrastructure and the source or target
  applications (Fig. 18.8). Because the abstraction
  layer and the manager manage the difference
  between all the applications requiring
  integration, there is almost no need for
  programming.
• The user sees only a business-like representation
  of the process and the metadata information as
  managed by the abstraction layer and the adapter.
• Although Thick Adapters are very complex to
  develop, greater complexity of EAI will ensure
  further sophistication, requiring no programming
  and providing as easy, business-like method to
  view integration of the enterprise Thick
  Adapters are the future.

Fig. 18.8 Thick Adapters place an abstraction
layer on top of the application interface.
16

• Static and Dynamic Adapters
• Static Adapters are the most common today, have
  to be manually coded with whats contained within
  the source and the target systems (e.g. they must
  be configured by hand to receive data form a
  source schema).
• Dynamic Adapters are able to learn about the
  source/target systems connected to them through a
  discovery process that the adapters go through
  when first connected to the application or data
  source. Equally important they can re-learn
  if changes occur.
• Using the API
• In addition to adapters, message brokers leverage
  APIs as a mechanism to access the services of
  source/target systems. It is similar to
  traditional message-oriented middleware, however
  broker could integrate multiple applications,
  while MOM is primarily for linking applications
  in one-to-one fashion.

17

• Topologies
• Message Brokers use a hub-and-spoke topology. As
  a hub, Broker rests between the source and target
  applications in topology of a star (Fig. 18.9).
  While this topology considered traditional,
  others include bus and multi-hub.
• In the bus configuration, the broker sits on the
  network bus and provides services to other
  systems connected to the bus (Fig. 18.10). Such
  topology is effective when brokers play smaller
  role within EAI.

Fig. 18.9 Hub-and-Spoke Configuration
18
Fig. 18.10 Bus Configuration
Multi-Hub Configuration a number of brokers are
linked, with source and target applications
linked to any of the brokers in the topology
(Fig. 18.11). The advantage is ability to scale,
making it possible to integrate a virtually
unlimited number of source and target
applications.
19
Fig. 18.11 Multi-Hub Configuration
20

• As message brokers mature and become more
  sophisticated, there is a clear movement away
  from the simple hub-and-spoke configuration
  toward the multi-hub configuration. Issues must
  be resolved in orchestrating load-balancing and
  implementing fail-safe mechanisms within the
  multi-hub topology.
• Future of EAI and Brokers
• Vendors need to standardize core features as
  include layers of value added software, with
  thin adapters giving way to thick. There is drive
  to hide the complexity behind abstraction layers
  and provide business face.
• Given this reality, message brokers may turn out
  to be more about business automation than
  middleware.