Multicast

1
Multicast Publish - Subscribe used to build
Event Notification on IP

• How to make the Internet work better.
• John Mathon
• TIBCO Software Inc

2
Who is TIBCO?

• Pioneer in Middleware, providing
  publish/subscribe technology since 1987
• 44th largest independent software company in 1995
  Software Magazine Software 100
• Market leader in industrial strength middleware
  for distributed computing
• Proven in the largest deployments

3
The Active Enterprise Publish Subscribe WAN
Architecture
Sales NT Server
Inventory SAP
ERP SAP
Customers
Publisher Applications Databases Devices
Business Partners

• Subscribers
• Browsers
• Applications
• Databases
• Devices

Marketing Notes / DBMS
Devices
4
The Information Bus
Oracle Informix Sybase Ingres
Fax, Email, Pager
S.W.I.F.T. SAP EDIFACT
Java ActiveX
CORBA COM
Legacy
Adapters
TIB Technologies
Infrastructure
..
OS/400
Windows
NT
Unix
OS/2
VMS
MVS
Monitoring Control
5
Information Bus

• Subject Based - Highly Scalable
• Location Transparency
• Meta-Data - Model Driven - XML
• Extensive Event Services - Routers, Initial
  Value, Historical Services, Entitlements,
  Accounting, Queuing, Load Balancing, Message
  Broker, Adapters, Application Monitoring, GUI
  Building Tools

6
The Tower of Middleware

• Challenges
• Integration
• Scalability
• Event-Driven
• Ability to Change/Flexibility
• Open
• Cost of Ownership
• Ease of Implementation

TIBCOs Investment 250M over 12
years Installed Base 2B worth of Integrated
Middleware
7
Representative Alliances
Network
Desktop
Integrators
E-commerce
Database
Hardware
Microsoft, Pointcast, BackWeb,and 11 others
VeriSign, CyberCash SUN
IBM, PW, Andersen
Hewlett Packard
Cisco, 3Com
Oracle, Informix
8
Multicast Protocols

• Unreliable Multicast - Available Today
• Reliable Multicast (PGM) - Proposed to IETF
• SBA - Subject Based Addressing - Available Today,
  not public domain.
• Market Research/Voting - Theoretical.

9
Multicast
Internet Group Membership Protocol (routers and
hosts) Anybody Listening to Group XYZ? Im
Listening to Group XYZ
Multicast Routing Protocol (among routers) Forms
a loop-free set of trees to interconnect all
members of a group Dynamically adjusts trees as
hosts join and leave a group Use tunnels to
circumvent non-multicast areas
10
Un-Reliable (Standard) Multicast

• Optimistic - Needs applications which tolerate
  high error rates, high latency corrections
• Problematic - At least 1 error rate in the best
  scenarios.
• Error Correction Unsystematic - Proprietary
  Solutions
• Uncontrolled streams of Data - High-Volume
  unsegmented bands of data. - Can overwhelm
  receiver.
• Useful for Streaming Audio, Video, Radio/TV like
  Applications.
• Difficult Addressing Content - Unclear Authority
  on Naming
• No Security, I.e. Authentication of Publishers
• No Guarantee of Order of messages

11
Reliable Multicast (PGM)

• Good - Updates Thousands of Subscribers in a
  fraction of a second.
• Good - Optimistic Algorithm - Proven in
  Industrial Environments.
• Good - Useful for More Data Oriented Applications
• Good - Guarantee of Order of messages from a
  single publisher

• Bad - Increased Latency over Unreliable
  Multicast.
• Bad - Uncontrolled streams of Data - High-Volume
  unsegmented bands of data. - Can overwhelm
  receiver..
• Bad - Difficult Addressing Content - Unclear
  Authority on Naming
• Bad - No Security, I.e. Authentication of
  Publishers

12
Subject Based Addressing

• Controlling Streams of Information - You dont
  have to take the whole banana. Just have a bite!
• More Efficient to User - Uses only the bandwidth
  required for what the user needs.
• Creates a Taxonomy of Fine Grained Information -
  Provides a naming scheme for information that
  makes it easier for the user to find information.
  Better than Multicast numeric addresses. (I.e.
  Ala Internic)
• Implements Authentication and Security
• More efficient usage of Multicast addresses.

13
Reliable Multicast and SBA

• Proven in Mission Critical Applications
• Advantageous in almost all circumstances
• Best for High Bandwidth / Large User Populations
  where some of the information is of interest to
  some of the users. I.e. N to N.
• But is usually as good as 1 to N, or 1 to 1 or
  even N to 1 implemented with Point-to-point or
  pure broadcast protocols.

14
Cost/Benefit Tradeoffs

• Where does multicast pay off vs repetitive
  polling or point-to-point push?
• Where does SBA pay off vs repetitive polling?
• Where does SBA/Multicast pay off vs repetitive
  polling?

15
Server Load

• Point to Point Load Increases Geometrically with
  of Publisher Messages and of Receivers
• Multicast load is linear with of changes
  (Publishes)
• Problem - What if everybody isnt online at the
  time of the updates?
• Problem - What if everybody isnt interested in
  all the content being broadcast?
• Advantage - Big Savings for many applications

16
One Message Updates Many Users, Caches, Proxies,
and Search Engines
Browser
Content
User
Cache/Proxy
Search Engine
User
17
Current New Approach

• Caches periodically query servers for new
  updates.
• Search Engines periodically go out and search and
  index new information.
• Hidden Content Missing

• Single Publish by Content source reaches all
  concerned caches and search engines.
• Search Engines more up to date.
• More efficient for publisher.

18
Flow Rates - Assume 1,000,000 sites changing on
average of once a day. Using SBA and PGM.

• Search Engines
• 1,000,000 messages a day which is about a tenth
  of what a typical trading room sees on a slow
  day.
• Caches, Proxies
• 1,000 pages cached, then 1000 messages a day.
  Also Trivial.

• Users Caches and Browsers.
• A few messages a day, typically when viewing
  highly active sites with constantly changing
  content.

19
Advantages

• Users
• UP TO DATE CURRENT CONTENT. WIN!
• Content Providers
• Reduced Load on Servers from Polls from Search
  Engines, Caches and Users.
• BIG WIN!!!
• Developers - Much simpler to build push
  applications.

• Search Engines
• No More Searching the Internet - or much more
  limited crawling and comparing. BIG WIN!
• Content more up to date, more comprehensive. BIG
  WIN!
• Proxies, Caches
• No polling required, but need to process updates.

20
One Message Queries Multiple Search Engines

• Publish Query
• and topic

• Subscribe to all queries

Search Engine(s)
User
Cache/Proxy
Local Search Engine
User

• Subscribe to limited set of topics

Hidden/Value Added content Search Engine(s)
21
Simple Market Research Protocol (SMRP)

• As Data is Broadcast, counts are kept and
  returned to distributor
• Anonymity. Only counts are returned.
• Advertisement is sent and an accurate count of
  number of impressions delivered
• Proxies/caches can extend MRP protocol where
  cacheing is performed

22
Market Research

• Delivered 4 impressions

Browser

• Delivered 25 impressions

Content

• Delivered 25 impressions

• Delivered 20 impressions

User
Cache/Proxy

• Delivered 1 impressions

Search Engine
User
23
Lessons Learned

• Everything is an Event - No special messages
• Multicast and SBA are essential to make even
  small systems perform.
• Internet Scale requires a non-centralised
  solution, I.e. TCP routing as an example.
• Multicast and SBA really work.

24
Lessons Learned

• Fine Tune the PGM algorithm and SBA algorithms.
  See how TCP routing has improved over the years.
• Dont just standardize the protocol but work on
  data representation issues as well.
• Build assuming events are prolific and diverse.