Messages in Paths

1
Messages in Paths

• Ninja Meeting
• 2/15/2000
• Sam Madden

2
Telegraph Extensions

• Message (Tuple) Based Transmission Layer

3
Telegraph Extensions

• Message (Tuple) Based Transmission Layer
• Aggregators/Broadcasters

4
Telegraph Extensions

• Message (Tuple) Based Transmission Layer
• Aggregators/Broadcasters
• Operator Typing System
• Interoperability Constraints

5
Telegraph Extensions

• Message (Tuple) Based Transmission Layer
• Aggregators/Broadcasters
• Operator Typing System
• Interoperability Constraints
• Mechanisms for Performance Evaluation

6
Telegraph Extensions

• Message (Tuple) Based Transmission Layer
• Aggregators/Broadcasters
• Operator Typing System
• Interoperability Constraints
• Mechanisms for Performance Evaluation

7
Messages vs. Streams
Streams
Messages

• Partitioned Data

• Unpartitioned Data

8
Messages vs. Streams
Streams
Messages

• Partitioned Data
• Push or Pull Data

• Unpartitioned Data
• Push out data

9
Messages vs. Streams
Streams
Messages

• Partitioned Data
• Push or Pull Data
• Higher Overhead

• Unpartitioned Data
• Push out data

10
Messages vs. Streams
Streams
Messages

• Partitioned Data
• Push or Pull Data
• Higher Overhead
• More Complicated Programming Model

• Unpartitioned Data
• Push out data

11
Messages vs. Streams
Streams
Messages

• Partitioned Data
• Push or Pull Data
• Higher Overhead
• More Complicated Programming Model

• Unpartitioned Data
• Push out data
• No
• Type Checking
• Admission Control
• Aggregation
• End-to-End Guarantees

12
Benefits of Messages

• Aggregation
• Improved End-To-End Guarantees
• Structural Type Checking (not Semantics)
• Admission Control
• Pull-based consumption
• Natural model for databases, Web

13
Architecture

• Messages are tuples
• ltIdenitifer id, int Length, Byte datagt
• Identifier may encapsulate semantic information,
  sequence number, etc.

14
Architecture

• Messages are tuples
• ltIdenitifer id, int Length, Byte datagt
• Identifier may encapsulate semantic information,
  sequence number, etc.
• Operators Specify Multiple Inputs
• ltIdentifier id, Class Cgt,
• Incoming messages are packaged as type C via
  (de)serialization methods.

15
Architecture 2

• Class C may be unstructured (e.g. String)
• Low overhead
• Structured classes allow
• Data verification
• Admission Control

16
Architecture 3

• Operators can request tuples from an input or
  push tuples down an output.
• Generates events on predceding/following node

17
Architecture 3

• Operators can request tuples from an input or
  push tuples down an output.
• Generates events on predceding/following node
• Events model implies VSpace

18
Architecture 3

• Operators can request tuples from an input or
  push tuples down an output.
• Generates events on predceding/following node
• Events model implies VSpace
• Maybe?

19
On Top Of Streams

• Streams (not RMI) are underlying transmission
  protocol

20
On Top Of Streams

• Streams (not RMI) are underlying transmission
  protocol
• Some operators still stream-based

21
On Top Of Streams

• Streams (not RMI) are underlying transmission
  protocol
• Some operators still stream-based
• Streams provide
• Low-level error detection
• Control over threads/operator
• Guaranteed reasonable performance