Title: Multimedia Data Stream Management System
1Multimedia Data Stream Management System
2Outline
- Definition
- Motivating Examples
- Nine Requirements
- Current Systems
- Comparison
- Brief Overview of current Stream Systems
- Preview of My Project
3What is it?
- Stream of multimedia data from a source (video
camera) - Query stored in a system (This query may itself
change - Process high volumes of data in real-time
4Motivating Examples
- Security Surveillance
- Crowd Security
- Air Security
- Burglary
- Baby Sitting
- Traffic Reports
- Science
- Animal behavior
- Ocean
5Reqirement 1 - Process Quickly
- Low latency
- Messages Processed In-Stream
- No Storage to perform operation
- Active System
- Avoid Polling
6Requirement 2 Query using SigmaQL for Streams
(StreamSigmaQL)
- Querying Mechanism
- Based on SQL
- Express Continuous Streams of Data
- Window Construct
- Time
- Frames
- Breakpoints
- Merge Operator
7Requirement 3 Handle Imperfections
- Data might be late delayed, missing, or out-of
sequence - Time out individual calculations or computations
- Challenges with Dealing with out-of-order data
- Mechanism for additional time
8Requirement 4 Generate Predictable Outcomes
- Generate deterministic and repeatable results
- Time-ordered deterministic processing throughout
entire pipeline - Important for fault tolerance and recovery
9Requirement 5 Integrate Stored and Streaming
Data
- Comparing present with past
- Capability to efficiently store, access, and
modify state information
10Requirement 6 Guarantee Data Safety
- Must use a high-availability solution
- Secondary System
- Synchronizes with primary frequently
- Takes over in case of failure
11Requirement 7 Partition and Scale Automatically
- Take advantage of distributed computing
- Support multi-threading
- Takes advantage of multi-processor
- Avoids blocking
- Load Balance across machines
- Automatic process
- Transparent
12Requirement 8 Process and Respond
Instantaneously
- Needs to respond in real time
- Highly optimized, minimal overhead execution path
- All system components have high performance
13Requirement 9 - Adaptability
- Change queries without restarting
- Accept all different types of multimedia streams
- Allow for custom configuration
- Work with different systems
- API
14DBMS
- Widely used
- Use SQL but not equipped for Streams
- Passive
- Do not keep data moving
- Difficult to handle out of order data
- Difficulty with predictable out comes
- Incur latency with seamless integration
15Rule Engine
- Example Prolog
- Active
- Handle imperfections
- Troubles with seamless integration
16Stream Processing Engine
- Handle all the requirements
- Not specifically designed to handle multimedia
constraints - Not Specifically designed to handle streams of
multimedia
17Chart
DBMS Rule Engine SPE MSPE
Keep data moving No Yes Yes Yes
SigmaQL No No No Yes
Handle Imperfections Difficult Possible Possible Yes
Predictable outcome Difficult Possible Possible Yes
High availability Possible Possible Possible Yes
Stored and Streamed data No No Yes Yes
Distribution and scalability Possible POssible Possible Yes
Real time Possible Possible Possible Yes
Adaptability Possible Difficult Possible Yes
18Aurora
- DSMS developed at MIT and Brown
19Aurora Query Network
20Stream Management System
21Simple Query Plan
Q1
Q2
State4
?
State3
?
Scheduler
State1
State2
?
Stream3
Stream1
Stream2
22NiagaraCQ
- Developed at Wisconsin
- First DSMS
- Uses a grouping strategy
- Not as complete as other two
23System Architecture
24TelegraphCQ
- Developed at Berkeley
- Stem storage point
- Eddy route tuples
- Good at handling multiple queries
- Adaptive
25Adaptivity (Telegraph)
Output Queues
STeMs for join
R
grouped filter (R.A)
EDDY
S
grouped filter (S.B)
R x S x T
T
Input Streams
- Runtime Adaptivity
- Multi-query Optimization
- Framework implements arbitrary schemes
26My Project
- Design a multimedia streaming database
- Outline the specifications
- The Scheduling algorithm
- The query structure
- The operators
- Etc.