Streaming Sensor Data

1
Streaming Sensor Data

• Fjord / Sensor Proxy
• Architecture for combining streaming data with
  static data sources
• Streaming (Windowed) Operators
• Multiquery Eddy
• Share memory and processing between stream
  queries
• TinyOS and Telegraph
• Enhancements to TOS for Sensor Data Processing

2
Fjords

• Query-plan like data structure for combining
  streaming (push) and traditional (pull) data
  sources.
• Operators assume non-blocking queue interface
  between each other.
• Queues implement push vs. pull
• Pull from A to B Suspend A, schedule B until
  it produces data. A cannot go forward until B
  produces data.
• Push from B to A A polls, scheduler thread
  invokes B until it produces data. A can process
  other inputs while waiting for B.
• Supports parallelism between operators via
  queues, state machines, and OS (e.g. NIC buffers,
  DMA) in operator transparent way.

3
Fjords (Continued)

• Key Insight Stream-based systems need to
  operate on traditional (pull-based) sources too!
• Example Combine traffic streams with web-based
  accident reports to correlate accidents with
  impact on freeway conditions.
• Existing streaming solutions cannot do this!

4
Fjord Operator Example
Example Zipper Join (similar to band-join,
Dewitt et. al. VLDB '91). Operator agnostic
with respect to data-flow direction on input and
output.
5
Sensor Proxy

• Energy-sensitive database operator
• Buffer sensor tuples and route to multiple user
  queries to hide query load from sensors
• Push aggregation operators into sensors to reduce
  communications load
• Dynamically adjust sample rate based on user
  demand
• Push results into Fjords so that other operators
  don?t block waiting on slow or dead sensors

6
Multiquery Eddy

• Observation Queries over streams apply only to
  now.
• Old and new queries always looking at same point
  in data set
• Idea only allocate on copy of each tuple
  route that copy to all user queries
• Second idea combine operators over the same
  data stream to increase efficiency

7
Multiquery Eddy(Example and Performance)
Tuple Throughput vs Number of Queries
8
TinyOS and Telegraph

• Goal Enhance TinyOS and TinyOS sensors with
  infrastructure to participate in Telegraph
  queries
• Establish a consistent catalog and sampling
  interface across all TinyOS sensors
• Enable selection predicate / aggregation
  push-down into sensors
• Partially distribute query plans across sensor
  networks