Collaborative Adaptive Sensing of the Atmosphere

1
Collaborative Adaptive Sensing of the Atmosphere

• Jim Kurose
• Prashant Shenoy
• Department of Computer Science
• Center for Collaborative Adaptive Sensing of the
  Atmosphere (CASA)
• University of Massachusetts
• Amherst MA 01003

IBM Research May 2004
2
Overview

• spectrum of sensor networks
• CASA Collaborative Adaptive Sensing of the
  Atmosphere
• when power is not a constraint?
• research challenges
• similarities, differences with power-
  constrained sensor nets
• architecture the big picture

3
Sensor nets wide range of characteristics

• power constrained or plugged in?
• data rate bit rate, duty cycle?
• reconfigurability retasking, retargeting how
  often?
• users single-purpose, many?
• in-situ or remote

4
Wide range of sensor nets embedded

• power constrained sensors
• mostly data push, re-tasking possible
• low bit rate data
• network design from scratch

5
Embedded Networked Sensing Apps

• embedded micro-sensors, on-board processing,
  wireless interfaces at very small scale
• in-situ sensing need to be there, monitor up
  close
• spatially, temporally dense environmental
  monitoring

Slide courtesy of D. Estrin
6
Wide range of sensor nets

• powered radars
• rapidly steerable beams
• data rates 2 Mbps - 100 Mbps per radar
• multiple data consumers
• network design space above IP?

7
Wide range of sensor nets
habitat monitoring
microclimate monitoring
animal tracking
vehicle tracking in sensor field
radar/weather
video surveillance
auto traffic monitoring
satellite observation (EODIS)
network traffic monitoring
8
In spite of differences much in common
retrieve stored data

• Data intermediary
• requests
• storage
• dist./centralized

• Driving application(s )
• ingest data
• request new data

sensor data
"Every problem in computer science can be solved
by adding another level of indirection" -- B.
Lampson
control
request new/future data
9
CASA collaborative adaptive sensing of the
atmosphere

• dense network of low power radars
• overcome blockage sense lower 3 km of earths
  atmosphere
• collaborating radars
• improved sensing
• improved detection, prediction
• responsive to multiple end-user needs

Todays meteorological radars
gap
Sample atmosphere when and where end-user needs
are greatest
10
CASA Collaborative, Adaptive

• collaborative
• improved sensing
• improved detection, prediction

11
CASA Collaborative, Adaptive

• collaborative
• improved sensing
• improved detection, prediction
• adaptive
• changing environment, user needs
• sensing plus actuation

12
Whats needed to solve this problem?
NSF Engineering Research Center Sept. 2003
Remote sensing Microwave engineering Networking Re
al-time systems Numerical prediction Emergency
management Radar meteorology Quantitative
inversion Climate studies Social impact Antenna
design
expertise
working together
core partners
13
Industry, government collaborators
14
CASA three generations, 10 year vision
Fleets Opportunistic Deployment
Clear Pre-Storm Environment
NetRad - Storms
1 2 3 4 5
6 7 8 9 10
Project Year
15
NetRad infrastructure

• Radar site
• power, battery backup
• currently T1 to DS3
• local computation, 1TB storage

16
Netrad requirements and architecture
data consumers emergency response, NWS, FAA,
meteorological data companies
data store intermediary
sensors radar
stored data
Per radar 1 Mbps (moment) 100 Mbps (raw)
radar data

• Meteorological
• detection
• prediction
• QPF

Real-time 30 second cycle
env state
policy
control

• resource control
• sensor targeting
• communication

new/future data
17
Challenge sensor (resource) scheduling

• multiple users
• different sensing needs (e.g., beam targeting)
• different utility
• policy
• environment (SNR) impacts sensing ability

• Meteorological
• tornado detection
• prediction
• QPF

env. state
policy

• resource control
• sensor targeting
• communication

18
Challenge sensor (resource) scheduling

• problem schedule/target beams to maximize
  utility
• commonality multimodal sensor configuration
• differences power, environment state, policy,
  sensor
• 
  characteristics

• Meteorological
• detection
• prediction
• QPF

env. state
policy

• resource control
• sensor targeting
• communication

Video surveillance networks similar?
19
Challenge congestion control, routing

• traffic carried over quasi-public network
• congestion
• priorities among sensor net flows
• differential service using endpoint control
• fairness wrt external flows
• routing overlay exploits alternate paths

20
Challenge congestion control, routing

• commonalities
• multi-path routing
• link nondeterminism
• flow priorities
• differences
• overlay versus underlay
• sharing with external users fairness

21
Challenge data errors, compression

• application-specific data reliability semantics
• beyond point-point (link, transport) reliability
  epidemic, multipath transfers facilitates
  reliability at application level
• missing data interpolated, cached locally?
• outlier detection removes corrupted data?
• reliable control
• (distributed) data compression

Many commonalities among all classes of sensor
nets
22
Challenge robustness, management

• Monitoring, management
• lesson SNMP followed IP by 8 years
• goal integration into sensor nets from day 1
• individual node management, reconfigurability
• security

• Robustness more important than raw performance?
• understand failure modes
• failure recovery
• fault tolerance
• storing state, initiating recovery

Many commonalities among all classes of sensor
nets
23
Challenge Leveraging Existing Systems
Infrastructure

• NSSL WDSS II (NEXRAD meteorological software)
• data formats NetCDF, NEXRAD Level II
• architecture NOAA Open Radar Data Acquisition
• Globus/grid
• managing and monitoring grid resources
• optimizing resource allocation
• real-time aspects

24
Challenge mixing in-situ and remote sensing

• mixing in-situ, remote sensing
• mix power-constrained powered
• dumb vs smart deployable sensors
• need to interconnect or scale sensor nets?
• internet analogy?
• how widely?
• how big?

in-situ
sensing
remote
sensing
25
Data Storage Issues

• Sensors produce a stream of data
• Weather sensors sequence of weather readings
• Video sensors sequence of frames
• Network measurement nodes packet headers
• Sensor data processed in real-time and archived
• Systems Issues
• What storage system is appropriate in sensor
  environments?
• What mechanisms are needed for real-time
  processing?

26
Technology Trends

• Each sensor has storage and communication
  capabilities
• Where should sensor data be stored?
• Observation storage is cheap, communication is
  not
• True for a variety of environments
• motes, weather sensors
• Store data locally whenever possible, transfer
  only when needed (or in the background)

27
Data Access Characteristics

• Sensors produce a stream of data that is archived
• Writes are append-only (append to a trace/log)
• Written data is structured (record-like)
• Writes (records) may be immutable
• Example archival of packet header logs
• Queries read data from archival storage
• Show me all instances when rainfall gt 1inch/hour
• What fraction of traffic was P2P traffic in the
  past 6 hours?
• When and where was an zebra seen in the past
  week?
• Reads on archival storage are random
• Reads will need to access data archived at
  multiple sensors

28
Existing Systems Inadequate

• Two possible approaches distributed file systems
  or distributed databases
• File systems
• No good support for random reads (need additional
  index structures)
• Writes log-structured file system is a
  possibility, but no support for record-like
  structures
• Databases too heavyweight
• RDBMs do not support stream processing stream
  databases do not support archival
• DB index structures not suitable for this
  environment

29
Sensor Storage Requirements

• Distributed data store
• Aggregate local storage at sensors into one
  logical store
• Support record-like structures (access records)
• Efficient random reads
• Local writes and mostly local index updates
• Index supports lightweight queries
• Distributed reads

30
Data Research Challenges

• What abstractions do sensor applications need
  from the store?
• How to design a system that supports local
  writes and global reads?
• What index structures are appropriate?
• Distributed indexing
• High volume updates writes more frequent that
  reads

Distributed search tree
31
Architecture stovepipes or layers?
habitat sensing net
atmospheric sensing nets
32
Architecture stovepipes or layers?
applications
habitat sensing
atmosp. sensing
geo sensing
habitat sensing net
physical
atmospheric sensing nets
33
Architecture stovepipes or layers?
applications
habitat sensing
atmosp. sensing
geo sensing
habitat sensing net
physical
atmospheric sensing nets
reusable components, e.g., x-kernel?
34
Summary

• spectrum of sensor networks
• what are the challenges when a sensor can be
  plugged in?
• end-user driven applications rule
• resource allocation
• protocols congestion, routing, data handling
• manageability, robustness
• data storage/querying
• unifying long term architecture?

Slides available at http//gaia.cs.umass.edu/kuros
e/talks