Situational Awareness for Fire Fighters (SAFIRE)

1
Situational Awareness for Fire Fighters (SAFIRE)

• Sharad Mehrotra
• Professor of Computer Science

2
SAFIRE Team

• Faculty
• Sharad Mehrotra
• Nalini Venkatasubramanian
• Researchers, Engineers, Programmers
• Naveen Ashish
• Chris Davison
• Dmitri Kalashnikov
• Jay Lickfett

• Imagecat Inc.
• Paul Amyx
• Charlie Huyck
• Ron Eguchi
• Graduate Students
• Subin Anthony
• Stefano Bonetti
• Chiara Chiappini
• Jeffrey Hu
• Dani Massaguer
• Ronen Vaisenberg
• Bo Xing

3
Outline

• Center for Emergency Response Technologies
• SAFIRE Project
• Goals, concept, deliverables, core research areas
• Progress Timeline
• SAFIRE system
• Research
• Testing validation

4
Bren School Center for Emergency Response
Technologies

• Established in 2007 by the School of Information
  and Computer Sciences

5
CERT Mission

• lead research, technology development
  coordination of ongoing projects on role of IT to
  improving emergency response.
• Provide a forum for collaboration between
  academia, industry, and government agencies.

6
CERT Research

• Robust computing, communication, and storage
  infrastructure
• Graceful performance degradation under extreme
  loads, catastrophic failures, surge demands
• Real-time Situational Awareness from multimodal
  inputs to support decision making
• Diversity of information sources, modalities, and
  needs, uncertainty in data, challenge of scale

7
CERT Current Projects

• NSF funded Responsphere Infrastructure
• Funded by NSF through research infrastructure
  program (1.8M, 2004-2010)
• NSF funded ITR-RESCUE
• Funded by NSF through its large ITR program
  (12.5M, 2003-2009)
• DHS funded SAFIRE project
• Funded by DHS through FEMA fire fighter safety
  program (1M, 2008-10).

8
CERT Infrastructure
1/3 of UCI sensing, communicating, storing, and
computing infrastructure with 300 sensors of
10 types, several comms nets 200 indoor net
cameras (with mics), 30 MicaZ motes (with light,
acoustic, acceleration, magnetic, etc sensors),
6 people counters, 6 mobile cameras , RFID
readers, outdoor cameras, gas sensor, mesh
routers, 802.11 (Wi-Fi), Ethernet, power-line
comms, 802.15.4 (Zigbee), mobile sensing
platforms, storing server, computing servers,
PCs, laptops, cellphones, PDAs, etc.
9
CERT Industrial Partners
5G Wireless Broad-ranged IEEE 802.11 networking AMD Compute Servers
Apani Networks Data security at layer 2 Asvaco 1st responder (LAPD), and threat analysis software
Boeing Community Advisory Board Member Canon Visualization equipment SDK
Convera Software partnership Cox Communications Broadcast video delivery
D-Link Camera Equipment and SDK Ether2 Next-generation ethernet
IBM Smart Surveillance Software (S3) and 22 e330 xSeries servers ImageCat, Inc. GIS loss estimation in emergency response
MicrosoftSoftware Printronix RFID Technology
The School Broadcasting Company School based dissemination Vital Data Technology Software partnership
Walker Wireless People-counting technology
10
CERT Government Partners
California Governors Office of Emergency Services California Governors Office of Homeland Security
City of Champaign City of Dana Point
City of Irvine City of Los Angeles
City of Ontario Fire Department City of San Diego
Department of Health and Human Services Centers for Disease Control Lawrence Livermore National Laboratory
Los Angeles County National Science Foundation
Orange County Orange County Fire Authority
U.S. Department of Homeland Security
11
Outline

• Center for Emergency Response Technologies
• SAFIRE Project
• Goals, concept, deliverables, core research areas
• Progress Timeline
• SAFIRE system
• Research
• Testing validation

12
SITUATIONAL AWARENESS FOR FIRE FIGHTERS (SAFIRE)

• Improve the safety of firefighters by providing
  decision makers with greatly improved situational
  awareness during response activities

13
Varying Situational Awareness Needs
Technological Requirements

• 1st responders _at_ crisis site
• dangers, hazmat in vicinity, ingress/egress
  routes, status of equipment, victim location
• Technology Needs information to enable immediate
  action, knowledge of surroundings, role-specific
  and fine granularity data
• IC _at_ crisis site
• Accurate assessment of the ongoing situation,
  response monitoring, and on-site planning
• Technology Needs mechanisms to monitor onsite,
  site-level granularity, information assimilation,
  decision-support
• EOC
• longer-term planning consequence management
• Traffic planning, shelter planning, structural
  health, public health, news/media
• Technology Needs inter-organization data
  sharing, simulations to help predict future,
  coarse level monitoring, decision support,
  ability to understand the big picture.

SAFIRE
14
SAFIRE Concept
15
How Situational Awareness can Help?
Example Scenario Structure Fire Abnormal event
such as flashover detected by sensors. Alert on
FICB causes IC to conduct roll call earlier than
normal. RIC or closest firefighters can be
directed to last known location.
16
SAFIRE Deliverables

• Core technologies for creating SA at IC level
• A prototype end-to-end SA system for IC
• Collect, manage, and transform diverse sensor
  data streams from incident site into actionable
  information.
• Connect to existing info systems (e.g., CAD)
• Intelligent interactive display system
• Real-time situational data, alerting analysis
  capability
• Testing validation of the SA system

17
SAFIRE Core Technology Areas
GIS
External Data Sources
hazmat
occupancy
FICB Visualization
SAFIRE System
Acoustic data
Environmental sensors
Sensor database
Video data
FF physio. location.

• Multimodal Sensing
• Robust Network Infrastructure
• Visualization and User Interfaces
• Sensor stream processing
• Integration of external data sources

18
Sensing Technologies
Polar Heart Rate Module

• SAFIRE has explored variety of sensor
  technologies
• Personnel accountability / health monitoring,
  Incident site / environment / hazard information,
  Resource tracking
• Location sensing
• Despite numerous efforts, effective indoor
  localization has remained a challenge!
• WIFI, UWB, IR, Acoustic, ultrasound, video-based
• Acoustic sensing e.g., first responder
  communication
• Benefit of human cognition

Polar T31 Heart rate strap transmitter
Mote
Environmental sensors Mobile Optical/Acoustic
sensors (D-Link TPZ cameras) Fixed Mote
Infrastructure (dB, Temperature, Light,
Humidity) Personal sensors Pulse Rate
(heartbeat), Acceleration, Temperature, Humidity,
location, , Orientation, Barometer
19
Location Sensing Technologies
Existing infrastructure
Ad-hoc h/w
Wireless Network-based
Inertial
Satellite-based
WiFi
GPS
UWB
speech
GSM
ultrasounds
IR
20
Existing infrastructure
Technology Technique Accuracy/Precision Cost Limits
GPS Microwave time-of-flight lateration 5-15 mt (95-99) Low Not indoors or in densely built areas
GSM (E911) Lateration or Angulation or Scene analysis 150-300 mt (95) Low Cell coverage
Wi-Fi (RADAR, Ekahau PS) Triangulation/ Scene analysis 3-4 mt (50) Low Calibration / fingerprinting cost
21
Ad hoc infrastructure
Technology Technique Accuracy/Precision Cost Limits
IR (Active Badge) Proximity 5-15 mt (95-99) High Sunlight interference, high base station cost
Ultrasounds (Active Bat) Time-of-flight lateration 9 cm (95) High High base station cost
MC-UWB (WPI PPL) Lateration 0.5 mt (99) High Requires outdoor base units, high hw costs
Inertial (WPI PPL) Local computation of location (Enhanced ) Low Fast drifting enhancer for existing systems
22
Localization Framework
Search Space of Possible localization Plans

• An integrative technology-agnostic localization
  framework
• Modeling location components
• Modeling location queries
• Location Queries
• Diverse Types
• Simple queries where am I
• Boolean queries Am I in Bren Hall
• Nearest neighbor queries
• Proximity queries Are Alice Bob within 100
  feet
• Quality requirement
• Granularity room level, building level, ...
• Certainty
• Solve queries using the best combination of
  location components
• Best tradeoff accuracy-precision/cost

Merging results of different components
Based on Lazaridis Mehrotra, SIGMOD 2006
23
Acoustic Sensing for Enhancing SA
Acoustic Capture
Acoustic Analysis
SA Application
Firefighters
Speech
Voice
Amb. Noise

• Type of Acoustic Analysis
• Human Speech Who spoke to whom about what from
  where and when.
• Ambient Sounds explosions, loud sounds,
  screaming, etc.
• Physiological Events cough, gag, excited state
  of speaker, slurring, ..
• Other features too loud, too quiet for too
  long,

23
24
Robust Networking Infrastructure

• Multitude of technologies
• WSN, UWB, mesh networks, DTN
• But .
• Rapidly deployable, self-configuring networks are
  still elusive!
• SAFIRE-Net a robust networking infrastructure
  for crisis site
• Exploit multiple networks that together provide
  connectivity, Exploit mobility when disconnected

25
Fire Incident Command Board (FICB)
Firefighter Status Dashboard
Available GIS layers
Mapping and Localization
Receive and display alert messages.
26
Integrating External Sources through EBox

• EBox -- one-stop information source for
  pre-compiled and evolving situational information
• GIS, floor plans, hazmat locations, hazmat
  descriptions, occupancy, real-time building
  sensors and instrumentation, other external data
  sources

User can select from list of available GIS
layers, documents, or sensor streams Data
automatically ingested by SAFIRE to provide an
integrated view
27
Comparison

• Commercial software offerings
• - Response Net, Virtual Emergency Services
• www.vemergency.com
• - Rapid Responder Prepared Response
• www.preparedresponse.com 
• - Crisis Response/Tactical Mapping Services
  Diligent Business Services
• www.diligentbus.com
• - Tactical Survey Group
• www.tacticalsurvey.com
• SAFIRE and FICB approach
• Exploit existing information and resource vs
  expensive, complex customized information
  gathering
• Richer representation of information
• Deeper analysis capabilities

28
Outline

• Center for Emergency Response Technologies
• SAFIRE Project
• Goals, concept, deliverables, core research areas
• Progress Timeline

29
Formation of the SAFIRE Advisory Board
Fire Agencies

• Advisory Board formed within 1 month of the grant
• Advisory board members involved in all aspects of
  SAFIRE
• Research prioritization
• System development
• Testing validation
• Technology transfer opportunities
• Four CERT Fire Fighter Forum in the past 1 year.

County of Los Angeles F. D. Ron Cabrera, Battalion Chief
Newport Beach F. D. Paul Matheis, Division Chief Chip Duncan, Battalion Chief
City of Ontario F.D. Rob Elwell, Battalion Chief Dave Schuler, Battalion Chief
Orange County Fire Authority Art Nevarez, Battalion Chief Michael Boyle, Battalion Chief Bill Lockhart, Captain Rich Toro, Comm. Supervisor Jonathan Wilby, Safety Officer
Other Agencies and Companies
UC Irvine Police Department
UC Irvine Environmental Health Safety
Deltin Corp.
Tactical Survey Group
Mesh Dynamics
30
Progress Component Level Research

• Sensing - incorporation of multiple new sensors
• CO sensors, localization, speech
• Networking Data collection Infrastructure
• New antenna array for increased coverage,
  multi-network store-and-forward architecture
• Stream management
• ability to incorporate variety of sensors ,
  multimodal sensor archival and retrieval
  functionality
• Visualization
• New functionalities in FICB simplified UI,
  annotations, ebox integration, etc.
• Ebox
• Prototype development, ontologies for resource
  selection, integration of static and dynamic data
  such as sensing infrastructure of buildings

Talk by Dr. Kalashnikov
Talk by Prof. Venkatasubramanian
Talk by Prof. Mehrotra
Illustrated through demo/video
31
Progress SAFIRE End-to-End SA System

• Sept. 2008 -- 1st Prototype
• May 2009 -- Significant redesign to add
  Robustness more dynamic configuration of
  components

32
Progress Testing Validation

• Technology Testing Exercise 16 SEP 08
• Bren Hall Evacuation w/Campus Police Department
  UCI Zone Crew 3
• Networking sensing technologies, 1st gen. FICB.
• Live Burn Exercise 23 FEB 09
• Live Burn with OCFA,LA Fire and Anaheim Fire
• Testing Sensing (human bio-sensing) data
  collection 2nd generation FICB

33
Progress Testing Validation
Talk by Dr. Davison

• Technology Testing Exercise 12 May 09
• Bren Hall Hazmat, multiple casualty incident
• Integrated technology exercise, 3rd gen. FICB.
• Table Top Exercise 15 May 09
• Part of CERT Fire Forum
• Participants LA County Fire, UCI EHS, Newport
  Beach Fire Department, Orange County Fire
  Authority, ImageCat Inc., Raytheon