MAST Distributed

1
MAST Distributed Sensing and Perception Kostas
Daniilidis University of Pennsylvania Universit
y Of Maryland October 15th, 2008
Distribution authorized to DoD and DoD
Contractors Only Critical Technology (March
2007). Other requests for this document Shall be
referred to Director, US Army Research
Laboratory, ATTN AMSRD-ARL-DP-P, Adelphi, MD
20783-1197
2
Distributed Sensing and Perception

• Distributed Inference
• Distributed SLAM
• Visual SLAM
• Partitioning
• Loop closing
• Distributed tracking camera calibration

3
1. Why distributed inference

• Scalability with network size
• Computing power required at each node is fixed
• Bandwidth required between any 2 nodes is fixed
• Robustness
• No node is system critical
• All nodes has full environment information
• Modularity and composability
• A network of modular components rather than a
  single complicated system
• Flexibility in deployment
• A system can be composed from modules based on
  situation and objectives
• Increased performance
• Nodes have global information available locally
• Nodes receive information about states they
  cannot observe

4
1. Our Requirements for Distributed Inference
Algorithms

• Models
• Dynamic
• Mobile platforms, mobile targets
• Mixed discrete continuous
• To describe more interesting/complex phenomena
• Hierarchical
• (next slide)
• Inference
• Asynchronous
• Observations are asynchronous
• Robust
• i.e. graceful degradation of belief quality in
  the face of communication failure Paskin05
• Efficient
• To be reasonably scalable
• No current work addresses all of these

5
1. Hierarchical Modeling

• Advantages
• Represent information on multiple abstraction
  layers
• Representation as well as learning/inference
  algorithms can be selected for each layer of the
  model
• Modular learning and reuse of models
• Better fit for human-network interaction

Transportation routines and destinations Liao04
(bidirectional information flow)
Office activity recognition Tu06 (information
flows bottom-up only)
6
1. Next StepDistributed Hierarchical Tracking

• Better distributed tracking through better
  modeling of, e.g.
• targets themselves
• target/target interaction (group behavior)
• target/environment interaction
• Graphical models are well suited for both
  distributed and hierarchical inference

Update
Prediction
Target Type
Target Context
Target Location
tk
tk1
7
2. Distributed SLAM Markov Random Fields
8
2. Reorder -gt Structure R Changes
R
- Sparser ! - Independent Parts
9
2. R Clique Tree Junction Tree
- R is not symmetric - Extra edges fill
10
2. Distributed SLAMSimulation
ResultsPlayer/Stage Simulation
GreenRobot 1, BlueRobot 2, Yellow common
landmarks
11
3. Visual SLAM

• SLAM with range sensors for indoor mapping is
  now a well studied problem
• Visual SLAM is useful for MAST because it is
  passive sensory modality and needs no
  infrastructure
• Challenges for MAST
• Data association in harsh environments
• Abundance of visual features makes it intractable
• Explosion of metric information
• Solution
• Map segmentation
• Loop closure for partitioning into stable submaps

Poorly tracked feature from specular reflection
on window
Accurately tracked features
Top down view
Robot
12
3. SLAM Structure

• The SLAM problem to simultaneously estimate the
  position of the robot and the landmarks in the
  world
• Uncertainty of landmark positions and the robot
  pose is captured by a covariance matrix.
  Correlations between features are captured by
  off-diagonal elements.
• The structure of the covariance matrix is
  block-diagonal
• Problem How to partition landmarks according to
  block structure?

13
3.1 Segmentation

• Graph cut algorithms used to partition into two
  sets while optimizing network flow
• Normalized graph cuts ltgt inter-set affinity
  maximization
• Widely used for image segmentation
• Can be directly used to partition covariance
  matrix

Images taken from Normalized Cuts and Image
Segmentation, J. Shi, J. Malik, PAMI 2000
14
3.1 Simulation example to demonstrate principle

• Simulated EKF SLAM
• Robot driven in circular pattern to observe
  landmarks
• Wall prevents measurements from crossing
• Partition result easily segments two rooms

15
3.1 Early results on indoor SLAM partitioning
Visual SLAM with simple feature
detectors Uses monocular camera and inverse
depth parameterization Operates in real-time
Video
16
3.2 Loop closing Perceptual aliasing and
variability

• Perceptual Aliasing Many places look the same.

Image similarities between images on a typical
route using a vanilla image retrieval index based
only on appearance
17
3.2 SIFT descriptors coresponding to one visual
word
18
3.2 Geometric Consistency
1. Given two images, Find a set of
correspondences (at least for a part of a
scene) 2. Use number of correspondences as
similarity measure (sensitive to outliers).
19
3.2 No match (false positive) vs match (repeated
traversal)
Similar appearance (horiz axis) in both but
different geometric consistency (vert axis)
Geometry
Appearance
20
3.2 Matched map using geometric consistency
Appearance and geometry
Appearance only
21
3.2 Fab-Map Public Oxford Datasets
New College
City Center

• 1.9 kilometers (2146 images)
• Tests robustness to perceptual aliasing

• 2 kilometers (2474 images)
• Tests robustness to scene changes traffic /
  pedestrians

Cummins Newman, The University of Oxford
22
3.2 Results for Fab-Map Datasets
Geometry enabled loop closing has best performance
City Center
New College
Precision
Precision
Recall
Recall
23
4. Robust Localization of Camera Networks

• Goal to recover orientations and positions of
  all cameras
• Difficulties
• Correspondence under wide baseline
• Point features are inadequate
• Our approach
• Spatio-temporal features
• we used tracks of moving objects as
  spatio-temporal features
• tracks are detected by each camera using
  multi-target tracking
• orientations and positions are computed by
  matching tracks between cameras
• Efficient since many spurious features are
  removed by the tracking algorithm before the
  matching step
• Robust against wide baseline and different
  photometric properties
• It can be applied to mobile robots with cameras
  for self-localization of a fleet of robots for
  solving SLAM

24
4. Experiments Camera Sensor Networks
CITRIC Mote Chen et al. ICDSC 2008
average reprojection error 4.9 pixels
25
4. Fusion-Based Localization

• Fuse video and radio data
• Recovers scaling factors
• Full localization of cameras
• For a clique of size 3, there exists a linear
  solution
• For a general case, a nonlinear approach
• Meingast et al. ICDSC 2008

(R,T)
q
(R,T,)
Full Localization
26
4. Experiments 6 Radio-Camera units
average reprojection error lt 3 pixels.
estimated scaling factors