Vehicle Recognition in Cluttered Environments

1
Vehicle Recognition in Cluttered Environments

• Masters Thesis Defense
• By Gerald Dalley
• Signal Analysis and Machine Perception Laboratory
• The Ohio State University
• 05 June 2002

2
Overview

• Problem Statement and Motivation
• Recognition Steps
• Range Image Generation
• Local Surface Estimation and Decimation
• Global Surface Reconstruction
• Surface Segmentation
• Graph Matching
• Conclusions and Future Work
• Questions

3
Problem Statement and Motivation

• Problem
• Recognize vehicles
• Military and civilian
• Forested environment
• Motivation
• Hostile forces tend to hide
• Camouflage and occlusion foil the human visual
  system

4
Range Image GenerationOverview

• Objects modeled
• Clutter models
• Camera flight paths (scenes)
• Noise generation

Range Image Generation
Local Surface Fitting
Surface Reconstruction
Surface Segmentation
Graph Matching
5
Range Image Generation Objects Modeled
6
Range Image Generation Clutter Models

• Visually realistic trees
• Look good, but
• Poor occlusion
• Very long runtimes

7
Range Image Generation Camera Flight Paths
(Scenes)
flyby
circle
unoccluded
8
Range Image Generation Noise Generation

• Isotropic additive Gaussian noise
• Standard deviations of
• 0mm
• 2mm
• 4mm
• 8mm
• 16mm
• 32mm

9
Local Surface Estimation and DecimationOverview

• Assumption Vehicles are composed primarily of
  large, low-order, low-curvature surfaces.
• Constraint 10 tank views ? more than 220,000
  range points (too many)
• Point Selection (Decimation)
• Principle Component Analysis
• Biquadratic Surface Fits

Range Image Generation
Local Surface Estimation
Surface Reconstruction
Surface Segmentation
Graph Matching
10
Local Surface Estimation and Decimation Point
Selection

• Method 1
• Randomly select 1 (for example) of the original
  points
• Make local surface estimates based on selected
  points
• Problems

No noise
s 0.3
Fit errors away from corner
Fit errors due to noise
Fit errors due to the corner
Fit errors due to the corner
11
Local Surface Estimation and Decimation Point
Selection (contd.)

• Method 2 In the region of interest
• Collect range image points into cubic voxel bins
  (128x128x128mm)
• Discard bins that have
• Too few points
• Points that do not represent biquadratic surfaces
  well
• Retain only the centroids of the bins and their
  surface fits

12
Local Surface Estimation and Decimation
Principle Component Analysis
w
u
13
Local Surface Estimation and Decimation
Biquadratic Surface Fits
pi
14
Global Surface ReconstructionOverview

• Motivations
• Post-Processing

Range Image Generation
Local Surface Fitting
Surface Reconstruction
Surface Segmentation
Graph Matching
15
Global Surface ReconstructionMotivations

• Easy, unambiguous nearest-neighbor identification
• Fast searches over small cardinality
• Makes rendering easier
• Avoids incorrect groupings of nearby surfaces

16
Global Surface ReconstructionPost-Processing
17
Surface SegmentationOverview

• Motivation Correspondence is hard
• Some Techniques Not Used
• Spectral Clustering
• An overview
• Normalized cuts
• Our affinity measure
• Results

Range Image Generation
Local Surface Fitting
Surface Reconstruction
Surface Segmentation
Graph Matching
18
Surface SegmentationSome Techniques Not Used
Robust Sequential Estimators (Mirza)
Regions of Constant Curvature (Srikantiah)
19
Surface SegmentationOverview of Spectral
Clustering

• Two surface points have an affinity

y1, where Ayili yi
Aij
20
Surface SegmentationNormalized Cuts
21
Surface SegmentationOur Affinity Measure
22
Surface SegmentationUnoccluded Results
23
Surface SegmentationWhich Objects Are These?
24
Graph MatchingOverview

• Match tree example
• Error measures
• Entropy
• Results
• What caused problems?

Range Image Generation
Local Surface Fitting
Surface Reconstruction
Surface Segmentation
Graph Matching
25
Graph MatchingMatch Tree Example
26
Graph MatchingError Measures

• Unary Error
• Area, Elongation, Thickness
• Orientation Error
• How poorly pairs of normals match up
• Centroid Distance Error
• How poorly pairs of centroids match up
• Cumulative Area Error
• What percentage of the model area is not matched
  up

27
Graph MatchingEntropy
28
Graph MatchingResults earthmover
29
Graph MatchingResults obj1
30
Graph MatchingResults sedan
31
Graph MatchingResults semi
32
Graph MatchingResults tank
33
Graph MatchingWhat Caused Problems?

• 19 total incorrect recognition results
• 12 over-segmentation
• 10 area errors (including non-existent segments)

34
Graph MatchingWhat Caused Problems? (contd.)

• 4 mis-aligned segmentation

35
Conclusions

• System features
• Modular design
• Handles pessimistic levels of clutter
• 100 recognition on earthmover and sedan
• Reliable segmentation is important when doing
  graph matching

36
Future Work

• Articulation
• Larger modelbase
• Iterative recognition
• Alternative segmentation methods
• Other affinity matrix normalizations
• Tensor voting
• Enhanced version of Srikantiahs algorithm
• Verification
• Alternative recognizers (e.g. SAI)
• E3D! (hopefully, for the remaining SAMPL crowd)

37
Your Questions...
38
EXTRA SLIDES
39
Range Image Generation Ray Tracing
xs
40
Global Surface Reconstruction PreliminariesVoro
noi Diagrams

• Voronoi cell locus of points closer to a given
  sample point than any other point

41
Global Surface Reconstruction PreliminariesMedi
al Axis

• Medial axis locus of points equidistant from at
  least two surface points (considering the
  original surface)

42
Global Surface Reconstruction Preliminaries
e-sampling

• e-sampling Samples are at most e times the
  distance to the medial axis

43
Global Surface ReconstructionCocone
p

• p pole of p point in the Voronoi cell
  farthest from p
• e lt 0.06 ?
• the vector from p to p is within p/8 of the true
  surface normal
• The surface is nearly flat within the cell

p
Voronoi cell of p
44
Surface SegmentationNormalized Cuts
45
Surface SegmentationProbabilistic Affinity
Framework
46
Surface SegmentationProbabilistic Position
Affinity
47
Surface SegmentationProbabilistic Position
Affinity
48
Surface SegmentationProbabilistic Normal
Affinity
49
Surface SegmentationProbabilistic Normal
Affinity
50
Graph MatchingError Measures
51
Graph MatchingError Measures (contd.)
52
Graph MatchingError Measures (contd.)