Vehicle Segmentation and Tracking From a LowAngle OffAxis Camera

1
Vehicle Segmentation and Tracking From a
Low-Angle Off-Axis Camera
Neeraj K. Kanhere
Committee members Dr. Stanley Birchfield Dr.
Robert Schalkoff Dr. Wayne Sarasua
Clemson University
July 14th 2005
2
Vehicle Tracking
Why detect and track vehicles ?

• Intelligent Transportation Systems (ITS)
• Data collection for transportation engineering
  applications
• Because it's a challanging problem!

Loop detectors
Tracking using Vision

• Low per unit cost
• Field experience

• No traffic disruption
• Wide area detection
• Rich in information

• No tracking
• Maintenance difficult

• Computationally demanding
• Expensive

3
Available Commercial Systems

• AUTOSCOPE (Image Sensing Systems)
• Has been around for more than a decade
• Dedicated Hardware
• Reliable operation
• Good accuracy with favorable camera placement

• VANTAGE (Iteris)
• New in market
• Accuracy has been found to be lower than
  Autoscope

4
Related Research

• Region/Contour Based
• Computationally efficient
• Good results when vehicles are well separated
• 3D Model Based
• Large number of models needed for different
  vehicle types
• Limited experimental results
• Markov Random Field
• Good results on low angle sequences
• Accuracy drops by 50 when sequence is processed
  in true order

• Feature Tracking Based
• Handles partial occlusions
• Good accuracy for free flowing as well as
  congested traffic conditions

5
Factors To be Considered
High angle
Low angle

• More depth variation
• Occlusions
• A difficult problem

• Planar motion assumption
• Well-separated vehicles
• Relatively easy

6
Overview of the Approach
Offline Calibration
Background model
Frame-Block 1
Feature Tracking
Estimation of 3-D Location
Frame-Block 2
Frame-Block 3
Grouping
Counts, Speeds and Classification
Block Correspondence and Post Processing
7
Processing a Frame-Block
Overlap
Block n
Block n1

• Multiple frames are needed for motion information
• Tradeoff between number of features and amount of
  motion
• Typically 5-15 frames yield good results

features in block
frames in block
8
Background Model
Time Domain Median filtering

• For each pixel, values observed over time
• Median value among observations
• Simple and effective for the sequences considered
• Adaptive algorithm required for long term modeling

9
Frame Differencing

• Partially occluded vehicles appear as single blob
• Effectively segments well-separated vehicles
• Goal is to get filled connected components

10

Offline Calibration
Control points

• Required for estimation of world coordinates
• Provides geometric information about the scene
• Involves estimating 11 unknown parameters
• Needs atleast six world-image correspondances

11
Calibration Process

• Using scene features to estimate correspondences
• Standard lane width (e.g. 12 feet on an
  Interstate)
• Vehicle class dimensions (truck length of 70
  feet)
• Relies on human judgment and prone to errors
• Approximate calibration is good enough

12
Estimation using Single Frame

• Box-model for vehicles
• Road projection using foreground mask
• Works for orthogonal surfaces

camera
vehicle
Road plane
13
Selecting Stable Features

• Shadows, partial occlusions will result into
  wrong estimates
• Planar motion assumption is violated more for
  features higher up
• Select stable features, which are closer to road
• Use stable features to re-estimate world
  coordinates of other features

14
Estimation Using Motion

• Estimate coordinates with respect to each stable
  feature
• Choose coordinates which minimized weighted sum
  of euclidean distance and trajectory error

• Rigid body under translation
• Estimate coordinates with respect to each stable
  feature
• Select the coordinates minimizing weighted sum of
  Euclidean distance and trajectory error

• Coordinates of P are unknown
• Coordinates of Q are known
• R and H denote backprojections
• 0 first frame of the block
• t last frame of the block
• ? Translation of corresponding point

15
Affinity Matrix

• Each element represents the similarity between
  corresponding features
• Three quantities contribute to the affinity
  matrix
• Euclidean distance (AD), Trajectory Error (AE)
  and Background- Content (AB)

• Normalized Cut is used for segmentation
• Number of Cuts is not known

16
Incremental Cuts

• We apply normalized cut to initial A with
  increasing number of cuts
• For each successive cut, segmented groups are
  analyzed till valid groups are found
• Valid Group meeting dimensional criteria
• Elements corresponding to valid groups are
  removed from A and process repeated starting from
  single cut

Avoids specifying a threshold for the number of
cuts
17
Correspondence Over Blocks

• Formulated as a problem of finding maximum wieght
  graph
• Nodes represent segmented groups
• Edge weights represent number features common
  over two blocks

an groups in block N bn groups in block N1
18
Results
19
Results
20
Conclusion

• A novel approach based on feature point tracking
• Key part of the technique is estimation of 3-D
  world coordinates
• Results demonstrate the ability to correctly
  segment vehicles under severe partial occlusions

Future Work

• Handling shadows explicitly
• Improving processing speed
• Robust block-correspondance

21
Questions ?
22
Thank You !