Background Subtraction

1
Background Subtraction

• Various Methods for Different Inputs

2
Purpose of Background Subtraction

• Reduce problem set for further processing
• Only process part of picture that contains the
  relevant information
• Segment the image into foreground and background
• Add a virtual background

3
Encountered Problems

• Lighting
• Shadows
• Gradual/Sudden illumination changes
• Camouflage
• Moving objects
• Foreground aperture
• Foreground object becomes motionless
• Bootstrapping

4
Lighting and Shadows

• Weight the luminance with other characteristics
• Depth of object
• Region/Frame information
• Adjust the background model with time
• Store a history of previous backgrounds

5
Comparison of Two Techniques

• Wallflower
• Uses three different components
• Pixel, Region, and Frame levels
• Uses many different statistical models to
  anticipate various changes in the background
• Gordon, Darrell, Harville, Woodfill Subtraction
• Two or more cameras to measure distances
• Uses distance to determine foreground and falls
  back on luminance

6
Wallflower Method Pixel Level

• Makes initial judgment whether a pixel is in the
  foreground
• Handles background model adaptation
• Addresses many of the classical problems
• Moved objects
• Time of day
• Camouflage
• Bootstrapping

7
Wallflower Region Frame

• Region level
• Refines the pixel level judgment
• Handles foreground aperture problem
• Frame level
• Sudden frame level change
• Uses previous models to figure out what caused
  the sudden change
• Light switching on/off

8
Results using Wallflower
9
Gordon, et al. Method

• Correctly identifies background depth and color
  when it is represented in a minority of the
  frames
• Addition of range solves many of the classic
  problems
• Shadows
• Bootstrapping
• Foreground object becomes motionless

10
Obtaining Initial Background Model

• Records the (R,G,B,Z) values at each pixel
• Attempts to determine background through the
  observed depth
• Marks a pixel as invalid if there is not enough
  information for the range
• valid pixel range determines whether the pixel
  is in the background, without the aid of the
  (R,G,B) values
• invalid pixel fall back on classic methods for
  background subtraction

11
Gordon, et al. Method (cont.)

• rm is invalid
• ri is valid and smoothly connected to regions
  with valid background data then a foreground
  decision can be made
• Solves the problem of the background being the
  same depth as part of the foreground
• Z-keying methods fail in these cases
• Kanade, Yoshida, Oda, Kano, and Tanaka, A
  Video-Rate Stereo Machine and Its New
  Applications, Computer Vision and Pattern
  Recognition Conference, San Francisco, CA, 1996.

12
Gordon, et al. Method (cont.)

• YValid(Ym) Y gt Ymin
• Shadows have a stronger effect on luminance than
  inter-reflections
• Separate ratio limits for shadows and reflections

13
Problems Using Only Range or Color
14
Which is better?

• Wallflower over Gordon, et al.
• Doesnt require extra cameras to record depth
• Gordon, et al. produces a halo around
  foreground objects
• Gordon, et al. over Wallflower
• Handles more problems
• Tree waving
• Bootstrapping

15
Other Innovative Methods

• Fast, Lighting Independent Background Subtraction
• Advantages
• Light has no basis on the decision of foreground
• Disadvantages
• Requires a known, static background
• Multiple cameras

16
Which Method to Use

• Type of background present
• Static or Dynamic
• Lighting
• Gradual/Sudden changes
• Lack of lighting
• Hardware used during recording
• Multiple cameras
• Speed required for application

17
Conclusion

• Record as much information as possible
• Background subtraction methods have mainly been
  looked at in particular situations
• Severe case Fast, Lighting Independent Method
• A method to use in every case is still being
  researched
• Currently combinations of previously released
  methods offer the best results for background
  subtraction