Landmark Selection and Visionbased Navigation of Mobile Robots

1
Landmark Selection and Vision-based Navigation of
Mobile Robots

• Ricardo Swain Oropeza
• Postdoctoral Research Associate
• Computer Science
• Beckman Institute
• University of Illinois at Urbana- Champaign

2
Overview

• The problem.
• Methods to solve the problem.
• Elements
• Robots.
• XVision.
• Landmark Selection.
• Visual Servoing.
• Conclusion.

3
The Problem
4
Methods to Solve the Problem

• Visual tracking.
• Visual landmark selection and characterization.
• Landmark reacquisition using prediction and
  search.
• Vision-based and hybrid motion control algorithms.

5
Robots
Existing fleet of Nomadics Super Scouts.

• Nomadics Super Scout
• Color camera.
• SICK laser scanner.
• Fiber optic gyroscope.
• Pan-tilt head.
• GPS.
• Digital compass.
• Ultrasonic.
• Contact bumpers.

6
Tracking XVision (Greg Hager)

• XVision a modular, portable framework for visual
  tracking.
• Purpose make real-time vision with cheap
  technology.
• Written in C and uses XWindows graphics.
• Suite of primitive trackers (gt 1khz).
• Small set of image-level trackings.
• Use of the sum square difference (SSD).

7
XVision Feature Tracking

• Graphics-like system
• Primitive features.
• Geometric constraints.
• Fast local image processing.
• Easily reconfigurable
• Goal
• Flexible, fast, easy-to-use.

8
System Architecture
9
Natural Landmark Selection

• M. Knapek (Technical University of Munich,
  Germany)
• D. Kriegman (University of Illinois at UC)
• R. Swain Oropeza (University of Illinois at UC)

10
Finding Distinctive Landmarks
Problem Given an image I, find one or more
landmarks which are both trackable using SSD and
unlikely to be confused with other possible
windows.
11
Landmark Selection Algorithm

• Detect k salient candidates (xi, yi) which are
  trackable
• Harris corner detector find maximal eigenvalues
  of variance.
• For each, compute k-jet of intensity
• Spatial derivatives Ix , Iy , Ixx , Ixy, Iyy ,
  Ixxx , Ixxy , Ixyy , Iyyy.
• Compute feature vector fi for window about
  (xi,yi)
• Selection of distinctive features.

12
Some Details

• Landmarks should be useful for visual servoing
  and should be the projection of viewpoint
  independent features (e.G. points, corners,
  junctions, etc).
• Recognition of each landmark is performed
  independently.
• Lower error rates using geometric constraints of
  multiple matches (e.G. epipolar constraints or
  quasi-invariants).

13
12 Most Distinctive Landmarks From 97

• Yellow Candidate markers.
• Green Distinctive landmarks.

14
Recognition of Distinctive Landmarks

• Green circle Recognized
• landmark.
• Yellow line Location of
• landmark in model image.

15
Experimental Protocol

• Static scene.
• Three trajectories
• Linear.
• Arc.
• Rotation about optical axis.
• 25 images per sequence at 10 cm increments.

16
Linear and Arc Sequences
17
Landmark Recognition in Linear Sequence
Note three errors
18
Rotational Sequence
19
Landmark Recognition Under Rotation
20
Landmark Selection Observations

• Distinctive landmarks are more readily
  recognized, but
• Some of the most distinctive landmarks correspond
  to viewpoint-dependent features such as
  T-junction/occlusion boundaries.
• Evolution of landmark description over time can
  be used to better evaluate candidates discard
  viewpoint-dependent candidates.
• Task constraints should guide selection of
  landmarks which are both perceptually distinctive
  and useful.

21
Visual Servoing

• D. Burchska (John Hopkins University)
• G. Hager (John Hopkins University)
• D. Kriegman (University of Illinois at UC)
• R. Swain Oropeza (University of Illinois at UC)

22
Vision-based Control

• Robot trajectory specified by image trajectory of
  tracked features.
• Same features are tracked and robot motion is
  controlled so feature trajectories are similar.
• Given
• Image from camera on curve.
• Corresponding points on image off curve.
• We can compute
• Relative body orientation.
• Line joining points.

23
Vision-based Motion Control

• Select natural landmarks in single image.
• User guides robot along a trajectory via
• Teleoperation.
• Person following using color.
• Person following using range.
• and robot records image trajectory of
  landmarks.
• Robot uses visual servoing to follow original
  trajectory backwards or forwards.

24
Vision-based Control With Stabilization
25
Landmark Stabilization Using Pan/tilt Head

• A robot moves (particularly rotates), landmarks
  are centered to prevent them from leaving the
  field of view.

26
Trajectory Following
Initial tracker window.
Selected landmarks.
27
The Scene and Trajectory
28
Following the Trajectory
29
The Image Trajectory
30
Conclusion

• Works in
• Visual tracking.
• Natural landmark selection.
• Visual servoing.
• Task and motion constraints in landmark
  selection.
• Multi-sensor motion control.
• Toward long sequences.
• Landmark reacquisition.
• Use of the Omnicam.

31
Omnicam
32
Contacts

• Ricardo Swain Oropeza
• swain1_at_uiuc.edu
• http//www-cvr.ai.uiuc.edu/swain1
• David J. Kriegman
• kriegman_at_uiuc.edu
• http//www-crv.ai.uiuc.edu/kriegman
• Mobile Robots Team
• http//www-cvr.ai.uiuc.edu/MR