View Planning

1
View Planning

• Candidacy Exam
• Paul Blaer
• December 15, 2003

2
The View Planning Problem
Find set of sensor configurations to efficiently
and accurately fulfill a reconstruction or
inspection task. Positions often found
sequentially so sometimes called the Next Best
View (NBV) Problem
3
Tasks

• Inspection

4
Tasks

• Inspection
• Surveillance

5
Tasks

• Inspection
• Surveillance
• 3D Models of Smaller Objects

6
Tasks

• Inspection
• Surveillance
• 3D Models of Smaller Objects
• 3D Models of Large Objects (such as buildings).

7
Tasks

• Inspection
• Surveillance
• 3D Models of Smaller Objects
• 3D Models of Large Objects (such as buildings).
• Mapping for Mobile Robots

8
View Planning Literature

• 1. Model Based Methods
• Cowan and Kovesi, 1988
• Tarabanis and Tsai, 1992
• Tarabanis, et al, 1995
• Tarbox and Gottschlich, 1995
• Scott, Roth and Rivest, 2001

• 2. Non-Model Based Methods
• Volumetric Methods
• Connolly, 1985
• Banta et al, 1995
• Massios and Fisher, 1998
• (Papadopoulos-Organos, 1997)
• (Soucey, et al, 1998)
• Surface-Based Methods
• Maver and Bajcsy, 1993
• (Yuan, 1995)
• Zha, et al, 1997
• Pito, 1999
• Reed and Allen, 2000
• Klein and Sequeira, 2000
• Whaite and Ferrie, 1997

• 3. Art Gallery Methods
• (Xie, et al, 1986)
• Gonzalez-Banos, et al, 1997
• Danner and Kavraki, 2000

• 4. View Planning for Mobile Robots
• Gonzalez-Banos, et al, 2000
• Grabowski, et al, 2003
• Nuchter, et al, 2003

9
Typical View Planning Constraints

• Fundamental Increase knowledge of the viewing
  volume.
• Scanning Ensure that the viewing volume can be
  scanned.
• Overlap Resample part of object already scanned
  and be able to ID that part.
• Tolerance Sample the object with a minimum
  accuracy.

• Self Termination
• Computational Burden Algorithm should be able
  to compute NBV in a computationally feasible
  amount of time
• Other constraints
• Few assumptions
• Generalizable

10
Automatic Sensor Placement from Vision Task
Requirements, C. K. Cowan and P. D. Kovesi, 1988

• Find camera view points for inspecting a scene.
• Requirements
• Resolution Constraint
• Focus Constraint
• Field of View Constraint
• Visibility Constraint
• View surface computed for each and intersected.
• Constraints Extend to Laser Scanners

11
The MVP Sensor Planning System for Robotic
Vision Tasks, K. A. Tarabanis, R. Y. Tsai, and
P. K. Allen 1995

• Given CAD model of the scene and task
  requirements.
• Compute view to fulfill tasks.
• Requirements
• Resolution
• Focus Constraint
• Field of View Constraint
• Feature Visibility Constraint solved in
  Computing Occlusion-Free Viewpoints, Tarabanis
  and Tsai, 1992
• Requirements written as inequalities.
• Optimization procedure run to maximize the
  quality of the viewpoints.

12
Planning for Complete Sensor Coverage in
Inspection,G. H Tarbox and S. N. Gottschlich,
1995 View Planning for Multistage Object
Reconstruction,W. R. Scott, G Roth and J.-F.
Rivest, 2001

• Model based approaches
• Camera and a laser with a fixed baseline.
• Measurability matrix, C(i,k), is computed.
• Tarbox and Gottschlich
• Next view based on glancing angles and
  difficulty to view.
• Scott, Roth, and Rivest
• Similar but add an incremental process and a
  constraint on sensor measurement error.

13
The Determination of Next Best Views,C. I.
Connolly, 1985 The Best-Next-View Algorithm
for Three-Dimensional Scene Reconstruction Using
Range Images, J. E. Banta, et al., 1995

• Connolly
• Volumetric Model-Based Approach. No prior
  information.
• Volume stored as Octree, regions labeled empty,
  object surface or unknown.
• Sphere around object is discretized into view
  points
• NBV is selected by picking viewpoints that see
  the most unkown voxels.
• Banta, et al.
• Similar to Connolly but voxels are only labeled
  as occupied or unoccupied.
• Views are chosen at points of maximum curvature
  on the object.

14
Occlusions as a Guide for Planning the Next
View,J. Maver and R. Bajcsy, 1993

• Occlusion based approach.
• No prior knowledge
• Camera-laser triangulation system.
• The planning is done in two stages
• Resolve occlusions from the laser stripe
  not being visible to the camera. Correct by
  rotating in scanning plane.
• Resolve occlusions from the laser line not
  reaching parts of the scene. Correct by rotating
  the scanning plane itself.

15
More Occlusion Based Methods

• Active Modeling of 3-D Objects Planning on the
  Next Best Pose (NBP) for Acquiring Range Images,
  H. Zha, K. Korooka, T. Hasegawa, and T. Nagata,
  1997
• NBV is computed by maximizing a linear
  combination of three weighted functions.
• Extending constraint for covering unexplored
  regions.
• Overlapping constraint for registration.
• Smoothness constraint for registration.

• A Best Next View Selection Algorithm
  incorporating Quality Criterion N. A. Massios
  and R. B. Fisher, 1998
• Voxels partitioned as empty, unseen, seen, or
  occlusion plane.
• An occlusion planes are computed along jump
  edges.
• A quality criteria based on the difference
  between the incident angle of the scanner and the
  normal of the voxel being scanned.
• NBVs are chosen to be in the direction of
  occlusion plane and also to maximize the quality
  of the voxels being imaged.

16
A Solution to the Next Best View Problem for
Automated Surface Acquisition,R. Pito, 1999

• No prior knowledge of the object.
• Void Volume stored as Void patches on the
  boundary.
• Observation Rays Computed From the Surface and
  projected into Positional Space.
• Potential Range Rays are projected into PS and
  collinear ORs are found.
• The NBV is scanner position that can view the
  most number of void patches while still viewing a
  threshold number of patches from the existing
  model.

17
Constraint-Based Sensor Planning for Scene
Modeling,M. K. Reed and P. K. Allen, 2000

• Constructs solid models from range imagery.
• No prior knowledge about the object is known
• Surface is tessellated surface from the range
  data and extruded to the bounding box.
• A surface is labeled as either imaged or
  occlusion.
• N largest targets by surface area are chosen and
  the set of positions from which the sensor can
  image the target is computed (the imaging set).
• A set of occlusion constraints are computed.
• Finally a set of possible views is computed by
  subtracting the occlusion constraints from the
  imaging set. The next view is chosen from that
  set.
• A new range image is incorporated into the model
  by intersecting it with the current model.

18
Autonomous Exploration Driven by
Uncertainty,P. Whaite and F. P. Ferrie, 1997

• Autonomous Exploration with a Laser Range Scanner
• Approximates Target with Superellipsoids.
• Parameters are estimated and Uncertainty Ellipse
  is Found.
• NBV is selected in the direction of least
  certainty.
• Restricted to single Superellipsoid.

19
View Planning for the 3D Modeling of Real World
Scenes,K. Klein, V. Sequeira, 2000

• No prior knowledge of the object being scanned.
• Surface represented as two meshes, a known mesh
  and a void mesh which is the boundary between the
  known and unknown regions.
• A cost benefit ratio is computed
• Benefit how close is each point viewed to its
  desired sampling density, and how much void
  volume is viewed.
• Cost how hard is it to get to that view point
  (manually computed).
• For calculation of the quality function at a
  given view point the mesh is partially rendered
  on to a view cube.
• A view is selected that has the best cost/benefit
  but maintains an overlap with known regions of at
  least 20.

20
Randomized Planning for Short Inspection
Paths,T. Danner and L. E. Kavraki, 2000

• Danner and Kavraki
• Extends the Gonzalez-Banos, et al.s (1997)
  randomized art gallery method to 3-D scenes.
• The visibility volume of points on the surface is
  computed.
• Random points within volume are chosen.
• Points are iteratively added to cover more of the
  surface.
• An approximation of TSP is used to connect the
  points and form the path.

21
Planning Robot Motion Strategies for Efficient
Model Construction, H. H. Gonzalez-Banos, et
al., 2000

• Goal Construction of a 2D map of the environment
• Uses a Sick laser range sensor
• Takes a single scan and extracts polylines to
  represent the obstacles
• NBV is solved by randomly picking locations in
  the free space and estimating How much new
  information will be gained.
• Best location chosen by maximizing the new
  information gained and minimizing distance
  traveled.

22
Autonomous Exploration via Regions of Interest,
R. Grabowski, P. Khosla, H. Choset, 2003

• Goal to construct 2D map of environment with
  Sonars.
• Data is fused into a occupancy map.
• Measurements with a low separation angle are
  highly coupled. Therefore next best views are
  chosen that have poses are not highly coupled
  (higher separation angles).
• After a view is taken, the regions that can see
  the same feature, but from a different angle are
  marked as regions of interest.

23
Planning Robot Motion for 3D Digitalization of
Indoor Environments, A. Nuchter, H. Surmann, J.
Hertzberg, 2003

• Goal to construct a 3D model of the environment
  with a Mobile Robot.
• Uses a pair of Sick laser scanners.
• Scans the ground plane and extracts straight
  lines, then adds unseen lines to close these
  lines off into a polygon that bounds the free
  space.
• NBV is chosen by randomly choosing views in the
  free space and evaluating how much of the unseen
  lines it can view.
• Views at a great distance and with a substantial
  change in angle are penalized.

24
Discussion

• Older methods relied on a fixed and known sensor
  work space.
• Interest is moving toward mobile robot platforms
  and exploration of complex indoor and outdoor
  environments.
• In complex exploration tasks, many problems
  become interrelated
• Localization
• Mapping
• Navigation and Path Planning
• Sensor Planning

Typical Model Acquisition Steps
Steps are missing
25
Open Problems and Future Research

• Improve efficiency to help with the move
  towards larger scenes
• Improve Accuracy and Robustness as we move
  towards more unstructured environments, sensor
  error will increase.
• Develop online planning methods take into
  account not only the changing model but the
  changing workspace of the sensor.
• Multisensor Fusion Approaches be able to
  construct our models out of multiple inputs and
  plan views that take into account the constraints
  and benefits of more than just the single sensor.

26
A Mechanism of Automatic 3D Object Modeling,X.
Yuan, 199

• No Prior knowledge of object
• Object represented by surface patches.
• Mass Vector Chain (MVC) is computed
• List of weighted normal vectors for each surface
  patch.
• Because the Guassian Mass of a convex object is
  zero, the sum of the MVC should also be zero.
• Direction of unviewed patches by the direction of
  the MVC of the patches scanned so far.

27
Uniform and Complete Surface Coverage with a
Robot-Mounted Laser Rangefinder, G. Soucey, F.
G. Callari, F. P. Ferrie, 1998

• No prior information.
• A stripe laser range finder is used.
• The scanner is swept across the object and the
  edge voxels tracked.
• Edge voxels are clustered to find the longest
  boundary of the surface.
• Assumption is that the region beyond the longest
  edge is going to be the largest region of
  unexplored space. Choose views that view that
  edge.

28
Planning Views for the Incremental Construction
of Body Models, S. Xie, T. W. Calvert, and B. K.
Bhattacharya, 1986

• 2-D map of the environment is assumed.
• Mobile robots goal is to construct 3-D models,
  but the 3-D world is projected into the 2-D
  plane. Simplifies to an art gallery problem.
• Two methods
• Shape of objects known
• Partitioned into simple polygons by connected
  edges of obstacles.
• Views are chosen by intersecting half-planes
  created by the edges of those simple polygons.
  Views are chosen greedily to cover as many edges
  as possible.
• Shape of objects unknown
• World is represented by the partial model and the
  Projected View Lines (PVLs).
• Views are picked within the polygons created by
  the edges of the partial model and the PVLs.

29
Automatic 3D Digitization Using a Laser
Rangefinder with a Small Field of View,D.
Papadopoulos-Organos and F. Schmitt, 1997

• No prior information
• Uses a triangulation based 3-D laser scanner.
• The object as it is acquired is stored in as
  voxels.
• Two types of planning are used
• Path Planning Using only translations in a
  zigzag pattern, avoiding the object as it is
  detected.
• Sensor planning The traditional view planning
  problem in which occlusions are resolved. Isnt
  dealt with directly.