Vision-Based Motion Control of Robots

1
Vision-Based Motion Control of Robots

• Azad Shademan
• Guest Lecturer
• CMPUT 412 Experimental Robotics
• Computing Science, University of Alberta
• Edmonton, Alberta, CANADA

2
Vision-Based Control
B
B
A
A
A
B
3
Vision-Based Control
Left Image
Right Image
B
B
B
4
Vision-Based Control

• Feedback from visual sensor (camera) to control a
  robot
• Also called Visual Servoing
• Is it any difficult?
• Images are 2D, the robot workspace is 3D 2D data
  ? 3D geometry

5
Where is the camera located?

• Eye-to-Hand
• e.g.,hand/eye coordination

• Eye-in-Hand

6
Visual Servo Control law

• Position-Based
• Robust and real-time pose estimation robots
  world-space (Cartesian) controller
• Image-Based
• Desired image features seen from camera
• Control law entirely based on image features

7
Position-Based
Desired pose
Estimated pose
8
Image-Based
Desired Image feature
Extracted image feature
9
Visual-motor Equation
Visual-Motor Equation
10
Visual-motor Jacobian
Joint space velocity
Image space velocity
11
Image-Based Control Law

• Measure the error in image space
• Calculate/Estimate the inverse Jacobian
• Update new joint values

12
Image-Based Control Law
13
Jacobian calculation

• Analytic form available if model is known. Known
  model ? Calibrated
• Must be estimated if model is not known
• Unknown model ? Uncalibrated

14
Image Jacobian (calibrated)

• Analytic form depends on depth estimates.

Camera Velocity

• Camera/Robot transform required.
• No flexibility.

15
Image Jacobian (uncalibrated)

• A popular local estimator
• Recursive secant method (Broyden update)

16
Calibrated vs. Uncalibrated

• Relaxed model assumptions
• Traditionally
• Local methods
• No global planning ?
• Difficult to show asymptotic stability condition
  is ensured ?
• The problem of traditional methods is the
  locality.

• Model derived analytically
• Global asymptotic stability ?
• Optimal planning is
• possible ?
• A lot of prior knowledge on the model ?

• Global Model Estimation (Research result)
• Optimal trajectory planning ?
• Global stability guarantee ?

17
Synopsis of Global Visual Servoing

• Model Estimation (Uncalibrated)
• Visual-Motor Kinematics Model
• Global Model
• Extending Linear Estimation (Visual-Motor
  Jacobian) to Nonlinear Estimation
• Our contributions
• K-NN Regression-Based Estimation
• Locally Least Squares Estimation

18
Local vs. Global

• Key idea using only the previous estimation to
  estimate the Jacobian
• RLS with forgetting factor Hosoda and Asada 94
• 1st Rank Broyden update Jägersand et al. 97
• Exploratory motion Sutanto et al. 98
• Quasi-Newton Jacobian estimation of moving
  object Piepmeier et al. 04

• Key idea using all of the interaction history to
  estimate the Jacobian
• Globally-Stable controller design
• Optimal path planning
• Local methods dont!

19
K-NN Regression-based Method
q2
q1
20
Locally Least Squares Method
(X,q)
21
Experimental Setup

• Puma 560
• Eye-to-hand configuration
• Stereo vision
• Features projection of the end-effectors
  position on image planes (4-dim)
• 3 DOF for control

22
Measuring the Estimation Error
23
Global Estimation Error
24
Noise on Estimation Quality
KNN
LLS
25
Effect of Number of Neighbors
26
Conclusions

• Presented two global methods to learn the
  visual-motor function
• LLS (global) works better than the KNN (global)
  and local updates.
• KNN suffers from the bias in local estimations
• Noise helps system identification

27
Eye-in-Hand Simulator
28
Eye-in-Hand Simulator
29
Eye-in-Hand Simulator
30
Eye-in-Hand Simulator
31
Mean-Squared-Error
32
Task Errors
33
Questions?
34
Position-Based

• Robust and real-time relative pose estimation
• Extended Kalman Filter to solve the nonlinear
  relative pose equations.
• Cons
• EKF is not the optimal estimator.
• Performance and the convergence of pose
  estimates are highly sensitive to EKF parameters.
  

35
Overview of PBVS
2D-3D nonlinear point correspondences
T. Lefebvre et al. Kalman Filters for Nonlinear
Systems A Comparison of Performance, Intl. J.
of Control, vol. 77, no. 7, pp. 639-653, May 2004.
36
EKF Pose Estimation
yaw
pitch
roll
State variable
Process noise
Measurement noise
Measurement equation is nonlinear and must be
linearized.
37
Visual-Servoing Based on the Estimated Global
Model
38
Control Based on Local Models
39
Estimation for Local Methods

• In practice Broyden 1st-rank estimation, RLS
  with forgetting factor, etc.

40
(No Transcript)