Visual Servoing in ALIVE

1
Visual Servoing in ALIVE
Qian Zhebin
2
Agenda

• vALIVE Overview (3)
• VDS Functional Specification (5)
• VDS Algorithms (15)
• Other Related Projects (7)

3
AUV ALIVE

• ALIVE, Autonomous Light Intervention Vehicle
  (European Development Project)
• First Intervention AUV Prototype
• AUV, Autonomous Underwater Vehicle
• To perform Workclass-ROV mission
• ROV, Remotely Operated Vehicle

4
ALIVE Front-side View
5
ALIVEs Mission

• Three Stages
• Transit, Inertial Navigation System-based
• To Safe Landing Zone (SLZ), 50-100m
• Approach, Sonar-based
• To 1.5-2m range from the docking panel
• Docking, Video-based
• To accurately grab the bars on the panel
• Highlights
• Cableless Tele-manipulation (with acoustic modem
  linking the Surface Control Unit)
• Auto Docking (Mechanical Attaching)

6
Docking Panel
7
Agenda

• ALIVE Overview (3)
• v VDS Functional Specification (5)
• VDS Algorithms (15)
• Other Related Projects (7)

8
Inside ALIVE Subsystems

• Inter-Module Connection is Ethernet, TCP/IP-based.

9
VDS

• VDS, Visual Docking System
• Interfacing with
• Navigation
• Sonar Docking System
• Camera (which is as a independent component with
  device driver)
• Vehicle Controller

10
VDS Input and Output

• Messaging Framework
• Messages between VDS and Others
• Procedure without RESPONSE
• Procedure with RESPONSE
• Protocol Specification
• Format
• Semantics
• Timing for response
• Abnormal processing
• Response time-out
• Bad message element

11
Docking Scenario

• MSC (Message Sequence Chart)

• Note Finally to Hover at a Target Relative Pose
  for Docking

12
Key Messages

• Input Current Image
• Output VEHICLE POSE COMMAND (VDS?VC), containing
• Discriminator
• Message type
• Six DoF of Relative Position between the Vehicle
  and the Panel
• X
• Y
• Z
• Pitch
• Yaw
• Roll

13
VDS Core

• VDS Algorithm 3-D Pose Estimation
• Engineering Requirements
• Real-time
• Configurable, environment varies
• Robust

14
Agenda

• ALIVE Overview (3)
• VDS Functional Specification (5)
• v VDS Algorithms (15)
• Other Related Projects (7)

15
VDS Algorithm

• Priori Knowns, from (re)configuration messages,
  or file (in algorithm test)
• Panels 3-D Model, some Points, pi
• Fixed focal length of Camera, f
• Initial Pose, Rough, (t, R)
• Algorithm Thresholds
• Input Current Image, (u, v)
• Output the Updated (t, R), translation and
  rotation

16
3-D Panel Model

• Derived from Panel CAD
• Three circle centers are selected to describe the
  model
• Panel Coordinate, three points (px, py, pz)
• pz 0

17
Current Image
18
Algorithm Steps

• Canny Edge Detection (basis)
• Contour Segmentation and Circle Detection with
  Center Calculated
• Feature Point Correspondence between Model and
  Current Image
• Pose Estimation (kernel)

19
Edge Detection

• Canny Edge Detection
• Input Current Image
• Output Edges
• Complex but essential and worthy

20
Circle Detection

• Edge Contour Segmentation
• Input binary edges
• Output several contour objects, with the centers
  calculated
• A stack is used to segment the continuous edges
  into contours
• Circle Detection
• Output 3 circle centers
• According to the circle definition with some
  tolerances
• The center history contexts used

21
Feature Correspondence

• The correspondence between the circle center
  (detected) in the image and a circle point p on
  the model is established.
• (ui, vi) ?? pi (pxi, pyi, pzi)
• Note 2 outer control loops and stabilization
  module make the matching easy

22
Models Projection

• !The task is to refine/solve for (t, R)
• Nonlinear system
• Note R is a matrix here, it has two equivalent
  forms in the vector and matrix.

23
Newtons Method

• s is parameter vector, like (t, R). s0 can be
  very rough. e is the error between projective
  model and image.
• In this case,

24
Partial Derivatives
25
Simulation and Tank Experiments

• Image is formed with pin-hole projection plus
  some noise
• Approaching procedure goes
• Error between estimated and real converges
• Accuracy Newtons Iteration
• Intra-frame
• Inter-frame

26
Conclusion

• Single-view solution
• Result Accuracy Newtons Iteration
• Intra-frame
• Inter-frame
• Missions last meter

27
Conclusion

• This single-view solution meets the requirements.
• Last meters mission done

28
Agenda

• ALIVE Overview (3)
• VDS Functional Specification (5)
• VDS Algorithms (15)
• v Other Related Projects (7)

29
Other related work

• Vehicle Stabilization with 2½D Visual Servoing
• Use of 2-D Hopfield Neural Network for Image
  Registration
• Point Target Detection from Low-SNR Sequences and
  Real-time Design
• MPEG-4 CODEC and DSP Optimization

30
2½D Visual Servoing
31
Use of HNN for Feature Correspondence

• Multiple Dimensional HNN and its Computational
  Complexity

32
Real-time Moving Point Target Detection
33
Multimedia Applications

• Video MPEG-4 Codec
• Codex Fitting onto TI C6x DSP
• Memory Hierarchy
• Program Optimization using Pipeline, VLIW, SIMD
• Industry Standard System Platform
• Device drivers
• Linux, Windows, and DSP/BIOS

34
Imagination at work expected