Robot Vision 3 exercise

1
(No Transcript)
2
Robot Vision 3 exercise
3
Scorpion profiles

• RobotVision3_Start.zip
• No configuration but images
• RobotVision3_Final.zip
• Final configuration with all tools
• Both profiles are on the Scorpion CD in the
  UnSupportedProfile folder

4
The Task

• To locate the object and find the x and y
  co-ordinates of the picking point centre of
  valve
• To measure the rotation (angle)
• To send the data (x, y and angle) to the robot

5
The system

• Camera VGA monochrome
• Communication link RS 232
• Operating procedure
• Robot is the master and Vision system the slave
• Robot sends trigger signal
• Vision system take an image and do the processing
• Vision system sends the result (status, position
  including angle) to Robot

6
Vision strategy part 1

• For high accuracyUse Calibration tool to
  minimize lens distortion
• To reduce varying light conditionsUse
  ImageConverter tool to minimize reflections
• For rough location and rotationUse
  TemplateFinder2 tool to locate the object and the
  rotation

7
Vision strategy part 2

• Use PolygonMatch tool to fine-tune the picking
  point and rotation given by the TemplateFinder2
  tool

8
Vision strategy part 3

• Create a reference system based on the result
  from the PolygonMatch tool, with visualisation in
  the original image

9
Complete Tools list
10
Lens calibration

• In robot vision application the camera often have
  a large Field Of View (FOW) and a relative short
  distance to the object
• This require a wide angle lens and consequently
  large lens distortion
• Without correcting the distortion, the inaccuracy
  in the outer part of the FOW might cause the
  robot to fail picking the object correctly

11
4 point robot calibration

• By using 4 point calibration you can establish a
  co-ordinate system equal to the one used by the
  robot
• By moving the robot to the centre of each point
  the robot co-ordinates are found
• In Scorpion the centre of each is found in the
  Scorpion reference system
• This enable you to communicate pick up points and
  rotation in the robot co-ordinate system

12
Understanding lens correction and robot
calibration

• Correction of lens distortion is independent of
  distance to the object
• The lens correction is done in one planeand
  valid for allrobot planes

• Robot calibration is only valid for a specific
  distance to the object (Robot plane)
• For each different robot picking plane a
  individual robot calibration must be done
• Scorpion can handle multiple robot planes

Robot plane 3
Robot plane 2
Robot plane 1
13
Pixel co-ordinate system
Origo X0, Y0
X0, Y640
Y

• Default co-ordinate system with Origin in upper
  left corner
• X and Y pixels according to camera resolution
• VGA 760 x 480(used in the exercise)
• XGA 1024 x 768

X
X480, Y640
X480, Y0
14
Configure the Calibration tool

• Open the Calibrator tool and name it
  GeoCalibrator
• Set number of rows and column count from image
• Set row and column spacing to 10 mm, and the unit
  to mm (millimetre)
• Click on compute

2
1
3
15
Calibrator advanced setup

• The calibrator use blob technology to find the
  black spots on the grid
• If Active is ticked ON in the Advanced tab,
  user defined configuration of the blob search
  parameter can be entered.
• Only to be used if the default values are not
  working satisfactory

16
Calibrator visualisation

• Both the actual corrections (Non-linear
  distortion) and residues (remaining distortion
  after correction) can be visualised
• You can increase the gain factor to improve the
  visualisation

17
The calibrator result

• When the relevant indicator is green (Gauging or
  Robot Vision) the accuracy is accepted for that
  type of application
• In the Result of Model fit panel you will find
  more details

18
Multi co-ordinate system
Y

• Pixel co-ordinate system (red)
• the default coordinate system
• Scaling is in pixels
• Calibration co-ordinate system (yellow)
• Generated by the Calibration tool
• Lens distortion is compensated
• Scaling is in mm

X
Y
X
19
Get image for Robot calibration

• The image used is in the folder calib.
• Go to tab Camera and double click on Robot
  Vision Image Settings and select the calib
  folder
• Click on Snapshot to get the image

20
Creating robot co-ordinates step 1

• Use the mouse (right click and select view info)
  to find the co-ordinates of the four points.
• Hint
• Use zoom to get accuracy
• Remember to select GeoCalibrator as reference
  system (right click and select reference system
  to check)

Point 1
Point 2
Point 3
Point 4
21
Creating robot co-ordinates step 2

• Open ExternalReference tool and name it
  RobotCoordinates
• Enter both local and robot co-ordinates in the
  External tool
• Remember to select the tool named GeoCalibrator
  as reference, since the local co-ordinates is
  relative to that system

22
Move reference system to image

• The origin of the robot coordinates are far
  outside the image.
• To see the result we can move it inside the image
  using the tool MoveReference
• Name the tool MoveRobotCoordToImage and use the
  inputs as on the image to the right

23
Robot coordinate system
Y
X

• We now have 3 coordinate systems
• 1 Pixel
• 2 GeoCalibrator
• 3 RobotCoordinates (moved)
• Remember to change back to the image folder
  images

Y
X
Y
X
24
Eliminating reflections

• Varying reflections from the valve may cause
  problems
• To eliminate the reflections with can make a new
  manipulated image.
• First add a new image in Image Setting under
  the camera tab and name it Valve
• Remember to tick OFF the active mark

1
2
3
4
25
Reducing the contrast

• By using the Image Converter we can create a new
  picture
• Select the ImageConverter tool and name it
  ReduceReflections
• Activate the Intensity part and write in the name
  of the image Valve created before
• To eliminate the reflections set the Filter High
  150 and set 150
• This means that all pixels with greyscale values
  over 150 are set to 150

26
How to use a template finder
1

• Select TemplatFinder2 tool and name it
  FindParts
• We will use image 2 (Enter 2 in the Image Index
  field under the General tab)
• To include the lens correction and to make it
  work in robot co-ordinates use the tool
  RobotCoordinates as reference
• The search area is the complete image, but to
  secure that parts close to the image border is
  found, make the search area slightly bigger than
  the image

2
27
Set resampling

• To speed up the processing time we will use
  resampling to reduce the information in the image
• Set the new image size to 125. This will give
  approximately 340 x 260 pixels, a reduction of 50
  

28
Look at resampled image
1

• Go to Resampling Tab
• When you have the image you want to use as the
  template push Acquire resampled image
• This picture have 50 less pixels than the
  original and is corrected for the lens distortion
• The more you reduce the number of pixels the
  faster the template will work but consequently
  less accurate

2
29
Create template image

• Use the cursor to zoom around the part
• Try to make a square image. This will work more
  efficient when rotation is required
• Push Copy selection to clipboard

30
Store Templates

• Click on the paste knob and the template is store
  in template 1
• If Paste square image is ticked on, pixels are
  added to make it 100 square
• You can add tabs for more images as templates by
  pushing

1
31
Visualisation

• Push apply and look in image to see if the part
  is found. If no results adjust the threshold to a
  lower value
• Adjust the visualisation as indicated to the
  right and in the image you will have the template
  axis and score

32
Understanding the results

• Go to the result tab and you will find all
  results
• Number of matches is number of parts found
• Match center (x,y) for each match plus angle
• Score for each match
• Template number and Template name (if any)

33
Set new centre and offset angle

• Centre of the template image is the default
  centre.
• You can change the centre by moving the cursor to
  the point where you want the centre, right click
  and select Set center

Before
After
34
Adjust offset angle

• The angle of the axis system is dependent of the
  rotation of the template image
• You can offset the angle to make the axis match
  any orientation
• You can also give the Template a name (Very
  useful with many templates)

35
Advanced Tab Template decimation

• Reduce the pixel content in the image to speed up
  the tool
• Start with Automatic settings. While watching the
  analysis time, adjust the factor manually 1 or 2
  steps up and down to find the fastest factor

36
Advanced Tab Template modification

• Handles rotation and scaling of the template
• We will use the template finder to roughly find
  the rotation (/- 10 degrees)
• Template finder can also handle variation in
  scaling
• Both rotation and scaling takes time and
  processing capacity

37
Advanced Tab Match description

• Select what to visualise in the description field

38
Advanced Tab Sorting results

• Set criteria for sorting the parts found
• Organise the matches according to score, x or y
  position or in rows and coloumns
• Useful when you want a robot to pick in a certain
  order

39
Valve located

• To check that a valve is located we can use the
  result from the TenplateFinder
• Use logic tool and select the parameter Number
  of matches
• The minimum condition should 1

40
Fine-tune the rotation angle and centre point
(picking point)

• Create a PolygonMatch tool named FindRotation
• The tool should reference to the TemplateFinder
  FindParts and will then generate a correction
  angle and centre offset value relative to the
  results from the TemplateFinder
• These values can be used to establish an accurate
  angle and picking point

41
How does it work

• Diagonal on the polygon, a number of trace lines
  are set. These trace lines detect points as for
  Linefinder and RadialArcFinder
• In this case we will set up the trace lines to
  find the mid point of the reflections on the tips.

42
Using multi polygons

• We will define 3 polygons
• Tip 1
• Tip2
• Circle
• Tip 1 and 2 will correct the rotation, and circle
  will correct the centre point

Circle
Tip 2
Tip 1
43
How to create a polygon

• Keep the Ctrl button down
• Then click with the mouse where you want the
  polygon to be
• Click Paste new and the polygon is stored in
  the tool
• The polygon is given a number and you can see the
  coordinates in Contents

click
click
click
click
44
Create all 3 polygons

• Paste new for each polygon

45
Configure the PolygonMatch tool

• Set visualisation as in the image to the right

46
Set the Tracelines

• To equally weight the results of each of the 3
  polygon tick on Use trace line count. Now each
  polygon will have 21 trace lines/points
• To make it work the trace lines must be longer.
  Set start point to 20 and end point to 20

47
Set the Edge detection

• To find the mid point of the reflections select
  Differentiation 2.
• The smooth base should be 10, equal to the
  average width of the reflections in pixels
• Polarity Light to dark since we are looking for
  light points
• And threshold must be reduced to 0.5

48
Polygon match result

• Activate the Child reference base and we have a
  corrected rotation system
• The axes may need an offset to fit as on the
  image

Before
After
49
The new rotation system co-ordinate system

• The rotation system in PolygonMatch is related to
  the centre point of the TemplateFinder
  FindParts
• It needs to be converted to Robot co-ordinates

50
Change to Robot co-ordinates

• Select the MoveReference Tool and name it
  ValveRefSystem
• Reference is made to FindRotation and this tool
  operates as the FindRotation reference system
• In the result tab select RobotCoordinates as
  result reference

51
Visualisation of rotation

• Now the Rotation system is in Robot co-ordinates
• In tab General you can select ImageIndex1 to
  move the visualisation to the original image

52
Customise text and visualisation

• We will use Python scripting to make a nice text
  string to present the result for the operator
• We will make a simple visualisation of the
  picking point in the original image

53
Python scripting

• Open a Python tool and name it CreatePositionStri
  ng
• The script on next page collect the x.y
  co-ordinates and angle from the Valve Reference
  System and create a string that is store in the
  text result field of the tool
• And create a circle marker at the picking point

54
Python script

• valvelocated GetValue('ValveLocated.Value')
• if valvelocated
• x GetValue('ValveRefSystem.Origo_x')
• y GetValue('ValveRefSystem.Origo_y')
• a GetValue('ValveRefSystem.Rotation')
• str0 'Position ((x).1f , (y).1f) mm
  Angle (a).1f degrees' vars()
• SetValue('CreatePositionString.Text',str0)
• print str0
• This line make maker at the picking point
• DrawMarker('RobotCoordinates', x, y, 'blue', 8
  , 1)
• else
• SetValue('CreatePositionString.Text','No
  position')
• Hint! Copy this page directly into the Python
  tool

55
Adding text string to the result panel

• Right click on the Result Panel and set Number of
  measured values to 1
• Right click again and for measured values select
  the right parameter
• Tool CreatePositionString
• Parameter Text

56
Operator interface
57
Sending data to the robot

• Each time a valve is located the x, y and angle
  values should be sent over RS232 to the robot
• In the event that a valve is NOT located a the
  figure 0 should be sent out
• These commands can be configured in the Valve
  located and Valve not found states defined
  under the Settings mode
• These commands are only executed when the state
  is true

58
Configure Valve located command
1

• Go to the Settings mode and open Valve
  located state
• Select the Command tab and click on new
• Click on the button to right of the Command
  line and select RS232Cmd from the list

2
4
3
5
6
59
Writing the parameters

• To send the x values use the following syntax
• 0.1fValveRefSystem.Origo_x
• configuration of a figure
• 0.1ffloating point with 1 decimal configuration
• ValveRefSystem.Origo_x Parameter to be sent

60
The complete parameter string is

• Add then the y value and angle
• Each figure is separated by a comma
• 0.1fValveRefSystem.Origo_x0.1fValveRefSystem.Or
  igo_y.0fValveRefSystem.Rotation

61
Valve not located command

• Add a RS232 command in the Valve not found State
  with the Parameter 0.

62
Checking communication

• In the communication Tab both out and in going
  communication can be monitored
• Select RS232 tab and check the results

63
Congratulation

• You have made an advanced Robot Vision system
  using Template Finder 2 and Polygon Match

64
(No Transcript)