Visual aid for car reversing and parking

1
Visual aid for car reversing and parking
Facilitators Dr Kolin Paul Prof Anshul
Kumar Computer Science and Engg. Department
Students Abhishek Bansal Sanchit Arora Computer
Science and Engg. Department
2
INTRODUCTION

• Uncertainty about an obstacle on the far side
  while reversing or parking a car.
• Dire need of a visual aid for estimation of path
  and distances on sides.
• An add-on, affordable and efficient device
  required.

3
OBJECTIVE

• Development of a low cost add on device for 
• cars that would help the driver while parking 
• and reversing in blind spot situations, by 
• providing him visual aid and trajectory 
• information through an LCD screen.  

4
APPROACH
5
BUILDING BLOCKS
6
Testing the usefulness of the system

• Initially on actual car.
• Built a wooden frame depicting the rear of the
  car.

7
Development of the processing unit

• Development of algorithm on laptop
• Worked on a laptop for testing purposes.
• Used the openCV library for basic image
  processing.
• Issues Image in the mirror
• Was inverted.
• Suffered from aberration.
• The inversion code was tested on a laptop.

8
(No Transcript)
9
Achievements before Mini P

• Such a system would enable the driver to see some
  reasonable distance behind the car (approx. 90
  inches).
• The view obtained through the LCD is better than
  that in existing systems, since we can also view
  the back and sides of the car.
• LCD working, but only at about 3 frames per
  second.

10
Issues Identified

• Frame rate
• Aberration
• Trajectory Plot
• Clarity of image through heat element in back
  pane
• Mounting issues
• Night Vision

11
Frame rate

• Frame rate at start of semester 3 fps
• Capture through OpenCv
• Display through Qt
• First impression Costly conversion
• Spent some time on Qt internals to try and
  capture image directly, but realized that
  bottleneck is the Qt display itself.

12

• Next Step Display through Opencv
• Required installation of X and Gtk
• Installed IPKG utility
• Used it to install required libraries.
• After some help
• Got a file system.
• Installed our kernel and camera support on it.

13
Achievement till mid semester

• Achieved a frame rate of about 8-9 fps.
• Clarity of the image has improved.
• Better video.

14
Aberration problem

• Camera focused on convex mirror -gt final image
  hampered by spherical aberration

15
Approach 1

• Modeling the mirror as part of a sphere

• Model the error as a radial distortion with a
  large radius of curvature.
• This type of modeling used to remove the
  distortion obtained at the edges of a photo due
  to the curvature of the camera lens.
• Essentially considering the mirror aberration to
  be a part of the camera lens aberration.

16
Model equations

• r norm(coord - xc yc 1')
• transformedcoord(1) xc (1 k1r k2r2
  k3r3 k4r4)(coord(1) - xc)
• transformedcoord(2) yc (1 k1r k2r2
  k3r3 k4r4)(coord(2) - yc)
• Where the symbols mean the following
• xc yc 1 - Homogenous coordinates of the
  estimated center
• Coord - Homogenous coordinates of a point in the
  original image
• r - Radius of curvature at that point
• transformedcoord- Homogenous coordinates of the
  above point in the transformed image
• k1, k2, k3, k4 - Transformation parameters

17
Results

• Model which we were using essentially used in
  case of cameras where the aberration introduced
  by the camera lens is not very large.
• Same model unable to give stable results when the
  aberration was large i.e. when the radius of
  curvature is reasonable.

18
Approach 2

• Finding a homograph by mapping image points to a
  predefined grid

• Marked points on the image which were corners in
  the floor tile grid.
• Provided world coordinates for these points.
• Found homograph by standard algorithm

19
Approach 3

• Simple model assuming radius of curvature of
  mirror in 1 direction

• Model the mirror as part of a cylindrical
  surface
• Map it to a plane which is like unfolding the
  image on to a flat plane.

20
Model equations

• r coord(2)-yc
• transformedcoord(1) coord(1)
• transformedcoord(2) sqrt(rr -
  coord(1)coord(1))yc
• where the symbols mean the following
• xc yc 1 - Homogenous coordinates of the
  estimated center
• coord - Homogenous coordinates of a point in the
  original image
• r - radius of curvature at that point
• transformedcoord- Homogenous coordinates of the
  above point in the transformed image

21
Results
22
DSP Setup

• The next task -gt set up the DSP
• Main aim
• DSP image processing
• ARM capture and display

23
DSP Gateway

• The DSP Gateway consists of two parts
• Linux device driver on the ARM
• DSP-side kernel library,
• To get the DSP running the following tasks had to
  be performed
• Enable DSP support in kernel
• Install Linux DSP tools
• Compile DSP side library
• Compile user space utilities (on the arm side)
  allow user to load and run programs on DSP
• Create appropriate device nodes in file system
  both for control and task dispatch.

24
Software used

• Latest version of DSP tools
• linuxdsptools_v1_00_00_06.bin
• Version of DSP gateway compatible with our kernel
• dspgw-3.3.1-arm/dsp.tar.bz2
• User space utility
• dspctl

25
Results

• Successfully able to run demo applications on DSP
• Proper communication between DSP and ARM
  established

26
Algorithm on OMAP

• The next task was to implement the algorithm on
  the OMAP board
• Implementation only on the arm side
• Frame rate reduced to about 4-5
• Problem Calculating transformation (floating
  point arithmetic) for each coordinate for each
  frame (320240)
• Calculation on the DSP part
• Slight improvement in frame rate
• Bottleneck transfer of data between the two
  processors

27
Final implementation

• Calculated the transformation before hand and
  saved in form of a matrix
• Only required to move image pixels from original
  to transformed coordinates
• Frame rate up to about 7 fps
• Image transformed
• Problems
• Startup time increased
• More memory required to store transformation
  matrix

28
Final achievements

• We were successfully able to remove the
  aberration in the image due to the convex mirror.
• Implemented the algorithm on the OMAP board
  achieving a frame rate of about 7 fps.
• Documented the whole procedure from setting up
  the OMAP board to making your own DSP
  applications
• Set up the SVN repository on our local server
  schnapps

29

• Thank you