Module Development Template

1
(No Transcript)
2
Introduction to MatLab Image Processing

• Most of the programming operations have as
  input or output a matrix or a vector.
• Images are often represented a matrices.
• - MatLab is a powerful tool to manipulate
  graphics and images.

3
Robot Vision Development in Practice

1. Build your robot (you are done)
2. Install a camera on it. Learn software that comes
   with camera, understand formats. JPG
3. Install Matlab on your laptop student version,
   much software for free on WWW Octave
4. Be able to display image from camera in one
   window so that you will see it and Matlab code or
   processing in other windows.
5. Guide your robot through stage or labyrinth and
   see with your own eyes what robot sees.
6. Think what is a good processing method for the
   image that you see.
7. Protytype this in Matlab using existing
   functions.
8. If necessary, use parallel Matlab on CUDA
9. If necessary, rewrite to C or Python or Java.

4
MATLAB

• This introduction will give
• a brief overview, its not a MATLAB tutorial !
• Some basic ideas
• Main advantages and drawbacks compared to other
  languages

5
MATLAB

• high-performance language for technical computing
• computation, visualization, and programming in an
  easy-to-use environment
• Typical uses include
• Math and computation
• Algorithm development
• Modelling, simulation, and prototyping
• Data analysis, exploration, and visualization
• Scientific and engineering graphics
• Application development, including Graphical User
  Interface building

6
Why MATLAB

• A good choice for vision program development
  because
• Easy to do very rapid prototyping
• Quick to learn, and good documentation
• A good library of image processing functions
• Excellent display capabilities
• Widely used for teaching and research in
  universities and industry
• Another language to impress your boss with !

7
Why not MATLAB
Has some drawbacks Slow for some kinds of
processes Not geared to the web Not designed
for large-scale system development
8
MATLAB Components

• MATLAB consists of
• The MATLAB language
• a high-level matrix/array language with control
  flow statements, functions, data structures,
  input/output, and object-oriented programming
  features.
• The MATLAB working environment
• the set of tools and facilities that you work
  with as the MATLAB user or programmer, including
  tools for developing, managing, debugging, and
  profiling
• Handle Graphics
• the MATLAB graphics system. It includes
  high-level commands for two-dimensional and
  three-dimensional data visualization, image
  processing, animation, and presentation graphics.
• (contd)

9
MATLAB Components

• The MATLAB function library.
• a vast collection of computational algorithms
  ranging from elementary functions like sum, sine,
  cosine, and complex arithmetic, to more
  sophisticated functions like matrix inverse,
  matrix eigenvalues, Bessel functions, and fast
  Fourier transforms as well as special image
  processing related functions
• The MATLAB Application Program Interface (API)
• a library that allows you to write C and Fortran
  programs that interact with MATLAB. It include
  facilities for calling routines from MATLAB
  (dynamic linking), calling MATLAB as a
  computational engine, and for reading and writing
  MAT-files.

10
MATLAB

• Some facts for a first impression
• Everything in MATLAB is a matrix !
• MATLAB is an interpreted language, no
  compilation needed (but possible)
• MATLAB does not need any variable declarations,
  no dimension statements, has no packaging, no
  storage allocation, no pointers
• Programs can be run step by step, with full
  access to all variables, functions etc.

11
What does Matlab code look like?
A simple example a 1 while length(a) lt 10 a
0 a a 0 end
which prints out Pascals triangle 1 1 1 1 2
1 1 3 3 1 1 4 6 4 1 1 5 10 10 5 1 1 6 15 20 15 6
1 1 7 21 35 35 21 7 1 1 8 28 56 70 56 28 8 1 1 9
36 84 126 126 84 36 9 1 (with a before each
line).
12
What does Matlab code look like?
Another simple example t 0pi/1002pi y
sin(t) plot(t,y)
13
What does Matlab code look like?
Another simple example t 0pi/1002pi y
sin(t) plot(t,y)
Remember EVERYTHING IN MATLAB IS A MATRIX !
creates 1 x 200 Matrix
Argument and result 1 x 200 Matrix
14
jpg
bmp
15
Formats of Images in MATLAB

• MATLAB can import/export several image formats
• BMP (Microsoft Windows Bitmap)
• GIF (Graphics Interchange Files)
• HDF (Hierarchical Data Format)
• JPEG (Joint Photographic Experts Group)
• PCX (Paintbrush)
• PNG (Portable Network Graphics)
• TIFF (Tagged Image File Format)
• XWD (X Window Dump)
• MATLAB can also load raw-data or other types of
  image data

• Data types in MATLAB
• Double (64-bit double-precision floating point)
• Single (32-bit single-precision floating point)
• Int32 (32-bit signed integer)
• Int16 (16-bit signed integer)
• Int8 (8-bit signed integer)
• Uint32 (32-bit unsigned integer)
• Uint16 (16-bit unsigned integer)
• Uint8 (8-bit unsigned integer)

16
Images in MATLAB

• Binary images 0,1
• Intensity images 0,1 or uint8, double etc.
• RGB images m-by-n-by-3
• Indexed images m-by-3 color map
• Multidimensional images m-by-n-by-p (p is the
  number of layers)

17
Image import and export

• Read and write images in Matlab
• gtgt Iimread('cells.jpg')
• gtgt imshow(I)
• gtgt size(I)
• ans 479 600 3 (RGB image)
• gtgt Igreyrgb2gray(I)
• gtgt imshow(Igrey)
• gtgt imwrite(lgrey, 'cell_gray.tif', 'tiff')
• Alternatives to imshow
• gtgtimagesc(I)
• gtgtimtool(I)
• gtgtimage(I)

18
Images and Matrices

• How to build a matrix (or image)?
• gtgt A 1 2 3 4 5 6 7 8 9
• A 1 2 3
• 4 5 6
• 7 8 9
• gtgt B zeros(3,3)
• B 0 0 0
• 0 0 0
• 0 0 0
• gtgt C ones(3,3)
• C 1 1 1
• 1 1 1
• 1 1 1
• gtgtimshow(A) (imshow(A,) to get automatic
  pixel range)

19
Matrices
20
Matrices

• Rows and columns are always numbered starting at
  1
• Matlab matrices are of various types to hold
  different kinds of data (usually floats or
  integers)
• A single number is really a 1 x 1 matrix in
  Matlab!
• Matlab variables are not given a type, and do
  not need to be declared
• Any matrix can be assigned to any variable

21
Matrices
Building matrices with A 2 7 4 A 2
7 4 A 2 7 4 3 8 9 B A A
2
7
4
2
7
4
2
7
4
3
8
9
?
22
Matrices
Building matrices with A 2 7 4 A 2
7 4 A 2 7 4 3 8 9 B A A
2
7
4
2
7
4
2
7
4
3
8
9
2
7
4
2
7
4
3
8
9
3
8
9
23
Matrices
24
Matrices
Some operators must be handled with care A 1
2 4 5 B A A prints 9 12
24 33 B A . A prints 1 4
16 25 Element by element multiplication
25
Submatrices
A matrix can be indexed using another matrix, to
produce a subset of its elements a 100 200
300 400 500 600 700 b 3 5 6 c
a(b) 300 500 600
26
Submatrices

• To get a subsection of a matrix, we can produce
  the index matrix with the colon operator
• a(25)
• prints
• ans 200 300 400 500
• This works in 2-D as well, e.g. c(23, 12)
  produces a
• 2 x 2 submatrix.
• The rows and columns of the submatrix are
  renumbered.

27
loops
for loops in MATLAB iterate over matrix
elements b 0 for i 3 9 17 b b
i end Result 29 Note The MATLAB way to
write that program would have been b sum( 3 9
17) Avoid loops if possible !
28
loops
The typical for loop looks like for i
16 end Which is the same as for i 1 2 3
4 5 6 end
29
loops
Once again AVOID LOOPS
30
Images
So why MATLAB and IMAGE PROCESSING ?
31
Images
Images can be treated as matrices !
32
Matrix
gtgt A1 2 33 2 1 A 1 2 3 3
2 1
gtgtbA(1,) b 1 2 3
gtgt BA' B 1 3 2 2 3 1
33
Images and Matrices

• Accesing image elements (row, column)
• gtgt A(2,1)
• ans 4
• can be used to extract a whole column or row
• gtgt A(,2)
• ans
• 2
• 5
• 8
• or a part of a column or row
• gtgt A(12,2)
• ans
• 2
• 5

• A
• 2 3
• 5 6
• 7 8 9

34
Image Arithmetic

• Arithmetic operations such as addition,
  subtraction, multiplication and division can be
  applied to images in MATLAB
• , -, , / performs matrix operations
• gtgt AA
• ans 2 4 6
• 8 10 12
• 14 16 18
• gtgt AA
• ans 30 36 42
• 66 81 96
• 102 126 150
• To perform an elementwise operation use . (.,
  ./, ., . etc)
• gtgt A.A
• ans 1 4 9
• 16 25 36
• 49 64 81

• A
• 2 3
• 5 6
• 7 8 9

35
Logical Conditions

• equal () , less than and greater than (lt and
  gt), not equal () and not ()
• find(condition) - Returns indexes of As
  elements that satisfies the condition.
• gtgt row colfind(A7)
• row 3
• col 1
• gtgt row colfind(Agt7)
• row 3
• 3
• col 2
• 3
• gtgt Indxfind(Alt5)
• Indx 1
• 2
• 4
• 7

• A
• 2 3
• 5 6
• 7 8 9

36
Flow Control

• Flow control in MATLAB
• - if, else and elseif statements
• (row1,2,3 col1,2,3)
• if rowcol
• A(row, col)1
• elseif abs(row-col)1
• A(row, col)2
• else
• A(row, col)0
• end

A 1 2 0 2 1 2 0
2 1
37
Flow Control

• Flow control in MATLAB
• - for loops
• for row13
• for col13
• if rowcol
• A(row, col)1
• elseif abs(row-col)1
• A(row, col)2
• else
• A(row, col)0
• end
• end
• end

A 1 2 0 2 1 2 0
2 1
38
Flow Control

• while, expression, statements, end
• Indx1
• while A(Indx)lt6
• A(Indx)0
• IndxIndx1
• end

• A
• 2 3
• 5 6
• 7 8 9

A 0 2 3 0 5 6 7
8 9
39
Working with M-Files

• M-files can be scripts that simply execute a
  series of MATLAB statements, or they can be
  functions that also accept input arguments and
  produce output.
• MATLAB functions
• Are useful for extending the MATLAB language for
  your application.
• Can accept input arguments and return output
  arguments.
• Store variables in a workspace internal to the
  function.

40
Working with M-Files

• Create a new empty m-file
• function Btest(I)
• row colsize(I)
• for r1row
• for c1col
• if rc
• A(r, c)1
• elseif abs(r-c)1
• A(r, c)2
• else
• A(r, c)0
• end
• end
• end
• BA

41
(No Transcript)
42
Examples
Try these MATRIX AND VECTOR OPERATIONS This is
how we can define a vector gtgt v1, 2, 3 Matlab
prints out the following v      1     2    
3 Similarly we can define a matrix gtgt M 1 2
3 4 5 6 7 8 9 The result is M      1    
2     3     4     5     6     7     8    
9 If you want to suppress the MatLab output then
you need to finish the line with semicolon as
follows. gtgtM 1 2 3 4 5 6 7 8 9
43
Projection

• Say you want to extract some rows and columns of
  a matrix. This is called a projection. We simply
  give the subset of rows and columns as
  parameters, as follows
• gtgt M11M(23 , 23)
• M11
•      5     6
•      8     9
• To specify all elements in a given dimension one
  can use ' So to get all rows but just columns 1
  and 2, we type
• gtgt A M( , 12)
• A
•      1     2
•      4     5
•      7     8

44
WORKING WITH IMAGES in MatLab
Lets talk about image files and their
formats.. Color vs GrayScale Basic Image
Processing functions Reading in an image gtgt
img1imread('Water lilies.jpg') Displaying an
image gtgt imshow(img1) Finding out size of an
image gtgt size(img1) gtgt size(img1) ans
600 800 3
imread
imshow
size
45
WORKING WITH IMAGES in MatLab

• Cropping an image
• gtgt imgsmallimg1(200300,300400,13)
• gtgt imshow(imgsmall)
• gtgt imgsmallimg1(150250,350450,13)
• gtgt imshow(imgsmall)
• gtgt size(imgsmall)
• ans
• 101 101 3
• Exercise 1. Find 2 images online
• 2. Crop them to the same size
• 3. Add the two images together.
• 4. Display resulting image
• Advanced Exercise

variable imgsmall
46
ReScaling

• We can rescale by changing the number of rows and
  columns, yet preserve the information in the
  image
• gtgt rows, cols, colors size(img1)
• rows 600 cols 800 colors 3
• Increase the number of rows
• gtgt stretchfactor 1.5
• gtgt rowVec linspace(1,rows,stretchfactorrows)
• gtgt newrowsround(rowVec)
• gtgt newimagimg1(newrows,,)
• gtgt imshow(newimg)
• Decrease number of columns
• gtgt stretchfactor 0.75
• gtgt colVec linspace(1,cols,stretchfactorcols)
• gtgt newcolsround(colVec)
• gtgt newimagnewimg(,newcols,)
• gtgtimshow(newimg)

linspace
round
47
Example Program Inverting an image

• To invert or to add two images we need to convert
  to double and then rescale the result back so
  that it looks like an image
• InvImg 1 - double(IMG1)/255
• NewImg uint8(round(InvImg255)))
• Imshow(NewImg)

uint8
48
WORKING WITH IMAGES in MatLab

• Color Masking
• Sometimes we want to replace pixels of an image
  of one or more colors with pixels from another
  image. It is useful to use a blue or green
  screen in some instances.
• Find an image with a big plot of one color. First
  we will replace that color. And then we will find
  another image for pixel replacement.
• Let us plot the color values of one chosen
  rowThis will tell us the pixel values of the
  color we want to replace.
• v imread(myimg.jpg)
• image(v)
• row input(which row?)
• red v(row,,1)
• green v(row,,2)
• blue v(row,,3)
• plot(red,r)
• hold on
• plot(green,g)
• plot(blue,b)

input
49
WORKING WITH IMAGES in MatLab

• Suppose we want to replace those values whose
  intensities exceed a threshold value of 160 in
  each color.
• v imread(myimg.jpg)
• thresh 160
• layer (v(,,1) gt thresh) (v(,,2) gt
  thresh) (v(,,2) gt thresh)
• mask(,,1) layer
• mask(,,2) layer
• mask(,,3) layer
• If you want to only mask a portion of the image
  you can use something like
• gtgt mask(700end,,) false
• Which sets the mask so that we do not affect rows
  700 and above
• To reset the color to red
• gtgtnewv v
• gtgtnewv(mask)(1) 255
• Or to replace pixels from a different image w
  usegtgt newv(mask) w(mask)

50
Histograms
51
Images in MATLAB
imread
imshow
52
imhist
53
Histogram Equalization
histeq
54
(No Transcript)
55
Noise
56

1. Adding Noise
2. Filtering Noise

imnoise
medfilt2
57
Add Image to noise
58
Median Filtering removes noise
59
Median Filtering removes Gaussian noise
imnoise
fspecial
imfliter
60
FSPECIAL creates special 2D filters
61
Imfilter multidimensional filtering
62
Thresholding
63
(No Transcript)
64
IM2BW converting to binary by thresholding
65
Repeated thresholding
66
Separating background by thresholding
67

• Extracting features
• Corner
• Center point

68
Edge Detection
69
edge
Example of kernel-based filtering - Sobel
70
Edge, Sobel and Prewitt
71
Roberts, Laplacian, Zero-cross, Canny edge
detection methods
72
Use of threshold in Sobel affects what is found
73
Laplacian of Gaussian filtering is often better
74

1. Canny Filter is even better this is a major
   invention with many applications in robotics.
2. Can one invent a better one?

75
Many variants of combining thresholding and edge
detection exist
76
Hough Transform
77

1. Hough Transform has links to Fourier, Radon and
   Neural Nets.
2. A deep concept

78
Camera Calibration
79
Calibration of Cameras
Matrix/vector multiplication
80
Sequences of operations
81
Noise, filtering and histogramming
82
Matlab versus other languages
83
Help
Help command
Help svd
84
C vs. Matlab
int j . for (j1jlt23jj2)
A4j3j
for i 12N for J 1N
A(I,J) (IJ-1) end end
85
C vs. Matlab (cont.)
int j while (jlt28) .
while Ngt 0, E E F
F AF/N N N 1 end
86
C vs. Matlab (cont.)
If (ij) Aij2 else if
(abs(i-j)!1) .
if i j A(i,j) 2 elseif abs(i-j)
1 A(i,j) -1 else A(i,j)
0 end
87
C vs. Matlab (cont.)

• Index to an array can be zero.
• double, float , int
• is very important

• Index into matrix cant be negative or zero.
• Dont need to worry about the data type
• not so important

88
Functions
function mean,stdev stat(x) n length(x)
mean avg(x,n) stdev
function mean avg(x,n) mean sum(x)/n
double stat(double x) . return stdev
89
Functions(cont.)
void Matrix2Vector( )

• function Matrix2Vector
• AvA(1,)
• for i2x
• AvAv A(i,)
• end
• AvAv'

90
Functions(cont.)
void AddF(int i) int main() addF(i) void
AddF(int i) ii1
File name testFunR.m
function testFun i2
AddF(i) i
function AddF(i)
ii1
A function declaration cannot appear within a
script M-file
91
Graphing
To make a graph of y sin(t) on the interval t
0 to t 10
In file PlotTest.m t 0.310 y sin(t)
plot(t,y,r)
92
Graphing(cont.)
graphing the fuction z(x,y) x exp( - x2 - y2)
In file MeshTest.m x,y meshgrid (-2.22,
-2.22) z x . exp(-x.2 - y.2) mesh(z)
93
Graphing(cont.)
Multiple windows in one figure
SubplotTest.m t 0.310 y sin(t) z
cos(t) subplot(2,1,1) plot(t,y)
subplot(2,1,2) plot(t,z)
94
Graphing(cont.)
Multiple windows in one figure
SubplotTest.m t 0.310 y sin(t) z
cos(t) subplot(1,2,1) plot(t,y)
subplot(1,2,2) plot(t,z)
95
Matrix Manipulation

• Singular value decomposition---U,S,Vsvd(A)

• Eigenvalues and eigenvectors---V,D eig(A)

• Orthogonal-triangular decomposition- Q,Rqr(A)

• LU factorization --L,U
  lu(A)

• Matrix rank --
  arank(A)

• Condition number -- acond(A)

96
Image Processing Toolbox

• Read an image ---- Iimread(filename)

• Display an image ---- imshow(I)

• Write an image ---- imwrite(I,
  filename,FMT)

97
Image Processing Toolbox(cont.)
ImageRWD.m function ImageTest Itifimread('image1.
tif') imwrite(Itif,'image1.bmp','bmp') Ibmpimre
ad('image1.bmp') subplot(1,2,1) imshow(Itif) su
bplot(1,2,2) imshow(Ibmp)
98
Image Processing Toolbox(cont.)
EdgeTest.m function EdgeTest Itifimread('image1.t
if') Bedge(Itif,'canny') subplot
(1,2,1) imshow(Itif) subplot(1,2,2) imshow(B)
99
Image Processing Toolbox(cont.)

• Pixel values and statistics --corr2,imhist

• Image enhancement histeq, medfilt2

• Linear filter -- conv2, filter2

• Image transform -- dct2, fft

• Binary image Op. --- dilate, erode

100
Neural Network

• Feed-forward backpropagatoin

• Hopfield Networks

• Self-organizing maps

• Radial Basis networks

101
Neural Networks(cont.)

• Create feed-forward NN

• Netnewff(-1 20 5,31,tansig,purelin,tr
  aingd)

• Neural Model tansig, logsig, purelin.

• Training method traingd, traingdm.

• Training

• net,trtrain(net,p,t)

• net.trainParam.???? show, lr, epochs,
  goal

• Simulation

• Asim(net,q)

102
Menu
function MenuDemo cell_list fig_number
1 title_figure 'Menu Demo' cell_list1,1
'Plot','PlotTest' cell_list1,2
'Mesh','MeshTest' cell_list1,3
'Image','ImageRWD' cell_list1,4 'Image
Edge','EdgeTest' cell_list2,1
'????','PlotTest' cell_list2,2
'????','PlotTest' cell_list2,3
'????','PlotTest' cell_list2,4
'Exit','disp(''Bye. To run again, type
"menu".'') close(' num2str(fig_number)
')' show_window(cell_list,fig_number,title_fi
gure,120,20,3)
103
Menu (cont.)

• cell_list1,1 name',function'

• show_window(cell_list,fig_number,title_figure,x,y,
  z)

104
EDGE DETECTION IN MATLAB
105
Fourier Transform in MATLAB
106
By the way

• MATLAB can also handle
• Movies
• 3D objects

107
Conclusion
MATLAB is a mighty tool to manipulate
matrices Images can be treated as
matrices MATLAB is a mighty tool to manipulate
images
108
In my opinion
MATLAB should be used to code software
prototypes Research is mostly about prototypes,
not runtime-optimized software MATLAB should be
used in research
109
In my opinion

• MATLAB prototypes must be re-coded (e.g. in C)
  if theres need for speed
• Algorithm development time is drastically shorter
  in MATLAB

110
end
111
Introduction to MATLAB and image processing

• Amin Allalou
• amin_at_cb.uu.se
• Centre for Image Analysis
• Uppsala University
• Computer Assisted Image Analysis
• April 4 2008

112
Prepared by Fred Annexstein University of
Cincinnati Some Rights Reserved
113
CIS 350 1 Introduction to MATLAB
Dr. Rolf Lakaemper