4' Image Operations

1
4. Image Operations

• 1. Images
• 2. Getting to know the Processing IDE
• 3. Interpolation

2
Processing Images
3
Basic Types

• Commonly used data types
• char k
• int I
• float d
• color c // actually a vector
• The red(), green(), and blue() functions can be
  used to extract individual values from a color
  object
• e.g. float r red (c)
• Conversely you can create a color from three
  scalars using the color constructor
• c color ( 255, 0, 1)

4
Type casting

• If you declare a variable as one type, you cant
  give it a value of another type
• int I
• I 1.6 //error!
• If for some reason you need to convert from one
  to the other you do it with a cast
• I (int) 1.6 //this converts 1.6 to the nearest
  integer and stores it in the variable I

5
Basic Arrays

• Declaring arrays in processing
• int numbers new int3
• Accessing arrays (pretty much like c/c/java)
• numbers0 90
• numbers1 150
• numbers2 30
• Alternatively
• int numbers 90, 150, 30
• int a numbers0 numbers1 // Sets variable
  a to 240
• int b numbers1 numbers2 // Sets variable
  b to 180

6
Variable-size Arrays

• int degrees 360
• float cos_vals new floatdegrees
• for(int i0 i lt degrees i)
• cos_valsi cos(TWO_PI/degrees i)

7
Setting the colour of pixels

• stroke (255, 0, 0)
• point (x, y)
• e.g. paint the screen black .
• for every pixel set the pixel colour to 0,0,0
  (black)
• stroke (0,0,0)
• for (int i0 iltwidth i)
• for (int j0 jltheight j)
• point (i, j)

width and height are special variables, that are
automatically assigned values in the processing
environment
8
Draw a Line (simple DDA)

• E.g. draw a line from 20, 20 to 200, 200
• int x1 20, y1 20 //start point
• int x2 200, y2 200 //end point
• y y1
• m (y2-y1) / (x2-x1) //gradient
• for (int xx1 xltx2 xx1)
• y y 2
• point (x, y)

Of course, there are existing operations to do
this e.g. background (0, 0, 0) line (20,
20, 200, 200)
9
Direct vs Event-driven Mode

• If your program isnt highly interactive e.g.
  simple image processing, simply state your
  instructions step by step
• But processing allows to work in a simple
  event-driven mode, similar to most windows
  programs.
• Event driven programs loop infinitely checking to
  see if some event (e.g. user input) has occurred.
• The programmer provides a callback function, for
  each event that he wishes to deal with. The
  system executes the callback function when the
  relevant event occurs.

10
An Event Loop
User
redraw
mouseMoved
mousePressed
Windows System
keyPressed
Event List
Operating System
Application
while (TRUE) e get_next_event() switch (e)
case MOUSEPRESSED call registered
mousePressed() function case KEYPRESSED
call registered keyPressed() function ...
11
Interactive programs

• The setup () function is where you state
  operations you want performed once when your
  program begins
• The draw () function gets called repetitively
  (unless you stop it)
• noloop() stops redrawing loop() restarts it
• Several mouse callbacks exist including
  mousePressed() , mouseReleased, mouseMoved(),
  mouseDragged()
• Mouse positions are returned in mouseX, mouseY
• The keyPressed() function gets called when you
  press a key. You can get the key pressed using
  the global variable keyPressed

12
Advanced Data Images

• An image is an array of pixels, each of which has
  an RGB value.
• An Image file usually contains
• Width and height of image
• An array of rgb values for individual pixels in
  the image
• In processing, PImage is a predefined
  data-structure for images
• It has the following attributes
• Pixels //array of pixels
• width
• height
• And the following functions
• get( x , y )
• set( x, y, color )
• These can also be applied to the window
  (framebuffer) which behaves like an image

13
Loading and drawing images

• Declare a new Image using the following code
• PImage img
• Other image related functions
• img LoadImage ( picture.gif)
• image (img, 0, 0)
• img.set (10, 10, color ( 255, 0, 0) )
• color c img.get(20, 20)

Instantiate a PImage object called img.
Load the file picture.gif into memory.
This draws the image with top-left corner
starting at 0, 0.
Set pixel 10, 10 to color 255, 0, 0 or red
Get the color of pixel at 20, 20 N.B. get and
set called without a PImage object will affect
the drawing window
For more info http//processing.org/reference/PI
mage.html
14
Iterate through an image

• for (int i0 iltim.width i)
• for (int j0 jltim.height j)
• im.set (i,j, C ) // c is a color
• N.B. the drawing window itself can be considered
  an image (remove im in the above code)

15
Sample Code

• PImage im
• void setup()
• imloadImage(../Image8.jpg)
• size(im.width, im.height)
• noLoop()
• void draw()
• color c, cc
• for(int x0 xltim.width x)
• for(int y0 yltim.height y)
• c im.get(x,y)
• //cc ? Provide operation here

• The following sample code is used for the
  examples in the rest of the lecture.
• What we essentially want to do in image
  processing is calculate a new colour cc for each
  pixel.
• This is based on the original colour c and some
  pixel operations which we will define.

In the following slides, we will provide
additional code that you can plug into this
sample program to get the desired effect.
16
Simple bit-copy functions

• Simply juggling pixels around can achieve some
  useful effects e.g.
• Rotating an Image by 90 degrees
• set(x, y, img.get(y, x))
• Mirroring an Image
• set(x, y, img.get(width-x, y))
• Flipping an Image
• set(x, y, img.get(x, height-y))
• Drawing an image twice as big
• set(x2, y2, img.get(x, y))
• set(x21, y2, img.get(x, y))
• set(x2, y21, img.get(x, y))
• set(x21, y21, img.get(x, y))

(repeat these for all pixels in the image)
17
Processing Images

• We change an image by applying a Procedure to
  some or all of its pixels.
• Examples
• Greyscale the Image
• Clear Image
• Threshold Image
• Mirror the image
• Line Drawing
• Anti-aliasing
• Filters
• Colour
• Edge detect
• Blur/Smooth

18
Operations on pixels greyscale

• An average or weighted average of the RGB
  intensities
• Just make sure the final result has values in the
  range from 255 to 255
• Unweighted
• cc (RGB) / 3
• Weighting depends on the sensor but usually green
  is given more importance
• cc 0.3R 0.59G 0.11B
• In processing place the following in the previous
  sample code
• cc color ( 0.3red(c) 0.59green(c)
  0.11blue(c) )

19
Negative image

• If 255 is full intensity then the negative value
  of any intensity c is 255 c
• e.g. the opposite of red (240, 0, 0) is
  (255-240, 255-0, 255-0) i.e. (15, 255, 255) cyan
• Code
• cc color ( 255-red(c), 255-green(c),
  255-blue(c) )

20
Thresholding

• Traditionally describes the operation of reducing
  an image to a 2 colour bitmap
• All pixels less than a certain threshold level of
  brightness are drawn as black. The rest are drawn
  white
• We can increase/decrease the threshold value to
  highlight areas of bright contrast at the expense
  of variations in dark areas, or vice versa.

Output
Input
High threshold all but the brightest areas are
rendered as black
Low threshold All these levels of brightness are
converted to white.
A halfway threshold.
21
Multiple Thresholding

• Alternatively we can set a number of discrete
  limits and render with a number of different
  shades e.g. 50, 100, 250, 200
• All intensities are rounded down to the closest
  of these thresholds
• This has some useful applications e.g. mosaic
  effects, colour depth reduction

22
Thresholding code

• Simple thresholding
• float g 0.3red(c) 0.59green(c)
  0.11blue(c)
• if (ggt128)
• cc color (255)
• Else
• cc color (0)
• Levels
• float g 0.3red(c) 0.59green(c)
  0.11blue(c)
• cc color (0)
• if (ggt50)
• cc color (50)
• if (ggt100)
• cc color (100)
• if (g gt150)
• cc color (150)
• //etc. ..

23
Simple Gamma Correction

• Thresholding is similar to gamma correction.
• Gamma correction increases/decreases the
  intensities of colours smoothly (no discrete
  threshold levels).

Linear
Originally, input and output values are the same
2. Pixel is rendered this level in final image.
Gamma corrected curve increasing brightness.
1. Original brightness of a sample pixel
24
Halftoning

• A method for simulating a greater range of
  colours using patterns of ink spots of varying
  concentration.
• Originally used in print media.
• This works because our visual system picks up
  localised averaged intensities of colour.

25
Digital Halftones

• Simple half tone patterns replace thresholded
  pixels.
• Code (just level 2) effectively involves the
  following
• if ( (ggt100) (glt150) )
• set(x2, y2, color(255))
• set(x21, y2, color(0))
• set(x2, y21, color(0)
• set(x21, y21, color(255))

0
1
2
3
4
Full code provided at http//isg.cs.tcd.ie/dingli
aj/Processing/halftone/
26

• We can also apply a 3x3 pattern to support 9
  different threshold values

27
Error Diffusion

• As we saw before, a pixel is thresholded (by
  rounding down its intensity to the nearest
  threshold level)
• The difference in actual intensity and threshold
  intensity is the error in the pixel.
• i.e. it is actually rendered less bright than the
  original
• some brightness is lost in the overall image
• To compensate for this error we can distribute it
  over its neighbouring pixels.

28
Error Diffusion

• Error current pixel value minus nearest
  threshold value
• Change the pixel values of neighbouring pixels in
  the original image before they are processed.
• a a ( 7/16 ) x Error
• b b ( 3/16 ) x Error
• c c ( 5/16 ) x Error
• d d ( 1/16 ) x Error

The image is drawn from left to right so a common
method is to diffuse the error to the right and
below the current pixel.
Full code provided at http//isg.cs.tcd.ie/dingli
aj/Processing/errordiffuse/ Comparision with
halftoning http//isg.cs.tcd.ie/dingliaj/Processi
ng/diffusion/
29
Dithering Patterns

• A more elegant way of halftoning
• On the left the dither pattern is twice the width
  and height of the original image
• Each original pixel is compared to a 2x2 block of
  pixels with the following values
• If the original value is higher than the value in
  the dither pixel, then it is enabled.

30
Mosaics

• Threshold values can be used to select patterns,
  or even other images to replace small areas of
  the original image.

http//isg.cs.tcd.ie/dingliaj/Processing/mosaic/
31
http//isg.cs.tcd.ie/dingliaj/Processing/picmosaic
.zip
32
(No Transcript)
33
(No Transcript)
34
Edge detection

• How does a pixel compare to its neighbours.
• 5 point laplacian operator
• if c (x, y) is the color of pixel at x,y
• Then we calculate
• 4c(x,y) c(x1, y) c(x-1,y) c(x, y1)
  c(x, y-1)
• i.e. 4 x current colour sum of neighbours
• If the pixel colour is the same as its neighbours
  the result is low or close to 0. Otherwise the
  value indicates the steepness of the change.

35
5-point laplacian
9-point laplacian
http//isg.cs.tcd.ie/dingliaj/Processing/laplac/
36
Original image showing threshold levels
Filtered image, note the high numbers in the
strong edge pixels
37
Linear Smoothing

• Uniform Filter For each pixel x, y, look at
  nearby pixels in a certain radius and average
  the values.
• Non-uniform filters take what is essentially a
  weighted average of nearby pixels

Rectangular
Circular
Pyramidal
Cone
http//isg.cs.tcd.ie/dingliaj/Processing/smooth/
38
Kuwuhara

• The Kuwuhara filter performs an edge preserving
  smooth
• For each pixel x,y, look at immediate adjacent
  regions
• Find the variance i.e. difference between min and
  max intensity
• For the region with the minimum variance
• Get its MEAN (average) colour and apply to x,
  y

http//isg.cs.tcd.ie/dingliaj/Processing/Kuwuhara/
5x5 Kuwuhara
39
Blending two images

• If a and b are pixels colors in imageA and imageB
  respectively
• And cc is the color of the blended images
• then
• cc (1-a)a ab
• Different values of the variable a will increase
  the emphasis on imageA or imageB
• We can do a smooth fade from imageA to imageB by
  gradually increasing a
• float r (1-a) red(imageA.get(x,y)) a
  red(imageB.get(x,y))
• float g (1-a) green(imageA.get(x,y)) a
  green(imageB.get(x,y))
• float b (1-a) blue(imageA.get(x,y)) a
  blue(imageB.get(x,y))
• cc color (r, g, b)
• a a0.01

http//isg.cs.tcd.ie/dingliaj/Processing/blend/