Image Digitization and Processing

1
Lecture 9

• Image Digitization and Processing

2
Lecture outline

• Physics and psychophysics of light
• Raster images
• Vector images
• Image processing
• Adobe Photoshop
• Assignment 3
• Project presentations

3
Light

• Visible light is a kind of electromagnetic
  radiation
• Different kinds of electromagnetic radiation fall
  within different frequency bands

4
The visible spectrum

• We perceive colour based on the wavelength (or,
  equivalently, frequency) of the light
• 1 nm 1 nanometer 1 x 10-9 m

5
Colours of surfaces

• Roughly speaking, light from the sun or a light
  bulb can be considered to be white light
• Contains all visible frequencies
• The absence of light is perceived as black
• Surfaces that are struck by light tend to absorb
  certain wavelengths of light more than others
• The remaining light is reflected and can strike
  the eye, thus leading to the perception of the
  colour of objects
• Surfaces therefore effectively filter light when
  they reflect it

6
Colours of surfaces cont.

• Three things can happen when light strikes a
  surface
• Transmission certain wavelengths of light pass
  through the object and can be seen on the other
  side
• Reflection certain wavelengths bounce off the
  object.
• Absorption certain wavelengths are absorbed by
  the object and converted to thermal energy

7
Examples

• A blue stone appears blue because it absorbs
  wavelengths other than blue, and reflects blue.
  It would appear black in red light, because red
  light would still be absorbed and no blue light
  would be available.
• A piece of paper appears white because it
  reflects all parts of the visible spectrum. It
  would appear green in green light, because only
  green light is present to be reflected.
• A piece of liquorice appears black because it
  absorbs most or all of the visible spectrum. It
  would still appear black in green light, because
  green light would still be absorbed.
• A piece of red stained glass appears red when
  viewed from either side of the light source,
  because it absorbs wavelengths other than red,
  which it both reflects and transmits.

8
Physiology of the eye

• Light entering the pupil of the eye strike the
  retina, which contains cells known as rods and
  cones
• Rods allow us to perceive light intensity as
  brightness (roughly)
• Cones allow us perceive light wavelength as
  colour (roughly)
• There are believed to be three kinds of cones,
  each sensitive to a different bandwidth of light
• Centred around red, green and blue, but are
  sensitive to colours other than just these

9
Cone sensitivity curve
10
Light perception

• Chemical reactions in the rods and cones cause
  impulses to be sent to the brain via the optic
  nerve
• Perception of colour and brightness are a
  combination of physics, physiology and psychology
  
• Analogous to sound
• Colour is a perceptual quality, not directly
  physical
• White and black are not actually colours
• What we perceive as white is a combination of all
  components of the visible spectrum
• What we perceive as black is the absence of light

11
Perceptual combination of wavelengths

• Our brains assign colours to certain combinations
  of colours
• e.g. yellow light is perceived as yellow, but so
  is the combination of green and red light
  striking our retina

12
Primary colours

• In fact, the perception of white can be induced
  by the combination at equal intensities of three
  widely separated wavelengths in the visible
  spectrum
• Any three such wavelengths are called primary
  colours
• Red, green and blue are primary colours

13
Combination of primary colours

• The combination of primary colours in varying
  intensities can lead to the perception of a wide
  range of colours

14
Secondary colours

• Colours that are produced by the addition of
  equal intensities of two primary colors of light
  are called secondary colours
• Yellow, magenta and cyan are secondary colours
  corresponding to the primary colours of red,
  green and blue

15
Complementary colours

• Any two colors of light which produce white are
  said to be complementary colours
• Example The complementary color of red light is
  cyan light.
• R C R (G B) W

16
Colour addition and subtraction

• Colour addition is what occurs when multiple
  colours are combined by the visual system
• Colour subtraction is what happens when a colour
  is removed from a set of wavelengths
• e.g. absorption by an object

17
Colour wheel

• A. Green B. Yellow C. Red D. Magenta E. Blue F.
  Cyan

18
Depth perception

• Visual system uses 3 methods to determine
  distance
• The size a known object has on your retina - If
  you have knowledge of the size of an object from
  previous experience, then your brain can gauge
  the distance based on the size of the object on
  the retina.
• Moving parallax - When you move your head from
  side to side, objects that are close to you move
  rapidly across your retina. However, objects that
  are far away move very little. In this way, your
  brain can tell roughly how far something is from
  you.
• Stereo vision - Each eye receives a different
  image of an object on its retina because their
  separation. This is especially true when an
  object is close to your eyes. This is less useful
  when objects are far away because the images on
  the retina become more similar the farther they
  are from your eyes.

19
Introduction to raster images

• Raster images
• Rectangular layout of sampled values called
  pixels
• Each pixel has only one uniform colour
• Also called bitmapped images
• The more pixels there are per cm of the image,
  the greater the resolution
• Must consider purpose when deciding upon
  resolution
• Standard screen resolution is 72 dots per inch
  (dpi)
• Even low-end printers use higher resolutions (300
  to 600 dpi)
• Insufficient resolution leads to a blocky effect
  known as pixelation
• Pixelation becomes particularly obvious when
  images are enlarged

20
Pixelation examples
21
Creating digital images

• Generated directly with a computer
• Sampling pixels
• Scanner
• Digital camera
• Other opical input device

22
Pixel values

• Each pixel has one or more numbers associated
  with it controlling its colour
• Binary (monochrome) images
• 1-bit pixels (black or white)
• Called bitmapped in PhotoShop
• Greyscale images
• 1 colour channel (values ranging from black to
  white)
• Colour images
• Multiple colour channels enabling the portrayal
  of a variety of colours

23
Colour, greyscale and binary images
24
Greyscale images

• Each pixel has one colour channel
• Shades of grey are displayed
• Values are found by measuring the intensity of
  light at each pixel
• Varies from black at the weakest intensity to
  white at the strongest
• Often 8-bits per sample (allows 256 intensities)
• Scale used is typically non-linear

25
Colour images

• Remember a large variety of colours can be
  simulated by using combinations of primary
  colours in various intensities
• Red, green and blue the most often used colours
  (called RGB)
• Each pixel has an independent channel
  corresponding to each of these colours

26
Colour images cont.

• Each channel is assigned an intensity value
• Often 8 bits per channel
• 256 intensities per channel (0 to 255)
• 3 channels (RGB) x 8 bits 24 bits per pixel
• Can be combined to form black, white, 254 shades
  of grey, and 16,777,216 colours
• Sufficient colour palette for most uses
• Can lead to very large files at high resolution
• Total number of bits per pixel called bit depth
  or colour depth
• Colour depth of 24 bits or greater called true
  colour

27
4 bit vs. 24 bit colour
28
RGB intensity histograms

• A statistical representation of an entire image
• Shows the relative occurrence of pixel intensity
  values for each of the red, green and blue
  channels
• X-axis one bin for each intensity level (0 to
  255 for 8 bit channels)
• Y-axis shows relative number of pixels at the
  given intensity

29
Brightness

• Brightness refers to
• Overall intensity of the image
• On intensity histogram
• Low brightness bin frequencies higher to the
  left of the histogram
• High brightness bin frequencies higher to the
  right of the histogram

30
Brightness examples
Right medium brightness Bottom low
brightness Bottom right high brightness
31
Gamma

• Gamma is a measure of mid-tone color brightness
• Increasing gamma will increase overall brightness
  with a slightly greater effect on midtones
• Windows systems use a higher gamma value than Mac
  OS systems
• The same image is noticeably darker on a Windows
  system than on a Mac OS system

32
Luminosity

• Luminosity is a measure of the overall brightness
  of a given pixel
• Colour images luminosity is the average of the
  RGB channels
• Greyscale images luminosity is the pixel
  intensity
• Colour images can be converted to greyscale using
  luminosity values

33
Contrast

• Contrast refers to
• The difference in visual properties that makes an
  object distinguishable from other objects and the
  background
• On intensity histogram
• Low contrast image levels clustered together or
  spread out evenly in one group
• High contrast image levels clustered in discrete
  groups

34
Contrast examples
Right medium contrast Bottom low
contrast Bottom right high contrast
35
Colour models

• RGB (red, green, blue) is the most common colour
  model, but there are others as well, including
• CMYK (cyan, magenta, yellow, black)
• HSB (hue, saturation, brightness)
• CIE Lab
• Greyscale
• Colour models affect
• Number of colours that can be displayed in an
  image
• Which colours can be displayed
• Number of channels
• File size of an image
• Colour models represent different coordinate
  systems

36
RGB model

• Uses red, green and blue in varying combinations
  to produce a wide variety of colour
• Remember these are primary colours
• Visual system perceives many colours through
  additive property of primary colours

37
RGB model cont.

• RGB model well suited to computer monitors
• Monitors can create colours by emitting red,
  green, and blue light, and letting the visual
  system integrate them
• Each pixel usually has 3 channels, with 8 bits
  each
• 24 bits per pixel

38
CMYK model

• Uses cyan (C), magenta (M), yellow (Y) and black
  (K)
• Based on the light-absorbing quality of ink
  printed on paper.
• As white light strikes, part of the spectrum is
  absorbed and part is reflected to your eyes
• In theory, pure C, M and Y pigments could combine
  to absorb all colour and produce black. For this
  reason these colors are called subtractive
  colours.

39
CMYK model cont.

• Since printing inks contain some impurities, CMY
  inks actually produce a muddy brown and must be
  combined with black ink to produce a true black
• CMYK model well suited for printers
• Each pixel usually has 4 channels, with 8 bits
  each
• 32 bits per pixel

40
HSB model

• Based on human perception of colour
• Hue Colour reflected from or transmitted through
  an object. It is measured as a location on the
  standard colour wheel, expressed as a degree
  between 0 and 360. Identified by the name of
  the colour (e.g. red, orange, or green) .
• Saturation (or chroma) Strength or purity of the
  color. Represents the amount of grey in
  proportion to the hue, measured from 0 (grey) to
  100 (fully saturated). On the standard colour
  wheel, saturation increases from the centre to
  the edge.
• Brightness Relative lightness or darkness of the
  colour, usually measured as a percentage from 0
  (black) to 100 (white).

41
Illustration of HSB model

• A. Saturation B. Hue C. Brightness D. All hues

42
CIE Lab model

• Designed to be device independent, creating
  consistent colour regardless of the device (such
  as a monitor, printer or scanner) used to create
  or output the image.
• Consists of
• Luminance or lightness component (L)
• Two chromatic components the a component (from
  green to red) and the b component (from blue to
  yellow).
• Luminance 100 (white)
• Green to red component
• Blue to yellow component
• Luminance 0 (black)

43
Colour matching

• No device can reproduce the full range of colours
  viewable by the human eye
• Each device operates within a specific colour
  space, which can produce a certain range,
  or gamut, of colours
• Different devices produce different gamuts
• Colours therefore shift in appearance as you
  transfer images between different devices and
  between different colour models
• Need to perform calibrations in order to
  standardize
• Set colour profiles are use to standardize
  colours across devices

44
Colour lookup tables

• Rather than allowing all possible colours, some
  file formats include a colour lookup table or
  palette in file header
• Avoids wasting bits by reserving bit combinations
  that represent colours that are not present in
  the image
• Generally limited to 8 bits (256 colours)

45
Alpha channel

• An additional alpha channel is sometimes added to
  each pixel
• Describes the transparency of each pixel when
  composited (superimposed) over another image
• Usually 8 bits
• 0 (fully transparent) to 255 (fully opaque)

46
Alpha channel example 1

• Alpha channel Composited
  images

47
Alpha channel example 2
Alpha channel Composited
images
48
Alpha channel example 3

• Alpha channel Composited
  images

49
Dithering

• Dithering is a way of making few colours appear
  to be many colours
• Differently coloured adjacent pixels are used to
  simulate colours and shades that do not actually
  exist in an image's colour palette
• Dithering fools the eye into seeing colours that
  are not really there
• Dithering produces visible artefacts
• Dithering is often used to compensate for loss of
  colours when the colour depth is reduced

50
Example of dithering
lt 24 bits 8 bits, dithered gt lt 8 bits,
undithered
51
Anti-aliasing

• Smooth lines and curves often appear jagged
• Called jaggies
• Occurs at low resolutions or when bitmaps are
  enlarged
• Anti-aliasing is a technique for smoothing out
  jagged edges in a bitmap
• An illusion of blending is created by placing
  similarly coloured pixels next to one another
• Can be done by an image processor or dynamically
  by a graphics card
• Two disadvantages
• Can increase file size because compression works
  best with solid colours
• Causes "fringe" effect, since edges are blended
  with their adjacent colours. A problem when using
  transparency, resizing or edge-detection
  algorithms.

52
Example of anti-aliasing
Without anti-aliasing With anti-aliasing
53
Interleaving

• Interleaving refers to the order in which pixels
  are stored
• Non-sequential interleaving useful
• Resistance to file errors due to corruptions of
  adjacent bytes will be spread throughout image
  rather than be concentrated in one spot
• Full area of an image can be seen during low
  bandwidth downloads before download is complete
• Done with audio too

54
Image compression

• Can reduce resolution or colour depth
• Non-lossy compression techniques
• Take advantage of redundancies in data to reduce
  file size without discarding any information
• Huffman encoding uses frequency tables to
  efficiently store bytes that often reoccur
• Run-length encoding abbreviates sequences of
  repeated bits
• Lossy compression techniques
• Use mathematical processes that discard
  information that will (hopefully) not be
  perceived
• In practice, often leaves artefacts
• Original image cannot be reconstructed exactly

55
GIF image file format

• 256 colour indexed colour palette
• Uses lossless compression
• Offers two special features
• Animation a single image can contain multiple
  frames that most web browsers can play
  sequentially without additional plug-ins
• Transparency can choose one (and only one)
  colour to be transparent (i.e. allow whatever is
  displayed "beneath" it to show through)

56
Strengths and weaknesses of GIFs

• Good for images composed primarily of lines and
  solid blocks of color (space efficient)
• Bad for images with complex, subtle gradations of
  colour

57
JPEG image file format

• Designed for use with images with smooth,
  continuous tones, like photographs
• Allows up to 24 bit colour depth
• No alpha channel
• Uses lossy compression
• Image quality is reduced each time a JPEG is
  saved
• Can choose a compression level from low to high

58
Strengths and weaknesses of JPEGs

• Good for photographs and other images with many
  different tones
• Performs poorly on solid blocks of colour because
  its attempt to smooth out the colours often
  results in blotchiness

59
Comparison of JPEG compression levels
60
BMP image file format

• Developed as standard Windows image format
• Uncompressed file format (usually)
• Takes up lots of disk space
• RGB, indexed colour and greyscale colour models
  supported
• No alpha channel

61
TIFF image file format

• Uses lossless compression
• Roughly 50 compression can be achieved
• CMYK, RGB, greyscale, indexed colour and Lab
  colour models supported
• Alpha channel supported
• Adobe Photoshop can save layers and other
  information in TIFFs

62
Problems with raster images

• Individual objects in raster images cannot be
  moved or altered independently without affecting
  the rest of the image (e.g. leaving a blank spot
  behind)
• Can be difficult even to select different
  components
• Raster images can sometimes appear jagged or
  individual pixels can be seen when low
  resolutions are used or it is necessary to
  convert between resolutions. This can also be a
  problem when raster images are magnified.
• Different resolutions can be used by monitors,
  printers, scanners, digital cameras and different
  file formats
• Image manipulation irretrievably changes image

63
Introduction to vector images

• Unlike raster images, vector image files do not
  store bitmaps of pixels
• Vector images consist of lines and curves
  expressed as mathematical objects that include
  colourings
• Images dynamically rendered to produce bitmap of
  pixels at time of display (called rasterization)
• Example a bicycle tire in a vector graphic is
  made up of a mathematical definition of a circle
  drawn with a certain radius, set at a specific
  location, and filled with a specific color

64
Comparison of vector and raster images

• Vector images
• Can easily select individual objects
• Can resize, warp or otherwise manipulate objects
  without introducing degradations
• Can display at any resolution without quality
  loss
• Can move or otherwise manipulate individual
  objects without altering rest of image
• Files usually much smaller than raster images
• Raster images
• Appear more natural
• Represent colours and colour gradations well
• Easy to generate (e.g. digital photograph or
  scan)
• Do not require overhead to dynamically render
• Analogous to difference between symbolic (e.g.
  MIDI) and audio (e.g. wave) files

65
Comparison of raster and vector images cont.
66
PNG image file format

• Can contain both bitmapped and vector-based image
  data
• Created specifically for the web in order to
  replace older GIF format, but still not widely
  adopted
• Allows colour depth up to 48 bits.
• Improved lossless compression relative to GIFs
• Almost always smaller (5-25) than identical GIF
  images
• Does not support animation
• Additional features
• Cross-platform colour and gamma correction
• Compensates for colour and brightness variations
  between different monitors
• Full alpha transparency
• Images can have graduated transparency

67
PostScript (PS) files

• Invented by Adobe in the 1980s
• A programming language optimized for printing
  graphics and text
• Describes images in a device independent manner
• The same PostScript file can be given to any
  PostScript printer without alterations
• Originally intended only for printing

68
EPS files

• EPS (Encapsulated PostScript) files can store
  single images that can be incorporated into
  larger .ps files
• Can contain vector graphics, bitmapped graphics
  and text
• Supports Lab, CMYK, RGB, indexed colour,
  Duotone and greyscale colour models
• Does not support alpha channels

69
PDF files

• Also developed by Adobe, but more recent
• Intended for platform independent transfer of
  text, image and multimedia information
• Preserves fonts, page layouts and both vector and
  bitmap graphics
• Can contain electronic document search and
  navigation features such as electronic links

70
PSD files

• Format used by Adobe PhotoShop
• Not portable, in general, to other applications
• Preserves information such as layers and
  operation history that would be lost in other
  formats
• Good for preliminary versions of images
• Final versions should be distributed in
  non-proprietary formats

71
Basic image processing transforms

• Flipping - lossless
• Rotation - lossless in multiples of 90 degrees
• Resizing - lossy
• Stretching - lossy
• Translation - lossy (what is left behind?)
• Cropping - lossless for the portion not cropped
• All these operations are lossless with vector
  images

72
Levels-based operations

• Can be local or global
• Brightness adjustments
• Contrast adjustments
• Colour balance adjustments
• Levels for each channel are adjusted according to
  some function(s)
• Posterization
• A number of brightness levels for each channel is
  specified, and all pixels are mapped to the
  closest matching levels

73
Levels-based operations cont.

• Inversion
• The brightness value of each pixel in the
  channels is converted to the inverse value on the
  256-step colour-values scale
• e.g. a pixel in a positive image with a value of
  255 is changed to 0 in the negative image, and a
  pixel with a value of 5 to 250
• Thresholding
• Pixels whose levels for a given channel or
  channels fall within a certain range (or ranges)
  are all converted to the same colour (or colours)

74
Sophisticated image processing operations

• Some operations referred to as applying filters
• This term does not coincide intuitively with the
  meaning of signal processing filters as discussed
  earlier
• Many possible operations
• Will introduce some examples here
• Many more examples under Filters menu in Adobe
  PhotoShop

75
Blur filters

• Averages levels of pixels in a limited
  neighbourhood
• Softens image, particularly sudden transitions
• Useful for removing salt and pepper (speckled)
  noise
• Before (left) and after (right) application of
  blur filter

76
Sharpen filters

• Increases contrast of adjacent pixels
• Causes blurry or out of focus images to become
  clearer
• Before (left) and after (right) application of
  sharpen filter

77
Edge detection

• Automatically detect edges of objects in an image
• Algorithms look for areas where significant
  colour changes occur
• Algorithms often make use of vector calculus
  (e.g. gradients, Laplacians)
• Useful for many other operations
• Automatic conversion to vector images
• Noise removal
• Object recognition
• Etc.

78
Edge detection example
79
Region detection

• Automatic segmentation of image into regions
• Contiguous similar pixels (usually based on
  similarity of levels)
• Areas contained within detected edges
• Colours falling within peaks in an intensity
  histogram
• Like edge detection, useful for other tasks like
  noise removal and object recognition

80
Region detection example
81
Vermeers Girl WithPearl Earing
82
Noise removal filters

• Blurring helps to remove random (Gaussian noise)
• Despeckling blurs all of the image except edges
  (found using edge-detection). This blurring
  removes noise while preserving detail.
• Region growing/shrinking helps to remove small
  regions that could be a hair or a scratch,
  example
• Many other approaches exist as well
• Most methods introduce artefacts of their own

83
Illumination

• Addition of a light source in or outside image
• Parameters
• Intensity
• Location
• Reflectivity of objects (image often given a mask
  denoting reflectivity)
• Of particular importance to 3-D images

84
Depth and texture

• Can simulate depth by providing different images
  to each eye
• Virtual reality glasses do this directly
• Classic 3-D glasses
• Image has red and blue channels slightly offset
  from one another
• Red and blue glasses filter out different parts
  of spectrum for each eye, so each eye sees a
  different image.
• The human visual system interprets this as three
  dimensional.

85
Depth and texture cont.

• Can also use classic techniques of perspective to
  simulate 3-D without needing to provide separate
  images to eyes
• Lighting can also be used to fool the visual
  system
• Brightness gradients help to achieve impression
  of depth

86
Example of simulating depth and texture
87
Object recognition

• The automatic recognition of objects in an image
• Makes use of statistical pattern recognition
  and/or artificial intelligence
• Very difficult and specialized
• Very successful in some cases
• e.g. OCR (optical character recognition)
• Recall OMR (optical music recognition) from last
  lecture

88
Image processing software

• Many image processors available
• Adobe PhotoShop
• GNU GIMP
• Voted for Photoshop last class

89
Using the scanner

• Converts photos into digital image files
• MTCL lab has scanner at instructors computer
• Desktop gt Macintosh HD gt Applications gt CanoScan
  Toolbox 4.1 gt CanoScan Toolbox X
• Need to boot locally when scan
• Probably easier to use library scanner

90
Using PhotoShop

• PhotoShop only available on instructors computer
• Desktop gt Macintosh HD gt Applications gt Adobe
  Photoshop 7 gt Adobe Photoshop 7.0
• On-line help
• Desktop gt Macintosh HD gt Applications gt Adobe
  Photoshop 7 gt Help gt help.html
• Many tutorials available in web
• Reference book in library
• Teague, J. C. 2003. Photoshop 7 at Your
  Fingertips Get In, Get Out, Get Exactly What
  You Need. San Francisco SYBEX.
• T385 T42 2003

91
PhotoShop Demo Topics

• Loading and saving images
• Toolbox
• Tool options bar changes based on tool selected
• Layers Palette
• Flattening layers
• History Palette
• Transforms
• Level adjustments
• Filters

92
Using GIMP

• Go menu gt Applications gt Gimp
• Must be installed in home directories
• Specify a local path for the tmp swap space
• e.g. Volumes/Macintosh HD/Workspace
• Takes long time to start up, probably hasnt
  crashed
• Usability pointers
• Double click tools to select them
• Must select image areas in order to be able to
  draw there

93
Class work

• Notes posted in PowerPoint not PDF now
• Assignment 3
• Download photos on course web page
• Questions 9 and 10
• Project proposals
• Questions
• Written proposals due
• 2 to 3 minutes per student