Image Formation Fundamentals

1
Image Formation Fundamentals

• CS491E/791E

2
How are images represented in
the computer?
3
Color images
4
Image formation

• There are two parts to the image formation
  process
• The geometry of image formation, which determines
  where in the image plane the projection of a
  point in the scene will be located.
• The physics of light, which determines the
  brightness of a point in the image plane as a
  function of illumination and surface properties.

5
A Simple model of image formation

• The scene is illuminated by a single source.
• The scene reflects radiation towards the camera.
• The camera senses it via chemicals on film.

6
Pinhole camera

• This is the simplest device to form an image of a
  3D scene on a 2D surface.
• Straight rays of light pass through a pinhole
  and form an inverted image of the object on the
  image plane.

7
Camera optics

• In practice, the aperture must be larger to admit
  more light.
• Lenses are placed to in the aperture to focus the
  bundle of rays from each scene point onto the
  corresponding point in the image plane

8
Image formation (contd)

• Optical parameters of the lens
• lens type
• focal length
• field of view
• Photometric parameters
• type, intensity, and direction of illumination
• reflectance properties of the viewed surfaces
• Geometric parameters
• type of projections
• position and orientation of camera in space
• perspective distortions introduced by the imaging
  process

9
Image distortion
10
What is light?

• The visible portion of the electromagnetic (EM)
  spectrum.
• It occurs between wavelengths of approximately
  400 and 700 nanometers.

11
Short wavelengths

• Different wavelengths of radiation have different
  properties.
• The x-ray region of the spectrum, it carries
  sufficient energy to penetrate a significant
  volume or material.

12
Long wavelengths

• Copious quantities of infrared (IR) radiation are
  emitted from warm objects (e.g., locate people in
  total darkness).

13
Long wavelengths (contd)

• Synthetic aperture radar (SAR) imaging
  techniques use an artificially generated source
  of microwaves to probe a scene.
• SAR is unaffected by weather conditions and
  clouds (e.g., has provided us images of the
  surface of Venus).

14
Range images

• An array of distances to the objects in the
  scene.
• They can be produced by sonar or by using laser
  rangefinders.

15
Sonic images

• Produced by the reflection of sound waves off an
  object.
• High sound frequencies are used to improve
  resolution.

16
CCD (Charged-Coupled Device) cameras

• Tiny solid state cells convert light energy into
  electrical charge.
• The image plane acts as a digital memory that can
  be read row by row by a computer.

17
Frame grabber

• Usually, a CCD camera plugs into a computer board
  (frame grabber).
• The frame grabber digitizes the signal and stores
  it in its memory (frame buffer).

18
Image digitization

• Sampling means measuring the value of an image at
  a finite number of points.
• Quantization is the representation of the
  measured value at the sampled point by an integer.

19
Image digitization (contd)
20
Image quantization(example)

• 256 gray levels (8bits/pixel) 32 gray levels
  (5 bits/pixel) 16 gray levels (4 bits/pixel)
• 8 gray levels (3 bits/pixel) 4 gray
  levels (2 bits/pixel) 2 gray levels (1
  bit/pixel)

21
Image sampling (example)

• original image sampled by a factor
  of 2
• sampled by a factor of 4 sampled by a
  factor of 8

22
Digital image

• An image is represented by a rectangular array of
  integers.
• An integer represents the brightness or darkness
  of the image at that point.
• N of rows, M of columns, Q of gray
  levels
• N , M , Q (q is the of
  bits/pixel)
• Storage requirements NxMxQ (e.g., NM1024, q8,
  1MB)

23
Image coordinate system
24
Image file formats

• Many image formats adhere to the simple model
  shown below (line by line, no breaks between
  lines).
• The header contains at least the width and height
  of the image.
• Most headers begin with a signature or magic
  number - a short sequence of bytes for
  identifying the file format.

25
Common image file formats

• GIF (Graphic Interchange Format) -
• PNG (Portable Network Graphics)
• JPEG (Joint Photographic Experts Group)
• TIFF (Tagged Image File Format)
• PGM (Portable Gray Map)
• FITS (Flexible Image Transport System)

26
Comparison of image formats
27
PGM format

• A popular format for grayscale images (8
  bits/pixel)
• Closely-related formats are
• PBM (Portable Bitmap), for binary images (1
  bit/pixel)
• PPM (Portable Pixelmap), for color images (24
  bits/pixel)
• ASCII or binary
  (raw) storage

28
ASCII vs Raw format

• ASCII format has the following advantages
• Pixel values can be examined or modified very
  easily using a standard text editor.
• Files in raw format cannot be modified in this
  way since they contain many unprintable
  characters.
• Raw format has the following advantages
• It is much more compact compared to the ASCII
  format.
• Pixel values are coded using only a single
  character !

29
Image Class

• class ImageType
• public
• ImageType()
• ImageType()
• // more functions ...
• private
• int N, M, Q //N rows, M columns
• int pixelValue

30
An example - Threshold.cpp

• void readImageHeader(char, int, int, int,
  bool)
• void readImage(char, ImageType)
• void writeImage(char, ImageType)
• void main(int argc, char argv)
• int i, j
• int M, N, Q
• bool type
• int val
• int thresh
• // read image header
• readImageHeader(argv1, N, M, Q, type)
• // allocate memory for the image array
• ImageType image(N, M, Q)
• // read image

31
Threshold.cpp (contd)

• cout ltlt "Enter threshold "
• cin gtgt thresh
• // threshold image
• for(i0 iltN i)
• for(j0 jltM j)
• image.getVal(i, j, val)
• if(val lt thresh)
• image.setVal(i, j, 255)
• else
• image.setVal(i, j, 0)
• // write image
• writeImage(argv2, image)

32
Reading/Writing PGM images
33
Writing a PGM image to a file

• void writeImage(char fname, ImageType image)
• int N, M, Q
• unsigned char charImage
• ofstream ofp
• image.getImageInfo(N, M, Q)
• charImage (unsigned char ) new unsigned char
  MN
• // convert the integer values to unsigned char
• int val
• for(i0 iltN i)
• for(j0 jltM j)
• image.getVal(i, j, val)
• charImageiMj(unsigned char)val

34
Writing a PGM image... (contd)

• ofp.open(fname, iosout)
• if (!ofp)
• cout ltlt "Can't open file " ltlt fname ltlt endl
• exit(1)
• ofp ltlt "P5" ltlt endl
• ofp ltlt M ltlt " " ltlt N ltlt endl
• ofp ltlt Q ltlt endl
• ofp.write( reinterpret_castltchar gt(charImage),
  (MN)sizeof(unsigned char))
• if (ofp.fail())
• cout ltlt "Can't write image " ltlt fname ltlt endl
• exit(0)
• ofp.close()

35
Reading a PGM image from a file

• void readImage(char fname, ImageType image)
• int i, j
• int N, M, Q
• unsigned char charImage
• char header 100, ptr
• ifstream ifp
• ifp.open(fname, iosin)
• if (!ifp)
• cout ltlt "Can't read image " ltlt fname ltlt endl
• exit(1)
• // read header
• ifp.getline(header,100,'\n')
• if ( (header0!80) / 'P' /
• (header1!53) ) / '5' /
• cout ltlt "Image " ltlt fname ltlt " is not PGM"
  ltlt endl

36
Reading a PGM image . (contd)

• ifp.getline(header,100,'\n')
• while(header0'')
• ifp.getline(header,100,'\n')
• Mstrtol(header,ptr,0)
• Natoi(ptr)
• ifp.getline(header,100,'\n')
• Qstrtol(header,ptr,0)
• charImage (unsigned char ) new unsigned char
  MN
• ifp.read( reinterpret_castltchar gt(charImage),
  (MN)sizeof(unsigned char))
• if (ifp.fail())
• cout ltlt "Image " ltlt fname ltlt " has wrong size"
  ltlt endl
• exit(1)

37
Reading a PGM image(contd)

• //
• // Convert the unsigned characters to integers
• //
• int val
• for(i0 iltN i)
• for(j0 jltM j)
• val (int)charImageiMj
• image.setVal(i, j, val)

38
How do I see images on the computer?

• Unix xv, gimp
• Windows Photoshop

39
How do I display an image from within my C
program?

• Save the image into a file with a default name
  (e.g., tmp.pgm) using the WriteImage function.
• Put the following command in your C program
• system(xv tmp.pgm)
• This is a system call !!
• It passes the command within the quotes to the
  Unix operating system.
• You can execute any Unix command this way .

40
How do I convert an image from one format to
another?

• Use xvs save option
• It can also convert images

41
How do I print an image?

• Load the image using xv
• Save the image in postscript format
• Print the postscript file (e.g., lpr -Pname
  image.ps)

42
Image processing software

• CVIPtools (Computer Vision and Image Processing
  tools)
• Intel Open Computer Vision Library
• Microsoft Vision SDL Library
• Matlab
• Khoros
• For more information, see
• http//www.cs.unr.edu/bebis/CS791E
• http//www.cs.unr.edu/CRCD/ComputerVision/cv_resou
  rces.html