Digital Cameras

1
Digital Cameras

• Engineering Math Physics (EMP)
• Jennifer Rexford
• http//www.cs.princeton.edu/jrex

2
Image Transmission Over Wireless Networks

• Image capture and compression
• Inner-workings of a digital camera
• Manipulating transforming a matrix of pixels
• Implementing a variant of JPEG compression
• Wireless networks
• Wireless technology
• Acoustic waves and electrical signals
• Radios
• Video over wireless networks
• Video compression and quality
• Transmitting video over wireless
• Controlling a car over a radio link

3
Traditional Photography

• A chemical process, little changed from 1826
• Taken in France on a pewter plate
• with 8-hour exposure

The world's first photograph
4
Image Formation
Digital Camera
Film
Eye
5
Image Formation in a Pinhole Camera

• Light enters a darkened chamber through pinhole
  opening and forms an image on the further surface

6
Aperture

• Hole or opening where light enters
• Or, the diameter of that hole or opening
• Pupil of the human eye
• Bright light 1.5 mm diameter
• Average light 3-4 mm diameter
• Dim light 8 mm diameter
• Camera
• Wider aperture admits more light
• Though leads to blurriness in theobjects away
  from point of focus

7
Shutter Speed

• Time for light to enter camera
• Longer times lead to more light
• though blurs moving subjects

• Exposure
• Total light entering the camera
• Depends on aperture and shutter speed

8
Digital Photography

• Digital photography is an electronic process
• Only widely available in the last ten years
• Digital cameras now surpass film cameras in sales

• Polaroid closing its film plants this year

9
Image Formation in a Digital Camera
10V
Photon

?
?
?
?
?
?
?

A sensor converts one kind of energy to another

• Array of sensors
• Light-sensitive diodes convert photons to
  electrons
• Buckets that collect charge in proportion to
  light
• Each bucket corresponds to a picture element
  (pixel)

10
CCD Charge Coupled Device
CCD sensor

• Common sensor array used in digital cameras
• Each capacitor accumulates charge in response to
  light
• Responds to about 70 of the incident light
• In contrast, photographic film captures only
  about 2
• Also widely used in astronomy telescopes

11
Sensor Array Image Sampling
Pixel (Picture Element) single point in a
graphic image
12
Sensor Array Reading Out the Pixels

• Transfer the charge from one row to the next
• Transfer charge in the serial register one cell
  at a time
• Perform digital to analog conversion one cell at
  a time
• Store digital representation

Digital-to-analog conversion
13
Sensor Array Reading Out the Pixels
14
More Pixels Mean More Detail
1280 x 960
1600 x 1400
640 x 480
15
The 320 x 240hand
The 2272 x 1704hand
16
Representing Color

• Light receptors in the human eye
• Rods sensitive in low light, mostly at periphery
  of eye
• Cones only at higher light levels, provide color
  vision
• Different types of cones for red, green, and blue
• RGB color model
• A color is some combination of red, green, and
  blue
• Intensity value for each color
• 0 for no intensity
• 1 for high intensity
• Examples
• Red 1, 0, 0
• Green 0, 1, 0
• Yellow 1, 1, 0

17
Representing Image as a 3D Matrix

• In the lab this week
• Matlab experiments with digital images
• Matrix storing color intensities per pixel
• Row from top to bottom
• Column from left to right
• Color red, green, blue
• Examples
• M(3,2,1) third row, second column, red intensity
• M(4,3,2) fourth row, third column, green
  intensity

1
2
3
2
1
18
Limited Granularity of Color

• Three intensities, one per color
• Any value between 0 and 1
• Storing all possible values take a lot of bits
• E.g., storing 0.368491029692069439604504560106
• Can a person really differentiate from 0.36849?
• Limiting the number of intensity settings
• Eight bits for each color
• From 00000000 to 11111111
• With 28 256 values
• Leading to 24 bits per pixel
• Red 255, 0, 0
• Green 0, 255, 0
• Yellow 255, 255, 0

19
Number of Bits Per Pixel

• Number of bits per pixel
• More bits can represent a wider range of colors
• 24 bits can capture 224 16,777,216 colors
• Most humans can distinguish around 10 million
  colors

8 bits / pixel / color
4 bits / pixel / color
20
Separate Sensors Per Color

• Expensive cameras
• A prism to split the light into three colors
• Three CCD arrays, one per RGB color

21
Practical Color Sensing Bayer Grid

• Place a small color filter over each sensor
• Each cell captures intensity of a single color
• More green pixels, since human eye is better at
  resolving green

22
Practical Color Sensing Interpolating

• Challenge inferring what we cant see
• Estimating pixels we do not know
• Solution estimate based on neighboring pixels
• E.g., red for non-red cell averaged from red
  neighbors
• E.g., blue for non-blue cell averaged from blue
  neighbors

Estimate R and B at the G cells from
neighboring values
23
Interpolation

• Examples of interpolation
• Accuracy of interpolation
• Good in low-contrast areas (neighbors mostly the
  same)
• Poor with sharp edges (e.g., text)

and
makes
and
makes
and
makes
24
Are More Pixels Always Better?

• Generally more is better
• Better resolution of the picture
• Though at some point humans cant tell the
  difference
• But, other factors matter as well
• Sensor size
• Lens quality
• Whether Bayer grid is used
• Problem with too many pixels
• Very small sensors catch fewer photons
• Much higher signal-to-noise ratio
• Plus, more pixels means more storage

25
Digital Images Require a Lot of Storage

• Three dimensional object
• Width (e.g., 640 pixels)
• Height (e.g., 480 pixels)
• Bits per pixel (e.g., 24-bit color)
• Storage is the product
• Pixel width pixel height bits/pixel
• Divided by 8 to convert from bits to bytes
• Example sizes
• 640 x 480 1 Megabyte
• 800 x 600 1.5 Megabytes
• 1600 x 1200 6 Megabytes

26
Compression

• Benefits of reducing the size
• Consume less storage space and network bandwidth
• Reduce the time to load, store, and transmit the
  image
• Redundancy in the image
• Neighboring pixels often the same, or at least
  similar
• E.g., the blue sky
• Human perception factors
• Human eye is not sensitive to high frequencies

27
Compression Pipeline

• Sender and receiver must agree
• Sender/writer compresses the raw data
• Receiver/reader un-compresses the compressed data
• Example digital photography

uncompress
compress
27
28
Two Kinds of Compression

• Lossless
• Only exploits redundancy in the data
• So, the data can be reconstructed exactly
• Necessary for most text documents (e.g., legal
  documents, computer programs, and books)
• Lossy
• Exploits both data redundancy and human
  perception
• So, some of the information is lost forever
• Acceptable for digital audio, images, and video

28
29
Lossless Huffman Encoding

• Normal encoding of text
• Fixed number of bits for each character
• ASCII with seven bits for each character
• Allows representation of 27128 characters
• Use 97 for a, 98 for b, , 122 for z
• But, some characters occur more often than others
• Letter a occurs much more often than x
• Idea assign fewer bits to more-popular symbols
• Encode a as 000
• Encode x as 11010111

29
30
Lossless Huffman Encoding

• Challenge generating an efficient encoding
• Smaller codes for popular characters
• Longer codes for unpopular characters

English Text frequency distribution
Morse code
30
31
Lossless Run-Length Encoding

• Sometimes the same symbol repeats
• Such as eeeeeee or eeeeetnnnnnn
• That is, a run of e symbols or a run of n
  symbols
• Idea capture the symbol only once
• Count the number of times the symbol occurs
• Record the symbol and the number of occurrences
• Examples
• So, eeeeeee becomes _at_e7
• So, eeeeetnnnnnn becomes _at_e5t_at_n6
• Useful for fax machines
• Lots of white, separate by occasional black

31
32
Joint Photographic Experts Group

• Starts with an array of pixels in RGB format
• With one number per pixel for each of the three
  colors
• And outputs a smaller file with some loss in
  quality
• Exploits both redundancy and human perception
• Transforms data to identify parts that humans
  notice less
• More about transforming the data in Wednesdays
  class

Uncompressed 167 KB
Good quality 46 KB
Poor quality 9 KB
33
Joint Photographic Experts Group (JPEG)
108 KB
34 KB
Lossy compression
8 KB
33
34
New Era of Computational Photography

• Beyond manual editing of photographs
• E.g., to crop, lighten/darken, sharpen edges,
  etc.
• Post-processing of a single photography
• De-blur photos marred by camera shake
• Changing the depth of field or vantage point
• Combining multiple pictures
• Creating three-dimensional representations
• Stitching together photos for a larger view
• Creating a higher-resolution picture of a single
  scene
• ltInsert your idea heregt

http//www.news.com/8301-13580_3-9882019-39.html
35
Conclusion

• Conversion of information
• Light (photons) and a optical lens
• Charge (electrons) and electronic devices
• Bits (0s and 1s) and a digital computer
• Combines many disciplines
• Physics lenses and light
• Electrical engineering charge coupled device
• Computer science manipulating digital
  representations
• Mathematics compression algorithms
• Psychology/biology human perception
• Next class compression algorithms