Digital Cameras

1
Digital Cameras

• Engineering Math Physics (EMP)
• Jennifer Rexford
• http//www.cs.princeton.edu/jrex

2
Image Transmission Over Wireless Networks

• Image capture and compression
• Inner-workings of a digital camera
• Manipulating transforming a matrix of pixels
• Implementing a variant of JPEG compression
• Wireless networks
• Wireless technology
• Acoustic waves and electrical signals
• Radios
• Video over wireless networks
• Video compression and quality
• Transmitting video over wireless
• Controlling a car over a radio link

3
Traditional Photography

• A chemical process, little changed from 1826
• Taken in France on a pewter plate
• with 8-hour exposure

The world's first photograph
4
Image Formation
Digital Camera
Film
Eye
5
Image Formation in a Pinhole Camera

• Light enters a darkened chamber through pinhole
  opening and forms an image on the further surface

6
Aperture

• Hole or opening where light enters
• Or, the diameter of that hole or opening
• Pupil of the human eye
• Bright light 1.5 mm diameter
• Average light 3-4 mm diameter
• Dim light 8 mm diameter
• Camera
• Wider aperture admits more light
• Though leads to blurriness in theobjects away
  from point of focus

7
Shutter Speed

• Time for light to enter camera
• Longer times lead to more light
• though blurs moving subjects

• Exposure
• Total light entering the camera
• Depends on aperture and shutter speed

8
Digital Photography

• Digital photography is an electronic process
• Only widely available in the last ten years
• Digital cameras now surpass film cameras in sales

9
Image Formation in a Digital Camera
10V
Photon

?
?
?
?
?
?
?

A sensor converts one kind of energy to another

• Array of sensors
• Light-sensitive diodes convert photons to
  electrons
• Buckets that collect charge in proportion to
  light
• Each bucket corresponds to a picture element
  (pixel)

10
CCD Charge Coupled Device
CCD sensor

• Common sensor array used in digital cameras
• Each capacitor accumulates charge in response to
  light
• Responds to about 70 of the incident light
• In contrast, photographic film captures only
  about 2
• Also widely used in astronomy telescopes

11
Sensor Array Image Sampling
Pixel (Picture Element) single point in a
graphic image
12
Sensor Array Reading Out the Pixels

• Transfer the charge from one row to the next
• Transfer charge in the serial register one cell
  at a time
• Perform digital to analog conversion one cell at
  a time
• Store digital representation

Digital-to-analog conversion
13
Sensor Array Reading Out the Pixels
14
More Pixels Mean More Detail
1280 x 960
1600 x 1400
640 x 480
15
The 320 x 240hand
The 2272 x 1704hand
16
Representing Color

• Light receptors in the human eye
• Rods sensitive in low light, mostly at periphery
  of eye
• Cones only at higher light levels, provide color
  vision
• Different types of cones for red, green, and blue
• RGB color model
• A color is some combination of red, green, and
  blue
• Intensity value for each color
• 0 for no intensity
• 1 for high intensity
• Examples
• Red 1, 0, 0
• Green 0, 1, 0
• Yellow 1, 1, 0

17
Representing Image as a 3D Matrix

• In the lab this week
• Matlab experiments with digital images
• Matrix storing color intensities per pixel
• Row from top to bottom
• Column from left to right
• Color red, green, blue
• Examples
• M(3,2,1) third row, second column, red intensity
• M(4,3,2) fourth row, third column, green
  intensity

1
2
3
2
1
18
Limited Granularity of Color

• Three intensities, one per color
• Any value between 0 and 1
• Storing all possible values take a lot of bits
• E.g., storing 0.368491029692069439604504560106
• Can a person really differentiate from 0.36849?
• Limiting the number of intensity settings
• Eight bits for each color
• From 00000000 to 11111111
• With 28 256 values
• Leading to 24 bits per pixel
• Red 255, 0, 0
• Green 0, 255, 0
• Yellow 255, 255, 0

19
Number of Bits Per Pixel

• Number of bits per pixel
• More bits can represent a wider range of colors
• 24 bits can capture 224 16,777,216 colors
• Most humans can distinguish around 10 million
  colors

8 bits / pixel / color
4 bits / pixel / color
20
Separate Sensors Per Color

• Expensive cameras
• A prism to split the light into three colors
• Three CCD arrays, one per RGB color

21
Practical Color Sensing Bayer Grid

• Place a small color filter over each sensor
• Each cell captures intensity of a single color
• More green pixels, since human eye is better at
  resolving green

22
Practical Color Sensing Interpolating

• Challenge inferring what we cant see
• Estimating pixels we do not know
• Solution estimate based on neighboring pixels
• E.g., red for non-red cell averaged from red
  neighbors
• E.g., blue for non-blue cell averaged from blue
  neighbors

Estimate R and B at the G cells from
neighboring values
23
Interpolation

• Examples of interpolation
• Accuracy of interpolation
• Good in low-contrast areas (neighbors mostly the
  same)
• Poor with sharp edges (e.g., text)

and
makes
and
makes
and
makes
24
Are More Pixels Always Better?

• Generally more is better
• Better resolution of the picture
• Though at some point humans cant tell the
  difference
• But, other factors matter as well
• Sensor size
• Lens quality
• Whether Bayer grid is used
• Problem with too many pixels
• Very small sensors catch fewer photons
• Much higher signal-to-noise ratio
• Plus, more pixels means more storage

25
Digital Images Require a Lot of Storage

• Three dimensional object
• Width (e.g., 640 pixels)
• Height (e.g., 480 pixels)
• Bits per pixel (e.g., 24-bit color)
• Storage is the product
• Pixel width pixel height bits/pixel
• Divided by 8 to convert from bits to bytes
• Example sizes
• 640 x 480 1 Megabyte
• 800 x 600 1.5 Megabytes
• 1600 x 1200 6 Megabytes

26
Compression

• Benefits of reducing the size
• Consume less storage space and network bandwidth
• Reduce the time to load, store, and transmit the
  image
• Redundancy in the image
• Neighboring pixels often the same, or at least
  similar
• E.g., the blue sky
• Human perception factors
• Human eye is not sensitive to high frequencies

27
Joint Photographic Experts Group

• Starts with an array of pixels in RGB format
• With one number per pixel for each of the three
  colors
• And outputs a smaller file with some loss in
  quality
• Exploits both redundancy and human perception
• Transforms data to identify parts that humans
  notice less
• More about transforming the data in Wednesdays
  class

Uncompressed 167 KB
Good quality 46 KB
Poor quality 9 KB
28
Conclusion

• Conversion of information
• Light (photons) and a optical lens
• Charge (electrons) and electronic devices
• Bits (0s and 1s) and a digital computer
• Combines many disciplines
• Physics lenses and light
• Electrical engineering charge coupled device
• Computer science manipulating digital
  representations
• Mathematics compression algorithms
• Psychology/biology human perception
• Next class compression algorithms