Sensory Information Processing

1
Sensory Information Processing

• Introduction to imaging sensors

2
Camera
How the image is recorded electrically? What is
the artifact of CCD? How to select the CCD?
CCD
CCD driver / Signal processing
Digital Capture
lens
What is principal parameter of the lens? What is
aberration? What is the image degradation by
lenses? How to select the lenses?
3
Camera
How the Synchronize and Sampling is performed?
CCD
CCD driver / Signal processing
Digital Capture
lens
How to drive the CCD? How the color value is
recovered?
4
Measuring the light

• History of light measurement
• Exposure meter for cameras(Human can not percept
  absolute amount of the light)

Leica MC meter (1954)
5
Before that..

• Rely on inspiration and experience..

Senoguide
6
Device to measurethe amount of light

• Transducing energy of light to electricity
• Photo diode
• Solar cell (similar to photo diode)
• Change of resistance according to the light
• CdS (sulfureted cadmium) cell
• Photo tube
• Light ? displacement, speed, etc..
• So weak, very difficult

7
CdS cell

• Change of resistance of sulfureted cadmium
  (compound semiconductor)
• Free electrons are emerged by the light, and
  conducts electric current well
• Wide range of sensitivity
• Slow reaction
• cheap

Inadequate for image sensor
8
CdS cell(2)
electrode

• Winding CdS area
• Wider width, shorter path

Maximum power consumption 100mW
Burn out
Electric current (mA)
Relative sensitivity
Wavelength(nm)
Voltage (V)
very good linearity
9
phototube(1)

• Photoelectric effect
• electron is kicked off from metal by photon

e-
photon
metal

• Characteristics of photoelectric effect
• No electron for the light with longer wavelength
  than threshold which depends on materials
• Number of electron is proportional to the
  intensity of light

10
phototube(2)

• Capturing the electron out from the metal
• Material Ag, Cs, Na, Te, Ga, As
• Very high speedof response up to 10ps

• Electron is attracted by electric potential by
  high voltage

11
Phototube(3)

• Very high sensitivity(photomultiplier)

• Dynodes (SbCs, AgHg etc.) emit lots of electron
  by hitting the accelerated electron.
• http//en.wikipedia.org/wiki/Photomultiplier

12
photomultiplier(4)

• Super-kamiokaNDE (observing facility of neutrino)

http//en.wikipedia.org/wiki/Kamiokande
13

• Broken photomultiplier of kamiokaNDE

14
photodiode(1)

• Major light sensing device
• Photoelectron emerged between p and n
  semiconductor
• No vaccum!

15
history of exposure meter

• Solar cell type require no battery
• 1950
• CdS type high sensitivity (but slow)
• 1960
• Photodiodes
• 1970

Leica MC meter
Leica MR meter
Current products
16
2-D image sensor

• How to capture the 2-D distibution of light?

Array of sensors? Connecting all sensors each?
17
2-D image sensors

• To reduce the wiring cost, scanning mechanism is
  necessary

Electric signal
18
Camera tube

• Magnetic field is used to deflection (scanning)

19
Camera and TV tubes

• Magnetic field for scanning

20
Image orthicon
21
CCD (charge coupled device)

• Electrostatic effect is used to transfer the
  electron
• Not limited to the imaging device
• CCD is the term of the way to transfer the
  electron
• http//en.wikipedia.org/wiki/Charge-coupled_device
  

22
Aside delay line (1)

• How to delay temporal signal?
• dead time component

23
Aside delay line (1)
24
Aside delay line (1)

• Delay line is a memory
• Recording information for a short time
• No sampling, no quantization

25
Delay line (2)

• SAW (surface acoustic wave) filter
• http//en.wikipedia.org/wiki/Surface_acoustic_wave
  

26
Delay line (3) BBD

• BBD Bucket Brigade delay

MN3004
Predecessor of CCD
27
Delay line (4) current standard
AD converter
CPU
DA converter
RAM

• Long time delay for sound, video is based on
  digital technology

28
CCD types

• Full frame
• Interline transfer
• Frame transfer
• Frame interline transfer
• http//laser.physics.sunysb.edu/jose/CCDInfo.html
  
• CCD university
• http//www.ccd.com/ccdu.html

29
Full frame

• While transfer, CCD should be covered
• (mechanical shutter is necessary)
• High aperture ratio
• Easy to manufacture
• Commonly used in digital SLR

30
Interline transfer

• Most popular
• Mechanical shutter is not necessary
• Very short exposure is possible
• Used in most video cameras

31
Frame transfer

• Image is transferred to the storage array quickly

32
Image degradation in CCD

• smear
• Highlight spreading to the transfer direction
  (vertical)
• blooming
• Highlight spreading to the peripherals
• Thermal noise
• Electron is emerged by the thermal agitation of
  free electron

33
smear
34
Source of smear

• Imperfect mask of transfer CCD areas, spilling
  out of electron to the transfer area

mask
mask
mask
mask
mask
35
Anti-blooming

• Discarding too much electrons by highlight

36
X-Y address image sensor
photodiodes are connected by wire and
transistor CMOS transistor is used, therefore,
commonly called as CMOS sensor
JFET type
CMOS image sensor
37
CMOS image sensor

• Low power consumption
• CCD needs high voltage to attract the electron
• Other circuit can be embedded on a chip(usual
  LSI process is used)
• AD converter, image processor can be embedded
• Noise was more than CCD
• Now it is well improved
• Flexibility to access to each pixel

38
Rolling shutter

• For most CMOS sensors, exposure timing is varied
  pixel by pixel
• Called rolling shutter
• If all pixels are exposed simultaneously, it is
  called global shutter

39
Shutter types and image skew
Lens shutter (interchanging the lens is not easy)
Focal plane shutter (suit for interchanging
lens) SLR (single lens reflex) etc.
40
Skew of moving object

• Shape of the object is skewed by the varied
  timing of exposure

41
Ancient example of skew

• Lartigue A.C.F grand prixWorlds most famous
  example of skew

42
Color image

• Sequencial shots of R, G, B
• filter wheel
• Split R, G, B to three image sensors optically
• 3 tubes, 3-CCD
• Each pixel is sensitive either R, G or B
• Primal color, complementary color
• Foveon

43
Filter wheel
44
3 tubes
prism
Camera tube
lens
45
3-CCDcamera
Each surface in the prism is interference
filter which consist of transparent materials
with different refractive index, and interference
of light reflect or transmit the energy of the
light without loss.

• No loss of light energy, high sensitivity
• Color reproduction is good

46
Single CCD
R
G
Cy
Ye
Cy
Ye
Cy
Ye
Cy
Ye
R
G
R
G
R
G
G
B
G
B
G
B
G
B
Mg
G
Mg
G
Mg
G
Mg
G
R
G
R
G
R
G
R
G
Cy
Ye
Cy
Ye
Cy
Ye
Cy
Ye
Mg
G
Mg
G
Mg
G
Mg
G
G
B
G
B
G
B
G
B
Cy
Ye
Cy
Ye
Cy
Ye
Cy
Ye
R
G
R
G
R
G
R
G
G
B
G
B
G
B
G
B
Mg
G
Mg
G
Mg
G
Mg
G
Primal color (Bayer pattern)
Complementary color

• Each pixel of CCD is colored
• Primal color
• ?natural color reproduction
• Complementary color
• ?high light efficiency

47
Color reproduction (2)

• Simple method (interpolation)
• See Fredo Durand and Bill Freemans slide
• http//groups.csail.mit.edu/graphics/classes/CompP
  hoto07/index2.html
• 03_colorInterpCP.ppt

Low resolution
G
G
G
G
R
R
R
R
G
G
G
G
G
G
G
G
R
R
R
R
G
G
G
G
G
G
G
G
R
R
R
R
G
G
G
G