FPGA

1
FPGA Camera Sensors
CISC 849
11/16/2006

• by Juergen Vogt

2
Outline

• Review
• The Whole System
• Software Overview
• What are Tone Mapping Zones
• What is the Math?
• Project plan
• Project Proposal
• Camera State Machine
• Realization
• Hardware Block Diagram
• Preparation with Matlab
• Obstacles to Implementation
• Preparing a Test Environment
• Simulation
• Results

3
The Whole System
Custom HW Custom SW
Off shelf HW Custom SW
Off shelf HW Off shelf SW
Camera
Control, Setup programming
Non real time storage evaluation
Real time acquisition recording
4
Software Overview
Async. Control
Camera VHDL
PCI-X Card VHDL
DMA
RAM
Async. Control
Interrupt
Imaging Software C
Display
HDD
5
Tone Mapping Zones

• Zones are on a logarithmic scale
• Real world zones have to be mapped to the zones
  of the display medium
• 14 bit HDR image maps to 14 zones
• 8 bit grey scale is only 8 zones

Real time 641 compression
6
What is the Math?
Computing the average world luminance
Scaling the luminance by a percentage a of the
average
Scaling by 1/L for large L Scaling by 1 for small
L
Scaling by 1/L for large L Scaling by 1 for small
L Highest luminance mapped to white
7
Project proposal

• Implementation of Global Tone Mapping in VHDL
• Proof of concept for real time filtering inside
  the camera
• Gaining a VHDL filter template for further use
• Reduce processing in acquisition system
• Camera could be used without PC in the future

8
Camera State Machine
Exposing
Storing
Reading
Sending to PC
Async. Receive
Different colors represent different clock
domains or asynchronous signals
9
Camera State Machine
Exposing
Storing
Reading
Pixel Processing
Sending to PC
Image Processing
Async. Receive
Different colors represent different clock
domains or asynchronous signals
10
Hardware Block Diagram
enable
Image data
Sending
Read out controller
RAM
Control signals
done
Pixel process
Image process
done
CCD control clocks
Multi MUX
CLK generator
11
Preparation with Matlab

• Implementation of the algorithm
• Porting to integer calculations
• Sample grayscale image
• 92 x 69 x 8 bit

12
Obstacles to Implementation

• The FPGA cannot calculate exponential or
  logarithmic functions.
• Solution
• Use build in PowerPC
• Use look up tables
• The FPGA can divide integers
• Only for and !!!
• Solution
• Use build in PowerPC

But the PowerPC libraries are additional
intellectual property (IP) packages (Xilinx
LogiCore)
13
Obstacles to Implementation
Look up table 256 x 0-5.5

• 256 x 0-55256 x 6 bit
• 192 byte

LUT may be Possible for test, but not real
CCD RAM750kbyte ROM Limited by LUTs !
Look up table size byte
14
Obstacles to Implementation

• Solution Compare value with range
• Matlab code comes in handy for automatic
  statement generation
• for full look up table
• fill table with current value
• if (value differs from last one)
• print VHDL statement with current position
• save current value for next loop
• end loop
• Result
• Out lt 1 when (Ilt25739) else
• 2 when (Ilt58167) else
• 3 when (Ilt79526) else
• 4 when (Ilt95479) else

15
Obstacles to Implementation

• By fixing the image key value a to a constant
  that fits the scene, we get an equation for
  global tone that only depends on two 8- bit
  variables and may therefore be realized with a
  65kb look up table.
• is note taken in to account

16
Preparing a Test Environment

• Since the CCD does not deliver images yet. The
  image tested in Matlab was imported as a fixed
  dataset.
• 6348 pixel with each 8 bit

8k RAM
8k ROM
Image data loaded on startup
Pixel process
Image process
done
17
Simulation
Exact Lwbar value from Matlab simulation 153.4091
18
Simulation
Comparison between Matlab and VHDL results
19
Results
Error indicator
Comparator with Matlab result
Image data loaded on startup
8k RAM
done
8k ROM
Pixel process
Image process
done
20
Results
Start
Error 1
Error 2
21
Summary

• Successful implementation of global tone mapping
• Successful calculation of Lwhite, but no chance
  to integrate this parameter into the tone mapping
  equation
• VHDL templates for other signal processing
  algorithms successfully generated
• Signal processing will require the integration of
  the PowerPC
• Look up tables deliver fast approximations but do
  not use resources efficiently
• Project shows limitations and possibilities