Real-time Image Processing System

1
Real-time Image Processing System

• Peter Chun
• Dr. Lev Kirischian
• Dr. Vadim Geurkov

2
Overview
3
Schematic
4
Topics on the table

• Controllers on FPGA
• Digital Camera (OV5017 and M4088)
• SRAM (IDT71V416 256K X 16-bit)
• VGA (DAC board THS8134)

5
Topics on the table (continue)

• Real-time Image Processing system
• Step1 VGA verification
• Step2 SRAM WRITE/READ verification using
  verified VGA
• Step3 Digital camera data acquisition on
  verified VGA using verified SRAM
• Static one-frame
• Continuous video capture
• Video capture with pixel update

Step by Step Guide How to construct one
6
VGA

• Hardware Interface
• DB15
• Number of pins you need
• 6
• Pin Descriptions
• Hsync Horizontal Sync
• Vsync Vertical Sync
• R red
• G green
• B blue
• GND ground

7
Hardware Interface DB15
8
VGA (continue)

• VGA arrangement

(0,0)
(0, 0)
Horizontal Synchronization
Vertical Synchronization
(640,480)
(640, 480)
9
Timing Requirements

• Horizontal

PARAMERTES A B C D E
TIME 31.77us 3.77us 1.89us 25.17us 0.94us
10
Timing Requirements

• Vertical

R
Red, Green, Blue
Horizontal Sync (Hsync)
Q
S
P
O
PARAMERTES O P Q R S
TIME 16.67ms 64us 1.02ms 15.25ms 0.35ms
11
VGA (continue)

• Example
• 20MHz

12
SRAM

• Signals
• Control signals
• we, oe, cs, address(length..0), bhe, ble
• Data signals
• Data(depth..0)
• NOTE
• depth is how many bits can you access at one
  time.
• length is how many of data can you store, which
  is equivalent to 2length.

13
SRAM (continue)

• read

14
SRAM (continue)

• write

15
SRAM (continue)

• Example
• 40MHz

16
Digital Camera

• Hardware Overview

17
Digital Camera (continue)

• Reference Signals
• Hsync
• indicates the duration of active horizontal pixel
  insertion. It remains high during valid line and
  reset while not valid.
• Vsync
• represents the duration of active frame. When
  valid, it remains low and asserts high briefly to
  indicate the start of new frame.
• Pclk
• establishes the availability of the pixels.
  Either at the rising or falling edge of pclk, the
  pixel data is ready to be outputted.

18
Digital Camera (continue)

• SetUp
• Exposure control, AGC (auto gain control), Gamma
  correction, Frame rate, resolution control
• Method
• Memory Access
• I2C protocol

19
Digital Camera (continue)

• Memory Access

100ns
100ns
50ns
50ns
50ns
20ns
20
SetUp Example

• case y is
• -- Initial state after resets
• when initial gt
• if(loop_num num_mul) then
• loop_num lt 0
• yltHWCTL_initial
• else
• loop_num lt loop_num 1
• end if
• -- Horizontal Window size control
• when HWCTL_initial gt
• yltHWCTL_ready
• when HWCTL_ready gt
• if(loop_num num_mul) then
• loop_num lt 0
• yltHWCTL_done
• else
• loop_num lt loop_num 1
• end if

• process(y)
• begin
• -- initial and reset values
• oeb lt '0'
• csb lt '1'
• web lt '1'
• a lt "0000"
• cam_data lt "00000000"
• if y initial then
• oeb lt '0'
• csb lt '1'
• web lt '1'
• elsif y HWCTL_initial then
• a lt "0110"
• cam_data lt "00000000"
• oeb lt '1'
• csb lt '0'
• web lt '0'
• elsif y HWCTL_ready then

21
Digital Design Tips

• Multiple processes
• For reference signals (pclk, vsync, href)
• Filtering unstable states of the signals

22
Digital Design Tips (continues)

• Address space division
• Address setup time exceeds synchronous clock
  events
• Long integer or std logic vector need attention
• Flip image
• Use decrement of address space

23
Digital Camera (continue)

• Pixel Timing

24
Digital Camera (continue)

• Horizontal Timing

25
Digital Camera (continue)

• Vertical Timing

26
Design evolution

• Frame based
• One memory
• Pixel based
• One memory
• Internal memory

27
Examples

• 40MHz
• 66MHz

28
Edge Detection

• Gradient based

29
Edge Detection (continue)

• An edge pixel is described using two important
  features
• Edge strength, which is equal to the magnitude of
  the gradient
• Edge direction, which is equal to the angle of
  the gradient.

30
Edge Detection (continue)

• Roberts cross Operator

385 pixels
288 pixels
31
Roberts Cross Operation
385 pixels
a1
b1
c1
d1
e1
f1

z1
y1
x1
a2
b2
c2
d2
e2
f2

z2
y2
x2
288 pixels




or
A
B
C
D
E
F

Z
Y
X
32
Example

• Real-time image processing system with edge
  detection