Block Loss Recovery Techniques for Image Communications

1
Block Loss Recovery Techniques for Image
Communications

• Jiho Park, D-C Park, Robert J. Marks, M.
  El-Sharkawi
• The Computational Intelligence Applications (CIA)
  Lab.
• Department of Electrical Engineering
• University of Washington
• May 29, 2002

2
Projections based Block Recovery Motivation

• Conventional Algorithms use information of all
  surrounding area.
• Using only highly correlated area

3
Alternating Projections

• Alternating Projections is projecting between two
  or more convex sets iteratively.

Converging to a common point
4
Projections based Block Recovery Algorithm

• 2 Steps
• Pre Process 1) Edge orientation detection
• 2) Surrounding vector extraction
• 3) Recovery vector extraction
• Projections 1) Projection operator P1
• 2) Projection operator P2
• 3) Projection operator P3

5
Pre Process 1 Edge Orientation Detection

• Edge orientation in the surrounding area(S) of a
  missing block(M). In order to extend the
  geometric structure to the missing block.
• Simple line masks at every i, j coordinate in
  surrounding area(S) of the missing block(M) for
  edge detection.

Horizontal Line Mask
Vertical Line Mask
6
Pre Process 1 Edge Orientation Detection

• Responses of the line masks at window W
• Total magnitude of responses
• Th gt Tv Horizontal line dominating area
• Th lt Tv Vertical line dominating area

7
Pre Process 2 Surrounding Vectors

• Surrounding Vectors, sk, are extracted from
  surrounding area of a missing block by N x N
  window.
• Each vector has its own spatial and spectral
  characteristic.
• The number of surrounding vectors, sk, is 8N.

8
Pre Process 3 Recovery Vector

• Recovery vectors are extracted to restore missing
  pixels.
• Two positions of recovery vectors are possible
  according to the edge orientation.
• Recovery vectors consist of known pixels(white
  color) and missing pixels(gray color).
• The number of recovery vectors, rk, is 2.

Vertical line dominating area
Horizontal line dominating area
9
Projections based Block Recovery Projection
operator P1

• Recovery vectors, ri, for i 1, 2
• Surrounding vectors, sj , for j 1 8N
• Surrounding vectors, S, form a convex hull in
  N2-dimensional space
• Recovery vectors, R, are orthogonally projected
  onto the line defined by the closest surrounding
  vector, si, j Projection Operator P1.

10
Projections based Block Recovery Projection
operator P1

• Projection operator P1

Convex hull (formed by surrounding vectors,
containing information of local image structure)
11
Projections based Block Recovery Projection
operator P1

• Surrounding vectors, sj , for j 1 8N
• Recovery vectors, ri, for i 1, 2
• The closest vertex, sdi , from a recovery vector,
  ri.
• or equivalently in DCT domain,
• P1

12
Projections based Block Recovery Projection
operator P2

• Convex set C2 acts as an identical middle.
• Projection operator P2

13
Projections based Block Recovery Projection
operator P3

• Convex set C3 acts as a convex constraint between
  missing pixels and adjacent known pixels, (fN-1
  fN).
• where,
• and is
  a N x N recovery vector in
• column vector form.

fN-1 fN

• Projection operator P3

14
Projections based Block Recovery Iterative
Algorithm

• Missing pixels in recovery vectors are restored
  by iterative algorithm of alternating projections
  
• N x N windows moving

Vertical line dominating area
Horizontal line dominating area
15
Projections based Block Recovery - Summary
Edge Orientation Detection
Surrounding Vector Extraction
Recovery Vector Extraction
Projection Operator P1
Projection Operator P2
Projection Operator P3
IterationI?
All pixels?
16
Simulation Results Lena, 8 x 8 block loss
Original Image
Test Image
17
Simulation Results Lena, 8 x 8 block loss
Ancis, PSNR 28.68 dB
Hemami, PSNR 31.86 dB
18
Simulation Results Lena, 8 x 8 block loss
Ziad, PSNR 31.57 dB
Proposed, PSNR 34.65 dB
19
Simulation Results Lena, 8 x 8 block loss
Ancis PSNR 28.68 dB
Hemami PSNR 31.86 dB
Ziad PSNR 31.57 dB
Proposed PSNR 34.65 dB
20
Simulation Results Each Step Lena 8 x 8 block
loss
(a) (b) (c)
21
Simulation Results Peppers, 8 x 8 block loss
Original Image
Test Image
22
Simulation Results Peppers, 8 x 8 block loss
Ancis, PSNR 27.92 dB
Hemami, PSNR 31.83 dB
23
Simulation Results Peppers, 8 x 8 block loss
Ziad, PSNR 32.76 dB
Proposed, PSNR 34.20 dB
24
Simulation Results Lena, 8 x one row block loss
Original Image
Test Image
25
Simulation Results Lena, 8 x one row block loss
Hemami, PSNR 26.86 dB
Proposed, PSNR 30.18 dB
26
Simulation Results Masquerade, 8 x one row
block loss
Original Image
Test Image
27
Simulation Results Masquerade, 8 x one row
block loss
Hemami, PSNR 23.10 dB
Proposed, PSNR 25.09 dB
28
Simulation Results Lena, 16 x 16 block loss
Original Image
Test Image
29
Simulation Results Lena, 16 x 16 block loss
Ziad, PSNR 28.75 dB
Proposed, PSNR 32.70 dB
30
Simulation Results Foreman, 16 x 16 block loss
Original Image
Test Image
Ziad, PSNR 25.65 dB
Proposed, PSNR 30.34 dB
31
Simulation Results Flower Garden, 16 x 16 block
loss
Original Image
Test Image
Ziad, PSNR 20.40 dB
Proposed, PSNR 22.62 dB
32
Simulation Results Test Data and Error

• 512 x 512 Lena, Masquerade, Peppers,
  Boat, Elaine, Couple
• 176 x 144 Foreman
• 352 x 240 Flower Garden
• 8 x 8 pixel block loss
• 16 x 16 pixel block loss
• 8 x 8 consecutive block losses
• Peak Signal Noise Ratio

33
Simulation Results PSNR (8 x 8)
Lena Masqrd Peppers Boat Elaine Couple
Ancis 28.68 25.47 27.92 26.33 29.84 28.24
Sun 29.99 27.25 29.97 27.36 30.95 28.45
Park 31.26 27.91 31.71 28.77 32.96 30.04
Hemami 31.86 27.65 31.83 29.36 32.07 30.31
Ziad 31.57 27.94 32.76 30.11 31.92 30.99
Proposed 34.65 29.87 34.20 30.78 34.63 31.49
34
Simulation Results PSNR (Row, 16 x 16)
(8 x Row) Lena Maskrd Peppers Boat Elaine Couple
Hemami 26.86 23.10 25.41 24.54 26.87 24.30
Proposed 30.18 25.09 28.31 26.06 30.11 26.12
(16 x 16) Lena Foreman Garden
Ziad 28.75 25.65 20.40
Proposed 32.70 30.34 22.62