Block Loss Recovery Techniques for Image Communications

1
Block Loss Recovery Techniques for Image
Communications

• Jiho Park, D-C Park, Robert J. Marks, M.
  El-Sharkawi
• The Computational Intelligence Applications (CIA)
  Lab.
• Department of Electrical Engineering
• University of Washington
• May 29, 2002

2
Projections based Block Recovery Motivation

• Conventional Algorithms use information of all
  surrounding area.
• Using only highly correlated area

3
Alternating Projections

• Alternating Projections is projecting between two
  or more convex sets iteratively.

Converging to a common point
4
Projections based Block Recovery Algorithm

• 2 Steps
• Pre Process 1) Edge orientation detection
• 2) Surrounding vector extraction
• 3) Recovery vector extraction
• Projections 1) Projection operator P1
• 2) Projection operator P2
• 3) Projection operator P3

5
Pre Process 1 Edge Orientation Detection

• Edge orientation in the surrounding area(S) of a
  missing block(M). In order to extend the
  geometric structure to the missing block.
• Simple line masks at every i, j coordinate in
  surrounding area(S) of the missing block(M) for
  edge detection.

Horizontal Line Mask
Vertical Line Mask
6
Pre Process 1 Edge Orientation Detection

• Responses of the line masks at window W
• Total magnitude of responses
• Th gt Tv Horizontal line dominating area
• Th lt Tv Vertical line dominating area

7
Pre Process 2 Surrounding Vectors

• Surrounding Vectors, sk, are extracted from
  surrounding area of a missing block by N x N
  window.
• Each vector has its own spatial and spectral
  characteristic.
• The number of surrounding vectors, sk, is 8N.

8
Pre Process 3 Recovery Vector

• Recovery vectors are extracted to restore missing
  pixels.
• Two positions of recovery vectors are possible
  according to the edge orientation.
• Recovery vectors consist of known pixels(white
  color) and missing pixels(gray color).
• The number of recovery vectors, rk, is 2.

Vertical line dominating area
Horizontal line dominating area
9
Projections based Block Recovery Projection
operator P1

• Recovery vectors, ri, for i 1, 2
• Surrounding vectors, sj , for j 1 8N
• Surrounding vectors, S, form a convex hull in
  N2-dimensional space
• Recovery vectors, R, are orthogonally projected
  onto the line defined by the closest surrounding
  vector, si, j Projection Operator P1.

10
Projections based Block Recovery Projection
operator P1

• Projection operator P1

Convex hull (formed by surrounding vectors,
containing information of local image structure)
11
Projections based Block Recovery Projection
operator P1

• Surrounding vectors, sj , for j 1 8N
• Recovery vectors, ri, for i 1, 2
• The closest vertex, sdi , from a recovery vector,
  ri.
• or equivalently in DCT domain,
• P1

12
Projections based Block Recovery Projection
operator P2

• Convex set C2 acts as an identical middle.
• Projection operator P2

13
Projections based Block Recovery Projection
operator P3

• Convex set C3 acts as a convex constraint between
  missing pixels and adjacent known pixels, (fN-1
  fN).
• where,
• and is
  a N x N recovery vector in
• column vector form.

fN-1 fN

• Projection operator P3

14
Projections based Block Recovery Iterative
Algorithm

• Missing pixels in recovery vectors are restored
  by iterative algorithm of alternating projections
  
• N x N windows moving

Vertical line dominating area
Horizontal line dominating area
15
Projections based Block Recovery - Summary
Edge Orientation Detection
Surrounding Vector Extraction
Recovery Vector Extraction
Projection Operator P1
Projection Operator P2
Projection Operator P3
IterationI?
All pixels?
16
Simulation Results Lena, 8 x 8 block loss
Original Image
Test Image
17
Simulation Results Lena, 8 x 8 block loss
Ancis, PSNR 28.68 dB
Hemami, PSNR 31.86 dB
18
Simulation Results Lena, 8 x 8 block loss
Ziad, PSNR 31.57 dB
Proposed, PSNR 34.65 dB
19
Simulation Results Lena, 8 x 8 block loss
Ancis PSNR 28.68 dB
Hemami PSNR 31.86 dB
Ziad PSNR 31.57 dB
Proposed PSNR 34.65 dB
20
Simulation Results Each Step Lena 8 x 8 block
loss
(a) (b) (c)
21
Simulation Results Peppers, 8 x 8 block loss
Original Image
Test Image
22
Simulation Results Peppers, 8 x 8 block loss
Ancis, PSNR 27.92 dB
Hemami, PSNR 31.83 dB
23
Simulation Results Peppers, 8 x 8 block loss
Ziad, PSNR 32.76 dB
Proposed, PSNR 34.20 dB
24
Simulation Results Lena, 8 x one row block loss
Original Image
Test Image
25
Simulation Results Lena, 8 x one row block loss
Hemami, PSNR 26.86 dB
Proposed, PSNR 30.18 dB
26
Simulation Results Masquerade, 8 x one row
block loss
Original Image
Test Image
27
Simulation Results Masquerade, 8 x one row
block loss
Hemami, PSNR 23.10 dB
Proposed, PSNR 25.09 dB
28
Simulation Results Lena, 16 x 16 block loss
Original Image
Test Image
29
Simulation Results Lena, 16 x 16 block loss
Ziad, PSNR 28.75 dB
Proposed, PSNR 32.70 dB
30
Simulation Results Foreman, 16 x 16 block loss
Original Image
Test Image
Ziad, PSNR 25.65 dB
Proposed, PSNR 30.34 dB
31
Simulation Results Flower Garden, 16 x 16 block
loss
Original Image
Test Image
Ziad, PSNR 20.40 dB
Proposed, PSNR 22.62 dB
32
Simulation Results Test Data and Error

• 512 x 512 Lena, Masquerade, Peppers,
  Boat, Elaine, Couple
• 176 x 144 Foreman
• 352 x 240 Flower Garden
• 8 x 8 pixel block loss
• 16 x 16 pixel block loss
• 8 x 8 consecutive block losses
• Peak Signal Noise Ratio

33
Simulation Results PSNR (8 x 8)
34
Simulation Results PSNR (Row, 16 x 16)