ROIBased Image and Video Coding and Transmission Over Packet Loss Channels

1
ROI-Based Image and Video Coding and Transmission
Over Packet Loss Channels

• Li Xiangjun
• Supervisor Dr. Cai Jianfei
• CeMNet, SCE
• Nanyang Technological University
• March 2004

2
Outline

• Introduction
• Proposed Layered Unequal Loss Protection (ULP)
  Scheme
• Improved Layered ULP
• Conclusions and Future Work

3
Introduction

• Problems of Packet Communications
• Packet delay hard to guarantee the timely
  transfer of a packet.
• Packet loss due to traffic congestion or
  physical layer channel impairment.

4
Error Control Techniques

• Network error control techniques
• -- Forward Error Correction (FEC), Automatic
  Repeat reQuest
• (ARQ), and Hybrid ARQ.
• Error resilient techniques
• -- Error isolation, Layered coding, and Error
  concealment.
• Joint source and channel coding (JSCC)
• -- Any scheme jointly considering source coding
  efficiency and transportation reliability can be
  considered as a JSCC scheme.

5
Research Objective

• Design efficient and reliable ROI-based image and
  video coding and transmission schemes over packet
  loss channels
• At current stage, we mainly focus on general
  priority-based joint source and channel coding.
  We will apply this work to the special case,
  where the priority is provided by ROI coding.

6
Proposed Layered ULP Scheme

• Motivation
• Existing ULP schemes usually have high
  computation complexity.
• Most existing ULP schemes do not consider the
  minimum quality requirement. A received image
  with very low PSNR is useless in practice.

7
Motivation (contd)
The reconstructed 512 by 512 Lena image with PSNR
22.17 dB. In this research, we choose 25 dB as a
PSNR threshold for the minimum quality
requirement.
8
Structure of The L-ULP Scheme
In the Layered ULP (L-ULP), each layer has equal
distortion gain while the first layer must
satisfy the minimum quality requirement.
9
Layer Division

• Find the optimal Equal Loss Protection (ELP)
  solution and compute the minimum distortion value
  .
• is the distortion value when nothing is
  received and is the distortion determined by
  the minimum quality requirement.
• Corresponding bit rate values can be obtained by
  linear interpolation.
• 
  .

10
Experimental Results
The performance comparison of transmitting the
Lena image over the packet-loss channel with the
exponential PMF under different packet loss
rates. The total bandwidth is 0.25 bpp.
11
Experimental Results (contd)
The performance comparison of transmitting the
Lena image over the packet-loss channel with the
exponential PMF under different bit rates. The
packet loss rate is 0.2.
12
Improved Layered ULP

• Motivation
• Although our proposed L-ULP can achieve
  low-complexity and meet the minimum quality
  requirement, its average PSNR performance is not
  as good as that of the Hybrid Loss Protection
  (HLP) or the ULP and needs to be further improved.

13
System Description

• IL-ULP is a combination of pre-interleaving with
  our proposed L-ULP.
• By using the pre-interleaving, we can delay the
  occurrence of the first unrecoverable loss in the
  source bitstream and thus improve the L-ULP
  performance.

14
Structure of The IL-ULP Scheme
An example of the L-ULP with three layers. Left
the original L-ULP. Right with the
pre-interleaving.
15
Experimental Results
The performance comparison between the IL-ULP,
L-ULP and the HLP transmitting the Lena image
over GEC channel. The packet size is 200 bytes,
burst length is fixed to 5 packets, and the total
bandwidth is 0.488 bpp. The values shown in this
figure are obtained over 1000 simulations.
16
Experimental Results (contd)
The complexity comparison.
17
Conclusions

• Developed the L-ULP scheme for progressive image
  transmission over packet loss channels.
• Integrated the L-ULP with the pre-interleaving to
  further improve the performance.

18
Future Work Image and Video Segmentation for ROI
Generation

• Visual attention (VA) is referred to the ability
  of rapidly focusing attention on interesting
  parts in a given scene. It is a good way for
  segmenting foreground and background.
• Apply visual attention models for image
  segmentation.
• Investigate the possible combination of visual
  attention algorithms with existing methods for
  video segmentation.

19
Future Work ROI-Based Image/Video Coding and
Transmission

• ROI Coding Methods
• Maxshift method
• General scaling method

20
Future Work ROI-Based Image/Video Coding and
Transmission

• ROI Coding Methods
• Bitplane-by-bitplane shift (BbBShift) method
• Partial significant bitplanes shift (PSBShift)
  method

21
Future Work ROI-Based Image/Video Coding and
Transmission

• How to allocate bandwidth optimally between the
  ROI and the background to achieve the best
  quality for the reconstructed image given the
  total bandwidth.
• Further study how to apply the L-ULP to ROI-based
  bitstreams. In particular, we will look into the
  optimal FEC allocation among the ROIs and the
  background.
• As for image and video transmission, we will also
  study ROI-based media adaptation. Media
  adaptation can be realized by using different
  coding parameters for different regions.

22
Future Work ROI-Based Image/Video Coding and
Transmission
(a) Original image
(b) ROI
(c) Decoded image
File size is 384 by 256, ROI is coded by maxshift
and the total bit rate is 0.5 bpp.
23
Q A

• Thanks!