Multiple Description Video Coding Through Adaptive Segmentation

1
Multiple Description Video Coding Through
Adaptive Segmentation

• Brian A. Heng and Jae S. Lim
• Research Laboratory of Electronics
• Department of Electrical Engineering and Computer
  Science
• Massachusetts Institute of Technology, Cambridge,
  MA

2
Motivation Real Time Streaming Video
Network

• Streaming video over packet networks
• Internet / wireless networks
• Best effort service
• Variable bandwidth and delay
• Packet loss inevitable
• Real-time applications
• Limited delay between sender and receiver
• Significant buffering not an option
• Retransmitted packets will arrive too late
• Link outages can underflow any reasonably sized
  buffer

3
Multiple Description Video Coding

• Multiple description (MD) video coding
• Encode into multiple complementary streams
• Acceptable video quality from each stream
• Higher quality from multiple streams
• Each stream equally important

• Properties of MD coding
• Increased resilience to loss
• Reduced coding efficiency

4
Examples of MD Coding

• Temporal Segmentation
• Video Redundancy Coding in H.263
• Even frames predicted from even frames
• Odd frames predicted from odd frames

0
2
4
6
1
3
5
7

• Spatial Segmentation
• Same approach along spatial direction
• Even lines predicted from even lines and odd from
  odd

• Other Approaches
• MD Quantization
• Correlating Transforms
• FEC based

5
Nonadaptive approach

• Tradeoff ? compression efficiency vs. error
  resilience
• Desire the most effective tradeoff
• Best approach depends on situation
• Source material
• Amount of motion
• Types of motion
• Spatial detail
• Network conditions
• Loss rates
• Burst characteristics
• User preferences
• Desired quality
• Desired robustness
• One single method ? not the best approach

6
Adaptive Segmentation

• Segmentation method chosen adaptively in
  intelligent manner
• Encoder has access to original source
• Calculate rate-distortion statistics during
  encoding
• Optimal method chosen using Lagrangian
  optimization

Simulate Loss of Stream 1
Calculate D1
Compute Lagrangian Cost Function
Calculate Rate R1 R2
Encode Block
Calculate D0
Simulate Loss of Stream 2
Calculate D2
7
System Implementation

• MD Codec
• Developed using the H.264 / MPEG4 AVC standard
• Encodes single I-frame followed by P-frames
• Periodically intra-codes macroblock lines
• Packetized using RTP by the H.264 encoder
• Adaptive Segmentation
• Adapts on a macroblock level (16x16 pixels)
• Chooses one particular method for each block
• Transmits choice to decoder
• Mode Selection
• Calculates rate-distortion statistics for each
  method during encoding
• Selects method using Lagrangian optimization

8
System Implementation

• Selecting a Set of Segmentation Methods
• Select methods which complement each other well
• Well rounded methods not as useful as set of
  specialized methods
• Methods used in current implementation
• Temporal segmentation (VRC) even/odd frame
  segmentation
• Spatial segmentation even/odd line segmentation
• Single description coding standard encoding, no
  segmentation
• Test Sequence
• Foreman sequence
• CIF resolution 288 rows x 352 columns
• 50 frames in length

9
Experimental Results Single Frame Loss
Foreman Sequence CIF resolution
10
Video Demo
11
Experimental Results Binomial Packet Loss
Foreman Sequence CIF resolution
12
Summary and Conclusions

• Introduced concept of adaptive segmentation for
  MD video coding
• Adaptively selects from a set of methods, making
  decisions which most effectively tradeoff
  compression efficiency for error resilience
• Adapts to characteristics of the video as well as
  rate-distortion goals
• Specialized methods have great potential in this
  system
• Initial results are promising
• The combination of a small number of simple
  algorithms can lead to significant increases in
  error robustness