RDOptimized Rate Shaping for Scalable Coded Streaming Video

1
RD-Optimized Rate Shaping for Scalable Coded
Streaming Video

• Rajyalakshmi Mahalingam
• Under the supervision of
• Prof. Dr. Eckehard Steinbach
• M.Sc. Wei Tu

2
Extending to Scalability

• Conventional Video Coding Systems does not
  fulfill the basic requirements of new flexible
  digital media applications.
• Scalable extension of H.264/AVC opens the door to
  some new video coding techniques with the
  following features.
• Adaptation to different bitrates
• Reduced spatial-temporal resolution
• Coding efficiency comparable to non-scalable
  video systems
• Main challenge is to make information easily
  retrievable for a variety of systems.

3
Outline

• Scalable Video Coding
• Problem at Hand
• Why Scalability?
• Approach
• RD Performance
• Conclusion

4
Scalable Video Coding (SVC)

• The SVC project aims to develop a fully scalable
  video codec based on the H.264/AVC codec as its
  backbone.
• The key features of the scalable extension of
  H.264/AVC are
• Hierarchical Prediction Structure
• Layered coding scheme with inter-layer
  prediction mechanisms
• Base layer compatibility with H.264/AVC
• Quality scalability using progressive
  refinement slices
• Usage and extension of the NAL unit concept of
  H.264/AVC

5
Conceptual Framework
Current digital video applications require at
least three types of scalability features
Spatial scalability Temporal scalability SNR
scalability These scalabilities can be used
individually or as combinations.
6
Architecture of H.264/SVC
The scalable H.264/AVC extension specifies a
layered structure.

• The generic structure is characterized by
• Mixed spatial and temporal scalability
• Independent motion compensation in individual
  loops

7
Problem at Hand

• Delivering quality video over wired and wireless
  networks poses many challenges.
• This is primarily because of the following
  problems.
• Bandwidth Fluctuations
• High Bit Error Rate
• Application Requirements
• Therefore, meeting bandwidth requirements and
  maintaining acceptable image quality
  simultaneously, is a challenge.

8
Why Scalability?

• The solution to the network uncertainties is not
  in choosing the perfect quality rate but in
  scaling the streams so that everyone can be
  pleased, that is, provide
• Graceful quality degradation
• Efficient bandwidth utilization
• Fairness in achieving quality
• SVC can combine certain layers in a flexible way
  in order to adapt to different bitrates, frame
  rates or spatial resolutions of the video content
  by removing parts of the bitstream.

9
Approach
We propose to use a RD-optimized rate adaptation
decision in case of network congestion for
multiple streams.
The rate shaping strategy relies on the scalable
vector that is sent as side information along
with the video streams.
The H.264 JSVM software is employed for
generating the side information.
10
Coding Efficiency
Sequence Foreman SVC at 30Hz with GOP 16
Sequence Foreman SLC at 30Hz with GOP 16
11
Coding Efficiency contd
In contrast to single-layer coding, scalable
video coding supports a rate interval instead of
a limited set of points. A substream for each
rate inside the supported interval can be
extracted.
Sequence Foreman SVC at 30Hz with GOP 16
12
Coding Efficiency contd
The achieved quality and the corresponding rate
ranges for both SVC and SLC are summarized.
The scalable vector consists of these RD points
within the available scalability levels and the
required bitrate.
13
RD Optimization
With multiple video streams, the decision which
packets (NAL units) to drop has to be jointly
made for all the streams. Our rate shaping
strategy is based on maximizing the Lagrangian
cost function J, given by J(n) Sk Ps(n)k -
?(n) Sk Rs(n)k that determines the optimum
strategy based on the current outgoing
transmission rate. The main approach is to max
(mean (PSNRi)) for all i 1K streams
14
Performance - Fixed Channel
We investigate the improvements in average
reconstruction quality for a fixed static channel.
For SVC Unoptimized mechanism, the video
sequences are scaled, first in the dimension of
quality resolution then spatial resolution and
finally in the temporal direction.
In the SVC Optimized strategy, the decision
which packets to drop is made in the RD-optimized
way.
15
Performance Curves contd
Performance obtained by SVC rate shaping concept
with SLC concept for a fixed static channel.
The blue curve represents RD approach in
single-layer coding whereas the pink curve is for
earlier introduced unoptimized strategy with
single-layer coding.
The performance gain for SVC generally decreases
at higher bitrates..
16
Performance Evaluation
Improvements and side information overhead for
the video sequences at outgoing link Rout 600
Kbit/s.
17
Performance Varying Channel
We investigate the improvements in average
reconstruction quality for a varying dynamic
channel.
Coding results for RD-optimized and unoptimized
dropping strategy for the SVC approach is
presented.
18
Performance Curves contd
Performance obtained by SVC rate shaping concept
with SLC concept for a fixed link static channel.
The blue curve represents RD approach in
single-layer coding whereas the pink curve is for
the unoptimized strategy with single-layer coding.
19
Performance Evaluation
Improvements and side information overhead for
the video sequences for the channel rate at the
10th GOP.
20
Achieving Quality
Achieving good PSNR performance and coding
efficiency doesnt guarantee that the video
sequences have nice reconstruction quality. The
RD approach basically deals with maximizing the
cost function. This leads to unfairness in the
reconstructed quality among the video sequences.
To overcome this problem, we try to achieve an
average reasonable quality for all the video
streams, i.e., max (min (PSNRi)) for all i 1K
streams
21
Quality Performance
Performance for 85 quality achievement
Performance for 90 quality achievement
22
Quality Performance contd
Performance for 95 quality achievement.
Quality achieved for Rout 500 Kbit/s.
23
Subjective Survey
Perceived video quality does not correlate well
with PSNR. Human visual characteristics must be
considered to provide better visual quality
measurements. Test videos from two different
video sequences were presented to users and were
asked to provide feedback based on their
perception quality on a respective scale.
Sequence Foreman
Sequence Football
24
Survey contd
The survey was an attempt to incorporate user
visualization of the video sequences with
different scalability levels suitable for
different needs and scenarios. The results of
the survey on a five pointed scale with five
graded as the best quality are presented below.
As seen, majority of the users preferred small
size and high frame rate videos.
25
Conclusion
The Scalable Video Coding standard is more robust
to network fluctuations and application
requirements. The performance results indicate
that Scalable Extension of H.264/AVC achieve
essentially the same reproduction quality as the
H.264/AVC standard while typically requiring less
bitrate. SVC, therefore, serves the basic needs
of the new flexible digital media in particular
for low bandwidth networks.
26
Future Outlook
Enhance the single network node model to more
practical situations with a hierarchy of many
active network nodes and perform rate shaping at
every node. Further, extend the scalable video
coding approach to MCTF based scalable video
codec which employs an open-loop architecture.
27
Thank you... Have a nice day!