A Flexible and Standards-Compatible Architecture for MVC Streaming

1
A Flexible and Standards-Compatible Architecture
for MVC Streaming

• Engin Kurutepe, Cagdas Bilen, Anil Aksay, Goktug
  Gurler, Thomas Sikora, Gozde Bozdagi Akar, Murat
  Tekalp
• A 3DTV collaboration between METU, KU and TUB
• ITG Fachtagung, HHI, Berlin, 22.06.2007

2
System Overview

• Standards-based, flexible, end-to-end Multi-View
  Video (MVV) Streaming system
• Server and streaming TUB
• Encoding decoding METU
• Client and display KOC
• MVC encoded MVV representation
• RTP/RTSP/SDP Server can serve various clients
• Legacy 2-D
• Static Stereo (3DV)
• Dynamic Stereo Head Tracking (3DV FVV)
• N-View (3DV FVV)
• Based on open-source libraries (Live Media and
  FFMPEG)
• An initial demo system was shown at GRM2 in
  Bodrum
• Final version will be demonstrated in IBC
  Amsterdam

3
Multi-View Encoding

• Simulcast Coding
• JM Reference Software (H.264 monoscopic standard)
  (GRM2 Demo System)
• Stereo/MultiView Coding
• MMRG Codec (Implemented in METU, without B
  picture support)
• JMVM 3.0 (Upcoming new standard)
• Complex prediction structures
• Encoders are not real-time

4
Possible MVC Coding Schemes in JMVM
General MVC Scheme
5
Possible MVC Coding Schemes in JMVM
Simplified MVC Scheme (IPPP)
6
Multi-View Decoding

• Simulcast Decoding
• JM Decoder Slow
• FFMPEG Decoder Fast, doesnt support
  Hierarchical B-frames correctly, compiled in
  Linux, can be used in Windows apps, used in some
  open source media players, encoders, decoders
  (GRM Demo)
• Multi-View Decoding
• JMVM Decoder Slow
• Modified JM Decoder Implemented but still slow
• Modified FFMPEG Decoder under development

7
Network Overview

• Standards-based streaming system
• Session announcement and initiation SDP
• Control of Streaming RTSP
• Streaming of media RTP

8
RTSP (Real Time Streaming Protocol) Basics
9
SDP (Session Description Protocol) Basics

• Session Description Protocol
• Provides information to initiate RTP streaming
• SDP Announcement lists all sessions available at
  the server
• Each session consists of several media-level
  sections (audio, video, application)
• Media-level sections can be extended with new
  attributes for future formats

10
SDP for MVV

• A Session for a whole MVV representation
• N video media-level sections for N-view MVV
• Client can request and play media-level sections
  independently
• Each view is streamed to a different port on the
  client over RTP
• Only streams for key-views (Intra Coded) are
  independently decodable.
• Dependencies between streams are announced
  through a new extended media-level attribute

11
New SDP Parameters

• The new x-reflist attribute is used to denote the
  immediate dependencies of a stream.
• ax-reflist96 97
• The dependencies are given in a space separated
  list
• The numbers correspond to rtpmap attributes of
  the reference streams, which are RTP payload
  types and are dynamically assigned for H.264/AVC
  streams
• The client should go through the reference list
  recursively.

12
MVV over RTP (Real Time Protocol)

• RFC 3984 is standard for H.264 streaming
• MVC bit stream is almost H.264 compliant
• Can be streamed in accordance with RFC 3984
• What about 2-D clients or clients with various
  3-D display systems?
• RFC 3984 is extended to allow flexibility
• Each view is handled as a separate H.264 Stream
• Views synchronized using RTP Timestamps
• Clients request streams according to their
  capabilities
• Varying levels of error protection can be applied
  to different streams

13
On-going work

• Server support for accurate seeking
• Support for selective streaming
• Real-Time decoding of MVC
• Visits of partners from KU and METU to TUB in
  August 2007
• Code will be submitted to 3DTV repository
• SDP extensions and RTP Payload format for MVC may
  be candidates for IETF standardization

14
Thank you.

• Question, suggestions, comments?