Multimedia Over IP Networks -- II

1
Multimedia Over IP Networks -- II

• Hao Jiang
• Computer Science Department
• Boston College
• Nov. 8, 2007

2
Real-time Transport Protocol

• Real-time transport protocol (RTP) is an
  Internet-standard protocol for transmitting
  real-time multimedia over IP network.
• RTP is (often) built on top of UDP.
• A RTP packet is embedded in a UDP packet and
  provides additional information to support
  real-time multimedia communications
• Timestamp.
• Sequence number and source id.

UDP header
RTP header
Multimedia Payload
3
The RTP Packet Header
Timestamp is a 32bit integer placed on the RTP
packet. It is incremented based on the local
system clock. For video it is usually 90kHz.
Sequence number is a 16bit integer incremented by
1 for each generated RTP packet.
PT payload type. For example, 7 is for LPC
audio, and 34 is for H263 video.
4
Example Motion JPEG Payload Packet

• Timestamp is incremented at 90KHz. The marker M
  bit is set at the last packet for each frame.

Main JPEG header following the RTP header
There are other headers to provide
information like quantization table, etc.
5
RTCP

• Real Time Control Protocol (RTCP) packets can be
  generated by both a sender and a receiver.
• RTCP uses a different UDP port from RTP.
• RTCP packets can be used to generate reports for
  network statistics such as round trip time,
  jitter and to synchronize different information
  sources.

V
P
IC
Length
PT
Format Specific Information
RCTP Packet Format
6
RTCP Receiver Report

• Payload type 201.
• RTCP RR packet reports include
• Loss fraction ratio of lost packets / expect
  packets .
• Cumulative number of packets lost.
• Highest sequence number received.
• Inter-arrival jitter
• D(i) (R(i) R(i-1)) (S(i) - S(i-1))
• J(i1) J(i) 15/16 D(i) 1/16
• Last send report timestamp (LSR) and delay since
  last send report (DLSR) NTP time
• round trip time current timestamp - LSR
  - DLSR

sender
LSR
Current Timestamp
DLSR
receiver
7
RTCP Sender Report

• Payload type 200.
• RTCP SR packet contains
• NTP timestamp.
• RTP timestamp.
• Reporter SSRC and others.
• The most important usage of RTCP SR packet is to
  synchronize different media streams.

8
RTP Example Application
encoding buffer
Audio Codec, Packing, Depacking
capturing
Internet
Playing back
Playback buffer
RTP Packet buffer
Client n
Client k
Client i
Audio Conferencing based on RTP
9
RTP Media Sender
Buffer until send time comes
Adding sequence number and other fields to make
RTP packets


Compressed media frames with relative time stamps
fragmentation
Send to UDP module
System clock


Media buffer
10
RTP Media Receiver
RTP packet with timestamp t gt t1and t lt t2
Packet buffer
(Packet with timestamp t1)
Play back buffer
Scheduled to play out
(Packet with timestamp t2)
decompress
We usually need another buffer for
de-fragmentation.
11
Timing
Sender clock
20ms audio
20ms audio
20ms audio
encoding
making a packet
Across The network
20ms audio
Receiver clock
What happens if receiver clock and sender
clock has different frequency?
12
Methods to Compensate for Clock Skew

• A method is to directly compute the skew
• sender timestamp n1 (t1 tx)/dtx
• packet arrive timestamp n2 (t2
  ty)/dty
• Assume we have 2 packets a and b, then
• n1(a) n1(b) t1(a) t1(b)/dtx
• n2(a) n2(b) t2(a) t2(b)/dty
• gt
• (n2(a) n2(b)) / (n1(a) n1(b))
• dtx/dty D(a,b)/(t1(a)
  t1(b))dtx/dty
• If (t1(a) t1(b)) is constant, we can do a
  long term average to remove the noise term.

13
Compensation for Jitter

• The leaky bucket model

Similar to a leaky bucket, we can use a buffer to
shape the network traffic to generate a smooth
playback.
Buffer

14
Compensation for Jitter
of Accumulated Packets
t1
t2
N/(t2-t1)
15
Compensation for Jitter
of Accumulated Packets
Production Curve
Consumption Curve
t1
t2
N/(t2-t1)
16
Compensation for Jitter
of Accumulated Packets
Production Curve
Consumption Curve
t1
t2
N/(t2-t1)
17
Compensation for Jitter
of Accumulated Packets
Production Curve
Consumption Curve
t1
t2
18
Media Synchronization

• In RTP, different media streams (audio and video)
  are sent separately.
• Media synchronization relies on the
  correspondence of timestamp and NTP time on the
  local machine.
• All the information sources need to be
  synchronized so that they have the same NTP time
  base.
• RCTP SR packet is used to align the timestamp
  with the NTP time in media synchronization.

19
Useful Links

• http//www.realnetworks.com/support/education/sdk.
  html
• http//www.w3.org/AudioVideo/Activity
• http//www.networksorcery.com/enp/protocol/rtp.htm