Adaptive Playout

1
Adaptive Playout
2
You are Here
Encoder
Decoder
Middlebox
Sender
Receiver
Network
3
How to recv and play?

• open socket
• while not done
• if socket is readable
• read packet from socket
• remove RTP header
• decode
• play back

4
Whats Wrong?

• packet ordering
• packet loss
• next packet arrive in-time?

Especially bad for audio applications
5
Overview
Decode
RTP Classifier
Network
6
Implementation

• Single Thread
• using select()
• Multi-Threads

7
Packet Buffer

• Sorted by sequence number
• When ADU is complete, send to decoder

RTP Classifier
8
Playout Buffer

• Stored decoded data in playout order
• Post-processing/Mixing may happens

Decode
9
Why Buffer?
10
Sending Packets
Packet
Time
11
Receiving Packets
Packet
Time
12
With Jitter
Packet
Time
13
With Jitter
Packet
Time
14
What causes Jitter?

• Network delay Transmission Delay (fixed)
  Propagation Delay (fixed) Queuing Delay
  (variable)
• Delay jitter is caused by variable queuing delay

15
Delay Jitter
Transit Time
small jitter
large jitter
Time
16
Spike
Transit Time
Time
17
Todays Question

• How big is the playout buffer?
• When to playback?

18
Types of Applications

• Non-interactive
• Buffer can be large
• Interactive
• As small as possible

19
Types of Applications

• Video
• Frames are discrete (easier problem)
• Audio
• Samples are continuous

20
Naive Answer

• How big is a buffer?
• Fixed at a small value, to reduce latency
• When to playback?
• Playback as soon as possible, to reduce latency

21
A Brief Introduction to Audio Conferencing
22
Audio Conferencing

• Live, interactive application
• Latency is important
• Each packet 20-30ms of audio

23
Silence Suppression

• Silence Detection
• if no sound, no need to send
• Talk spurt
• consecutive audio packets (between silence)
• hundreds of ms

24
Demo
25
Recall RTP Header

• marker bit
• depends on payload
• e.g. beginning of frame

26
RTP and Talkspurt

• First packet of a talkspurt will have marker bit
  set to 1

27
RTP and Talkspurt

• Deduce talkspurt from sequence number and
  timestamp

2 40
1 20
3 60
5 190
SeqNo TimeStamp
28
Consequences of Talkspurt

• Opportunity to adjust playout delay
• if jitter is large, increase delay
• if jitter is small, decrease delay

29
Fixed Playout Delay
SEND
RECV
PLAY
30
Adaptive Playout Delay
SEND
RECV
PLAY
31
Adaptive Playout Delay
SEND
RECV
PLAY
32
Trade-Off

• Latency vs. Packet Loss

33
Latency vs Loss-Rate
Loss Rate
Latency
34
Adaptive Playout Mechanisms for Packetized Audio
Applications in WAN

• R. Ramjee, J. Kurose, D. Towsley, H. Schulzrinne
• INFOCOM 1995

35
Variables and Notations
36
Variables and Notations
Tbuf(i)
Tnet(i)
Tarrive(i)
Tplay(i)
Tdelay(i)
Tsend(i)
37
1st Packet in Talkspurt
We can estimate as
38
How to estimate Vnet(i)
39
How to estimate Enet(i)

• Method 1 Jacobsons Method

40
Spike
Tnet
Time
41
Problems

• Does not react to spike fast enough

42
How to estimate Enet(i)

• Ramjees Method

SPIKE
NORMAL
43
Ramjees Idea
if Tnet(i) suddenly increase
SPIKE
NORMAL
if slope is small enough
44
In Spike Mode
SPIKE
Tnet
i
45
In Normal Mode
NORMAL
Tnet
i
46
Evaluations
Loss Rate
Delay
47
Problems with Ramjees Method
Transit Time
Time
48
Packet Audio Playout Delay Adjustment
Performance Bounds and Algorithms

• S. Moon, J Kurose, D. Towsley
• Multimedia Systems 1998

49
Recall Previous Methods
50
How to Set Tdelay(i)

• Moons Method
• Collect statistics on packets that have arrived.
• Find t such that q of last w packets have
  Tnet(i) lt t.

51
Example (w 50, q 90)
num of packets
1 2 3 4 5 6 7 8 9
10 11 12
delay
52
Setting Tdelay(i)
NORMAL
SPIKE
53
Setting Tdelay(i)
Transit Time
Time
54
Performance Bound

• Given a trace of packets, and a loss rate, find
  the minimum average playout delay.
• Use Dynamic Programming

55
A Packet Trace
k
M talkspurts
1,k
2,k
3,k
j,k
nk,k
Tnet
13
15
10

56
More Notations

• M
• Number of Talkspurt
• Npacket(k) or nk
• Number of packets in talkspurt k
• Ntotal
• Total number of packets

57
Definition

• minimum average playout delay for choosing i
  packets to be played out from k-th talkspurt

k
M talkspurts
58
How to find
59
Definition

• minimum average playout delay for choosing i
  packets to be played out from k-th to M-th
  talkspurt

k
M
..
M talkspurts
60
Base Case

• D(k, 0)
• D(M, i)

minimum average playout delay for choosing i
packets to be played out from k-th to M-th
talkspurt
61
Recursive Case
k
M
..
..
1,k
2,k
3,k
nk,k
..
j
62
Performance Bound

• Given a trace of M talkspurts and n packets, and
  a loss rate e, find the minimum average playout
  delay.
• Answer Minimum possible average playout delay is
  D(1, (1-e)n)

minimum average playout delay for choosing i
packets to be played out from k-th to M-th
talkspurt
63
Evaluations
Loss Rate
Delay
64
Summary

• Playout Adjustment for Audio Conferencing
• Weighted Average Methods vs. Statistical Methods
• An Analysis of Minimum Playout Delay

65
Practical Complications

• Clock Drifts
• Route Change

66
Advanced Techniques

• Speed-up Playback