Network Coding for Large Scale Content Distribution

1
Network Coding for Large Scale Content
Distribution

• Christos Gkantsidis
• Georgia Institute of Technology

Pablo Rodriguez Microsoft Research
IEEE INFOCOM 2005
Presented by Ryan
2
Outline

• Introduction
• Related Works
• Model for Cooperative Content Distribution
• Performance Evaluation
• Conclusion and Future Works

3
Introduction

• Large Scale Content Distribution
• Typical content distribution solutions
• CDN Content Delivery Network
• Placing dedicated equipment around the network
• e.g. Akamai
• Cooperative content distribution solutions
• Self-scalable
• Preventing sudden surge of traffic to the source
• e.g. BitTorrent

4
Introduction

• Network Coding
• Allowing intermediate nodes to encode packets
• Making optimal use of the available network
  resources

5
Introduction

• An example
• Without a global coordinated scheduler
• Node B, receiving Packet 1 or 2 from Node A?

6
Introduction

• Contributions in the Paper
• Proposing a practical system based on network
  coding
• Not require the knowledge of the underlying
  topology and centralized scheduling
• Robust to extreme situations with sudden server
  and nodes departures
• Better performance comparing to source coding and
  no encoding schemes

7
Related Works

• Tree-Based Cooperative Systems
• Creating and maintaining shortest-path multicast
  trees
• Bandwidth-limited (by the bottleneck link on the
  path from the server)
• e.g. SplitStream

8
Related Works

• Mesh Cooperative Architectures
• Improving the download rates by using parallel
  downloads
• Under-utilizing the network resources (the same
  block traveling over multiple competing paths)
• e.g. BitTorrent

9
Related Works

• Erasure Codes
• Reconstructing the original content of size n
  from roughly a subset of any n symbols from a
  large universe of encoded symbols
• Network Coding
• Based on theoretical calculations (with the
  detailed knowledge of the topology and a
  centralized scheduler)

10
The Model

• Server
• Dividing the file into k blocks
• Uploading blocks at random to different clients
• Clients (Users)
• Collaborating with each other to assemble the
  blocks and reconstruct the original file
• Exchanging information and data with only a small
  subset of others (neighbors)
• Symmetric neighborhood and links

11
The Model

• Upon arrival
• Contacting a centralized server (like the tracker
  in BitTorrent) to get a random list of users in
  the system
• Connecting to the returned users to construct the
  neighborhood

12
The Model

• Content Propagation
• 1) No Coding
• 2) Source Coding
• 3) Network Coding

13
The Model

• No Coding and Source Coding
• Based only on local information for deciding
  which block to transfer
• Random
• A random block
• Local Rarest
• The rarest block in the neighborhood

14
The Model

• e.g. BitTorrent system
• A combination of the Random and Local Rarest
  schemes
• Random for the first few blocks
• Local Rarest afterwards

15
The Model

• Network Coding
• The node generates and sends a linear combination
  of all the information available to it

16
The Model

• Recovering the original file after receiving k
  blocks (associated coefficient vectors are
  linearly independent to each other)
• Just solving the system of linear equations

17
The Model

• Incentive Mechanisms
• Discouraging free-riding
• Scheme 1
• Preference to mutual exchanges
• Scheme 2 (Tit-for-tat)
• Bounding the absolute difference of uploading
  minus downloading from one to another

18
Performance Evaluation

• Round based simulator
• Input
• Overlay topology
• Users upload and download capacities
• Servers capacity
• Capacity number of blocks that can be
  downloaded/uploaded in a single round
• Size of file to distribute
• Metric
• Download finish time

19
Performance Evaluation

• Connecting to 4 peers when joining
• Max number of neighbors 6
• Discovering new neighbors when the utilization of
  the download capacity is below a certain
  threshold (10)

20
Performance Evaluation

• Homogeneous topologies
• 200 users with capacity 1
• Servers capacity 1
• File size 100 blocks

No Coding
Source Coding
Network Coding
21
Performance Evaluation

• Topologies with clusters
• Two clusters, 100 users each
• Capacity
• Within cluster 8
• Cluster to cluster 4
• Server
• Capacity 4
• Departing at round 30
• File size 100 blocks

22
Performance Evaluation
No Coding
Source Coding
Network Coding
23
Performance Evaluation

• Heterogeneous capacities
• 10 fast users with capacity 4
• 190 slow users with capacity 1
• Servers capacity 4
• File size 400 blocks

No Coding
Source Coding
Network Coding
24
Performance Evaluation

• Minimum finish time for the fast users 50
  rounds

25
Performance Evaluation

• Dynamic Arrivals
• 40 empty nodes every 20 rounds
• Capacity 1
• Staying in the system 10 more rounds after
  finishing
• Servers capacity 1
• File size 100 blocks

26
Performance Evaluation
27
Performance Evaluation

• Robustness to node departures

28
Performance Evaluation

• Leaving after serving 5 extra blocks
• Network coding 100 finish
• Source coding 40 finish
• No coding 10 finish

Network Coding
Source Coding
No Coding
29
Performance Evaluation

• Incentive mechanisms
• Max difference 2 (tit-for-tat)

30
Conclusion

• A new content distribution system
• Not require knowledge of the whole network
  topology
• Easy to schedule content propagation
• Good performance in simulations
• Download finish time
• Robust to server and users departures
• Avalanche a real system implementation using
  network coding

31
Future Works

• Speed of encoding and decoding
• Encoding O(k)
• Decoding inverting a matrix O(k3),
  reconstructing the file O(k2)
• Dominated by reconstruction
• Many reads of large blocks from the harddisk
• Protection against malicious nodes
• Introducing arbitrary blocks
• Making the reconstruction of the original file
  impossible

32

• THANK YOU