Rome, Italy

1
SP3 Content distribution, storage and P2P
network analysisDanny Bickson, HUJI

• Rome, Italy
• March 6-7, 2006

2
Storage, Content distributionand P2P analysis

• Main goals
• Ultimate AKAMAI content distribution
• Ultimate Raid distributed storage
• Analysis of P2P network behavior
• Participants HUJI, SICS, KTH and EPFL

3
Contribution to Evergrow Goals
P2P behavior analysis
Julia content distribution network Epidemic
algorithms for data replication Efficient
dissemination using BP techniques
Symmetric replication
4
Content distribution

• Work pursued in several directions
• The Julia content delivery network
• Replication and data placement algorithms using
  the belief propagation technique
• Gossip (epidemic) algorithms for data replication

5
Content distribution

• The Julia content delivery network
• Designed to be completely decentralized, scalable
  to very large network, reducing network load
• Prototype implemented using C and deployed over
  the PlanetLab testbed
• Experiments done against the state of the art
  BitTorrent client using more than 250 machines
• Extensive simulation was done to verify the
  algorithm behavior for larger networks
• Main results
• Average finishing times same as in BitTorrent
• Reduced network load and communication cost

6
(No Transcript)
7
(No Transcript)
8
Content distribution

• Replication and data placement using the belief
  propagation algorithm
• Efficient dissemination of information across
  multiple paths
• Data indexing for overlay networks (reported in
  WDAS 06 and ECCS 05 P2P workshop)
• Solving theoretical network flow problems using
  message passing techniques

9
Content distribution

• Gossip (epidemic) algorithms for data replication
• Multiple chunks reside on multiple nodes and the
  goal is to replicate all the chunks to all the
  nodes
• Well known results for gossiping a single message
• Fault tolerant algorithm for gossiping multiple
  chunks with the lowest time complexity (O(kln(n)
  rounds)
• Won the best paper award in IPDPS 05

10
Storage

• Symmetric Replication
• O(1) message for every join and leave operation
  to maintain any replication degree.
• Applicable to all existing structured
  peer-to-peer systems
• Increase the security by using distributed voting
• Adds routing proximity to reduce latencies.
• Particularly suitable for use with erasure codes

11
Analysis of P2P Systems

• Providing analytical models of large-scale P2P
  overlay systems under churn
• Combines the expertise of Computer Scientists and
  Physicists
• Enables reasoning about future Internet scales
  that are beyond our current numerical simulation
  abilities
• Enhances current configuration of actual P2P
  deployments

12
Analysis of P2P Systems

• Example Result Lookup length as a function of
  churn (Precise Prediction)

13
Future plans - tomorrow

• Work package 3m utilizing gossiping and belief
  propagation techniques to optimize content
  distribution
• Work package 3n exploring dynamics of large
  scale overlay networks for reliability and fast
  recovery

14
2006/2007 Themes
BP for data correlation, localization, IR and
IF Global properties estimation Analysis of P2P
networks for improving reliability and fast
recovery
Message passing techniques The Julia content
delivery network Gossip algorithms Data
replication using BP algorithm
Self-managing replicated storage and
backup Distributed backup on top of DKS