Generic Dependability

1
Generic Dependability

• For overlay networks
• My name is Barry Porter

2
Overlay networks

• An abstraction over another network
• Typically a subset of the other network

3
Overlay networks

• An abstraction over another network
• Typically a subset of the other network

4
Overlay networks

• An abstraction over another network
• Typically a subset of the other network

5
Proprietary Generic
Recovery

• On the failure of a node
• An overlays own recovery method is invoked

OpenCOM.zip DTAO.zip
OpenCOM.zip DTAO.zip
GTrack MSPastry.pdf
GTrack MSPastry.pdf
SGC
SGC
T-F17 LOTR
T-F17 LOTR
RON.pdf Phoenix.pdf Anderson.jpg
RON.pdf Phoenix.pdf Anderson.jpg
Winamp MikTeX
Winamp MikTeX
Kazaa TBCP.pdf
Kazaa TBCP.pdf
GSView.exe GR.ppt leeroy1.wmv SBEmail.swf
GSView.exe GR.ppt leeroy1.wmv SBEmail.swf
6
Proprietary Generic
Recovery

• On the failure of a node
• An overlays own recovery method is invoked

OpenCOM.zip DTAO.zip
OpenCOM.zip DTAO.zip
GTrack MSPastry.pdf
GTrack MSPastry.pdf
SGC
SGC
T-F17 LOTR
T-F17 LOTR
RON.pdf Phoenix.pdf Anderson.jpg
RON.pdf Phoenix.pdf Anderson.jpg
Winamp MikTeX
Winamp MikTeX
Kazaa TBCP.pdf
Kazaa TBCP.pdf
GSView.exe GR.ppt leeroy1.wmv SBEmail.swf
GSView.exe GR.ppt leeroy1.wmv SBEmail.swf
7
Proprietary Generic
Recovery

• On the failure of a node
• An overlays own recovery method is invoked

OpenCOM.zip DTAO.zip
OpenCOM.zip DTAO.zip
GTrack MSPastry.pdf
GTrack MSPastry.pdf
SGC
SGC
Winamp MikTeX
T-F17 LOTR
T-F17 LOTR
RON.pdf Phoenix.pdf Anderson.jpg
RON.pdf Phoenix.pdf Anderson.jpg
Winamp MikTeX
Kazaa TBCP.pdf
Kazaa TBCP.pdf
GSView.exe GR.ppt leeroy1.wmv SBEmail.swf
GSView.exe GR.ppt leeroy1.wmv SBEmail.swf
8
Proprietary Generic
Recovery

• On the failure of a node
• An overlays own recovery method is invoked

OpenCOM.zip DTAO.zip
OpenCOM.zip DTAO.zip
GTrack MSPastry.pdf
GTrack MSPastry.pdf
SGC
SGC
Winamp MikTeX
T-F17 LOTR
T-F17 LOTR
RON.pdf Phoenix.pdf Anderson.jpg
RON.pdf Phoenix.pdf Anderson.jpg
Winamp MikTeX
GSView.exe GR.ppt leeroy1.wmv SBEmail.swf
Kazaa TBCP.pdf
GSView.exe GR.ppt leeroy1.wmv SBEmail.swf
Kazaa TBCP.pdf
9
Proprietary Generic
Recovery

• On the failure of a node
• An overlays own recovery method is invoked

10
Proprietary Generic
Recovery

• On the failure of a node
• An overlays own recovery method is invoked

11
Proprietary Generic
Recovery

• On the failure of a node
• An overlays own recovery method is invoked

12
Proprietary Generic
Recovery

• On the failure of a node
• An overlays own recovery method is invoked

13
Proprietary Generic
Recovery

• On the failure of a node
• An overlays own recovery method is invoked

14
Proprietary Generic
Recovery

• On the failure of a node
• An overlays own recovery method is invoked

15
Generic Recovery

• Solution requirements
• Must scale with the overlay
• Should be self-repairing
• Should require little input from the overlay
• Should introduce little overhead

16
Dependability as a Service

• Simple architecture
• Each service is a component

Dependability Agent
FailureDetectionService
BackupService
RecoveryService
17
Dependability as a Service

• Solution requirements
• Must scale with the overlay
• Services are distributed
• Each node is concerned only with its neighbours

Backup
Failure
Recovery
18
Dependability as a Service

• Solution requirements
• Should be self-repairing
• Dependability agents spread themselves through
  the overlay
• Services are all soft-state

Has agent?
No deploy
19
Dependability as a Service

• Solution requirements
• Should require little input from the overlay
• Overlays provide a simple API

getNodestate() gets application-specific state
of node for backup addNodestate(object o)
restores above state getOwnAccessinfo() gets
contact address of this node getAccessinfoList()
gets the neighbour list of this
node setAccessinfoList(list l) allows service
to modify neighbour list send(object
accessInfoRecord, byte message) control
channel
20
Dependability as a Service

• Solution requirements
• Should introduce little overhead
• Main overhead is during recovery (ends when
  recovery is complete)
• Pays off later
• Have to store backups at your node
• Can take into account your free resources
• Small service interruption while backup is taken
• Needs some work to improve this
• Ambient messaging Only from failure detection
• Overlays would otherwise have to do this anyway

21
Dependability as a Service

• Overlay deployment environments
• Open Overlays, iOverlay, Macedon
• Manage a set of hosts and can deploy an overlay
  network over some or all of them, often
  dynamically

22
Dependability as a Service

• Overlay deployment environments
• Better recovery options

OpenCOM.zip DTAO.zip
OpenCOM.zip DTAO.zip
GTrack MSPastry.pdf
GTrack MSPastry.pdf
Winamp MikTeX
Winamp MikTeX
Kazaa TBCP.pdf
Kazaa TBCP.pdf
23
Dependability as a Service

• Overlay deployment environments
• Better recovery options

OpenCOM.zip DTAO.zip
OpenCOM.zip DTAO.zip
GTrack MSPastry.pdf
GTrack MSPastry.pdf
Winamp MikTeX
Winamp MikTeX
Kazaa TBCP.pdf
Kazaa TBCP.pdf
24
Dependability as a Service

• Overlay deployment environments
• Better recovery options

OpenCOM.zip DTAO.zip
OpenCOM.zip DTAO.zip
GTrack MSPastry.pdf
GTrack MSPastry.pdf
Winamp MikTeX
Winamp MikTeX
Kazaa TBCP.pdf
Kazaa TBCP.pdf
25
Dependability as a Service

• Overlay deployment environments
• Better recovery options

26
Dependability as a Service

• Overlay deployment environments
• Better recovery options

27
Dependability as a Service

• Overlay deployment environments
• Better recovery options

28
The Divergent Grid

• Host failure is not the only concern
• Overlays ignore heterogeneity of the network

29
Dependability as a Service

• Simple architecture
• Each service is a component

Dependability Agent
FailureDetectionService
BackupService
RecoveryService
CloningService
30
The Cloning Service

• Monitors the resources of a host
• If they become too low, clone the node
• Note This does not mean replicate
• How?
• An overlay node can often be split into
  self-standing units of workload
• DHTs Stored resources
• Trees Child connections
• Gnutella Resource table connections

31
The Cloning Service

• Monitors the resources of a host
• If they become too low, clone the node

accessinfo
Identity of a node copied
nodestate
Workload of a node split
list getNodestate() returns a list of nodestate
units
addNodestate(object o, boolean b) adds a single
unit to a node
32
The Cloning Service

• Monitors the resources of a host
• If they become too low, clone the node

Mountees
KaZaA.zip
Crais
Mountains
Horses
33
The Cloning Service

• Monitors the resources of a host
• If they become too low, clone the node

Mountees
Mercs3
KaZaA.zip
Forklift.mov
Leeroy1
Crais
Mountains
Horses
34
The Cloning Service

• Monitors the resources of a host
• If they become too low, clone the node

Mountees
Mercs3
KaZaA.zip
Forklift.mov
Leeroy1
H2 PC
Crais
Space
Mountains
Farscape
Horses
35
The Cloning Service

• Monitors the resources of a host
• If they become too low, clone the node

Mountees
Mercs3
KaZaA.zip
Forklift.mov
Leeroy1
H2 PC
Crais
Space
Mountains
Farscape
Horses
36
The Cloning Service

• Monitors the resources of a host
• If they become too low, clone the node

Leeroy1
H2 PC
Mountees
Space
Mercs3
KaZaA.zip
Forklift.mov
Crais
Mountains
Farscape
Horses
37
The Result
Leeroy1
H2 PC
Mountees
Space
Mercs3
KaZaA.zip
Forklift.mov
Crais
Mountains
Farscape
Horses
Chord node
TBCP node
Gnutella leaf peer
Gnutella super peer
38
Summary

• Benefits of the approach
• Applicable to many overlays
• Can offer better recovery options while the
  overlay implementation remains the same
• Can add extra services dynamically
• Drawbacks
• Some overhead in backup taking
• Recovery when not using external hosts is not
  quite as good as proprietary equivalent