Delay-

1
Delay- Disruption-Tolerant Networking

• Stephen Farrell
• stephen.farrell_at_cs.tcd.ie
• N4C Kick-Off
• Luleå, Sweden
• May 13th 2008
• These slides (will be) at https//down.dsg.cs.tcd
  .ie/misc/n4cko.pdf

2
Contents

• Whats all this about then?
• Motivating DTNs
• Pre-History
• Some similar networks/protocols
• Some details
• The Bundle Protocol (BP) and Licklider
  Transmission Protocol (LTP)
• Some open issues
• Routing, security,
• Outlook
• Where to take this?

3
TCP Breaks!
Pretty certainly after 5 mins LTT (User
timeout) Actually after 50ms LTT (100ms RTT)
things get bad and generally fail after 1.5s LTT
E.g. SSH fails between 1s and 10s LTT (higher
layer timer LoginGraceTime)
4
Delay- Disruption-Tolerant Networking
5
Classic DTN Experiment
Demmer, M., et al., Implementing Delay Tolerant
Networking, Intel technical report,
IRB-TR-04-020, Dec.28 2004. http//www.cs.berkele
y.edu/demmer/papers/dtn-irb-tr-04-020.pdf
6
A Bit of DTN History

• Various DTN-like things have existed for a long
  time
• IPNSIG (www.ipnsig.org)
• 1999, Original InterPlanetary Network concept
• IEEE Journal on Selected Areas in Communications
  (JSAC), special edition on DTNs, June 2008
• DTN An Architectural Retrospective, Fall
  Farrell

7
Some Possible Options for DTNs

• Roughly from application layer downwards
• Sámi Network Connectivity Project
• Application Layer Gateway/SOAP
• E-mail
• wwwoffle
• TP-Planet
• Fidonet
• Postmannet
• CPIP

8
Sámi Network Connectivity Project

• DTN-like applications
• Partners LTU, Tannak, (Folly)
• Deployments 2006/7
• PRoPHET

9
Application Layer Gateways

• One could construct a DTN based on a web services
  model
• Dont think its been done though
• Everthing in the Bundle Protocol could be
  replicated with SOAP
• More on the Bundle Protocol in a bit...
• Basically this would be fine with so long as
  delays and disruptions dont affect every use of
  TCP
• If some TCP sessions succeed, even perhaps very
  occasionally, then you could get DTN functions
• So why not SOAP?

10
E-mail

• (For completeness)
• E-mail application is delay tolerant
• But none of SMTP, IMAP or POP are
• All sit on TCP
• DNSBLS, Greylisting etc would muck things a bit
• Essentially, RFC 2821/2822 has too much legacy
  stuff going on
• Same as SOAP but much worse (since mail is
  deployed)

11
wwwoffle

• An offline browser/proxy
• http//www.gedanken.demon.co.uk/wwwoffle/
• Locally installed web proxy, continues to serve
  content even if network connection has gone away
• Dial-on-demand, cached outbound requests,
  pre-fetching based on inbound responses Can
  display date content cached
• Generic DTN GUI issues
• Problems
• Pre-fetching needs rendering in proxy (i.e. peek
  into HTML, Javascript etc.)
• FTP etc not as easy and therell always be
  another protocol (e.g. BitTorrent)
• Cookies how to consult privacy preferences with
  no user there?
• Captchas hmm
• These are all generic problems with putting
  browser stuff into proxies and have been seen a
  number of times without being solved

12
TP-Planet

• Good idea extend work done on TCP to try handle
  delay/disruption
• Could leverage lots of networking expertise,
  tools, (conference papers-)
• TP-Planet changes
• Slow-start use environmental knowledge (e.g.
  LTT) to pick initial window size, and also window
  size adjustments
• Insert NIX packets at low high IP priority
  packet drops suggest congestion then apply AIMD
• Session interruptions handled without MD TCP
  assumes all packet drops due to congestion,
  TP-Planet doesnt
• BUT as done it requires that there be a
  end-to-end connection for the session duration
• So this is really looking at a TCP-like protocol
  for high latency, connected cases
• In general wont work for an orbiter relay to a
  lander on the far side of the moon

13
Fidonet

• In the days of dial-up and high long-distance
  charges, and before ISPs
• Idea was to make server-server phone calls in the
  middle of the night (cheaper)
• Hierarchical naming routing tables
• 1105/6.42 USA, Portland Oregon, host 6,
  point 42
• http//www.fidonet.org/
• No longer used (mostly) since ISPs do this
• Lesson even cute technology dies when theres
  something easier
• Might happen to DTNs, e.g. if WiMAX coverage
  fills enough of the DTN niche

14
Postmanet

• DVDs via snail-mail used as physical layer
• Not a totally bad bitrate! 4.7GB/day456kbps
• Argument capacity of physical storage media will
  increase faster and more cheaply than deployed RF
• Send artefacts to carry interstellar messages!

15
CPIP

• RFC 1149
• http//www.blug.linux.no/rfc1149/

Script started on Sat Apr 28 112409
2001 vegard_at_gyversalen ping -i 900
10.0.3.1 PING 10.0.3.1 (10.0.3.1) 56 data
bytes 64 bytes from 10.0.3.1 icmp_seq0 ttl255
time6165731.1 ms 64 bytes from 10.0.3.1
icmp_seq4 ttl255 time3211900.8 ms 64 bytes
from 10.0.3.1 icmp_seq2 ttl255 time5124922.8
ms 64 bytes from 10.0.3.1 icmp_seq1 ttl255
time6388671.9 ms --- 10.0.3.1 ping statistics
--- 9 packets transmitted, 4 packets received,
55 packet loss round-trip min/avg/max
3211900.8/5222806.6/6388671.9 ms vegard_at_gyversalen
exit Script done on Sat Apr 28 141428 2001
16
Applications

• Deep Space Networking
• Environmental Monitoring
• Networks for Developing Regions
• Underwater/Acoustic Networking
• Military Tactical Networking

17
Pre-amble Causes of Delay/Disruption

• Laws of physics
• Light trip time
• Conserving power
• Batteries have a crappy Moores law
• Intermittent availability
• Lectures 12-1 Tues and 2-4 Thurs only!
• Nothing happening
• Most of the time, sensors must be really bored!
• Bad things happening
• DDoS an edge router and what happens?

18
Deep-space networking

• Mars-probes have sort of done networking
• MEX as relay for MER (800M issue!)
• Spacecraft operators hate this idea!
• They care about mission success only
• Spacecraft in extended mission more likely to be
  usable
• I know a JPL guy who cant wait for MER to get
  a wheel stuck!

19
More deep-space

• Deep space network (DSN) is critical here
• 3 main sites (Canberra, Goldstone, Madrid) with
  70, 35 14m antennae
• Bottleneck in all deep-space missions
• Interesting question here about range
• GEO satellites TCP with performance enhancing
  proxies seem to work, mostly
• Cislunar applications if theres a BIG pipe
  to/from Earth we might want to keep that busy
• Beyond Mars, perhaps DTNs are silly, given there
  are no plans for multiple spacecraft

20
Scheduling for Space Applications
Views (10yr, 3month, 1 week, 1-day) of eumulated
MER-B communications schedules just after
landing in 2004
21
SeNDT

• Sensor networking with delay tolerance
• https//down.dsg.cs.tcd.ie/sendt/
• http//www.trinityisland.com/
• Approach was to deploy sensors that communicate
  via WiFi with a dutycycle of about 10 (i.e.
  power-off 90) and use boats as data-mules
• Sensors in the lake during 2007
• Instrumenting boats...still TBD-(
• Waterproofing-)

22
Developing world applicaitons

• DTNs might provide a nice way to meet some of the
  challenges involved in networking in developing
  regions
• Wizzy digital courier (South Africa)
• http//www.wizzy.org.za/
• Postmanet
• http//www.cs.princeton.edu/rywang/distance/
• TierStore

23
Acoustic Networking
http//www.spawar.navy.mil/
24
US marine corps.
25
FAPH Reliable Deliveryacross heterogeneous and
completely disconnected networks

• Mobile Ops Scenario with mobiles relaying data
  from disconnected nodes
• Five mobile nodes
• Fifteen stationary nodes deployed including three
  at Night Vision Lab
• 802.11 on all nodes long-haul tactical radios on
  mobiles
• Nodes send identical messages to HQ via DTN,
  end-to-end IP, and telemetry net. A-B comparison
  of DTN v e2e IP

SUVs 1, 3, 5
Mosby Road
Jordans Crossing
SUVs 2, 4
SUVs 1,3,5
Vaughn Road
SUVs 2, 4
All SUVs start here and do3 loops at 10 MPH (9
min loop)
HQ
Reception histograms for DTN E2E IP (blue for
DTN, orange for E2E IP) Delivery ratio for
reports v. time Delivery ratio for reports v.
node id
DTN Delivers Even for Stationary Nodes which
NEVER have an E2E Path -- IP Can Not
True position (silver/black circle)
Last reported location (solid disc)
Green ? recent report, Dark/Empty ?
older/timed-out report
26
DTN Details
27
DTN Research Group

• DTNRG is an IRTF research group The IRTF is the
  research arm of the IETF
• IETF http//www.ietf.org/
• IRTF http//www.irtf.org/
• DTNRG http//www.dtnrg.org/
• Specs, Code, Papers, Project links
• DTNRG is an open group just get on the mailing
  list (dtn-interest) and off you go...
• Two main protocols being developed
• Bundle Protocol (BP)
• Licklider Transmission Protocol (LTP)

28
Bundle Protocol - Quick history

• 1998 Vint Cerf a bunch of JPL folks started
  on Interplanetary Internet (IPN) work
• Became clear (2002) that its hard to do many
  experiments on the solar system
• Luckily the generalisation of an IPN also has
  terrestrial applications generalised to Delay
  Tolerant Networking
• DARPA (US DoD) started funding (2005) Disruption
  Tolerant Networking projects (US20M)
• DTN expanded whichever way you prefer

29
Bundle protocol

• The bundle protocol (BP) is the main focus of the
  work in DTNRG
• BP is a delay/disruption tolerant overlay network
  protocol
• More on that in a moment
• Multiple implementations exist, some interop.
  happened in Nov06 and again (though more
  limited) in March 08

30
BP 06 interop end-to-end topology
Demmer-mac .11, RI
BBN, java .31, RI ext CL
Demmer-pc .12
Pocket, C .63
tcp
udp
udp
udp
jpl1 .23, ION
mitre2 .52, RI
War, C .61
770 .42, RI on Nokia770
tcp
udp
udp
udp
mitre .51, RI ext Rtr
9300, C .41, Symbian
Charon, C .62, GA Tech
tcp
31
(No Transcript)
32
BP 06 implementations
TKK DTN2 MITRE BBN GA Tech ION
Language C C C C, Java C C
Platform Symbian cell phone MacOS and Linux on PC and (Nokia 770) Linux on PC, external router Linux on PC, external CL adapter .NET on Win32 Linux on PC, PDA Linux on PC
Custody transfer ? ? ? ? ?
Status rpts ? ? ? ? ?
TCP CL ? ? ? ?
UDP CL ? ? ? ? ?
DTN2 reference code is available from
http//www.dtnrg.org/
33
(No Transcript)
34
BP documents

• Mature
• DTN Architecture RFC 4838
• Note not really the architecture for all of DTN,
  but actually fairly specific to the bundle
  protocol
• BP spec RFC 5050
• Maturing...
• draft-irtf-dtnrg-bundle-security
• draft-irtf-dtnrg-sec-overview
• draft-irtf-dtnrg-prophet
• Co-authored by Anders Lindgren formerly of LTU
• draft-irtf-dtnrg-tcp-clayer
• Less mature...
• multicast, last-hop, header compression,
  retransmission
• Missing...
• More on routing, only have one so far on PRoPHET
• Key Management, one expired draft-)

35
DTN and layering

• What layer is best to try tackle delay and/or
  disruption?
• No single answer, as usual
• BP chooses the overlay approach on the basis that
  highly challenged networks/nodes may have to use
  strange communications layers
• Original IPN concept of regions
• Late binding of names

36
BP is an Overlay Nework Protocol
37
A bundle
Primary Bundle Block -----------------------
-----------------------------------------
Version Proc. Flags
() ----------------------
------------------------------------------
Block length ()
------------------------
-----------------------------------------
Destination scheme offset () Destination
SSP offset () --------------------------
--------------------------------------
Source scheme offset () Source SSP
offset () ----------------------------
------------------------------------
Report-to scheme offset () Report-to SSP
offset () -----------------------------
-----------------------------------
Custodian scheme offset () Custodian SSP
offset () -----------------------------
-----------------------------------
Creation Timestamp time ()
--------------------------------
----------------------------------
Creation Timestamp sequence number ()
---------------------------------
---------------------------------
Lifetime ()
----------------------------------
------------------------------
Dictionary length ()
------------------------------------
----------------------------
Dictionary byte array (variable)
--------------------------------------
---------------------------
Fragment offset ()
----------------------------------------
-------------------------
Total application data unit length ()
--------------------------------------
--------------------------- Bundle Payload
Block -------------------------------------
--------------------------- Block type
Proc. Flags () Block length()
---------------------------------------
------------------------- /
Bundle Payload (variable)
/ -------------------------------------------
------------------------
38
BP Concepts (1)

• Naming URIs, including dtn scheme
• Endpoint identifiers EIDs
• Late binding routers each can lookup
  names/interface mappings
• Or notor whatever
• Open issues abound here (as always)
• Contacts duration with a positive channel
  capacity associated
• Persistent, on-demand, scheduled, opportunistic,
  
• Uni-/bi-directional

39
BP Concepts (2)

• Time time is carried in-band in the BP
• Mistake or not? Time will tell.
• Requirement for clock synchronisation seems a bit
  odd in a DTN (though some work has been done
  there)
• Used to expire bundles as part of congestion
  avoidance
• Custody intermediate routers re-transmit as
  bundle gets closer to desination (hopefully)
• Good area for experimentation

40
BP Concepts (3)

• Signalling reporting activities as the bundle
  transits the DTN
• Potential expansion effects gt potential DoS
• But was found to be v. helpful in interop
• Is in-band signalling or an ICMP-like approach
  better?
• Congestion mostly reduces to storage congestion

41
BP Concepts (4)

• Fragmentation pro-active or reactive
• Reactivedifficult! Idea is to not waste
  bandwidth following a link disruption
• But makes it v. hard to do security stuff
• Toilet paper!
• Expedited delivery Like reactive fragmentation
  but for delivery to application
• May risk data integrity violations
• Convergenc layer the layer below the bundle
  protocol
• Could be LTP, Proximity-1, UDP, TCP, ethernet,
  Irridum,

42
DTN Security

• Whats different in securing a DTN?
• Not all hosts that process a bundle are DTN nodes
• Possibilities for hard-to-detect traffic
  manipulation multiplied
• Resource consumption more serious
• Ingress control more important
• Same as any ad-hoc network every router has to
  be a firewall

43
BP Cryptographic Services

• Lower layers are just fine too!
• IPSec, TLS, WPA, etc. all good
• BP security primitives defined for bundle
  integrity and confidentiality
• Have now been implemented so maybe not bad specs!
• But no key management yet
• BP security isnt quite end-to-end
• End-to-end-ish-ness and hop-by-hop-ish-ness

44
Future-ish BP Things

• Extension blocks for handling
• Multicast
• Previous-hop
• Bundle-in-bundle encapsulation
• Retransmission block
• Convergence layer specifics
• Routing
• Although PRoPHET exists, other schemes will be
  needed as well

45
BP Summary

• BP architecture and protocol are well-defined,
  stable and have been interopd
• Experimental RFCs are a fine basis for subsequent
  experimentation
• DTN-ish projects that previously rolled their own
  protocols should (and are starting to) use the BP
• Will there be interest in developing
  standards-track versions?
• Maybe, but this is not yet commercial
• FP7 ICT (N4C!) and NSF FIND programmes might
  engender more commercial interest

46
Licklider Transmission Protocol (LTP)

• LTP is a point-to-point protocol for DTNs
• Designed as a BP convergence layer for deep space
  (v. high latency) links
• Think of it as somewhere from layer 2 up to maybe
  layer 4!
• Encoding is terse and binary
• LTP is highly stateful
• Needed to avoid negotiation exchanges
• Can also (I claim!) be used for terrestrial DTN
  applications (e.g. SeNDT)

47
LTP Background

• Named for J.C.R. Licklider
• CCSDS have defined CFDP (CCSDS File Delivery
  Protocol) that is quite like LTP, but
• LTP spec. is much less OSI-like
• Internet approach better for more open
  development environments
• CFDP securityhmm
• CCSDS http//www.ccsds.org/
• So LTP is sort-of another go at CFDP in a more
  open development environment

48
LTP Documents

• draft-irtf-dtrng-ltp-motivation
• Background and reasons for
• draft-irtf-dtnrg-ltp
• Core LTP protocol
• draft-irtf-drnrg-ltp-extensions
• Extensions (security)
• Documents are currently with the RFC editor
• Usual IRSG/IESG/IANA/RFC-ed procedural wrangling
  on-going

49
An LTP Session
50
LTP Layering

• LTP runs on top of some MAC layer or deep space
  lower layer
• LTP assumes lower layer cues are provided so
  that some infrastructure (e.g. ephemeris handler
  scheduler or proximity detector) tells the
  stack when to expect to receive or transmit with
  a given peer

51
An LTP Segment

• Bit 0 1 2 3 4
  5 6 7
• --------------------------
  --------------
• Version number
  Segment Type Flags
• ------------------------------
  ----------------
• 
  
• / Session ID
  \
• \
  /
• Header ------------------------------
  ----------------
• Header Extension Cnt.
  Trailer Extension Cnt.
• ------------------------------
  ----------------
• 
  
• / Header
  Extensions \
• \
  /
• V -------------------------------
  ----------------
• 
  
• 
  
• 
  
• Segment Content
  
• /
  \

52
LTP Features

• Sessions/Segments
• A single block is sent per session using
  multiple segments
• Segment size is limited by the underlying MTU
• Session-ID is src-ID number
• Recommended to use a (P)RNG for the number
• Red/green parts Partial Reliability
• Data is ACKed (red) or not (green)
• Not ACKing is easier, but doesn't fulfill all
  appn. Requirements
• Red part first (if any), then green (if any)
• Each segment in a session is entirely red or
  entirely green

53
Red/Green Motivation

• Lots of science data formats (e.g. images) put
  important information (e.g. codec, timing,
  orientation) at the start, followed by lots of
  less important detail (pixels)
• Loose the codec information and the rest is
  useless
• Losing a pixel or two (hundred) isn't that bad
• Want to have ACKs for the start of the block but
  not the entire block
• Caller or config. determines which, if any,
  segments are red. Others are green.

54
LTP (Security) Extensions

• Both header and trailer extensions allowed
• LTP authentication extension
• Header ciphersuite
• Trailer MAC/Signature
• LTP cookie extension
• DoS would be very bad for a deep space host
• Cookies aiming to protect against off-path
  attacks using a header extension, once turned on,
  only segments with cookie accepted
• No confidentiality related extensions so far
• Could be, but feeling is other layers will
  commonly do this so not yet
• And of course, weve no good ideas about key
  exchange either-)

55
LTP Interop

• Same beer picture-)
• But only two LTP implementations (Ohio TCD) in
  06
• And two (TCD JPL) in 08
• Nominal operation in both directions
• Both layered on top of UDP
• 1st exchange took 30-45 mins of work
• A few minor bugs on both sides (mostly mine,
  mostly htonl related)
• Features tested
• Red/Green mixtures
• Lost segment handling
• Extension skipping
• Not tested
• Corrupt segments
• Extension specific handling (not in Java version)
• Re-ordered segments

56
LTP Code

• DTNRG site has links http//www.dtnrg.org/
• Manis code (Java)
• http//irg.cs.ohiou.edu/ocp/ltp.html
• My dodgy code (C, sort-of-)
• https//down.dsg.cs.tcd.ie/ltplib/
• CVS snapshot is currently best
• JPL ION code
• Not released yet? Ohio Uiversity plan that. (I
  think)

57
LTP-T

• LTP is point-to-point, but exensiblehmm
• Could make a transport protocol out of a sequence
  of LTP links
• So I didits called LTP-T
• https//www.cs.tcd.ie/publications/tech-reports/re
  ports.05/TCD-CS-2005-69.pdf
• My code includes an LTP-T implementation

58
An LTP-T Session
59
DTN Routing

• There are lots of schemes
• Inventing new ones is easy
• None of them are proven
• All (probably) have scaling issues
• What to do?
• Do experiments, let others simulate things (they
  will)
• This is hopefully the main technical focus of the
  DTNRG for the next while, so get onto the
  dtn-interest list as well
• Zhang, Z., Routing in Intermittently Connected
  Mobile Ad Hoc Networks and Delay Tolerant
  Networks Overview and Challenges, IEEE
  Communications Surveys and Tutorials, 8(1), 2006.

60
Other DTN Open Issues

• Key management
• Congestion handling
• Scalability
• Policy stuff (ingress, egress, routes, )
• Traffic analysis

61
Next Steps in DTNRG

• Progress routing as a priority in DTNRG once
  existing document queue processed
• Maintain open source code for protocols and base
  experiments on those
• Build a lasting testbed for DTN based on real
  application and user requirements...
• http//www.n4c.eu/

62
Next Steps in DTN

• Experiments
• Establish lasting DTN testbed(s)
• Determine (or create) commercial interest

63
Next Steps in DTN
N4C Components

• Add N4C slide

64
Summary

• There are some highly-challenged networking
  scenarios where current Internet protocols cant
  be used
• There are a variety of reasons for this
• There are some applications that we may want to
  use in such environments
• DTN represents an approach to trying to provide
  some level of networking, approaching, but never
  equalling the current Internet experience

65
Thanks!

• Questions