Research Opportunities in IP Wide Area Storage

1
Research Opportunities in IP Wide Area Storage

• George Porter
• Li Yin
• Department of EECS
• U.C. Berkeley

2
Outline

• Trends
• Challenges for wide-area storage
• Programmability inside networks
• Common techniques to hide latency
• Functionality that will benefit applications
• Network Support for that functionality
• Reconsidering the programmability model and
  application space
• Feedback

3
Outline

• Trends
• Challenges for wide-area storage
• Programmability inside networks
• Common techniques to hide latency
• Functionality that will benefit applications
• Network Support for that functionality
• Reconsidering the programmability model and
  application space
• Feedback

4
Storage Wide-Area Networking
Metro Area

• SAN technology works well in relatively small
  area (metro-wide)
• There is a desire to implement storage
  applications in the wide-area
• Comparable performance as small area storage
  applications

5
Outline

• Trends
• Challenges for wide-area storage
• Programmability inside networks
• Common techniques to hide latency
• Functionality that will benefit applications
• Network Support for that functionality
• Reconsidering the programmability model and
  application space
• Feedback

6
Challenges in Wide-Area Storage

• Speed of light is constant
• Long distance implies propagation delay
• Network dynamics
• Variation of cross traffic load
• Routes changes
• Increasing storage capacity
• Transmit huge amount of data across wide-area

7
Challenges in Wide-Area Storage

• Simple operation Host writes data to the remote
  target disk
• Send data to the remote target
• Write disk
• Where is the bottleneck?
• Disk?
• Network Link?
• Distance?
• Performance Degradation

Extra Delay Caused By the Network
Time
8
Challenges in Wide-Area Storage

• Three Cases
• Case 1 Limited link bandwidth
• Case 2 Small data set with high bandwidth
• Case 3 Large data set with high bandwidth

9
Challenges in Wide-Area Storage

• Case 1 Limited link bandwidth

More data to be transmitted
Time
Time
Extra Delay Caused By the Network
10
Challenges in Wide-Area Storage

• Case 1 Limited link bandwidth

• As more data to be transmitted
• The performance degradation caused by the
  transmission delay gets larger
• Propagation delay does not matter
• As the disk getting faster, more bandwidth is
  required to shift the bottleneck away from the
  network

11
Challenges in Wide-Area Storage

• Case 2 Small data set with high link bandwidth
• In this case, the throughput is very sensitive to
  the distance, especially when it becomes of the
  order of the disk latency

Larger Distance
Time
Time
12
Challenges in Wide-Area Storage

• Case 3 Large data set with high link bandwidth

More Data to be Transmitted
Time
Time
Time
Extra Delay Caused By the Network
13
Challenges in Wide-Area Storage

• Case 3 Large data set with high link bandwidth

• In this case, disk is the bottleneck, the network
  only introduces the propagation delay which can
  be ignored as more data to be transmitted
• As the disk getting faster, more bandwidth is
  required to shift the bottleneck away from the
  network

14
Challenges in Wide-Area Storage

• Where is the bottleneck?
• Link Bandwidth
• Size of data to be transmitted
• Disk Speed
• Key issue in the wide-area storage is how to
  reduce the latency
• Latency introduced by the network
• Latency introduced by the storage

15
Outline

• Trends
• Challenges for wide-area storage
• Programmability inside networks
• Common techniques to hide latency
• Functionality that will benefit applications
• Network Support for that functionality
• Reconsidering the programmability model and
  application space
• Feedback

16
Common Techniques to Hide Latency

• Caching
• Parallelism
• Pipelining
• Prefetching

17
Code at edge vs- in the fabric
A

• Location of data separated from use of data
• Idea is to put processing near the data it acts
  on
• Better visibility into network conditions,
  dynamics
• Big performance gains if we can act on streams of
  data in the datapath
• Network processors are more powerful today
• A good match?

18
Outline

• Trends
• Challenges for wide-area storage
• Programmability inside networks
• Common techniques to hide latency
• Functionality that will benefit applications
• Network Support for that functionality
• Reconsidering the programmability model and
  application space
• Feedback

19
Gather
Synchronous
Asynchronous

• N-to-1 disk copies (KaZaa)
• Recreate dataset from multiple sources/disks
  (scientific experiment)
• Restore backup

• Digital animation editing
• Large dataset visualization

20
Gather

• Techniques
• Caching
• Parallelism
• Prefetching

• Network Primitives
• FS semantic information
• Store block location state in router
• View into network routes/conditions
• Table lookup in router
• Modify disk requests to point to correct
  locations
• Join data streams to deliver coherent data to app
• Orthogonal path selection

Synchronous

• Digital animation editing
• Large dataset visualization

21
Gather
Synchronous
Asynchronous

• N-to-1 disk copies (KaZaa)
• Recreate dataset from multiple sources/disks
  (scientific experiment)
• Restore backup

• Digital animation editing
• Large dataset visualization

22
Gather

• Join data streams to deliver coherent data to app
• Orthogonal path selection
• Volume state in routers
• Replicate SCSI requests
• Reorder SCSI responses

• Techniques
• Pipelining
• Avoid congestion/optimize for bandwidth

• Network Primitives
• FS semantic information
• Store block location state in router
• View into network routes/conditions
• Table lookup in router
• Modify disk requests to point to correct locations

Asynchronous

• N-to-1 disk copies (KaZaa)
• Recreate dataset from multiple sources/disks
  (scientific experiment)
• Restore backup

23
Scatter
Synchronous
Asynchronous

• Disaster-recovery application
• Experimental data unloading

• State dissemination
• CDN/web server updating?
• Gaming?
• Updating mapping tables

24
Scatter

• Techniques
• Delay-sensitive path selection
• Congestion avoidance
• Synchronization

• Network Primitives
• Network monitoring
• FS semantic information
• Store block location state in router
• View into network routes/conditions
• Table lookup in router

Synchronous

• State dissemination
• CDN/web server updating?
• Gaming?
• Updating mapping tables

25
Scatter
Synchronous
Asynchronous

• Disaster-recovery application
• Experimental data unloading

• State dissemination
• CDN/web server updating?
• Gaming?
• Updating mapping tables

26
Scatter

• Techniques
• Disk location/selection
• Load balancing
• Physical distance knowledge

• Network Primitives
• Network monitoring
• FS semantic information
• Store block location state in router
• View into network routes/conditions
• Table lookup in router

Asynchronous

• Disaster-recovery application
• Experimental data unloading

27
Outline

• Trends
• Challenges for wide-area storage
• Programmability inside networks
• Common techniques to hide latency
• Functionality that will benefit applications
• Network Support for that functionality
• Reconsidering the programmability model and
  application space
• Feedback

28
Useful Network PrimitivesWhat is reasonable and
possible?

• FS semantic information
• Store block location state in router
• View into network routes/conditions
• Table lookup in router
• Modify disk requests to point to correct
  locations

• Join data streams to deliver coherent data to app
• Orthogonal path selection
• Volume state in routers
• Replicate SCSI requests
• Reorder SCSI responses
• Others?

29
Outline

• Trends
• Challenges for wide-area storage
• Programmability inside networks
• Common techniques to hide latency
• Functionality that will benefit applications
• Network Support for that functionality
• Reconsidering the programmability model and
  application space
• Feedback

30
Your Feedback?