Scalable Performance

1
Scalable Performance

• Distributed Systems Lecture 9

2
Overview

• How to maintain high-performance that scales
  millions or billions of entities
• Case Study Internet and WWW

3
Performance Tricks

• Systems Approach
• The KISS principle Keep it Simple Stupid
• Layering
• Identify bottlenecks
• Hide Latency Exploit workload properties
• Improve Throughput Concurrency
• Case Study Internet
• Guarantees about state Soft vs. Hard
• Model of consistency Absolute vs. Eventual
  Consistency

4
Internet Design

• Really Really Simple
• End-to-end system design
• The system does not provide any guarantee
• No Reliability
• No Ordering
• No Congestion Control
• Only Routing
• No QoS?

5
Why?

• Ordering, Reliability and Congestion Control
• Typically involve state
• May evolve over time
• Must be application-specific
• Architect Services at the end-systems

6
Layering in the Internet

• Large Distributed System
• IP only provides unreliable, unordered,
  non-flow-controlled message delivery
• Layer these services on top when needed
• Workload characterization
• Performance for the common case
• Reliability, Ordering, and Congestion-control
  when needed

7
TCP Screw Up

• Combines
• Reliability
• Ordering
• Congestion Control
• Multimedia Applications
• Ordering but no reliability
• RTP
• Congestion Control
• Wireless Networks
• Multimedia Applications
• TCP Vegas, Reno, Taheo

8
Conclusions

• Keep the core system really simple
• IP Layer
• Layer additional Services on top
• Keep Services at the ends of the system
• Easy to upgrade
• Easy to maintain

9
Internet Routing Architecture

• All routers must have consistent routes
• But If a router is out of date, its not the end
  of the world
• But system must be available
• Reliability is provisioned at the ends
• Question Whats the consistency model?

10
Eventual Consistency

• Just tell your neighbor every so often about your
  routing state
• He tells his neighbors and so on
• Eventually, everyone will be consistent
• Good enough?
• Ford-Fulkerson
• They call it Link-State-Routing
• Call it gossip

11
Gossip-based Consistency

• Talk to a subset of your friends
• They talk to another subset
• Will everyone be eventually consistent?

12
Scalable Performance

• Lets assume you are writing a web-server
• Apache, IS, Zope
• Principles
• Identify Bottlenecks
• Hide Latency Workload Characterization
• Throughput Use Threads

13
Workload

• Workload?
• Same thing over and over again
• Use processes vs. threads?
• Modern Webservers do more than fetch and reply
• Can use stages?
• Fast path vs. Slow Path

14
Bottlenecks

• http --gt Get file --gt Reply
• Hard disk access!
• Interleave requests

15
Concurrency

• Fixed vs. Thread Pools
• Staging of Jobs
• Queues between stages