Scalable Content-aware Request Distribution in Cluster-based Network Servers

1
Scalable Content-aware Request Distribution in
Cluster-based Network Servers
Jianbin Wei 10/4/2001
2
Introduction

• Content-based request distribution can improve
  the throughput of the whole cluster server by
  cache affinity and get good balancing
  simultaneously.
• A single front-end quickly becomes the bottleneck
  of whole performance and limits the scalability
  of the distributed servers.
• TCP splicing strategy forwards the request based
  on the workload of back-end servers. Use weighted
  round-robin to get good load balancing.
• Decouple the role of the front-end into
  dispatcher and distributor.
• The dispatcher decides the distribution policy.
• The distributors perform the policy result by
  distributing the request.

3
Performance of LARD and TCP splicing
4
Solution

• Decouple the role of the front-end into
  dispatcher and distributor.
• The dispatcher decides the distribution policy.
• The distributors perform the policy result by
  distributing the request.
• The key insight is that the function of
  distributor can be executed parallel while the
  overhead of dispatcher accounts very small.
• With multiple front-end nodes, it must choose
  which distributor to process the request, which
  is difficult to achieve good load balancing with
  simple strategy.

5
Models
6
Prototype Implementation

• A persistent TCP control connection exists
  between any two nodes of the cluster and
  multiplexes the messages exchanged between any
  two components.
• The layer 4 switch maintains the load of each
  cluster node and decides the first part of client
  request.
• After receiving update message from the
  dispatcher that indicates which cluster server
  node is chose to service the request, it
  redistributes these requests.
• Upon connection termination, a close message is
  sent to the switch to discard the connection
  state.

7
Cluster Operation
8
Operation (cont.)

• The client contacts the switch, which decided
  which distributor to service the request
• The distributor contacts the dispatcher to get
  the assignment of this request
• The distributor distributes the request to the
  selected server node and sends this information
  to the switch
• The server response goes to the client directly.
• Any TCP acknowledgments sent by the client to the
  distributor are forwarded to the server node
  using a forwarding module at the distributor
  node. (DNS round-robin)

9
Technologies

• When using layer 4 switch
• The choice of distributor is made by the switch,
  using WRR strategy
• After a connection is handed off by the
  distributor, the switch is notified and the
  subsequent forwarding of TCP acknowledgments to
  the corresponding server node is handled by the
  switch.
• To decrease the communication between the
  dispatcher and the distributor,
• it hashes the URL to a 32 bit integer
• Sends several messages at the same time with
  dynamic degree of batching.

10
Experimental Results
11
Experimental Results (cont.)

• It shows the throughput results as the number of
  server nodes are increased.
• It indicates that the dispatcher is far from
  being a bottleneck.
• Due to the resource share between distributor and
  server component, the performance of N nodes is
  achieved by N-1 server nodes in LARD strategy.
  However, it affords greater scalability of the
  cluster server.
• When it runs the real workload, it shows the
  similar results.

12
Conclusion

• This strategy is a scalable architecture for
  content-aware request distribution in Web server
  clusters.
• Drawback because of the high overhead of
  handoff, it is necessary to decrease this cost to
  improve the performance and scalability of
  cluster server. (Scalable Web Server Cluster
  Design with Workload-Aware Request Distribution
  Strategy WARD, L. Cherkasova and M. Karlsson, HP
  Lab)