Scaleout vs. Scaleup Robert Barnes Microsoft

1
Scaleout vs. ScaleupRobert BarnesMicrosoft
2
Why 1BTPD?

• We had two main obstacles in bringing TP to
  Microsoft
• Lack of products and infrastructure
• Industry mindshare and/or credibility
• After starting to address the first obstacle, we
  decided to build something to demonstrate
  Microsoft has a credible platform for large-scale
  applications

3
Why 1BTPD?

• We liked Jim Grays cyber-brick machine and
  wanted to build one - only the technology wasnt
  there yet
• The closest thing to it was a collection of
  laptops - so we wanted to tie together 1000
  laptops. We decided DebitCredit would be a good
  workload to run, and 10KTPS DebitCredit was born
• 10 K Transactions per Second

4
(No Transcript)
5
Why DebitCredit?

• We needed a DB-only benchmark
• Do not want to configure one million terminals
• A simple transaction profile to stress DTC
• Many little transactions
• Need distributed transactions to stress DTC
• TPC-B is dead (can not report numbers)
• TPC-C is not server-only (need 1 million
  terminals) and requires distributed data
  transparency
• So, we returned to the Datamation Benchmark (1985)

6
10KTPS ? 1BTPD

• Work began on the benchmark in mid 1995
• A vendor released a 30K TPM-C number
• Even though 10KTPS is 600Ktpm and DebitCredit is
  very different from TPC-C transactions, we needed
  a different metric
• 11574.07 tps 694444.2 tpm 41666652 tph
  1 Billion Transactions
  Per Day
• thanks, Jim

7
Scaling
8
Scaleable SystemsSMP and Loosely Coupled Systems
Scaleup with SMP8P is new standard
(SHV) Scaleout with Loosely Coupled
Systems (LCS) using inexpensive parts
SMP
Super Server
Departmental
Server
LCS of PCs or SHVs
Personal
System
9
SMP Advantages

• Single system image
• no change to operations (relative to
  uniprocessor)
• no change to applications
• Simple system resources
• shared memory
• shared disk
• shared net
• Load balancing in OS kernel
• 8x SMP soon commodity SHV

SMP
Super Server
Departmental
Server
Personal
System
10
SMP Problems

• Problems
• More then 8 processors not a commodity today
• More than 16 processors may require
  partitioning, scale out needed anyway
• scale-down problem (starter systems expensive
  or fork lift upgrade)
• Potential single point of failure
• Eventually stops scaling

SMP
Super Server
Departmental
Server
Personal
System
11
SMP Performance Diminishing returns
Linear extrapolation for 16
12
LCS Advantages

• Advantages
• No hardware limit to scale, given a scaleable
  interconnect and an application that partitions
• Uses high volume,commodity components
• No single point of failure
• Upgrade by incremental growth

Load Balancing
Common Function
13
LCS Problems

• Operations complexity
• When not designed for LCS, costs can be very
  expensive
• Change in system architecture e.g. Exchange, SQL
  Server (relative to uniprocessor)
• Need additional system services
• Load Balancing
• LCS Membership
• Configuration replication
• Failure Retry (within LCS)
• Parallelism needed for data and applications
• explicit parallelism exposes partitions to the
  application
• implicit parallelism requires transparency

Load Balancing
Common Function
14
givememoney.com

• The givememoney.com business is brokering their
  goal is to not directly handle any products
• Site is currently lt10 stores, plan to be hundreds
  in a few years
• Books, music, video,computers,software,games,surpl
  us
• It takes over 90 days to add a new store
• Over 90 servers make up current site
• Today, checkouts/day 10K
• Business plan calls for 500K by 2002

15
givememoney.com FE
5
2
5
2
2
Cache Server
ASP
SSL
ASP
SSL
FARM B
FARM A
Basket/Ad/Surplus
ASP File Server
SQL Product Server
SQL Product Server
ASP File Server
Receipt/Fulfillment
Games/Music
Videos
Games/Music
Videos
Monitor and cache
Comp/Soft
Books
Comp/Soft
Books
Music
Music
Search Servers
Search Servers
16
givememoney.com

• We dont have a scaling problem if we need
  more capacity, we just add a server. We have a
  management problem

17
Scale Out Business Need

• givememoney.com is at 10K orders/day, business
  plan is to grow to 500K by 2002
• Moores law says scale improves by 4X in 3 years,
  givememoney.com wants 50X , over 10X Moores law.
• Even if you believe Moores law solves the growth
  problem, it requires many forklift upgrades
• We still have to scale up as it minimizes the
  number of servers needed

18
Consolidation

• Not consolidation vs. scale out
• Consolidation motivations
• Business
• Site (Space)
• Operations
• Storage
• Reasons for never wanting to have 1 huge system
• Single point of failure
• Dis-economies of scale up
• Mixed workload has operations challenges similar
  to scale out
• Difficulty of dealing with growth

19
Scaleout is a software challenge

• Scaleup is hardware architecture OEM driven
• Optimized by Software
• e.g. minimize locking, parallel threads, reduce
  contention on critical sections, etc.
• Ultimately, scale up begins to look like scale
  out (partitioning)
• Scaleout is software architecture driven
• Optimized by hardware
• e.g. SAN (system area nets), low latency/high
  bandwidth interconnects
• Primarily an application design and operational
  management problem
• Possible today, but mostly left to application
  design/development and ad hoc management tools

20
Summary

• Scaleup begins to look like scaleout as you add
  processors
• Scaleout with commodity hardware is economically
  compelling
• Every large successful web site uses scaleout
  it is practical, but hard
• Operations complexity is the key barrier to
  scaleout

21
CyberWall