Direct Attached Storage Performance and Tuning

1
Direct Attached Storage Performance and Tuning

• Darryl Gardner
• xSeries Performance Engineer
• October 25, 2007

2
Agenda

• 2007 RAID Adapter Roadmap
• RAID Adapter Performance
• ServeRAID Tuning Recommendation
• Solid State Drive Performance

3
2007 RAID Adapter Roadmap
4
Current SAS RAID Adapters (ARC Code Stack)

• ServeRAID 8i (x366, x460)
• I/O Processor 600 MHz Intel 80321
• SCSI Adapter Adaptec
• Memory 128MB or 256MB DDR 200 MHz
• Battery Standard
• PCI-X 1.0 PCI-X 1.0 (Special ZCR Slot)
• Half Length, Half Height
• Internal SAS Ports 6
• Max Drive support 6 - 2.5 10K RPM SAS
• RAID Levels 0,1,1E,5,6,5EE,00,10,1E0,50,60
• Max Stripe Size 1M
• Price

• ServeRAID 8i (x260)
• I/O Processor 600 MHz Intel 80321
• SCSI Adapter Adaptec
• Memory 128MB or 256MB DDR 200 MHz
• Battery Standard
• PCI-X 1.0 PCI-X 1.0 (Special ZCR Slot)
• Half Length, Half Height
• Internal SAS Ports 8
• Max Drive support 12 - 3.5 10K/15K RPM SAS
• RAID Levels 0,1,1E,5,6,5EE,00,10,1E0,50,60
• Max Stripe Size 1M
• Price

5
Current SAS RAID Adapters (ARC Code Stack)

• ServeRAID 8K
• I/O Processor 250 MHz MIPS
• SAS Adapter Adaptec
• Memory 256MB DDR II 533 MHz
• Battery Standard
• PCI-E x4
• On Planar
• Internal SAS Ports 4
• External SAS Ports 4
• Max Drive support 18 - 3.5 10K/15K, RPM SAS (6
  internal, 12 external), 8 2.5 10K RPM SAS
• RAID Levels 0,1,1E,5,6,5EE,00,10,1E0,50,60
• Max Stripe Size 1M
• Price

• ServeRAID 8K-l
• I/O Processor 250 MHz MIPS
• SAS Adapter Adaptec
• Memory 32MB DDR II 533 MHz
• Battery Not Available
• PCI-E x4
• On Planar
• Internal SAS Ports 4
• External SAS Ports 4
• Max Drive support 6 - 3.5 10K/15K RPM SAS, 8
  2.5 10K RPM SAS
• RAID Levels 0,1,,10
• Max Stripe Size 1M
• Price

6
Current SAS RAID Adapters

• ServeRAID 8S
• I/O Processor 800 MHz Intel 80333
• SAS Adapter Adaptec 9410W
• Memory 128 MB DDRII 400 MHz
• Battery Optional
• PCI-E only
• 3/4 Length, Full Height
• Internal x4 SAS Ports 2
• External x4 SAS Ports 1
• Max Drive support 36
• RAID Levels 0,1,1E,5,5EE,6,10,50,60
• Max Stripe Size 1M
• Positioned to provide internal RAID function for
  servers that do not support ServeRAID 8K
  (Guanshan, Mabolas)

• MegaRAID 8480
• I/O Processor 500 MHz Intel 80333
• SAS Adapter LSI 1064
• Memory 256 MB DDRII 400 MHz
• Battery Standard
• PCI-E only
• Half Length, Full Height
• Internal x4 SAS Ports 0
• External x4 SAS Ports 2
• Max Drive support 48
• RAID Levels 0,1,5,10,50
• Max Stripe Size 128K

7
4Q07 SAS RAID Adapters Hardware Design Comparison

• ServeRAID MR10K
• I/O Processor 533MHz PPC 440
• SAS Controller LSI 1078 (single chip design)
• Memory 256 MB DDRII 667 MHz
• Battery Yes
• RAID On Motherboard (ROMB)
• External SAS Ports 4
• Internal SAS Ports 4
• Max Drive support 117
• RAID Levels 0,1,5,6,10,50,60
• Cache memory upgrade to 1078 subsystems for full
  RAID on Motherboard implementation

• ServeRAID MR10i
• I/O Processor 533MHz PPC 440
• SAS Controller LSI 1078 (single chip design)
• Memory 256 MB DDRII 667 MHz
• Battery Yes
• RAID On Motherboard (ROMB)
• External SAS Ports 0
• Internal SAS Ports 8
• Max Drive support 16
• RAID Levels 0,1,5,6,10,50,60
• Low cost PCI-E RAID Controller for Internal
  System RAID 5
• System Supported follow-ons to x3200, x3250

8
4Q07 SAS RAID Adapters Hardware Design Comparison

• ServeRAID MR10M
• I/O Processor 533MHz PPC 440
• SAS Controller LSI 1078 (single chip design)
• Memory 256 MB DDRII 667 MHz
• Battery Yes
• RAID On Motherboard (ROMB)
• External SAS Ports 8
• Internal SAS Ports 0
• Max Drive support 226
• RAID Levels 0,1,5,6,10,50,60
• High Performance PCI-E RAID Controller for
  External Direct Attached Storage
• System Supported All

9
Comparison EXP400, EXP3000
10
RAID Adapter Performance Comparisons
11
Single Threaded Sequential Performance (File
Copy, OS load, video rendering)

Peak sequential performance will not be achieved
with a single thread
12
Small Random Read Performance (Web/File/Print
Server, OLTP database)
-Random performance scales as drives are added
13
Small Random Write Performance (Web/File/Print
Server, OLTP database)
-Random performance scales as drives are added
14
Small Random Read/Write Performance
(Web/File/Print Server, OLTP database)
15
Small-Medium Random Read I/O Latency(Web/File/Pri
nt Server, CAD/CAM)
Marauder (SR 8S) has higher latency than Sanford
(MR 8480).
16
Small-Medium Random Write I/O Latency(Mail
Server, OLTP Database)
Marauder (SR 8S) has higher latency than Sanford
(MR 8480)
17
Medium Sequential Read Peak (Video/Audio
streaming, Television Broadcasting)
18
Medium Sequential Write Peak (Video/Audio
streaming, Television Broadcasting)
Peak RAID-5 sequential write performance of SR MR
10k/10i/10M will be reduced by 15-20 in Athena
due to extra PCIe latency on Athena
19
Large Sequential (Video/Audio streaming,
Television Broadcasting, Business Intelligence)
To achieve maximum performance on SR MR 10M, use
all SAS PHYs
20
ServeRAID 10 Tuning Recommendations
21
ServeRAID 10 - Stripe Size Tuning
-When throughput is SAS PHY limited (using only 4
SAS PHYs) increasing stripe size will allow for
slightly more efficient use of SAS PHYs - Peak
Sequential Read performance was measured with a
64K stripe size
22
ServeRAID 10 - Stripe Size Tuning
-In general, larger stripe sizes will yield
higher peak sequential write performance
23
ServeRAID 10 - Stripe Size Tuning
-With the default no read ahead setting, single
threaded sequential read performance increases
with smaller stripe sizes
24
ServeRAID 10 - Stripe Size Tuning
-64K and 128K stripe sizes are the sweat spot for
small-medium block random workloads
25
ServeRAID 10 - Stripe Size Tuning
-64K and 128K stripe sizes are the sweat spot for
small-medium block random workloads
26
ServeRAID 10 Hardware Read Ahead Tuning
- Use adaptive read ahead setting to achieve
highest single threaded sequential read
performance - System must be rebooted before the
setting will take effect
27
ServeRAID 10 Software Read Ahead Tuning
-Use a larger read ahead parameter in Linux to
improve sequential read performance -Command line
syntax blockdev --setra 16384 /dev/sdb1
28
Solid State Drive (SSD) Performance
29
Linux Boot Performance (Measured)

• It takes 2 minutes longer to boot Linux on USB
  flash drives than it does on SAS drives
• Linux boot time on SATA flash drives are equal to
  or better than SAS
• Two SATA flash drives in a RAID-1 configuration
  boots Linux slightly faster than a single SATA
  flash drive configuration

30
Windows Boot Performance (Measured)

• The Windows boot performance is comparable on SAS
  and SATA flash drives

31
Linux OS Install Performance (Measured)

• SATA flash takes about 51/2 to 61/2 minutes
  longer to install Linux than SAS drives.
• USB flash drives takes more than 7 times longer
  than SAS drive to install Linux
• USB flash drive Linux OS install performance may
  not be acceptable all customers

32
Sequential Read Performance (Iometer)

• SAS drives provide higher sequential read
  performance than the flash drive alternatives
• SATA flash drives provide much higher sequential
  read performance than USB flash drives .

33
Sequential Write Performance (Iometer)

• The 4GB USB flash drives single threaded
  sequential write performance is comparable to a
  10K SFF SAS drive
• SATA flash drives single threaded sequential
  write performance is comparable to a 15K SFF SAS
  drive
• In multi-threaded sequential write workloads, SAS
  drives outperform flash drives by 7 to 17 times.

34
Random Performance (Iometer)

• Flash drives have a medium to very large
  advantage over SAS drives in random read
  workloads.
• However, the random write performance of the
  flash drives are significantly lower than SAS
  drives
• For server applications that consist of 100
  random reads and can fit on the smaller flash
  drives, a flash drive may be appropriate
• However, if there is even a small amount of
  random write I/O (which is very common in many
  random server workloads) the overall performance
  of the flash drives will drop off significantly

35
Random Read/Write Performance (Iometer)

• In workloads that consist of a 21 mix of random
  reads and random writes, the overall performance
  of the flash drives are much lower than SAS
  drives, due to their low random write performance.

36
Random Performance (Iometer)

• Due to its low random write performance It is
  expected that SanDisks SATA flash performance in
  random workloads will not match that of the SAS
  drives unless the workload consist of less than
  5 writes.

37
Additional Charts
38
Benchmark Tool Recommendations
39
Benchmark Tool Recommendation

• Iometer is the recommended tool to evaluate disk
  subsystem performance
• Windows Setup
• Adjust the outstanding I/Os parameter to
  increase of I/O threads
• Set the Outstanding I/O parameter to at least 2
  I/O per drive (for example, in a 10 drive
  configuration, set the outstanding I/O parameter
  to 20)
• Linux Setup
• Add dynamos to increase of I/O threads
• Use at least 2 dynamos per drive (for example, in
  a 10 drive configuration, use at least 20
  dynamos)
• Linux Iometer automation scripts available on
  request

40
Recommended Iometer Workloads

• 4K Random Read, 4K Random Write, and 4K OLTP
  workloads generally emulate some of the following
  server workloads
• Web server (Reads)
• Web e-Commerce (Reads)
• File server (Windows) (Read/Writes)
• Print server (Reads)
• Customer Relationship Management (Reads)
• SQL, DB2, Oracle- OLTP/Database (Read/Writes)

41
Recommended Iometer Workloads

• 64K Random Read, 64K Random Write, and 64K OLTP
  workloads generally emulate some of the following
  server workloads
• Mail -Lotus Domino, Exchange, etc (read, write)
• SQL, DB2, Oracle- OLTP-Database (read, write)
• File Server (Linux) (read)

42
Recommended Iometer Workloads

• 64K Sequential Read and 64K Sequential Write
  workloads emulates some of the following server
  workloads
• Video Rendering (Writes)
• Video Streaming (Reads)
• Audio Streaming (Reads)
• Television broadcasting (Reads)

43
Recommended Iometer Workloads

• 1M Sequential benchmark emulates some of the
  following server workloads
• Video Rendering (Writes)
• Video Editing (Read/Writes)
• Video Streaming (Reads)
• Audio Streaming (Reads)
• Television broadcasting (Reads/Writes)
• SQL, DB2 - Business Intelligence-Database (Reads)