Technology Strategies for Deploying Oracle Real Application Clusters

1
Technology Strategies for Deploying Oracle Real
Application Clusters

• Vic Andrade, Solutions Architect
• vic.andrade_at_hp.com

2
What is a Cluster?
A cluster is a group of independent computers
working together as a single system

• AvailabilityContinues running in case of a
  hardware or software failure
• ScalabilityProvides incremental growth to handle
  increased workload
• PerformanceDistributes workload among nodes for
  optimal performance

3
Clustering Models
A) Active / Standby Mode

• Only one server (the primary node) performs work,
  and the other (the standby, or secondary node)
  stands by waiting for a failure in the primary
  node
• If the primary node fails, the clustering
  solution transfers the primary node's workload to
  the standby node and terminates any users or
  workload on the standby node

4
Clustering Models (cont.)
B) Active / Active Mode
Clients

• All nodes in the cluster perform meaningful work
• If any node fails, the remaining node(s)
  continues handling its workload and takes on the
  workload from the failed node

Client Network
Sales Database
Marketing Database
Heartbeat
Sales Data
Marketing Data
5
Disk Systems
A) Non-Shared Disk Systems

• MPP computers
• Uniprocessor or SMP nodes
• Restricted disk access
• High-speed interconnect mandatory
• high bandwidth, low latency

6
Disk Systems (cont)
B) Shared Disk Systems

• Clustered computers
• Uniprocessor or SMP nodes
• Unrestricted disk access
• High-speed interconnect optional
• Limited by disk bandwidth
• 9i RAC/8i OPS maps naturally to this HW
  architecture

7

9i Real Application Clusters (RAC) More than just
a new name for OPS!
OPS
RAC

• scalable
• near linear scaling

• hard to scale
• negative scaling

real applications will work without change
application must be cluster-aware
manage just as a single instance
very complex to manage
8
Cache coordination challenge
9
Performance Ramifications of Cache Coherency
100 msec
Delay (approx)
20 msec
1 msec
.01 msec
data not in cache (fetch from disk)
data dirty(locked for writing) in remote cache
data in local cache
10
Choice of Supported Platforms

• Various Unix on various RISC machines
  (Tru64/Alpha, Solaris/Sparc, HPUX/PA-RISC,
  AIX/Power)
• Windows on Intel IA32
• Various Linux on Intel IA32
• Various Linux on Intel IA64 (Itanium)
• HPUX on Itanium
• Windows on Itanium
• Windows, Linux on Opteron?

11
Software/Hardware Layer Architecture
CFS or Raw/LVM
CFS or Raw/LVM
CFS or Raw/LVM
CFS or Raw/LVM
12
Raw Devices, Cluster File Systems, and Cluster
Managers

• Raw devices/Partitions Physical Disk accessed at
  lowest level. Managed by OS LVM, but not an OS
  File System. Fastest Performance due to less OS
  overhead. Oracle is managing access to the Shared
  Disks
• Cluster File System Uses the OS File system to
  manage the physical disk. All files, including
  Oracle data files, can be shared amongst cluster
  nodes. Can make for easier management of the
  entire cluster. OpenVMS was the first
  implementation supported by Oracle, followed by
  Tru64.
• Cluster Manager Negotiates the failover should a
  node encounter problems. Tracks who is
  controlling the shared resources when a member
  tries to re-join the cluster, along with a quorum
  mechanism.

13
Storage solutions for Linux and Oracle

• Raw devices ?

• Logical volume manager ?

• Cluster File System ?

14
Storage solutions for Linux, Windows and Oracle
- Oracle Cluster File System (OCFS)

• features and benefits
• developed and supported by Oracle
• simplifies storage and Oracle management
• file level backup
• no limitation on number of devices
• minimal performance impact
• available for Windows and Linux
• Supported by Oracle
• limitations
• Currently, only recommended for database files,
  not for Oracle Binaries
• OCFS requires Red Hat Errata 12 or higher
• Currently OCFS doesnt support Asynch_IO, will be
  supported in 1.0-9

15
Software Stack Summary for 9i RAC
See Metalink and Docid 183408.1 for additional
details
16
Comparison of Common Interconnects technologies
Latency
Throughput
Name
(in Microsec)
(in Gb/S)
Memory Channel
3
1
Hyper-Fabric
22
4
Fast Ethernet
3000
0.1
Gigabit Ethernet
3000
1
InfiniBand
lt500
gt2.5
Myrinet
7
2
Available with Linux
Not yet available on Linux, Windows
17
Scale Up vs. Scale Out?

• Scale Up
• Most common in RISC based SMP (2 to 128 CPU
  single systems) running UNIX
• Lowest memory latencies better performance
• Large memory addressability
• Scalability limited by each frame model
• Optional SW and Physical Partitioning schemes
  available
• Scale out
• Most common in Intel IA 32 based systems running
  Linux
• Offers improved reliability and continuity of
  service
• Can grow modularly as needed, minimizing
  disruption
• Unlimited scalability
• Explore the limits of each later

18
Scale Up vs. Scale Out? (Continued)

• Windows (IA32)
• Scaling up to 8 processors/ node
• Limited to a 4 GB Address space (up to 3 GB for
  Oracle)
• Shared memory limit of 1.7 GB for IA32
• Oracle variables require Registry editing (limits
  multiple instances on a machine)
• Unlimited number of nodes? realistically 3-6
  (?)
• Linux (Not All Distributions are supported)
• RH, Suse IA32 max RAM of 64GB, 16GB more
  realistic
• Shared memory limit of 1.7 GB for IA32, 2.7 GB
  with some tricks (Limits the SGA)
• Limited to 4-8 CPUs/node, Latest kernel says 16
  CPUs.
• Unlimited number of nodes? realistically 3-6
  (?) for IA32, but much higher with IA64/Itanium

19
Intel Itanium 2based systems addressthe ability
to scale up and out
faster than IA-32 and more cost effectively than
RISC through more than just 64-bits and fast
floating point
benefits
faster OLTP
quickerWeb serving
faster securetransactions
better Java object codeperformance
IA32 scales out because it needs to, RISC/IA64
scales out if it has or wants to.
20
Does RAC Scale in Real Applications?

• Transaction Processing Council (TPC-C, TPC-H)
• TPC-C attempts to simulate OLTP activity both
  read and write
• TPC-H attempts to simulate Decision Support
  more query intensive, but bulk data loading
• Oracle Applications Standard Benchmark
• Mix of transactions from OE, HR, FI, AR, AP, GL
  etc. Some OLTP, DSS, batch, with a goal of
  getting as many users as possible within a
  defined response time.
• SAP Sales Distribution (SD) User benchmark
• High volume of inserts, with goal of trying to
  get as many users as possible within a lt2s
  response time

21
TPC-C 8 node Cluster 32 CPUs
Oracle on HP Computers
22
TPC-H 300GB Cluster
Oracle on HP Computers
23
E-Business Suite Scalability with Oracle9i RAC
90 Scalability
Oracle Applications Benchmark on RAC
4,368
Users
2,296
Running on HP Computers, with 4x 550 Mz PA-RISC
processors (Version 11.5.3)
24
E-Business Suite Scalability with Oracle9i RAC
80 Scalability
Oracle Applications Benchmark on RAC
18,368
Users
5,656
Running on IBM Computers, Suse Linux, 1 Node with
8x Xeon 2GHz, 4 Node with 8x Xeon 2.8 GHz
(Version 11.5.6) - Not identical
systems
25
SAP Scalability with Oracle9i Real Application
Clusters
SD 3-Tier Parallel Benchmark Results Now Official
82 Scalability
Users
Certified
Running on HPQ AlphaServer Computers
26
Scale out and Scale Up

• HP and Oracle first to break 1 million tpm-c
• UNIX and Linux
• Both with same of Intel Itanium processors
• HP sets the bar for Oracle
• 1 cluster TPC-C
• 1non-cluster TPC-C
• Linux RAC TPC-C
• 16 nodes x 4 processors Intel Itanium2 1.5 Ghz
• Red Hat Linux AS 3
• HP StorageWorks MSA100
• Gigabit Ethernet interconnect

TPC-C Benchmark Results
27
Example Price Points 8 CPUs
Single SMP RISC/Unix vs. Clustered IA32/Linux

• rp4440 with 8 x PA8800 _at_1 GHz, 16 GB RAM est.
  tpmC 155K
• rx7620 with 8 x Itanium2 _at_ 1.3 GHz, 16 GB RAM
  est. tpmC 180K
• rp 7410 with 8 x PA8700 _at_ 875 MHz, 16 GB RAM
  est. tpmC 105K
• 2 x DL580/4 each with 4 x Xeon _at_ 2.8 GHz, 8 GB
  RAM est. tpmC 142K

28
Example Price Points 8 CPUs
29
Example Price Points
Single SMP RISC/Unix vs. Clustered IA32/Linux

• rx8620 with 16 x Itanium2 _at_ 1.5 GHz, 32 GB RAM
  est. tpmC 366K
• rp7420 with 16 x PA8800 _at_ 1.0 GHz, 32 GB RAM
  est. tpmC 257K
• 4 x DL580/4 each with 4 x Xeon _at_ 2.8 GHz, 8 GB
  RAM est. tpmC 265K
• 8 x DL580/4 each with 4 x Xeon _at_ 1.4 GHz, 8 GB
  RAM actual tpmC 138K

30
Example Price Points 16 CPUs
31
Example Price Points OASB
Single SMP RISC/Unix vs. Clustered IA32/Linux

• rx5670 with 4 x Itanium2 _at_ 1.5 GHz, 64 GB RAM,
  RH Linux 5992 Users
• rx5670 with 4 x Itanium2 _at_ 1.5 GHz, 64 GB RAM,
  HPUX 6440 Users
• 2 x DL580/4 each with 4 x Xeon _at_ 2.8 GHz, 32 GB
  RAM Suse Linux 7504 Users

32
Summary Price vs. Performance

• On Average, RISC performance is 1.2-2x IA32
• SW license for RAC adds about 50 to Oracle
  licensing costs
• RAC is more complicated to set up and administer
• Certification process is less defined more
  integration effort
• Not all ISVs support the use of RAC (SAP)
• similar clock rates

• HW Acquisition costs for IA32 servers is less
  than RISC based systems
• RAC can provide greater reliability and
  scalability
• Increasing performance (scaling out) can be done
  with minimal disruption
• Some vendors have predefined bundles to jump
  start RAC deployment

33
Summary Price vs. Performance

• Tradeoff in HW costs for SW licensing fees
• Tradeoff in acquisition costs, deployment effort,
  and ongoing administration costs
• Still learning from existing RAC deployments,
  many good POCs conducted
• Ultimately What do you need and how do you run
  your IT Operations?

34
(No Transcript)