Title: Keeping Up with z/OS
1Keeping Up with z/OS Alphabet Soup
- Darrell Faulkner
- Computer Associates
- Development Manager
- NeuMICS
-
2Objectives
- Integrated Coupling Facility (ICF) and
Integrated Facilities for LINUX (IFL) - PR/SM and LPs
- Intelligent Resource Director (IRD)
- IBM License Manager (ILM)
- Capacity Upgrade on Demand (CUoD)
- Conclusions
3Acronyms
- CF - Coupling Facility
- CP - Central Processor
- CPC - Central Processor Complex
- ICF - Integrated Coupling Facility
- IFL - Integrated Facility for LINUX
- PR/SM - Processor Resource/Systems Manager
- HMC - Hardware Management Console
- LLIC - LPAR Licensed Internal Code
- Logical CP - Logical Processor
- LP - Logical Partition
- LPAR - Logical Partitioning (LPAR mode)
- PU - Processor Unit
4ICF and IFL
- Beginning with some IBM G5 processor models, the
ability to configure PUs (Processing Units) as
non-general purpose processors - Benefit - Does not change model number hence no
software licensing cost increase
IFL - Integrated Facility for LINUX
ICF - Integrated Coupling Facility
5z900 Models 2064-(101-109)
6z900 Model 2064-105
5 PUs Configured as CPs Model 105
CPs Defined Model Number
7z900 Model 2064-105
8ICFs and IFLs
IBM SMF Type 70 subtype 1 record - CPU
Identification Section
- There is one section per EBCDIC name that
identifies a CPU type. 'CP' and 'ICF', with
appropriate trailing blanks, are examples of
EBCDIC names describing a General Purpose CPU and
an Internal Coupling Facility CPU, respectively. - Name SMF70CIN
- Length 16 EBCDIC
- Description CPU-identification Name
- Offsets 0 0
- As of z/OS Version 1 Release 2, both IFLs and
ICFs are represented by ICF in the SMF type 70
CPU ID Section
IFL - Integrated Facility for LINUX
CP Central Processor
ICF - Integrated Coupling Facility
9PR/SM LPAR
- Allows up to 15 images (LPs) per CPC
- Different control programs on images
- (z/OS, z/VM, Linux, CFCC etc.)
- Each LP (image) assigned CPC resources
- Processors (CPs) (referred to as logical CPs)
- Memory
- Channels
- Each LP either DEDICATED or SHARED
Logical CP Logical Processor
LP Logical Partition
CPC Central Processor Complex
10PR/SM Benefits
- Protection/isolation of business critical
applications from non-critical workloads - Isolation of test operating systems
- Workload Balancing
- Different operating systems -- same CPs
- Ability to guarantee minimum percent of shared CP
resource to each partition - More white space the ability to handle spikes
and unpredictable demand
11LP Configuration Decisions
- LP definitions entered on HMC
- Dedicated or not-dedicated (shared)
- Logical processors (initial, reserved)
- Weight (initial, min, max)
- Capped or not-capped
- CPC memory allocation
- I/O Channel distribution/configuration
- More
HMC Hardware Management Console
CPC Central Processor Complex
LP Logical Partition
12Dedicated LPs
- LPs logical CPs are permanently assigned to
specific CPC physical CPs - Less LPAR overhead (than shared LPs)
- Dedicated LPs waste physical (CPC) processor
cycles unless 100 busy - When less than 100 busy, the physical CPs
assigned to dedicated LPs are IDLE
Logical CP Logical Processor
LP Logical Partition
CPC Central Processor Complex
13LPAR MODE - Dedicated
PR/SM LPAR LIC
CPC MEMORY
ZOS1
ZOS2
ZOS1 Image - 3 Dedicated Logical
Processors ZOS2 Image - 2 Dedicated Logical
Processors Same problem as basic mode - Unused
cycles wasted
LCP Logical CP Logical Processor
14Shared LPs
HMC Image Profile
ZOS1
15Shared LPs
HMC Image Profile
ZOS2
16LPAR Mode - Shared
PR/SM LPAR LIC
LCP Logical CP Logical Processor
17LPAR Dispatching
- What does LLIC (LPAR Licensed Internal Code) Do?
- LCPs are considered dispatchable units of work
- LCPs placed on ready queue
- LLIC executes on physical CP
- it selects a ready LCP and
- dispatches it onto real CPs
- z/OS executes on physical CP until timeslice
expires (12.5-25 milliseconds) or until z/OS
enters a wait state - Environment saved, LLIC executes on freed CP
- If LCP still ready (used timeslice), it is placed
back on ready queue
LLIC LPAR Licensed Internal Code
LCP Logical CP Logical Processor
CP Central Processor
18Selecting Logical CPs
- Priority on the ready queue is determined by
PR/SM LIC - Based on LP logical CP actual utilization
versus targeted utilization - Targeted utilization is determined as a function
of LCPs and LP Weight - LP weight is a user specified number between 1
and 999 (recommended 3 digits)
LP Logical Partition
LLIC LPAR Licensed Internal Code
LCP Logical CP Logical Processor
CP Central Processor
19LP Weights -Shared Pool
PR/SM LPAR LIC
CPC MEMORY
ZOS1
ZOS2
400
100
ZOS1 Image
ZOS2 Image
- Total of LP Weights 400 100 500
- ZOS1 LP Weight 100 400/500 80
- ZOS2 LP Weight 100 100/500 20
LCP Logical CP Logical Processor
LP Logical Partition
20LP Weights GuaranteePool CP Share
- Weight assigned to each LP defined as shared
- All active LP weights summed to Total
- Each LP is guaranteed a number of the pooled
physical CPs based on weight of Total - Based on shared logical CPs defined for each LP
LP weight, LLIC determines the ready queue
priority of each logical CP - Weight priority enforced only when contention!
LP Logical Partition
LLIC LPAR Licensed Internal Code
CP Central Processor
LCP Logical CP Logical Processor
21LP Target CPs
PR/SM LPAR LIC
CPC MEMORY
ZOS1
ZOS2
400
100
ZOS1 Image
ZOS2 Image
- ZOS1 LP Weight 80
- Target CPs 0.8 5 4.0 CPs
- ZOS2 LP Weight 20
- Target CPs 0.2 5 1.0 CPs
22LP Logical CP share
- ZOS1 LP is guaranteed 4 physical CPs
- ZOS1 can dispatch work to 5 logical CPs
- Each ZOS1 logical CP gets 4/5 or 0.8 CP
- ZOS1 effective speed 0.8 potential speed
- ZOS2 LP is guaranteed 1 physical CP
- ZOS2 can dispatch work to 3 logical CPs
- Each ZOS2 logical CP gets 1/3 or 0.333 CP
- ZOS2 effective speed 0.333 potential speed
23Impact of Changing Weights
- An active LPs weight can be changed
non-disruptively using system console - Increasing an LPs weight by x, without any
other configuration changes, increases its pooled
CP share at the expense of all other shared LPs - This is because the TOTAL shared LP weight
increased, while all other sharing LPs weights
remained constant - LPn weight LPn weight gt
- TOTAL (TOTAL x)
LP Logical Partition
CP Central Processor
24Changing LPAR Weights
100
400
100
25LP Target CPs
PR/SM LPAR LIC
CPC MEMORY
ZOS1
ZOS2
400
200
26LP Logical CP share
- ZOS1 LP is guaranteed 3.335 physical CPs
- ZOS1 can dispatch work to 5 logical CPs
- Each ZOS1 logical CP gets 3.335/5 or 0.667 CP
- ZOS1 effective speed 0.667 potential speed
- ZOS2 LP is guaranteed 1.665 physical CP
- ZOS2 can dispatch work to 3 logical CPs
- Each ZOS2 logical CP gets 1.665/3 or 0.555 CP
- ZOS2 effective speed 0.555 potential speed
27Changing Logical CP Count
- An active LPs logical CPs can be increased or
reduced non-disruptively - Changing the number of logical CPs for a shared
LP increases or decreases the LP work potential - Changes z/OS and PR/SM overhead
- Does not change the CPC pool share
- Changes the LP logical CP effective speed
CPC Central Processor Complex
28Adding Logical CPs
400
100
ZOS1 Image
ZOS2 Image
- Total LP Weights 400 100 500
- ZOS1 LP Weight 100 400/500 80
- ZOS2 LP Weight 100 100/500 20
29Adding Logical CPs
400
100
LCP
LCP
LCP
LCP
LCP
LCP
LCP
LCP
LCP
ZOS1 Image
ZOS2 Image
- ZOS1 Weight 80
- Target CPs 0.8 5 4.0 CPs
- ZOS2 LP Weight 20
- Target CPs 0.2 5 1.0 CPs
30Adding Logical CPs
- ZOS1 LP is guaranteed 4 physical CPs
- ZOS1 can dispatch work to 5 logical CPs
- Each ZOS1 logical CP gets 4/5 or 0.8 CP
- ZOS1 effective speed 0.8 potential speed
- ZOS2 LP is guaranteed 1 physical CP
- ZOS2 can dispatch work to 4 logical CPs
- Each ZOS2 logical CP gets 1/4 or 0.25 CP
- ZOS2 effective speed 0.25 potential speed
ZOS2 Effective logical CP speed DECREASED!!
31Subtracting Logical CPs
400
100
-
ZOS1 Image
ZOS2 Image
- Total LP Weights 400 100 500
- ZOS1 LP Weight 100 400/500 80
- ZOS2 LP Weight 100 100/500 20
32Subtracting Logical CPs
400
100
ZOS1 Image
ZOS2 Image
- ZOS1 Weight 80
- Target CPs 0.8 5 4.0 CPs
- ZOS2 LP Weight 20
- Target CPs 0.2 5 1.0 CPs
33Subtracting Logical CPs
- ZOS1 LP is guaranteed 4 physical CPs
- ZOS1 can dispatch work to 5 logical CPs
- Each ZOS1 logical CP gets 4/5 or 0.8 CP
- ZOS1 effective speed 0.8 potential speed
- ZOS2 LP is guaranteed 1 physical CP
- ZOS2 can dispatch work to 2 logical CPs
- Each ZOS2 logical CP gets 1/2 or 0.5 CP
- ZOS2 effective speed 0.5 potential speed
ZOS2 Effective logical CP speed INCREASED!!
34Logical CPs - How Many?
- Both z/OS and PR/SM overhead minimized when LCP
count is equal to the physical CP requirements of
the executing workload - The number of LCPs online to an LP is correct
sometimes - When the LP is CPU constrained, too few
- When the LP is idling, too many
- When the LP is about 100 busy, just right!
- Ideally, effective LCP speed 1.0
CP Central Processor
LP Logical Partition
LCP Logical CP Logical Processor
35LP Configuration Decisions
- LP definitions entered on HMC
- Dedicated or not-dedicated (shared)
- Logical processors (initial, reserved)
- Weight (initial, min, max)
- Capped or not-capped
- CPC memory allocation
- I/O Channel distribution/configuration
- etc
HMC Hardware Management Console
CPC Central Processor Complex
36LP Hard Capping
- Initial weight enforced
- LLIC will not allow LP to use more than
guaranteed shared pool even when other LPs idle - Dynamic change to capping status
- Capped or not capped
- Capped weight value
- In general, not recommended
LLIC LPAR Licensed Internal Code
LP Logical Partition
37Intelligent Resource Director
- IRD brings four new functions to the parallel
SYSPLEX that help insure important workloads meet
their goals - WLM LPAR Weight Management
- WLM Vary CPU Management
- Dynamic Channel-Path Management
- Channel Subsystem I/O Priority Queueing
38IRD PR/SM WLM
- IRD WLM CPU management allows WLM to dynamically
change the weights and number of online logical
CPs of all z/OS shared LPs in a CPC LPAR cluster - IRD WLM Weight Management
- Allows WLM to instruct PR/SM to adjust shared LP
weight - IRD WLM Vary CPU Management
- Allows WLM to instruct PR/SM to adjust logical
CPs online to LPs
Logical CP Logical Processor
LP Logical Partition
39Pre-IRD LP Management
- Dedicate CPs to important LPs---wasteful
- Over share CPs---incurring LPAR overhead
- Capping
- Super Operator
- Dynamically change LP weights/caps
- Dynamically vary LCPs offline/online
LP Logical Partition
CP Central Processor
40IRD Prerequisites
- Running z/OS in 64-bit mode
- Running z/900 in LPAR mode
- Using shared (not dedicated) CPs
- No hard LP caps
- Running WLM goal mode
- LPs must select WLM Managed
- Access to SYSPLEX coupling facility
LP Logical Partition
41What is an LPAR Cluster?
An LPAR cluster is the set of all z/OS shared LPs
in the same z/OS parallel SYSPLEX on the same CPC
z900
z900
CPC Central Processor Complex
42What is an LPAR Cluster?
4 LPAR clusters in this configuration (color
coded)
z900
z900
dedicated
ZOSD
ZOSX
Linux
ZOSY
dedicated
z/VM
ZOSZ
ZOS1
ZOSA
ZOS2
shared
ZOSB
shared
ZOS3
ZOSC
ZOS4
ZOSD
ZOS5
ZOSE
SYSPLEX2
SYSPLEX1
43WLM LPAR Weight Management
- Dynamically changes LP Weights
- Donor Receiver Strategy
- WLM Evaluates all SYSPLEX Workloads
- Suffering Service Class Periods (SSCPs)
- High (gt1) SYSPLEX Performance Index (PI)
- High Importance
- CPU delays
LP Logical Partition
WLM Workload Manager
44WLM Policy Adjustment Cycle
- IF The SSCP is missing goal due to CPU Delay
- and WLM cannot help the SSCP by adjusting
- dispatch priorities within an LP
-
- THEN WLM and PR/SM start talking ---
- Estimate impact of increasing SSCPs LP weight
- Find donor LP if there will be SSCP PI
improvement, - Donor LP must contain heavy CPU using SCP
- Evaluate impact of reducing donor LPs weight
- - Cannot hurt donor SCPs with gt importance
- WLM changes weights via new LPAR interface
SSCP Suffering Service Class Periods
WLM Workload Manager
PI Performance Index
SCP Service Class Period
45Rules and Guidelines
- 5 from donor, 5 to receiver
- No recent LP cluster weight adjustments
- Must allow time for impact of recent adjustments.
Avoid see-saw effect - Receiver and Donor LPs will always obey specified
min/max weight assignments - Non-z/OS images unaffected because total shared
LP weight remains constant!
LP Logical Partition
46Goals Should Reflect Reality
- MAKE SURE YOUR SCP GOALS AND IMPORTANCE REFLECT
REALITY AT THE LPAR CLUSTER LEVEL! - Because WLM thinks you knew what you were doing!
SCP Service Class Period
WLM Workload Manager
47Goals / Reality Continued
- In the past, the WLM goal mode SCPs on your
test or development LPs had no impact on
production LPs - If part of the same LPAR cluster,
- IRD will take resource away (decrease weight) of
production LP, - Add resource (increase weight) of test LP to
meet the goal set for a SCP of higher importance
on test LP - Develop service policy as though all SCPs are
running on a single system image
SCP Service Class Period
WLM Workload Manager
LP Logical Partition
48Workload Manager Level of Importance
CICS WEBSITE DAVES STUFF
Importance 1
Importance 2
Importance 3
Importance 4
GUTTER WORK BOBS STUFF
Importance 5
- WLM uses the Level of Importance YOU assign to
make resource allocation decisions!
49WLM Vary CPU Management
- Varies logical CPs online/offline to LPs
- Goals
- Higher effective logical CP speed
- Less LPAR overhead and switching
- Characteristics
- Aggressive Vary logical CP online
- Conservative Vary logical CP offline
- Influenced by IRD LP weight adjustments
Logical CP Logical Processor
LP Logical Partition
50Vary CPU Algorithm Parameters
- Only initially online logical CPs eligible
- Operator varied offline not available
- If z/OS LP switched to compatibility mode,
- all IRD weight and vary logical CP adjustments
undone. - LP reverts to initial CP and weight settings
CP Central Processor
LP Logical Partition
Logical CP Logical Processor
51What is Online Time?
RMF interval 30 minutes
ZOS2
PRESENT (IRD)
PAST (pre-IRD)
---------- 30 MINUTES ----------
---------- 30 MINUTES ----------
LCP 0
LCP 0
LCP 1
LCP 1
LCP 2
LCP 2
Note LCP 2 varied offline during interval
Interval timeonline time
LPC Dispatch Time
In the past, RMF only indicated that LPC 2 was
not online at end of interval. Now, RMF reports
the online time for each LPC for each partition.
Previously, the length of the interval was the
MAX time that each LCP could be dispatched. RMF
reports on the actual dispatch time.
52Vary CPU - CPU Percent Busy
- Before IRD CPU Vary Management
- CPU Time Interval Time - Wait Time
- and CPU Busy CPU Time 100
- Interval Time No. Processors
- After IRD CPU Vary Management
- CPU Time Online Time - Wait Time
- and CPU Busy CPU Time 100
- Total Online Time
- New SMF70ONT field, total time processor online
to LP during RMF interval
53Dynamic Channel-PathManagement (DCM)
- Goals
- Better I/O response (less pend) time
- I/O configuration definition simplification
- Reduces the need for gt 256 channels
- Enhanced availability
- Reduced management
- Operates in both goal and compatibility modes
54DCM Prerequisites
- Systems running on z900 or later CPC
- Running z/OS 1.1 in 64 bit mode
- Both LPAR and Basic mode supported
- To share managed channels on same CPC
- Systems in LPAR cluster
- Balance mode either compatibility or goal
- Goal mode requires WLM goal mode and that global
I/O Priority queuing selected
CPC Central Processor Complex
DCM Dynamic Channel-path Management
55IRD DCM Balance Mode
Before IRD DCM
After IRD DCM
PCT Busy
PCT Busy
Channel Path
Channel Path
Stated simply, the goal of IRD DCM is to evenly
distribute I/O activity across all channel paths
attached to the CPC
CPC Central Processor Complex
DCM Dynamic Channel-path Management
56DCM Balance Mode
- Moves channel bandwidth where needed
- Simplified configuration definition
- For managed CUs, define 2 non-managed paths plus
n managed paths to meet peak workload - DCM balanced mode removes paths from non-busy CUs
and adds paths to busy CUs - Currently manages paths to DASD CUs
- New metric I/O Velocity
CU Control Unit
DCM Dynamic Channel-path Management
57I/O Velocity
- LCU
productive time - LCU productive
time channel contention delays - Calculated by WLM and the IOS
- Uses the same CF structure for CPU mgmt
- Calculated for every LCU in LPAR cluster to
compute a weighted average - DCM attempts to ensure all managed LCUs have
similar I/O velocities
I/O Velocity
DCM Dynamic Channel Path Management
LCU Logical Control Unit
CF Coupling Facility
58DCM Goal Mode
- All LPs in I/O cluster in WLM goal mode
- During policy adjustment routine WLM selects SSCP
- IF I/O delays the problem increasing I/O
priority does not help adding alias for PAV
volumes will not help I/O requests suffering
channel contention - THEN WLM estimates impact of increasing LCU I/O
velocity and if benefits SCP PI, sets explicit
velocity for LCU - Explicit velocity overrides Balance mode
SCP Service Class Period
PI Performance Index
LCU Logical Control Unit
SSCP Suffering Service Class Periods
59Channel Subsystem (CSS)I/O Priority Queueing
- Dynamically manages channel subsystem I/O
priority against WLM policy goals - Only meaningful when I/Os are queued
- Supports prioritization of
- I/Os waiting for a SAP
- I/Os waiting for a channel
- Previously, these delays were handled FIFO
SAP System Assist Processor
60Channel Subsystem I/O Priority Queueing (contd)
- Uses Donor Receiver strategy
- CSS I/O priority setting (waiting for a SAP)
- System SCP assigned highest priority
- I/O delayed SCP missing goal next
- When meeting goals
- Light I/O users higher
- Discretionary work has lowest priority
- UCB and CU I/O priority setting (waiting for a
channel) - System SCP assigned highest priority
- I/O delayed SCP missing goal next
- Less important SCP is donor
SAP System Assist Processor
SCP Service Class Period
61IBM License Manager (ILM)
- PROBLEMS
- Software charges based on CPC size
- CPCs getting bigger
- Workloads more erratic (spikes)
- eBusiness
- SOLUTION
- New LP setup option Defined Capacity
- Software priced at LP Defined Capacity
CPC Central Processor Complex
LP Logical Partition
62IBM License Manager (ILM)
- z/900 servers running z/OS V1R1
- New external Defined Capacity for shared LPs
- Defined capacity expressed in MSUs
- Millions of Service Units per hour
- Rolling 4 hour average MSU use compared with
defined capacity by WLM - When rolling 4 hour MSU usage exceeds defined
capacity, WLM tells PR/SM to soft cap LP
(really a temporary hard cap)
MSU Millions of Service Units
LP Logical Partition
63Rolling Four Hour Average
Actual MSUs Consumed
Average Actual
Avg Rolling
Rolling Avg Detail
64IBM License Manager (ILM)
- For software pricing, IBM uses following
- Dedicated LPs - logical CPs engine MSU
- PR/SM hard cap - shared pool engine MSU
- Defined Capacity - defined capacity
- Basic mode - models MSU rating
- www.ibm.com/servers/eserver/zseries/srm/
Logical CP Logical Processor
MSU Millions of Service Units
65IBM License Manager (ILM)
- If ZOS1 set with defined capacity of 100 MSU
175
White Space Ability to handle spikes and
unpredictable demand
100
Defined Capacity
66Capacity Upgrade on Demand
- CUoD supports non-disruptive activation of PUs as
CPs or ICFs in a CPC - Specify Reserved Processors
- When an upgrade is made (105 to 106)
- LPs with reserved processors can begin using
immediately after operator command - IBM recommends specifying as many reserved
processors as model supports
CPC Central Processor Complex
LP Logical Partition
67z900 Model 2064-105
5 PUs Configured as CPs Model 105
CPs Defined Model Number
68Shared LPs
HMC Image Profile
ZOS1
69z900 Model 2064-106
All contain a 12 PU MultiChip Module (MCM)
5 1 PUs Configured as CPs Model 106
CPs Defined Model Number
70LP Weights - LP Weight
PR/SM LPAR LIC
CPC MEMORY
ZOS1
ZOS2
400
100
ZOS1 Image
ZOS2 Image
- Total of LP Weights 400 100 500
- ZOS1 LP Weight 100 400/500 80
- ZOS2 LP Weight 100 100/500 20
71LP Target CPs
PR/SM LPAR LIC
CPC MEMORY
400
100
ZOS1 Image
ZOS2 Image
- ZOS1 Weight 80
- Target CPs 0.8 6 4.8 CPs
- ZOS2 LP Weight 20
- Target CPs 0.2 6 1.2 CPs
72LP Logical CP share
- ZOS1 LP is guaranteed 4.8 physical CPs
- ZOS1 can dispatch work to 5 logical CPs
- Each ZOS1 logical CP gets 4.8/5 or 0.96 CP
- ZOS1 effective speed 0.96 potential speed
- ZOS2 LP is guaranteed 1.2 physical CP
- ZOS2 can dispatch work to 3 logical CPs
- Each ZOS2 logical CP gets 1.2/3 or 0.40 CP
- ZOS2 effective speed 0.40 potential speed
Logical CP Logical Processor
CP Central Processor
73Conclusions
- Large system resource, configuration, and
workload management tasks shifting towards
intelligent, automated, dynamic WLM functions - Responsibility of capacity planners and
performance analysts shifting towards better
understanding of business workloads relative
importance and performance requirements
74NeuMICS Support
- April 2002 PSP
- CAP - New IRD and ILM Planning Applications
- PER - New MSU and Soft Capping Analysis
- October 2001 PSP
- RMF6580 - IRD, ILM, and CUoD
- April 2001 PSP
- RMF6560 - z/OS (64 bit) Multisystem Enclaves, USS
Kernal, and 2105 Cache Controllers - October 2000 PSP
- RMF6540 - ICF, IFL, and PAV support
75References
- Parallel Sysplex Overview Introducing Data
Sharing and Parallelism in a Sysplex
(SA22-7661-00) - Redbook z/OS Intelligent Resource Director
(SG24-5952-00) - IBM e-server zSeries 900 and z/OS Reference Guide
(G326-3092-00) - zSeries 900 Processor Resource/Systems Manger
Planning Guide (SB10-7033-00)
76Trademarks
The following terms are trademarks of the
International Business Machines Corporation in
the United States, or other countries, or both
- PR/SM
- RMF
- SMF
- Sysplex Timer
- S/390
- VSE/ESA
- zSeries
- z/OS
- CICSDB2
- ESCON
- FICON
- IBM
- IMS
- MVS
- MVS/ESA
- OS/390
- Parallel Sysplex
- Processor Resource/Systems Manager
77Thanks!
- Questions ???
- Darrell Faulkner
- NeuMICS Development Manager
- Computer Associates
- Email darrell.faulkner_at_ca.com