Paul Jaffray

1
Session 5823 Working with Windows NT Performance
Data

• Paul Jaffray
• Technical Services Manager
• Promedica Health System, Inc.
• Wednesday March 6th, 2002

2
Abstract

• Real world look at working with NT performance
  data
• Application of MVS techniques to NT
• High level overview of NT performance data
• Tools used for collection and analysis
• What data do we look at
• Discussion What does everyone else do?

3
Intro

• I am not an expert on NT or Performance
• I took my experience from managing SMF data on
  MVS and creating various performance reports for
  the mainframe and applied those processes to NT
• I am still learning how NT works and what the
  various counters represent
• Still developing ROTs for various counters

4
What Data Is Available?

• Various Performance Objects
• Each object contains related Performance Counters
• Processor Processor Time
• ProcessorInterrupts/Sec
• ProcessorDPC Rate
• Lots and lots of counters
• But how good is the data?

5
Performance Objects

• Processor
• System
• Memory
• Network Interface
• Physical Disk
• Process
• Print Queue
• Server
• Thread

• MSExchangeDB
• MSExchangeDS
• MSExchangeMTA
• Lotus.Notes.MTA
• SQLServer
• SQLServerDatabases
• AppleTalk
• ADSM Client Performance
• ColdFusion Server

6
Whats Missing

• No response time counters
• Exchange
• Delivery times are available
• SQL
• Transactions per Second

7
How Can I View the Data?

• Task Manager
• Real Time
• Local machine only
• Provides summary performance data
• Not customizable
• How to start
• Right-click the task bar and select Task Manager
• Crtl-Alt-Del and select Task Manager

8
How Can I View the data?

• Performance Monitor
• Real time or historical via logs
• Also has alerting facility
• Local machine or remote
• You can select which objects and counters to view
• How to start
• StartRunPERFMON
• StartProgramsAdministrative ToolsPerformance

9
So Now What?
I wanted a way to consolidate performance data
from every server to a single point and combine
into a daily file for processing. This file
should then be saved where it can be combined
into a monthly file. A number of daily files and
all monthly files should be archived so they can
be retrieved and reported against in the future.
10
Challenges

• No SMF facility standard with Windows
• No IFASMFDP or DFSORT
• No JCL/TSO/ISPF
• How to automate
• How to locate and copy the data
• How to handle and report errors
• How to analyze data and create reports

11
Solution (so far)

• NTSMF
• PrimalScript
• VBScript code
• Scheduled Tasks
• Text file with a list of servers with data
  location
• SAS
• MXG

12
Creating the SMF Files

• On each server NTSMF gets data from standard NT
  interfaces, builds records and writes them to a
  current file
• NTSMF Directory Structure
• C\NTSMF\Data\Current
• Holds active file
• C\NTSMF\Data\Previous
• Yesterdays data file(s)
• C\NTSMF\Data\Archive
• Settable number of days worth of data

13
Gathering The Data

• A script, driven by a list of servers to collect
  from, looks for files on each of those servers
  where the date in the file name is equal to
  yesterdays date and copies those files to a
  directory on the system the script is running on
• All files are combined together in a single daily
  file that is available locally for SAS/MXG
  processing and is copied to a server for archival
  and later consolidation

14
Error Logging

• Status messages are written to the NT Application
  Log
• Start and Stop times for the script
• Servers not found
• Data not found
• Messages are also gathered together and emailed
  to me as the last step in the script

15
Reporting On the Data

• Processing can be done with SAS on MVS or Windows
• MXG provides SAS Macros to process the raw NTSMF
  data into SAS observations
• SAS can then be used to summarize and report on
  the data

16
Some Gotchas

• Cross domain trusts
• Must be in place to copy files from servers in
  other domains
• Permissions
• File access
• Registry updates

17
Some Gotchas

• A crashed server can dump several K worth of
  nulls onto the end of a .smf file
• These nulls cause SAS to stop processing the file
• If they exist in the daily file they have to
  found and removed

18
Some Gotchas

• Disk performance numbers must be turned on by
  issuing the diskperf command from the command
  prompt
• C\gt diskperf yv
• Some network numbers require Network Monitoring
  Agent to be installed

19
Some Gotchas

• MemoryPage Faults/Sec
• Can be very misleading as it includes Transition
  faults
• Should use MemoryPage Reads/Sec
• Accurate reflection of demand paging

20
Some Gotchas

• Microsofts Outlook security patch in response to
  the ILOVEYOU virus broke the email part of my
  initial script
• New version of script requires the SMTP service
  to be loaded on the machine that runs the script

21
Some Gotchas

• Manually have to maintain the Server List file
• Can be coordinated with the setting of the NTSMF
  options that must be set once the collector
  software has been loaded

22
What Numbers Do We Look At?

• Daily 3 reports
• Configuration report
• Shows all servers we collect data from and their
  configuration
• Disk Numbers
• Free space shortages and high queuing
• System Hourly Summaries
• Hourly report for each server of key values

23
What Numbers Do We Look At?

• Processor Processor Time
• Looking for busy processors
• SQL based applications seem to suffer at 60
• Others?
• SystemProcessor Queue Length
• Indicates work waiting for CPU
• Doesnt seem to work like I would expect

24
What Numbers Do We Look At?

• MemoryAvailable Bytes
• Shows physical memory installed but not used by
  operating system or application
• Large number indicates an over-configured server
• MemoryPage Reads/sec
• Actual page faults resulting in a page in from
  disk
• Page Faults/sec counter is misleading
• Should be less than 200/sec?

25
What Numbers Do We Look At?

• PhysicalDiskAvg. Disk Queue Length
• How backed up is the IO subsystem
• Shouldnt be greater than 3 or 4?
• PhysicalDiskDisk Bytes/sec
• How much data is being moved
• How much is too much depends on the interface

26
What Numbers Do We Look At?

• PhysicalDiskAvg. Disk Sec/Read
• Average number of seconds per read
• Less than 20ms good, more bad
• PhysicalDiskAvg. Disk Sec/Write
• Same as above except for writes

27
Discussion

• Comments
• Questions
• What are others doing?

28
Tools NTSMF

• Windows NT/2000 performance data collector
• Creates SMF files containing records of
  selected performance counters
• Demand Technology Software
• www.demandtech.com

29
Tools PrimalScript

• The professional scripting environment of choice
  for developers working in any scripting language
• Useful for working with VBScript
• SAPIEN Technologies Inc.
• www.sapien.com

30
Tools SAS

• Data Access/Management/Analysis/Presentation Tool
• Allows processing and summerization of raw data
• SAS Institute Inc.
• www.sas.com

31
Tools MXG

• SAS-based software package that processes the
  "SMF" data records created by computer operating
  systems
• Turns raw data into observations in SAS datasets
• Merrill Consultants
• www.mxg.com

32
Some Books

• Microsoft Windows 2000 Performance Tuning
  Technical Reference
• ISBN 0-7356-0633-1
• Windows 2000 Performance Guide
• ISBN 1565924665
• Tuning Netfinity Servers for Performance
• IBM Redbook / ISBN 0-13-040612-0