SCANALL Replacement

1
SCANALL Replacement

• By DeFrance Clarke
• 360/902-4942 CLAG235_at_LNI.wa.gov
• to Pacific Northwest User's Group
• November 16, 2006

2
Overview

• Problem SCANALL takes too long
• Proposed Solution
• Extract
• Format
• Load
• Using the data
• Results
• Discussion and Your Questions

3
Problem Statement

• Batch SCANALL takes 30 seconds of CPU and at
  least 35 MIPS.
• On a loaded system multiple programmers can add
  significant load.
• In addition, when the system is loaded just
  paging through Natural code is tedious at best.

4
JCL for Batch SCANALL
//
// SCAN AN ENTIRE
NATURAL LIBRARY FOR A FIELD.
// KEEP THE FIELDNAME TO 20 CHARACTERS OR
LESS. //
//
SCANALL EXEC NATPRF,
// DBID200,
//
TIME5
//DDCARD DD
DSNMML235.T.CNTLLIB(DDCARD1H),DISPSHR
// INPUT SEQUENCE SRCH1,SRCH2,LIBR,PG
M1,PGM2,SW (SWY NO DETAIL) // NOTE
V, lt LAST CHAR OF SRCH1
//NATURAL.CMOBJIN DD
CPAYPPR,,XREF235,,,N

//NATURAL.SYSIN DD
GLOBALS MT300

SCANALL
FIN
//
5
Typical Run Unloaded System

• -
  --TIMINGS (MINS.)--
• -JOBNAME STEPNAME PROCSTEP RC EXCP CONN
  TCB SRB CLOCK
• -TCLAG235 SCANALL NATURAL 00 676
  436 .49 .00 2.9
• IEF404I TCLAG235 - ENDED - TIME05.41.13
• .49 min CPU
• ---------------------- x 208 MIPS/CPU 35 MIPS
  Adabas MIPS
• 2.9 min Elapsed

6
Proposed Solution

• Extract Natural Source from Production Library
• FTP the extracted source to Windows server
• Load the Natural Objects as text files on a
  shared drive
• Make the shared drive accessible to programmers
  readonly
• Modify the Windows client to enable searching
  inside the Natural code
• Schedule the Extract/Load cycle weekly

7
http//sww.wa.gov/dis/vendorinfo/
8
Extract

• SYSTRANS Utility - Overview
• The information provided in this section deals
  with all environments supported by Natural.
• Note that the utility SYSTRANS will no longer be
  available with the next major release of Natural.
  Its functionality is covered by the Natural
  Object Handler.
•   Utilities  
• Object Handler - Overview
• The Natural Object Handler is designed to process
  Natural and non-Natural objects for distribution
  in Natural environments. This is done by
  unloading the objects in the source environment
  into work files and loading them from work files
  into the target environment.

9
Extraction JCL - 1
//DELWORK EXEC PGMIDCAMS
//SYSPRINT DD SYSOUT(,)
//SYSIN DD

DELETE OCP235.P.SYSTRAN1.TXT
IF MAXCC EQ 8 THEN SET MAXCC
0 /

//
// UNLOAD USING SYSOBJH
UTILITY
//
//UNLOAD EXEC NATPRF,
//
DBID200,
// PARMS'IMD,MT1780'
//DDCARD DD
DSNDBUD155.ADAPROD.DDCARD(MULTI),DISPSHR
//
DD DSNDBUD155.ADAPROD.DDCARD(DB200),DISPSHR
//
DD DSNDBUD155.ADAPROD.DDCARD(PREFETCH),DISPSHR
//
DD
ADARUN PREFSBL32000,PREFTBL32000,PREFN
REC20 /
10
Extraction JCL - 2

• //CMPRINT DD SYSOUT
  
• //CMPRT10 DD SYSOUT
  
• //CMWKF01 DD DSNOCP235.P.SYSTRAN1.TXT,
• // DISP(NEW,CATLG,CATLG),
  
• // DCB(LRECL100,RECFMVB,BLKSIZE0),
  
• // SPACE(CYL,(200,50),RLSE),
  
• // DATACLASSEQ,
  
• // STORCLASNORMAL,
  
• // MGMTCLASD3DB7A0D
  
• //SYSIN DD
  
• SYSOBJH

11
Extraction JCL - 3

• //SYSIN DD
  
• SYSOBJH
  
• UNLOAD LIB XREF235, OBJT N, FMDATE TO -21, SC
  B,
• WHERE TRANS INCL
  
• UNLOAD LIB XREF235, OBJT D, FMDATE TO -21,
  
• EXCEPT ( TODATE 1998-12 )
  
• WHERE TRANS INCL
  
• .
  
• FIN
  
• STASH OLD CARDS DOWN HERE
  
• SYSTRANS
  
• U
  
• N,N,N,Y,Y,N,N,N
  
• L
  
• XREF235,Y,,,Y,,Y,
  
• .
  
• FIN
  
• /
  

12
Extraction JCL - 4
//
// SEND THE LISTING FILE TO LNI FTP
SERVER //FTPPUT EXEC FTP
//SYSIN DD
LNIFTP.LNI.WA.GOV
(EXIT ANONYMOUS

W02CP235 PRODUCTION
CD CLAG235
PUT 'OCP235.P.SYSTRAN1.TXT'
SYSTRANS.TXT QUIT
/ //
13
SYSTRANS Data Format
HENAT4104200611040632085MVS/ESA
4AEMB C
XREF235 ADDPDE03
F S Y D01NAT4104F XREF235 ADDPDE03
RAB2235 RAB2235 NATURAL S D02
2006101916455792006101916455790000006275
D03MVS/ESA X0528235 D04
S0010 JOB X0614235, SUBMITTED BY
PROCEDURE ADDP S0020 ACCOUNT
DOWNLOAD PROCESS - DELETE DATA BASED ON PCY
DRIVER FILE S0030 BOYD BETTS,
OCTOBER 1997 S0040 S0050 THIS
PROGRAM WILL DELETE DATA FROM THOSE FILES WHO
HAVE AN C
SYSTEM SI-RCPT-HIST V S
D01NAT414 V SYSTEM SI-RCPT-HIST
GALE235 GALE235 D02
2006101909100302006101909100300000000831 D03
D040 285 A S
Generation started S at
2006-10-19 091003 S by user GALE235
S S 1A1SI-RCPT-ID
A0100N SHDSI RCPT/ID S
1A2SI-RCPT-STAT-CODE A0010N
14
Loading the Natural Objects

• Objectives
• Read the FTPed file with all objects extracted
• Create a separate file for each object named
• ltobject-namegt.NAT for Natural Source
• ltobject-namegt.VU for DDM view
• Add a header line with header information
• Name, Library, Date, User who saved, Version
• Change the Modify date/time to date Saved

15
Loading the Natural Objects - 2

• Perl script
• Available on request
• Perl is available from
• www.perl.com
• http//open-perl-ide.sourceforge.net

16
Using the Data

• Registry Changes are needed
• Recognize .NAT and .VU files as text
• Enable searching inside
• Accomplished with small .EXE file (119KB)
• Windows Search is primary access
• By name
• By contents (case insensative)
• By date

17
Results

• Search of 13,000 Natural Programs
• C drive - 212 minutes, i.e. 132 seconds
• S drive - 500 minutes
• Copy of all .NAT and .VU files from S to C -
  527 minutes
• Another search of S right after the copy from S
  to C and it ran in 152 minutes
• So caching on the S drive is significant.