Ok, finally I can report to the group thet I've been able to
successfully sort SMF records.
First however, I'd like to thank you all for your input. Each response
has given me some insight that collectivly have helped me to accomplish
my goal.
Here are my observaions:
1. I tried to use IFASMFDP to check for invalid records. That
resulted in complete failure, since the program installed on the TK
systems must be very old and has only one purpose (apparently), and that
is to dump SMF records from a MAN(x) dateset to a sequential dataset
(DUMPOUT) and then clear out the input dataset (DUMPIN). It does not
accept any SYSIN parameter statements to alter this behavior, and so I
lost some data, because I didn't realize that the input was ALWAYS
cleared out. (No big deal since the whole thing is experimental anyway.)
2. I looked for an alternative on the CBT tape. I found a newer version
called SMFDUMP, howevr I wasn't able to assemble it, because it is
designed for OS/390 and up, requiring HASM, etc. I could have tried to
adapt it for my purpose, but I decided not to try. 3. I looked for a
FIXVBS program on the CBT tape, but couldn't find that name. Instead I
found COPYVBS, which seemed to perform some chchecks on VBS type
recordes, (eg. check if BDW and RDW are acaccuratetly describing the
record/block on the track.)
This was interresting, because at first it seemed to work fine. (RC =
00, no ABEND). Then I checked the EXCP stats and found only 1 EXCP had
been used to read a dataset of about 25 tracks. That seemed suspicious,
and so I printed the output dataset using IDCAMS PRINT. Only som 28
records had been copied, again without any error message or other
problem indication. I then used IDCAMS REPRO to copy the input dataset.
That worked and reported some 208 records.
I copied some 40 SMF archive datasets with IDCAMS REPRO. Some were bad
or empty (because I damaged them by using IFASMFDP on them.) Of the good
ones that were left I created one single archive by concatenating all of
them and feeding them to IDCAMS REPRO. The result was about 5,108
records.
I still couldn't figure out why COPYVBS woul ony read 1 track (1 EXCP),
so I did some cosmetic changes to the program. First I added register
equates by using a macro I found in one of Sam Golob's programs
(SAVEALL). I used this because it also allows for proper save area
chaining, program entry. (and exit using ENDALL). Then I changed all
the register uses to use the equated register symbolic names. (Now I
could figure out quickly which registers are used in the program.)
Next I added a record counter using R5, and code to format that counter
at the end of the program and print a message. I decided to count each
record written to the output file.
When running this program on my 5,108 record archive it only copied
about 28 records. So now had to figure out why it wouldn't go past
that first track. It took a while but I found that R10 was used as base
for the input buffer, a well as some other area (which didn't change
after the second EXCP, and therefore the program recognized EOF. After
making the base for that area R12, the program happily read and wrote
all 5,108 records with no further problems.
4. Now I tried the SORT again. I found that the "CORE=" parameter
yields an error message "IER067I - INVALID EXEC OR ATTACH PARAMETER".
The DIAG parameter causes a lot of basically useless messages to be
written to the log, and causes the S0C1 abend if any errors are found.
If the DIAG parameter is not present the program ends with RC=16.
The last message "IER053A - OUT OF SEQ" is the culprit for the RC=16.
By changing the SORT statement to properly reflect the SMF record fields
of DATE and TIME, I was able to overcome this issue.
SORT FIELDS=(19,16,BI,A,11,4,BI,A,7,4,BI,A)
END
Was my original statement, copied from SYS1.SAMPLIB. It's wrong!
This is the one I'm using now:
SORT FIELDS=(07,04,PD,A,03,4,BI,A)
END
First field is the record date packed decimal, second is the record time
binary.
This works now, but before the whole sort was successful there were stil
some issues. 5. The SORT program had definite troubles reading
records greater than 7K. So I reblocked the archive to 7K on a 2314
volume.
The sort program had no trouble writing large records, even larger the
32760. So I moved my sortout to a 3380, because using a 3350 volume
caused some S002-0c, or S002-18 abends. 6. Finally the sort ended
reporting all 5,108 records had been read, and written!
Here is the message log (Ok, by now there are more records i te
archive)!
001 COPYVBS STARTED COPYVBS_09/05/10_17.06
COPYVBS I/O: WRITTEN: 6,309 RECORDS
998 COPYVBS ENDED
IER036I - B = 7296
IER037I - G = 52
IER038I - NMAX = 790
IER050I - END MERGE PH
IER054I - RCD IN 6309,OUT 6309
IER052I - EOJ
I verified the sorting sequence using IDCAMS PRINT, and the program
SMFFRMT I found in SYS1.SAMPLIB.
Here is my final SORT JCL: [:D]
//SMFSORT0 JOB 'ROGER W. SUHR',CLASS=A,MSGCLASS=A,
// NOTIFY=RWS01
/*JOBPARM LINES=9999
//*-------------------------------------------------------------------
//*
//S005 EXEC IEFBR14,DS1='SMF.SMFREBLK',
// DS2='SMF.SMFSORT'
//*
//S010 EXEC COPYVBS
//SYSUT2 DD DISP=(,CATLG,DELETE),DSN=SMF.SMFREBLK,
// DCB=(RECFM=VBS,LRECL=6996,BLKSIZE=7000),
// UNIT=2314,SPACE=(CYL,(50,50),RLSE),
// VOL=SER=SORT01
//*
//S020 EXEC SORT,COND=(0,NE),RGN=6144,
// PARM='MSG=AP'
//SYSIN DD *
SORT FIELDS=(07,04,PD,A,03,4,BI,A)
END
//SORTIN DD DISP=SHR,DSN=SMF.SMFREBLK
//SORTOUT DD DISP=(,CATLG,DELETE),DSN=SMF.SMFSORT,
// UNIT=SYSDA,VOL=SER=WORK03,
// SPACE=(CYL,(50,50),RLSE),
// DCB=(LRECL=32756,BLKSIZE=32760,RECFM=VB)
//*
//SORDATA DD UNIT=2314,SPACE=(CYL,(1,1)),
// DCB=(RECFM=VBS,LRECL=600,BLKSIZE=200)
//HDRDATA DD UNIT=2314,SPACE=(TRK,(5,5)),
// DCB=(RECFM=VBS,LRECL=600,BLKSIZE=200)
//*
//S990 EXEC IEFBR14,COND=(0,NE),DS1='SMF.SMFREBLK'
Procs used:
//SORT PROC CLS='*',RGN=4096
//*
//SORT EXEC PGM=SORT,REGION=&RGN.K,
// PARM='MSG=AP'
//SYSOUT DD SYSOUT=&CLS.
//SYSUDUMP DD SYSOUT=&CLS.
//SYSPRINT DD SYSOUT=&CLS.
//SORTLIB DD DSNAME=SYS1.SORTLIB,DISP=SHR
//SORTWK01 DD UNIT=2314,SPACE=(CYL,(5,5))
//SORTWK02 DD UNIT=2314,SPACE=(CYL,(5,5))
//SORTWK03 DD UNIT=2314,SPACE=(CYL,(5,5))
//SORTWK04 DD UNIT=2314,SPACE=(CYL,(5,5))
//SORTWK05 DD UNIT=2314,SPACE=(CYL,(5,5))
//SORTWK06 DD UNIT=2314,SPACE=(CYL,(5,5))
//SORTWK07 DD UNIT=2314,SPACE=(CYL,(5,5))
//SORTWK08 DD UNIT=2314,SPACE=(CYL,(5,5))
//IEFBR14 PROC DS1=,
// DS2=,
// DS3=,
// DS4=,
// DS5=
//*
//DEL EXEC PGM=IEFBR14
//DD1 DD DSN=&DS1.,
// DISP=(MOD,DELETE,DELETE),UNIT=SYSDA,SPACE=(TRK,0)
//DD2 DD DSN=&DS2.,
// DISP=(MOD,DELETE,DELETE),UNIT=SYSDA,SPACE=(TRK,0)
//DD3 DD DSN=&DS3.,
// DISP=(MOD,DELETE,DELETE),UNIT=SYSDA,SPACE=(TRK,0)
//DD4 DD DSN=&DS4.,
// DISP=(MOD,DELETE,DELETE),UNIT=SYSDA,SPACE=(TRK,0)
//DD5 DD DSN=&DS5.,
// DISP=(MOD,DELETE,DELETE),UNIT=SYSDA,SPACE=(TRK,0)
//*
//COPYVBS PROC G='0'
//*
//COPYVBS EXEC PGM=COPYVBS
//*
//STEPLIB DD DISP=SHR,DSN=RWS01.LINKLIB
//SYSPRINT DD SYSOUT=*
//SYSUDUMP DD SYSOUT=*
//SNAP DD SYSOUT=*
//SYSUT1 DD DISP=SHR,DSN=SMF.DAILY.DATA(&G.)
//SYSUT2 DD DUMMY,DCB=(RECFM=VBS,LRECL=32756,BLKSIZE=32760)
Thanks again for all your input, it is very much appreciated!
Roger W. Suhr