Discussion:
Sorting SMF records
Small
14 years ago
Permalink
Has anyone attempted to sort SMF records using the MVT sort utility?
I have some trouble doing this, so I wonder if this old sort program is too limited to handle the large records.

Any advice would be appreciated.

Thanks
Roger
Kevin Leonard
14 years ago
Permalink
Post by Small
Has anyone attempted to sort SMF records using the MVT sort
utility? I have some trouble doing this, so I wonder if this
old sort program is too limited to handle the large records.
Sort block size is limited by the work file block size. If you
haven't already, try using tape work files.

--
cedarlakebuzzards
14 years ago
Permalink
Keep in mind that the sort work files are 2314 packs, which are low
capacity when compared to moderm pack. (max blksize is about 7k, max
cylinders is 200). You may want to try allocating as many sortwkxx as
you can. Tape sort work is also an option.

I ending creating more sort work volumes for a total of 8. I believe
Turnkey is gen'd with a maximum of 32 2314 drives.
Post by Small
Has anyone attempted to sort SMF records using the MVT sort utility?
I have some trouble doing this, so I wonder if this old sort program
is too limited to handle the large records.
Post by Small
Any advice would be appreciated.
Thanks
Roger
Small
14 years ago
Permalink
Ok, I understand that. Here is what's happening:

1. I reblock my SMF data using IEBGENER to a 2314 volume with a BLKSIZE of 7004 (LRECL=7000,RECFM=VBS). At this point I don't have to worry about capacity, since I only have about 800 TRKS worth of data.
2. I run SORT with 8 SORTWK daasets and PARM='MSG=AP,BALN,DIAG'
(I also tried CRCX, but the result was the same.)
3. Sort runs for a little while and seems to read all the data, but then ABENDS with S0C1. It appears that someone intentionally branched to an area initialized with x'00000000', hence the S0C1.
4. The last messages from the DIAG output are:
IER940I - GENERATED CORE END ADDR - 000B4D38
IER941I - INPUT BFR TBL ADDR - 000B4C70
IER988I - IERAOU LOC. AT 0B7F20
IER988I - IERAGK LOC. AT 0B7930
IER944I - DCB TBL ADDR - 000B48AC
IER988I - IERABQ LOC. AT 0B7EB8
IER988I - IER9BN LOC. AT 0B6068
IER942I - OUTPUT BFR ADDR - 000B84A0 , 000CA4A0
IER988I - IERAGH LOC. AT 0B7E80
IER988I - IER9GC LOC. AT 0B7C78
IER945I - I/P CCW ADDR - 000B4758
IER988I - IERADI LOC. AT 0B6010
IER053A - OUT OF SEQ

And that's how far I've gotten.

I know I've done this way back in 1980 or so, but we were using SYNCSORT at the time already.

This is not a critical isue, but since I'm toiling around with this I'd like to see if I can get this to work, for curiosity!

Thanks for all your help

Roger

BTW: Is it possible to attach files to these messages in theis forum?
...
halfmeg
14 years ago
Permalink
Post by Small
1. I reblock my SMF data using IEBGENER to a 2314 volume with a
BLKSIZE of 7004 (LRECL=7000,RECFM=VBS). At this point I don't have
to worry about capacity, since I only have about 800 TRKS worth of
data.
SORT is suppose to handle variable length spanned records. The SORT manual notes them as VRE instead of VBS however. Don't know if unblocked input would make a difference.
Post by Small
2. I run SORT with 8 SORTWK daasets and PARM='MSG=AP,BALN,DIAG'
(I also tried CRCX, but the result was the same.)
3. Sort runs for a little while and seems to read all the data, but
then ABENDS with S0C1. It appears that someone intentionally
branched to an area initialized with x'00000000', hence the S0C1.
There were some items in SORT which may have changed since original TK3 was distributed. Memory size for SORT to use was perhaps one of them.
Post by Small
<snip>
Believe memory size for SORT use is in the DIAG messages as well. Is it too small? For 7K size records it specified 100K minimum IIRC.
Post by Small
<snip>
BTW: Is it possible to attach files to these messages in theis forum?
Don't believe the forum supports attachments currently.

However you can get assistance with your exact SORT JCL and DATA files if you zip them up and upload them to the files section for others to work with.

Or, post a message with SORT JCL and JCL you use to prepare your SMF data so that folks may use their own data for input.

Some of the other problems you are having, IDCAMS on MVSCAT for instance may be due to bugs in the DASD image on original TK3 image ( 3375, 3380 & 3390 had various glitches ). 3270 error may be from attempting to attach other than 3270-2 ( ie 3270-2E and 3270-3,4,5? ) may give the error display initially. This might not be it exactly as my memory on the issues are not in the forefront of my mind currently.

Phil
yvette hirth
14 years ago
Permalink
Post by Small
1. I reblock my SMF data using IEBGENER to a 2314 volume with a
BLKSIZE of 7004 (LRECL=7000,RECFM=VBS). At this point I don't have
to worry about capacity, since I only have about 800 TRKS worth of
data.
can you move this to a 3380/3390 volume, and can you "unspan" your data?

a larger "track size" really means little wrt Hercules, as all the
"disks" are files. but using an 80 or a 90 would give you the ability
to use larger blocksizes.

to me, the "spanned" DCB attribute might be causing the issue.
traditionally, spanned attributes arose when either the record could be
bigger than the LRECL, or in some cases to attempt to maximize space
utilization. if spanned is not needed and can be removed, that will
simplify the i/o logic and reduce the potential number of issues.

see here:

http://publib.boulder.ibm.com/infocenter/zos/v1r9/index.jsp?topic=/com.ibm.zos.r9.cbcpx01/recfmv.htm

i remember VS/VBS being more frequently used during 3330 and 3350 days.
once the larger 3380 track size became popular, there didn't seem to
be any great need for VBS.

if you can (i.e., if the "S" spanned attribute is optional), try setting
sortout to LRECL=7000,RECFM=VB (or even RECFM=V) and see what happens.
or set LRECL=<as big as possible based on track size>.

yes, more i/o will occur, and your job will probably run slower, but
it's a good debugging technique for the problem you're reporting.

yvette hirth
Rick Fochtman
14 years ago
Permalink
-----------------------------------------<snip>-------------------------------------
Post by yvette hirth
Post by Small
1. I reblock my SMF data using IEBGENER to a 2314 volume with a
BLKSIZE of 7004 (LRECL=7000,RECFM=VBS). At this point I don't have
to worry about capacity, since I only have about 800 TRKS worth of
data.
can you move this to a 3380/3390 volume, and can you "unspan" your data?
a larger "track size" really means little wrt Hercules, as all the
"disks" are files. but using an 80 or a 90 would give you the ability
to use larger blocksizes.
---------------------------------<unsnip>-------------------------------
OS/360 didn't support any DASD device beyond the 3330-11. 3380/3390 are
out of the question because of that limitation.

-----------------------------------------<snip>--------------------------------
...
-------------------------------------<unsnip>-----------------------------------
It would help immensely if the SORT JCL and control statements could be
posted. OS/360 SORT/MERGE handled VBS records very nicely but the
control statements had to be perfect to do so. It may also be that the
data contains an incomplete spanned record, not at all unusual.

Also, if you could find the FIXVBS program on the CBTTAPE.org web site,
that might be useful in this situation. FIXVBS will delete any "partial"
logical records that might be causing the problem. It should work just
fine on OS/360, since that's where it was originally developed.

Rick
<mailto:yvette-RPrnXxQTqyxWk0Htik3J/***@public.gmane.org?subject=Re:%20%5Bturnkey-mvs%5D%20Re:%20Sorting%20SMF%20records>
<mailto:turnkey-mvs-***@public.gmane.org?subject=Re:%20%5Bturnkey-mvs%5D%20Re:%20Sorting%20SMF%20records>
dennis_stone2001
14 years ago
Permalink
...
I have used the following JCL to sort dumped SMF data on MVS 3.8 using the OS/360 sort utility, no special mods or anything. My sort work devices are 2314's. Hope this helps!

//SORTSMF JOB ACCT,DSTONE,CLASS=1,MSGCLASS=T,NOTIFY=DSTONE
//*
//SORT EXEC PGM=SORT,REGION=4096K PARM='MSG=AP'
//SORTLIB DD DISP=SHR,DSN=SYS1.SORTLIB
//SYSOUT DD SYSOUT=*
//SYSIN DD *
SORT FIELDS=(6,1,BI,A)
END
//SORTIN DD DISP=OLD,DSN=DSTONE.SMFDUMP
//SORTOUT DD UNIT=SYSTS,SPACE=(19069,(5000,500),RLSE),
// DISP=(,DELETE),
// DCB=(LRECL=32756,BLKSIZE=19069,RECFM=VBS)
//SORTWK01 DD UNIT=SORTWK,SPACE=(CYL,(199))
//SORTWK02 DD UNIT=SORTWK,SPACE=(CYL,(199))
//SORTWK03 DD UNIT=SORTWK,SPACE=(CYL,(199))

Dennis Stone
halfmeg
14 years ago
Permalink
Post by Small
<snip>
I have used the following JCL to sort dumped SMF data on MVS 3.8
using the OS/360 sort utility, no special mods or anything. My sort
work devices are 2314's. Hope this helps!
<snip>
//SORT EXEC PGM=SORT,REGION=4096K PARM='MSG=AP'
<snip>
The core size used by OS/360 Sort, included with TK3, was specified when the SORT utility was generated. Between TK2 and TK3 the size was upped to 256K. DIAG can be added to the PARM to determine what amount of core you are utilizing for the sort. Like so:

//SORT EXEC PGM=SORT,REGION=4096K PARM='MSG=AP,DIAG'

Even though REGION is 4096K, SORT output tells us how much it uses in the DIAG section.

IER963I - MAX. SYSGEN CORE - 262144
IER964I - CALC. CORE PH 1 = 261990
IER964I - CALC. CORE PH 2 = 259744
IER964I - CALC. CORE PH 3 = 244089

Even without adding DIAG to the PARM the JES2 SYSOUT indicates just over 256K was used: See VIRT size below of 272K.

IEF374I STEP /SORT / STOP 10247.1218 CPU 0MIN 00.21SEC SRB 0MIN 00.03SEC VIRT 272K SYS 200K

If instead we override the default SORT core size, like so:

//SORT EXEC PGM=SORT,REGION=4096K PARM='MSG=AP,CORE=4194304,DIAG'

to utilize the full amount of the 4096K region, we see.

IER963I - MAX. SYSGEN CORE - 4194304
IER964I - CALC. CORE PH 1 = 4194206
IER964I - CALC. CORE PH 2 = 259744
IER964I - CALC. CORE PH 3 = 244089

The JES2 SYSOUT below now shows VIRT size of 4100K for the step.

IEF374I STEP /SORT / STOP 10247.1219 CPU 0MIN 00.23SEC SRB 0MIN 00.02SEC VIRT 4100K SYS 212K

The above tests were performed using the JCL in SYS2.JCLLIB(TESTSORT) which includes it's own test data to sort.
"Below are 3 runs of sort against the same data. Results in
Min:Sec (10 Byte sort field, 500,000 Records, 80 Bytes long)

SORTSML used the defaults genned of 64K. Elapsed time
4:02
SORTMED used core and region size of 512K. Elapsed time
2:19
SORTBIG used core and region size of 1700K. Elapsed time
1:46"

More info in the post

http://tech.groups.yahoo.com/group/hercules-390/message/16976

Phil - YMMV
Rick Fochtman
14 years ago
Permalink
----------------------------------------<snip>------------------------------------
...
----------------------------------<unsnip>-----------------------------------------
Dennis, in MVS 3.8, all support was dropped for 23xx devices except for
the 2305-1 and 2305-2. Your SORTOUT DD statement appears like you're
using 3350 devices for SYSTS. Why don't you stick to a single device
type for all your DASD, instead of trying to support a multitude? What's
your IPL device?

Your sort control statement looks as though you're trying to sort on a
single byte. Is that really the case?

Also, your SORTOUT will be deleted at the end of the job. Is that what
you want? In the SORTOUT DD statement, save yourself some headaches; use
DCB=DSTONE.SMFDUMP. VBS or VS is not an issue in MVS; both are as solid
as the Rock of Gibraltar and have been since early releases of OS/360.

After a little research, I've found that MVS 3.8 did indeed support the
3350 volume.

If you'd like to converse offlist, send me your Hercules config file and
we'll put our heads together. <rfochtman-***@public.gmane.org>

Rick
(Retired Sysprog)
Small
14 years ago
Permalink
Ok, finally I can report to the group thet I've been able to
successfully sort SMF records.

First however, I'd like to thank you all for your input. Each response
has given me some insight that collectivly have helped me to accomplish
my goal.

Here are my observaions:

1. I tried to use IFASMFDP to check for invalid records. That
resulted in complete failure, since the program installed on the TK
systems must be very old and has only one purpose (apparently), and that
is to dump SMF records from a MAN(x) dateset to a sequential dataset
(DUMPOUT) and then clear out the input dataset (DUMPIN). It does not
accept any SYSIN parameter statements to alter this behavior, and so I
lost some data, because I didn't realize that the input was ALWAYS
cleared out. (No big deal since the whole thing is experimental anyway.)
2. I looked for an alternative on the CBT tape. I found a newer version
called SMFDUMP, howevr I wasn't able to assemble it, because it is
designed for OS/390 and up, requiring HASM, etc. I could have tried to
adapt it for my purpose, but I decided not to try. 3. I looked for a
FIXVBS program on the CBT tape, but couldn't find that name. Instead I
found COPYVBS, which seemed to perform some chchecks on VBS type
recordes, (eg. check if BDW and RDW are acaccuratetly describing the
record/block on the track.)

This was interresting, because at first it seemed to work fine. (RC =
00, no ABEND). Then I checked the EXCP stats and found only 1 EXCP had
been used to read a dataset of about 25 tracks. That seemed suspicious,
and so I printed the output dataset using IDCAMS PRINT. Only som 28
records had been copied, again without any error message or other
problem indication. I then used IDCAMS REPRO to copy the input dataset.
That worked and reported some 208 records.

I copied some 40 SMF archive datasets with IDCAMS REPRO. Some were bad
or empty (because I damaged them by using IFASMFDP on them.) Of the good
ones that were left I created one single archive by concatenating all of
them and feeding them to IDCAMS REPRO. The result was about 5,108
records.

I still couldn't figure out why COPYVBS woul ony read 1 track (1 EXCP),
so I did some cosmetic changes to the program. First I added register
equates by using a macro I found in one of Sam Golob's programs
(SAVEALL). I used this because it also allows for proper save area
chaining, program entry. (and exit using ENDALL). Then I changed all
the register uses to use the equated register symbolic names. (Now I
could figure out quickly which registers are used in the program.)

Next I added a record counter using R5, and code to format that counter
at the end of the program and print a message. I decided to count each
record written to the output file.

When running this program on my 5,108 record archive it only copied
about 28 records. So now had to figure out why it wouldn't go past
that first track. It took a while but I found that R10 was used as base
for the input buffer, a well as some other area (which didn't change
after the second EXCP, and therefore the program recognized EOF. After
making the base for that area R12, the program happily read and wrote
all 5,108 records with no further problems.
4. Now I tried the SORT again. I found that the "CORE=" parameter
yields an error message "IER067I - INVALID EXEC OR ATTACH PARAMETER".
The DIAG parameter causes a lot of basically useless messages to be
written to the log, and causes the S0C1 abend if any errors are found.
If the DIAG parameter is not present the program ends with RC=16.
The last message "IER053A - OUT OF SEQ" is the culprit for the RC=16.
By changing the SORT statement to properly reflect the SMF record fields
of DATE and TIME, I was able to overcome this issue.
SORT FIELDS=(19,16,BI,A,11,4,BI,A,7,4,BI,A)
END
Was my original statement, copied from SYS1.SAMPLIB. It's wrong!
This is the one I'm using now:
SORT FIELDS=(07,04,PD,A,03,4,BI,A)
END
First field is the record date packed decimal, second is the record time
binary.
This works now, but before the whole sort was successful there were stil
some issues. 5. The SORT program had definite troubles reading
records greater than 7K. So I reblocked the archive to 7K on a 2314
volume.
The sort program had no trouble writing large records, even larger the
32760. So I moved my sortout to a 3380, because using a 3350 volume
caused some S002-0c, or S002-18 abends. 6. Finally the sort ended
reporting all 5,108 records had been read, and written!

Here is the message log (Ok, by now there are more records i te
archive)!

001 COPYVBS STARTED COPYVBS_09/05/10_17.06
COPYVBS I/O: WRITTEN: 6,309 RECORDS
998 COPYVBS ENDED


IER036I - B = 7296
IER037I - G = 52
IER038I - NMAX = 790
IER050I - END MERGE PH
IER054I - RCD IN 6309,OUT 6309
IER052I - EOJ


I verified the sorting sequence using IDCAMS PRINT, and the program
SMFFRMT I found in SYS1.SAMPLIB.

Here is my final SORT JCL: [:D]

//SMFSORT0 JOB 'ROGER W. SUHR',CLASS=A,MSGCLASS=A,
// NOTIFY=RWS01
/*JOBPARM LINES=9999
//*-------------------------------------------------------------------
//*
//S005 EXEC IEFBR14,DS1='SMF.SMFREBLK',
// DS2='SMF.SMFSORT'
//*
//S010 EXEC COPYVBS
//SYSUT2 DD DISP=(,CATLG,DELETE),DSN=SMF.SMFREBLK,
// DCB=(RECFM=VBS,LRECL=6996,BLKSIZE=7000),
// UNIT=2314,SPACE=(CYL,(50,50),RLSE),
// VOL=SER=SORT01
//*
//S020 EXEC SORT,COND=(0,NE),RGN=6144,
// PARM='MSG=AP'
//SYSIN DD *
SORT FIELDS=(07,04,PD,A,03,4,BI,A)
END
//SORTIN DD DISP=SHR,DSN=SMF.SMFREBLK
//SORTOUT DD DISP=(,CATLG,DELETE),DSN=SMF.SMFSORT,
// UNIT=SYSDA,VOL=SER=WORK03,
// SPACE=(CYL,(50,50),RLSE),
// DCB=(LRECL=32756,BLKSIZE=32760,RECFM=VB)
//*
//SORDATA DD UNIT=2314,SPACE=(CYL,(1,1)),
// DCB=(RECFM=VBS,LRECL=600,BLKSIZE=200)
//HDRDATA DD UNIT=2314,SPACE=(TRK,(5,5)),
// DCB=(RECFM=VBS,LRECL=600,BLKSIZE=200)
//*
//S990 EXEC IEFBR14,COND=(0,NE),DS1='SMF.SMFREBLK'

Procs used:

//SORT PROC CLS='*',RGN=4096
//*
//SORT EXEC PGM=SORT,REGION=&RGN.K,
// PARM='MSG=AP'
//SYSOUT DD SYSOUT=&CLS.
//SYSUDUMP DD SYSOUT=&CLS.
//SYSPRINT DD SYSOUT=&CLS.
//SORTLIB DD DSNAME=SYS1.SORTLIB,DISP=SHR
//SORTWK01 DD UNIT=2314,SPACE=(CYL,(5,5))
//SORTWK02 DD UNIT=2314,SPACE=(CYL,(5,5))
//SORTWK03 DD UNIT=2314,SPACE=(CYL,(5,5))
//SORTWK04 DD UNIT=2314,SPACE=(CYL,(5,5))
//SORTWK05 DD UNIT=2314,SPACE=(CYL,(5,5))
//SORTWK06 DD UNIT=2314,SPACE=(CYL,(5,5))
//SORTWK07 DD UNIT=2314,SPACE=(CYL,(5,5))
//SORTWK08 DD UNIT=2314,SPACE=(CYL,(5,5))

//IEFBR14 PROC DS1=,
// DS2=,
// DS3=,
// DS4=,
// DS5=
//*
//DEL EXEC PGM=IEFBR14
//DD1 DD DSN=&DS1.,
// DISP=(MOD,DELETE,DELETE),UNIT=SYSDA,SPACE=(TRK,0)
//DD2 DD DSN=&DS2.,
// DISP=(MOD,DELETE,DELETE),UNIT=SYSDA,SPACE=(TRK,0)
//DD3 DD DSN=&DS3.,
// DISP=(MOD,DELETE,DELETE),UNIT=SYSDA,SPACE=(TRK,0)
//DD4 DD DSN=&DS4.,
// DISP=(MOD,DELETE,DELETE),UNIT=SYSDA,SPACE=(TRK,0)
//DD5 DD DSN=&DS5.,
// DISP=(MOD,DELETE,DELETE),UNIT=SYSDA,SPACE=(TRK,0)
//*

//COPYVBS PROC G='0'
//*
//COPYVBS EXEC PGM=COPYVBS
//*
//STEPLIB DD DISP=SHR,DSN=RWS01.LINKLIB
//SYSPRINT DD SYSOUT=*
//SYSUDUMP DD SYSOUT=*
//SNAP DD SYSOUT=*
//SYSUT1 DD DISP=SHR,DSN=SMF.DAILY.DATA(&G.)
//SYSUT2 DD DUMMY,DCB=(RECFM=VBS,LRECL=32756,BLKSIZE=32760)

Thanks again for all your input, it is very much appreciated!



Roger W. Suhr
halfmeg
14 years ago
Permalink
Post by Small
<snip>
1. I tried to use IFASMFDP to check for invalid records. That
resulted in complete failure, since the program installed on the TK
systems must be very old and has only one purpose (apparently), ...
<snip>
Can't find it currently, but believe I read that there were only 2 options, CLEAR or DUMP. DUMP must include an automatic CLEAR afterward based on your experience.
Post by Small
2. I looked for an alternative on the CBT tape. I found a newer
version called SMFDUMP, howevr I wasn't able to assemble it, because
it is designed for OS/390 and up, requiring HASM, etc. I could have
tried to adapt it for my purpose, but I decided not to try.
HASM alternatives:

www.z390.org
www.tachyonsoft.com
www.dignus.com
...
Please send corrected copy of program to CBTTape (see bottom of www.cbttape.org) to contribute or update to CBT.

VBSFIX is in CBT file 316 along with SPANCPY and some other SMF stuff.
Post by Small
4. Now I tried the SORT again. I found that the "CORE=" parameter
yields an error message "IER067I - INVALID EXEC OR ATTACH
PARAMETER".
'CORE=' has to have a value. Your JCL below:

//S020 EXEC SORT,COND=(0,NE),RGN=6144,
// PARM='MSG=AP'

would need the following to allow SORT to use more than it's default genned 256K of memory to create the initial runs.

//S020 EXEC SORT,COND=(0,NE),RGN=6144,
// PARM='MSG=AP,CORE=6291456'

Your SORT PROC JCL below:

//SORT PROC CLS='*',RGN=4096
//*
//SORT EXEC PGM=SORT,REGION=&RGN.K,
// PARM='MSG=AP'

would need the following otherwise SORT only uses 256K as genned for TK3.

//SORT PROC CLS='*',RGN=4096
//*
//SORT EXEC PGM=SORT,REGION=&RGN.K,
// PARM='MSG=AP,CORE=4194304'

I have not attempted your jobstream or the utilization of a SORT proc as you have so there may be some snafu which you have encountered when you attempt to use the 'CORE=' parm.
Post by Small
The DIAG parameter causes a lot of basically useless messages to be
written to the log, and causes the S0C1 abend if any errors are
found. If the DIAG parameter is not present the program ends with
RC=16.
Normally you wouldn't use DIAG unless you were having problems. The CORE utilization for the various passes of SORT are detailed in them however. Increasing the size of SORT's CORE usage can shorten the run time of the SORT by creating larger initial runs and allowing fewer merge passes to take place.
Post by Small
The last message "IER053A - OUT OF SEQ" is the culprit for the
RC=16. By changing the SORT statement to properly reflect the SMF
record fields of DATE and TIME, I was able to overcome this issue.
SORT FIELDS=(19,16,BI,A,11,4,BI,A,7,4,BI,A)
END
Was my original statement, copied from SYS1.SAMPLIB. It's wrong!
The sample SORT in SYS1.SAMPLIB is setup to utilize exits E15 & E35. You have dropped that from your sort. If you run the SMFE15 & SMFE35 members of SYS1.SAMPLIB, then run the SMFSORT, you should not have a problem. I didn't have 5,000+ records when I tried it, but it worked.
http://publib.boulder.ibm.com/infocenter/zos/v1r9/index.jsp?topic=/com.ibm.zos.r9.ieag200/iea2g28060.htm

//SYSIN DD *
SORT FIELDS=(19,16,A,11,4,A,7,4,A),FORMAT=BI,SIZE=E4000
MODS E15=(E15,700,EXITLIB,N),E35=(E35,1500,EXITLIB,N)
END
/*

which is the same as TK3 SYS1.SAMPLIB(SMFSORT).
Post by Small
<snip>
5. The SORT program had definite troubles reading records greater
than 7K.
Can you elaborate. A TAPE SORT permits record sizes up to around 24,000.
Post by Small
So I reblocked the archive to 7K on a 2314 volume.
The sort program had no trouble writing large records, even larger
the 32760. So I moved my sortout to a 3380, because using a 3350
volume> caused some S002-0c, or S002-18 abends.
If you have your input on AWS or 2314 which can be zipped and emailed would appreciate a copy to test with. ( opplr limited to 10MB per attachment by hotmail.com )
Post by Small
6. Finally the sort ended reporting all 5,108 records had been read,
and written!
Sounds like a good end result but jumping through hoops isn't so hot.
Post by Small
<snip>
Thanks again for all your input, it is very much appreciated!
Thank you for posting your endeavors. Many times we never hear from someone after they present a problem and solution(s) are proposed.

Jay Moseley's IEFU29 web page

www.jaymoseley.com/hercules/installmvs/iefu29.htm

mentions some other CBTTape SMF stuff;

File 94 SMF Dataset Audit Facility
File 523 General SMF Record Selection Program

Phil - 94 & 316 are distributed in the TK3 CBT DASD HLQ CBT429 - 523 is too new for TK3 inclusion
Small
14 years ago
Permalink
Ok, so I allocated the reblocked data on WORK03, Which is a 3390.

UCB=01B0,Volume=WORK03,date=10.256,time=20:40:19,VTOC=OS,parm=
VTOC: CCHH=X'00000001'-X'0001000E',DS4VTOCI=X'08',DSCB/TRK=49,DIR/TRK=44
VTOC: Trks/cyl=15,Track length=57326,Number of cyls=01113
WORK03 free 1015 cyls 0 trks max: 931 cyls 0 trks 14
datasets
Number of free VTOC index records (VIRS)= N/A , blank DSCBS= 1405

//*
//S010 EXEC COPYVBS
//SYSUT2 DD DISP=(,CATLG,DELETE),DSN=SMF.SMFREBLK,
// DCB=(RECFM=VBS,LRECL=23996,BLKSIZE=24000),
// UNIT=SYSDA,SPACE=(CYL,(50,50),RLSE),
// VOL=SER=WORK03

(SYSUT1 DD is in the proc and points to the latest SMF archive.)

//*
//S020 EXEC SORT,COND=(4,LT),RGN=6144,
// PARM='MSG=AP,CORE=6291456'
//EXITLIB DD DISP=SHR,DSN=RWS01.LINKLIB
//SYSIN DD *
SORT FIELDS=(07,04,PD,A,03,4,BI,A)
END
//SORTIN DD DISP=SHR,DSN=SMF.SMFREBLK
//SORTOUT DD DISP=(,CATLG,DELETE),DSN=SMF.SMFSORT,
// UNIT=SYSDA,VOL=SER=WORK03,
// SPACE=(CYL,(50,50),RLSE),
// DCB=(LRECL=32756,BLKSIZE=32760,RECFM=VB)
//*

I didn't include the MODS statement for the exits, because I didn't want
to introduce
another variable. Once this works, I'll try the exits again.

IEF285I JES2.JOB00010.SO0102 SYSOUT
IEF285I JES2.JOB00010.SO0103 SYSOUT
IEF285I JES2.JOB00010.SO0104 SYSOUT
IEF285I SMF.DAILY.DATA.G0061V00 KEPT
*-------45
IEF285I VOL SER NOS= WORK03.
IEF285I SYS1.UCAT.MVS KEPT
*--------0
IEF285I VOL SER NOS= MVSCAT.
IEF285I SMF.SMFREBLK CATALOGED
*-------62
IEF285I VOL SER NOS= WORK03.
IEF373I STEP /COPYVBS / START 10256.2026


This resulted in the following messages:

001 COPYVBS STARTED COPYVBS_09/06/10_11.57
COPYVBS I/O: WRITTEN: 10,754 RECORDS
998 COPYVBS ENDED RC= 4

IER059A - REC LNG INVALID FOR DEVICE
From SORT
As you can see, I did get the CORE= parameter to work, with your help.
Apparently this must be calculated exactly and matched to the region.

So now I'm trying different record lengths .

//S010 EXEC COPYVBS
//SYSUT2 DD DISP=(,CATLG,DELETE),DSN=SMF.SMFREBLK,
// DCB=(RECFM=VBS,LRECL=19096,BLKSIZE=19100),
// UNIT=SYSDA,SPACE=(CYL,(50,50),RLSE),
// VOL=SER=WORK03
//*

I also tried 3350, 3375 with half, and third track blocks. All gave me
the above message
until I came down to this:

//S010 EXEC COPYVBS
//SYSUT2 DD DISP=(,CATLG,DELETE),DSN=SMF.SMFREBLK,
// DCB=(RECFM=VBS,LRECL=07156,BLKSIZE=07160),
// UNIT=2314,SPACE=(CYL,(50,50),RLSE),
// VOL=SER=SORT01
//*
//S020 EXEC SORT,COND=(4,LT),RGN=6144,
// PARM='MSG=AP,CORE=6291456'
//EXITLIB DD DISP=SHR,DSN=RWS01.LINKLIB
//SYSIN DD *
SORT FIELDS=(07,04,PD,A,03,4,BI,A)
END

That resulted in:

001 COPYVBS STARTED COPYVBS_09/06/10_11.57
COPYVBS I/O: WRITTEN: 10,754 RECORDS
998 COPYVBS ENDED RC= 4

IER036I - B = 7296
IER037I - G = 163
IER038I - NMAX = 790
IER050I - END MERGE PH
IER054I - RCD IN 10754,OUT 10754
IER052I - EOJ

As for speed, and efficiency - I don't recall the run times with the
unmodified CORE,
but with elapsed time of about a second, I don't think I have to worry
about that right now.

************************************************************************\
************************************************************
* 3. Jobstep of job: SMFSORT0 Stepname: SORT Program name:
SORT Executed on 13.09.10 from 21.01.10 to 21.01.11 *
* elapsed time 00:00:01,17 CPU-Identifier:
RWS1 Page-in: 0 *
* CPU time 00:00:00,65 Virtual Storage used:
6144K Page-out: 0 *
* I/O Operation
*
* Number of records read via DD * or DD DATA: 2
*
* DMY.......0 DMY.......0 DMY.......0 148.......0 136.....214
137......32 132......36 133......30 135......24 138.....100 *
* 134.....206 136.....212 141.......0 141.......0 DMY.......0
131.....209 1B1.......0 1B0......46 135.......0 137.......0 *
*
*
* Charge for step (w/o SYSOUT):
1,08 *
************************************************************************\
************************************************************

It's nice to see though, that all of the region was actually used!

More results in my next post!

Thanks again

Roger
halfmeg
14 years ago
Permalink
Post by Small
<snip>
IER059A - REC LNG INVALID FOR DEVICE
It would be nice if SORT told you what it really meant, as in which DD it was complaining about.
Post by Small
<snip>
// DCB=(RECFM=VBS,LRECL=07156,BLKSIZE=07160),
OS360 Sort, SM023, which is what we have available only 'knows' about 2301, 2311 & 2314 DASD types. Even though QSAM is fetching records from the sort input DD, SORT seems to want to limit the LRECL to the max record size it is using for SORTWORK packs, in this case around 7200-7300 for 2314 tracks. If 3.8j supported 2301 drums, we could possibly get around 20,000 byte inputs accepted from DISK.

It may be that no matter what DASD type the input resides on, 3330,3350, 3380, 3390, etc... the limiting factor may be the SORTWORK in use. SORT module IERRCI ( IIRC ) has this stuff hardcoded in it.
Post by Small
<snip>
As for speed, and efficiency - I don't recall the run times with the
unmodified CORE, but with elapsed time of about a second, I don't
think I have to worry about that right now.
<snip>
It's nice to see though, that all of the region was actually used!
<snip>
More results in my next post!
Before TK3 was released in 2002, I ran some tests with SORT using various CORE sizes with the DATAMATION Sort records ( 100 LRECL, KeyLength 10, 1 million records), as input. I don't have the results off hand, but 256K did make enough difference that it was made the default in TK3.

When or if there is time here I may run some more tests with various Sorts.

Phil

halfmeg
14 years ago
Permalink
Post by Small
<snip>
Dennis, in MVS 3.8, all support was dropped for 23xx devices except
for the 2305-1 and 2305-2.
<snip>
Not quite so. 2314s are supported in MVS 3.8j otherwise the SORT wouldn't have a DASD sortwork it could utilize.

The OS/360 SORT we have available supports only 3 types of DASD for sortwork areas. Maximum sortwork devices permitted are 6 if 2301 or 2311 are utilized and 17 if 2314 specified. A HACK is available to use any other DASD but SORT acts as if the geometry of the pack is a 2311 so there is no increase in sortwork space by using the HACK.

A quick look at the SYSGEN macro UNITNAME shows the following esoteric names:

3350
3330-1
3330
3340
2314
2305-1
2305-2

Additional DASD support (3375, 3380, 3390) was added to MVS 3.8j before TK3 release via mods from Jim Morrison.

Phil - believe 2311 is included in newest DASD mods
Loading...