OpenVMS SORT statis

[David Moczygemba]

Environment:
OpenVMS 7.3-2, alpha (3CPU ES40)

What are some things which could impact "Sort tree size"? Though our change logs shed no light, a regular sort job has changed it's pattern taking much longer than it used to. Two sample stats are below.

Any feedback is greatly appreciated! Thanks, -David

(Good sort from last month)
$ SORT/KEY=(POSITION:38,SIZE:32) /STAT -
ra_data1:TR_REV_401B1.0703 -
ra_data1:TR_REV_401B1.0703

OpenVMS Sort/Merge Statistics

Records read: 21161897 Input record length: 622
Records sorted: 21161897 Internal length: 622
Records output: 21161897 Output record length: 622
Working set: 8388608 Sort tree size: 446622
Virtual memory: 552864 Number of initial runs: 25
Direct I/O: 904702 Maximum merge order: 20

Buffered I/O: 139 Number of merge passes: 2
Page faults: 35113 Work file alloc: 30873474
Elapsed time: 00:27:50.52 Elapsed CPU: 00:10:32.74

(Bad sort from this month)
$ SORT/KEY=(POSITION:38,SIZE:32) /STAT -
ra_data1:TR_REV_401W1.0704 -
ra_data1:TR_REV_401W1.0704

OpenVMS Sort/Merge Statistics

Records read: 2277280 Input record length: 622
Records sorted: 2277280 Internal length: 622
Records output: 2277280 Output record length: 622
Working set: 8388608 Sort tree size: 2
Virtual memory: 77104 Number of initial runs:569235
Direct I/O: 3168711 Maximum merge order: 20

Buffered I/O: 23675 Number of merge passes: 29960
Page faults: 5270 Work file alloc: 3026946
Elapsed time: 12:32:07.18 Elapsed CPU: 09:59:26.48

[Dean McGorrill]

directio is up, bufio is up, virtual memory
is down. I'd probably start and check to see if the process quotas have been changed. the cpu time is the wonder, from 10+ minutes to almost 10hours! take a close
look at the files to, see if they are normal
to what you usually process.

[Robert Gezelter]

David,

I agree with Dean, and will further note that the number of passes increased dramatically. I too would wonder about the process quotas, and a variety of other factors, including the quota settings on the batch queues.

I would also like to understand what other load was on the system at the time of the sort, and whether this behavior is reproduceable on an otherwise idle machine.

I have investigated several SORT anomalies for clients over the years, and changes in quotas, settings, and external workload have caused performance issues.

I hope that the above is helpful.

- Bob Gezelter, http://www.rlgsc.com

[David Moczygemba]

As best I can tell the queue must have had some working set settings which were removed in the last few days. Though I wasted a lot of today trying higher and higher working set values, the key was lower. Specifically: a working set of 777680 seemed relatively optimum for the 12GB file we needed to run. It sorted in 15 minutes once that change was in place. I also had some minor challenges with precedence between UAF settings, queue settings and PQLM settings but all is working now. Thanks so much! -David

$ SET WORK/LIMIT=777680/QUOTA=777680/EXTENT=777680
$!starting sort on key1..from nr, to nr, rcd date & connect time....
$ SORT/KEY=(POSITION:38,SIZE:32) /STAT -
ra_data1:TR_REV_401B1.0704 -
ra_data1:TR_REV_401B1.0704

OpenVMS Sort/Merge Statistics

Records read: 20017984 Input record length: 622
Records sorted: 20017984 Internal length: 622
Records output: 20017984 Output record length: 622
Working set: 777680 Sort tree size: 472358
Virtual memory: 584640 Number of initial runs: 22
Direct I/O: 819499 Maximum merge order: 20
Buffered I/O: 119 Number of merge passes: 2
Page faults: 36600 Work file alloc: 26821314
Elapsed time: 00:15:22.86 Elapsed CPU: 00:09:46.95

[David Moczygemba]

As best I can tell the queue must have had some working set settings which were removed in the last few days. Though I wasted a lot of today trying higher and higher working set values, the key was lower. Specifically: a working set of 777680 seemed relatively optimum for the 12GB file we needed to run. It sorted in 15 minutes once that change was in place. I also had some minor challenges with precedence between UAF settings, queue settings and PQLM settings but all is working now. Thanks so much! -David

$ SET WORK/LIMIT=777680/QUOTA=777680/EXTENT=777680
$!starting sort on key1..from nr, to nr, rcd date & connect time....
$ SORT/KEY=(POSITION:38,SIZE:32) /STAT -
ra_data1:TR_REV_401B1.0704 -
ra_data1:TR_REV_401B1.0704

OpenVMS Sort/Merge Statistics

Records read: 20017984 Input record length: 622
Records sorted: 20017984 Internal length: 622
Records output: 20017984 Output record length: 622
Working set: 777680 Sort tree size: 472358
Virtual memory: 584640 Number of initial runs: 22
Direct I/O: 819499 Maximum merge order: 20
Buffered I/O: 119 Number of merge passes: 2
Page faults: 36600 Work file alloc: 26821314
Elapsed time: 00:15:22.86 Elapsed CPU: 00:09:46.95