%SORT-W-SYSERROR and LIB-F-INSVIRMEM, in a CONVERT with /NOSORT

[yaron1]

Hi,

I did a CONVERT on an indexed file and got in the log the following message:
%SORT-W-SYSERROR, system service error
-LIB-F-INSVIRMEM, insufficient virtual memory
%SORT-W-SYSERROR, system service error
-LIB-F-INSVIRMEM, insufficient virtual memory

Seems to be a sort system error or warning due to insufficient virtual memory. This is strange because I did the CONVERT with the /NOSORT qualifier why do I get this message then. The converted, output file seems to be O.K. This convert is done rarely and if the sort is meaningless here and this is just a warning, then can I just ignore it? I havent yet replaced the file. The convert command is something like:
CONVERT/FDL=

/STAT/NOSORT.
The machine is VAX 4500A and the OS is VMS version 5.5. If I do SHOW PROCESS/QUOTAS/MEMORY I get:
CPU limit: Infinite Direct I/O limit: 240
Buffered I/O byte count quota: 82000 Buffered I/O limit: 240
Timer queue entry quota: 50 Open file quota: 200
Paging file quota: 90999 Subprocess quota: 2
Default page fault cluster: 64 AST quota: 48
Enqueue quota: 999 Shared file limit: 0
Max detached processes: 0 Max active jobs: 0

Process Dynamic Memory Area
Current Size (bytes) 51200 Current Total Size (pages) 100
Free Space (bytes) 42512 Space in Use (bytes) 8688
Size of Largest Block 42048 Size of Smallest Block 8
Number of Free Blocks 4 Free Blocks LEQU 32 Bytes 1

So in short, can just ignore this message and replace the file? Thanks for the answers.
Yaron.

[labadie_1]

Can you post
$ mc sysgen sh virtualp
$ sh mem/phy
and the pgflquota of the process doing the sort ?

[Hein van den Heuvel · Accepted solution]

>> Seems to be a sort system error or warning due to insufficient virtual memory.

Correct. It is 'just a warning'. It means that sort was led to believe it had a certain amount of virtual memory to play with (looking at WSEXTEND and PAGFILQUO notably). Make sure that the working set extent is no more than one third of the page file quota.

Sort calls LIB$GET_VM to request the virtual memory needed. If this request to LIB$GET_VM exceeds the virtual memory limits of the process, LIB$GET_VM returns an INSVIRMEM error. The Sort Utility signals the INSVIRMEM error, but tries to recover by decreasing the request until the LIB$GET_VM call succeeds. It sizes the request down by 7/8 in each iteration.

>> This is strange because I did the CONVERT with the /NOSORT qualifier why do I get this message then.

The FDL must have requested one or more Alternate key. The /NOSORT only tells convert that the PRIMARY key needs no sorting. To really get no sort, you need to tell convert to RMS directly through /NOFAST (and/or /MERGE)

BTW If there is more than 1 alternate key, please make a test with /SECONDARY=x to avoid creating one CONVWORK files per alternate key.

>> The converted, output file seems to be O.K.


>> PROCESS/QUOTAS/MEMORY I get:
>> Paging file quota: 90999

We'll need a $SHOW WORK output as well.

>> So in short, can just ignore this message and replace the file?


Yes.

Regards,
Hein van den Heuvel (at gmail dot com)
HvdH Performance Consulting

[yaron1]

Thanks for the answers.
Here is SHOW WORKING_SET
Working Set /Limit= 8192 /Quota= 16000 /Extent= 65500
Adjustment enabled Authorized Quota= 16000 Authorized Extent= 65500

[Hein van den Heuvel]

And is that wsextent significantly lower than pagfilquo is I replied it should be? No.
So you get the warning as deserved.
Grow pagfilquo (2x or 3x wsextent) or shrink wsextent (1/2 or 1/3 of pagfilquo)?and be happy!

;-)

Cheers,
Hein.

[John Gillings]

As Hein says:

>Grow pagfilquo (2x or 3x wsextent) or shrink wsextent (1/2 or 1/3 of pagfilquo)?and be happy!

Typically Virtual Memory size (limited by PGFLQUOTA) should be significantly larger than Physical Memory size (limited by WSEXTENT).

The most common reason for an unexpected imbalance is PQL_MWSEXTENT being set to WSMAX by AUTOGEN. Generally the best solution is to increase PGFLQUOTA, BUT you need to make sure you have enough pagefile to cover the allocation.

Consider that your PGFLQUOTA listed above is 90999 - not much more than 40MB (yes MEGA bytes).

Consider the cost of that much disk space, weighed against he cost of dealing with this ONE instance of this error.

Please give this system adequate resources to do the work you're asking of it!

(maybe we should start the SPCV? "Society for Prevention of Cruelty to VMS systems" The poor machines are trying to work under such tight constraints, all those INSVIRMEM's must be painful, yet it's so cheap and simple to give them enough room to move)