QMAN messages about Low Virtual Memory

[M C_1]

All - I am attempting to troubleshoot an issue that has been present on our cluster for quite some time. Every so often we get a message on the console similar to the following:

Message from user QUEUE_MANAGER on
%QMAN-W-LOWMEMORY, the queue manager process may require more virtual memory than is currently available

I would say on average we get the message about once every 1 hour or so. However, its pretty sporatic, sometimes every 30 minutes and sometimes every 2 hours.

I read the articles in DSN that point to modifying SYS$STARTUP:VMS$CONFIG-050_JOBCTL.COM, however this does not seem to apply as the calculation appears to be correct. I also checked and verified that we have plenty of page file space.

HP documents that this was fixed in 7.1

Our config consists of 2 ES40's clustered running VMS 7.2-2. I have verified that this also occurs on another system running VMS 7.3-2.

A reboot will stop the messages for a while (10 or more days) but then they come back.

Any ideas?

MC

[Bojan Nemec]

Hi,

Help/message lowmemory says:

User Action: There may be a need to run AUTOGEN to increase the SYSGEN parameter VIRTUALPAGECNT.

So first check the VIRTUALPAGECNT sysgen parameter. Between the calculation from SYS$STARTUP:VMS$CONFIG-050_JOBCTL.COM and the VIRTUALPAGECNT, the lower is used. You can test what will AUTOGEN do, by running @sys$update:autogen getdata testfiles and examine the AGEN$PARAMS.REPORT file.

Bojan

[John Gillings]

Bojan,

Sorry, VIRTUALPAGECNT is an obselete parameter on Alpha, so that's a blind alley for this symptom.

MC,

Perhaps you're pushing the queue manager particularly hard and therefore have an abnormally high demand for virtual address space? What is your total pagefile space -see F$GETSYI("PAGEFILE_PAGE")?

Try executing the commands from SYS$STARTUP:VMS$CONFIG-050_JOBCTL.COM to see what pagefile quota you get.

$page_size = f$getsyi("page_size") / 512
$total_page = f$getsyi("pagefile_page") * page_size
$min_jobctl_pagelets = 40960
$if total_page .lt. 2*min_jobctl_pagelets
$then pagefile = min_jobctl_pagelets
$else pagefile = total_page/2

Now compare it with the actual quota for JOB_CONTROL and QUEUE_MANAGER using

$ pgflquota=F$GETJPI(pid,"PGFLQUOTA")

(where "pid") is the PID of JOB_CONTROL and then QUEUE_MANAGER.

Track the page file consumption over time by running a periodic job - say every 5 minutes which finds the PAGFILCNT. Does it increase over time or just at startup and then plateaus?

You can restart the queue manager without rebooting (BUT BEWARE - THIS WILL KILL ALL EXECUTING BATCH AND PRINT JOBS)

Logged in as SYSTEM on all nodes:

$ STOP/QUEUE/MANAGER/CLUSTER ! (on one node)
$ STOP JOB_CONTROL ! (on each node)
$ @SYS$SYSTEM:STARTUP JOBCTL
$ START/QUEUE/MANAGER/CLUSTER

Also beware that after STOP JOB_CONTROL, you won't be able to login until it's restarted, so make certain you're logged in as SYSTEM on all nodes.

You should be able to increase the quota by installing more page file.

On the other hand, if QUEUE_MANAGER is reporting the error but not actually failing then another possibility is the messages are bogus, resulting from having TOO MUCH pagefile quota! When you have very large pagefiles, the quota may be high enough to overflow the 32 bit arithmetic used to work out if queue manager is getting close to filling the page file. As long as QUEUE_MANAGER has plenty of PAGFILCNT and there is plenty of free space in the page file at the time the message is issued, you can safely ignore them.

This issue has been reported and should be fixed in a future version. In the mean time, a fairly clumsy way to avoid the error might be to rearrange your startup procedures so that secondary page files aren't installed until after QUEUE_MANAGER has been started. That way PGFLQUOTA will only be calculated on the primary page file on the system disk. Given memory sizes of modern systems, it's unlikely that your startup is dependent on page files, so this should be perfectly safe.

[Lawrence Czlapinski]

Bojan, VIRTUALPAGECOUNT is obsolete on Alphas since at least VMS Alpha 7.1-1H2. ES40's are Alphas.
"CONSHY» HELP/MESSAGE "LOWMEMORY"


LOWMEMORY, the queue manager process may require more virtual memory
than is currently available

Facility: QMAN, Queue Manager

Explanation: The queue manager is still progressing through its work, but
a lack of virtual memory has been detected. This message
indicates a potential problem if preventive action is not
taken.

User Action: On VAX systems, try running AUTOGEN to increase the SYSGEN
parameter VIRTUALPAGECNT."
Unfortunately it doesn't give a suggestion for what to do on an Alpha.
MC, It seems likely that the process thinks it is running low on virtual space.
What is the WSDEF, WSQUO and WSEXT for your QUEUE_MANAGER process? How many pages in working set for process QUEUE_MANAGER? QUEUE_MANAGER usually doesn't page fault much so I'm thinking the help isn't specific enough about what is really needed on an Alpha. You could also check PGFLQUO for the account that is running it.
What is your SYSGEN WSMAX? Perhaps WSMAX is too low for your QUEUE_MANAGER. Seems unlikely but you should still check it.
Lawrenc