Determining Pagefile usage

[Shael Richmond]

Is there a way to determine what processes are actually using the pagefiles?

I found a show_pagefile.com on openvms.org but it's not accurate. I ran it on a system that the entire pagefiles were free yet it showed usage. It looks like it just takes the pgflquota - pgflquota_left.

This is on OpenVMS 8.2.

I had an issue where the pagefile was full and luckily I was able to get on and create a new one. But I am trying to figure out if I just sized the pagefile wrong or I had a bad process.

[Hoff]

The aggregate PGFLQUOTA value "promised" to each process is the quantity of pagefile that should be around if there's a "run on the pagefile". In the vast majority of cases, systems are run with less pagefile as there's seldom a case where a single run-away process can consume enough to matter (before slamming into its PGFLQUOTA) and the aggregate of processes aren't all asking for their maximal quota at the same time.

As for your question...

SHOW PROCESS /ACCOUNT /ID=xxx will show the peak virtual address space. Oversimplifying a little here, the peak virtual address space minus the working set resides in the pagefile.

Accounting typically logs the peak virtual address space value when the process logs out.

f$getji offers up PAGFILLOC (where the process is allocating its storage), as well as the VIRTPEAK value. So you could loop through (using WORLD privilege and some lexicals) to sniff all the processes.

I'd probably look to AMDS or Availability Manager here, as these tend to be better at poking around inside the running system than most other tools.

Took a quick look at the show_pagefile.com command tool over at http://dcl.openvms.org/stories.php?story=03/06/03/0307237 and that looked OK. Entire pagefiles free? How did you determine that? The whole area is a bit of an approximation, due to how processes are coming and going and how virtual memory on a multi-user box works...

[Shael Richmond]

If the free and total are the same then there is no pagefile usage??


Paging File Usage (8KB pages): Index Free Size
DISK$CCPPRA_PS:[CCPPRA]PAGEFILE.SYS;1
253 156280 156280
DISK$AXPVMSSYS:[SYS0.SYSEXE]PAGEFILE.SYS
254 93776 93776

Total size of all paging files: 250056
Total committed paging file usage: 40454

This isn't the site where I had an issue but based on your reply, more then likely my pagefile was too small and a peak period caught up with me. The system has been up for 195 days without issue previously.

[Hoff]

There's free, total, and the reserved/allocated/committed value.

Free is the remaining room.

Total is how much was configured.

The old reservable value was the "run on the bank" value -- if all the processes show up requesting their respective full pgflquota allocations. On older releases, the available or reservable value could show up as negative, and it was useful to know that having some negative amount here could be OK; but increasing toward a double-overcommit -- larger negative value -- usually led to trouble.

The negative value was viewed as confusing, and was changed at V7.3 into the commit display you see now.

Here, you don't have all that much pagefile configured (well, by my standards), but then there's not all that much of it in use right now. That you have committed less than total -- this is the way the old reservable stuff is displayed on V7.3 and later -- where having it set larger can be viable. It's still the same basic display. (Albeit somewhat more confusing for those of us that figured out the old display.)

How many processes are active peak, and how much memory? If there are few processes and buckets of physical memory, you won't need much pagefile, for instance.

AUTOGEN (with FEEDBACK) is your friend here, too. AUTOGEN can help dial-in the size of the files you need, assuming the page, swap and dumpfile size adjustments haven't been disabled in MODPARAMS.DAT.

Also watch the pagefault rates. Assuming you don't run out of room in the pagefile and grind to a halt, these faults can be a bigger performance factor.

Ian's PF tool might also be useful here.
http://www.openvms.org/stories.php?story=06/06/09/7721822

[labadie_1]

>>> Took a quick look at the show_pagefile.com command tool over at http://dcl.openvms.org/stories.php?story=03/06/03/0307237 and that looked OK.

I think this is correct until Vms 7.3, but memory management changed then and since, I do not know of a good simple way.

[Shael Richmond]

>There's free, total, and the >reserved/allocated/committed value.
>Free is the remaining room.
>Total is how much was configured.

Ok I am confused. If free is amount remaining and total is amount configured then if free=total then no pagefile is in active use???


>How many processes are active peak, and >how much memory? If there are few >processes and buckets of physical memory, >you won't need much pagefile, for instance.

My systems have 1gb memory.
The site with no pagefile usage is about 100 processes. The site with issues has about 265 processes.

I normally have two pagefiles. One 2,500,000 blocks off the system disk and another 1,500,000 on the system disk. The one on the system disk varies depending on how old(how big the disks were) and how many nodes.

>AUTOGEN (with FEEDBACK) is your friend >here, too. AUTOGEN can help dial-in the >size of the files you need, assuming the >page, swap and dumpfile size adjustments >haven't been disabled in MODPARAMS.DAT.

I ran this and it verified my conclusion that I was short of pagefile space. For some reason I only had one pagefile on the site with a problem. I've made the second one permanent now!

[Hoff · Accepted solution]

>>>Ok I am confused. If free is amount remaining and total is amount configured then if free=total then no pagefile is in active use???<<<

Correct; nothing has been written to the pagefile. The pagefile might well have space reserved, but -- like an airline that routinely overbooks its flights or a restaurant that routinely overbooks its tables -- nothing bad happens so long as not everybody shows up to claim the space.

If you've just connected the new file, the processes are likely not using it. There have been changes over the years, though the classic way was to cause the process to choose the pagefile with the most free space when the process logged in.

Here's a quick intro to virtual memory:
http://64.223.189.234/node/228

>>>My systems have 1gb memory.
The site with no pagefile usage is about 100 processes. The site with issues has about 265 processes.<<<

Which implies the system is using backing storage from elsewhere (eg: installed images, commons, file-backed sections), and the system is running the application largely in memory, and there's minimal use of storage in the pagefile.

>>>I normally have two pagefiles. One 2,500,000 blocks off the system disk and another 1,500,000 on the system disk. The one on the system disk varies depending on how old(how big the disks were) and how many nodes.<<<

Ok. I tend to look at the disk spindle traffic, too. This to spread any paging I/O load around.

There have been various schemes over the years, whether fewer and bigger files, or more and smaller. I've tended toward fewer and larger, and toward watching the disk spindle queue depth.

>>>I ran this and it verified my conclusion that I was short of pagefile space. For some reason I only had one pagefile on the site with a problem. I've made the second one permanent now!<<<

Ok.

[John Gillings]

Shael,

> Ok I am confused. If free is amount
>remaining and total is amount configured
>then if free=total then no pagefile is in
>active use???

Correct. However, processes may still "think" some of their pages are in the page file. Processes kick modified pages out of their working set when space is required for another page. As far as they're concerned it's in the pagefile (PGFLQUOTA has been decremented). When they fault the page back into the working set, it's no longer thought of as "in the pagefile".

Now consider the same page from the perspective of the memory management system. Instead of immediately writing the page to the page file, the it's placed on the Modified Page List (MPL). If the page is requested, it will be returned from the MPL as a soft fault. Only when the MPL gets too big, are pages flushed to the pagefile, at which point the page file registers usage and decrements the FREE count. If it's then requested, you will register a hard fault, because the page needs to be physically read from the pagefile.

Somewhere in the documentation is a state diagram showing how pages move from working sets through various page lists, page and/or swap files. It's worth finding and studying.

On modern systems with large memories, the MPL can often grow large enough to hold much of the "page file pages", so no actual page file is used. This is a fairly good position to be in.

If you're seeing free=total, but have processes with apparent pagefile usage for a significant time, and you're seeing soft faulting (free list faults and modified page list faults) that may indicate your working set quotas and/or extents are too small. Increasing process working sets will eliminate the soft faulting, but is memory neutral because the pages will simply remain in the process working set, rather than cycling through the MPL or Free List.