Monitoring process creation stats (creating high pagefaulting?)

[Thomas Griesan]

Hey people,

I suspect the amount of process creation is the major consumer of resources on my cluster.
I'm unable to find a straightforward way to monitor this activity.

Pagefaulting is high and when I look at mon proc/topfault the process dissappears before I can look at it any closer.

There is quite a bit of demand zero faulting which suggests a lot of image activation.

To affect process creation, of course this is an application issue mostly, but I need to understand and document this behavior.

There is a ton of free memory (28% free of 4 Gig, or 72% used) and frankly I'm also a little puzzled why the Page Read I/O rate would be this high.

I've attached a notepad file with paste of "mon page".

Does process creation and image activation require reads from the page file on disk (i.e. hard faults)?

This makes me wonder if WSdef or even WSquota should be set pretty low so that process creation isn't as expensive?

Thanks TONS!!!
Tom

[Ian Miller. · Accepted solution]

image activation does involve page faults from the image file which are recorded as page reads.

Have a look at MON IO. You will probably see lots of file opens.

Briefly switch on image accounting (uses disk space quickly) and see which processes and which images.

SET ACC/ENABLE=IMAGE
wait a bit
SET ACC/DISABLE=IMAGE

[Thomas Griesan]

Ian - THANKS!!! Yep, lots of file opens for what I'd have expected and in relation to Direct IO. E.G. 40 file-opens/sec compared to 275 DIOs on average (one node). Its a small itanium cluster/appl.

One more thing, tho. :-)

I don't see the processes gobbling up much working set whey have quite large def/quo/ext.

Would it makes sense to turn down WSdef and/or WSquota?

I guess I'll have to read up on this, but what I mean is that when a process is created does it automatically map it's full WSdef and therefore cause lots of pagefaulting? Or does it just page in what it requires at the time...?

And in this cluster's case only use a small amount of memory and then terminate.

I was hoping to tune this somewhat to help the system, but I expect the process just pages in what it needs when created.

Thanks AGAIN!!!
Tom

[Volker Halle]

Tom,

as Ian said, briefly turn on image accounting, so you get an idea, which images get activated and how often. If it's a small number of images, you may be able to install them /SHARED or even /RESIDENT. Also take into account any shareable libraries used.

The workingset is filled as the process starts pagefaulting and reading in the pages from the images on disk.

You may consider to use T4 - a free performance data collector created and used by OpenVMS engineering. It's on your disk in SYS$ETC - see T4 home page:

http://h71000.www7.hp.com/OpenVMS/products/t4/index.html

Volker.

[Jan van den Ende]

Tom,


I suspect the amount of process creation is the major consumer of resources on my cluster.


One quick way to find out:
ACCOUNTING /SINCE= /BEFORE=
If you see many similar processes in short succession, take a shorter interval, do ACCO/FULL , and watch the process duration (ending time - Start time )

About a year ago we had a SAN monitoring package (HP OV SAN management agent; of course, at the request of our SAN managers) which caused 10 process creations PER SECOND in a 4 node cluster. Running time ca 0.1 sec each. Useful work done? Zero, Na...

[Volker Halle]

Tom,

looks like there is a performance data cell in the system, which keeps a counter of process creations:

$ ANAL/SYS
SDA> EXA PMS$GQ_PROCESSES_CREATED

Volker.