topic Re: wsinc default value in Operating System - OpenVMS

wsinc default value

Wim Van den Wyngaert — Thu, 03 Nov 2005 05:11:43 GMT

I was investigating a process on VMS 7.3 doing a lot of pagefaults and found out that wsinc has a default (and actual) value of 2400 pagelets.

I compared this with a 6.2 node and noticed that it 2400 there too. Don't have older versions here.

Should't this parameter be a lot higher these days ? E.g. 24000 ?

I have some processes that are created with default pql settings but that allocate about 600.000 pages. So, they have to fault as crazy to get their working set increased.

Side effects if I would increase it ?

Wim

Re: wsinc default value

Mike Reznak — Thu, 03 Nov 2005 05:53:01 GMT

Hi,

we have the default value everywhere. So that means our support have never sugested any change.

Mike

Re: wsinc default value

Marc Van den Broeck — Thu, 03 Nov 2005 06:11:34 GMT

On Vax, the default value was 150 (512 byte) pages. So actually the default value did increase from 150 to 2400. But indeed in environments where processes have to grow to 600.000 pages you might increase this value.
But then this is active for all processes and it migth be an overkill for several (temporary?) processes.

Rgds
Marc

Re: wsinc default value

Wim Van den Wyngaert — Thu, 03 Nov 2005 06:18:33 GMT

Marc,

No overkill because the memory is not taken when not used (by the temp process).

But the sysgen help sys_p pfrath says that wsinc is assigned every 10 cpu seconds. Should't that be every 1/10 second ?

My process is having a working set of 580.000 pagelets and only used 52 seconds.

And it took about 3 hours to get there. Each time a lot of activity is done by the process, it increased in size.

Wim

Re: wsinc default value

Marc Van den Broeck — Thu, 03 Nov 2005 07:12:49 GMT

Wim,

On Alpha systems, WSINC specifies the number of pagelets by which the limit of a working set is automatically increased at each
adjustment interval (which is quantum end).

Isn't PFRATH the value per 10 seconds above which 'high fault rate' is decided? And if so, then every quantum the limit is increased with WSINC?

Rgds
Marc

Re: wsinc default value

Mike Reznak — Thu, 03 Nov 2005 08:50:29 GMT

Another explanatin

e.g. Default WSINC of 150 and default PFRATH of 120, means your process
needs to exceed 120 faults/sec for 95 CPU seconds, probably about 2
minutes to you and me before it gets to WSQUOTA. This may not be a lot
on a single build taking upwards of 30 minutes, but if you are invoking
the compiler multiple times, the process starts back down again at
WSDEFAULT.

But on Alpha systems, WSINC also has the AUTOGEN attribute. So that means that autogen should suggest change if it's needed.

Mike

Re: wsinc default value

Wim Van den Wyngaert — Thu, 03 Nov 2005 08:58:26 GMT

QUANTUM=200ms
AWSTIME=200ms
PFRATH=8
are my settings.

So, every 200 ms wsinc pages will be added if the number of pagefaults in 200ms converted to pagefaults per 10 sec is bigger than 8. So, 1 pagefault * 50 (factor to convert it) = 50. 50 > 8. So 1 pagefault is enough to get the wsinc.
In fact, pfrath values 1 thru 49 (or 50?) have the same result.

But I need at least 200 times wsinc to get the working set I need. So, about 200 * 200 ms or 40 cpu seconds are needed to get there. That is, if I fault in every 200 ms.

Wim

Re: wsinc default value

Mike Reznak — Thu, 03 Nov 2005 09:01:43 GMT

This article doesn't bring much more light into the right value setting but is quite interesting.

http://www3.sympatico.ca/n.rieck/docs/openvms_tuning_notes.html

Mike

Re: wsinc default value

Wim Van den Wyngaert — Thu, 03 Nov 2005 09:06:26 GMT

Mike,

This is what I have on AWSA.

A process is checked to determine the necessity for working
set growth at AWSTIME intervals. AWSTIME is expressed in
units of 10 milliseconds and its default value is 20. The
amount a process pagefaults in AWSTIME is compared to an
upper threshold for pagefaults per 10 CPU seconds defined
by the SYSGEN parameter 'PFRATH'. For example, if AWSTIME
is set to 20 and PFRATH 120, a process would get WSINC WSLEs
if it pagefaulted 3 pages in AWSTIME as shown in the chart
below:

AWSTIME set to 20 = 200 milliseconds
3 pagefaults in 200 milliseconds = 15 pagefaults per CPU second
15 pagefaults per CPU second = 150 pagefaults per 10 CPU second
150 > 120 (Value of PFRATH)
----------------------------------------------
Process gets WSINC number of WSLEs

If it is determined that the process has exceeded the
pagefault threshold (PFRATH) within the specified interval
(AWSTIME), it will receive WSINC WSLEs when it has validated
more than 75% of the WSLEs previously given to the process.

Re: wsinc default value

Ian Miller. — Thu, 03 Nov 2005 12:22:11 GMT

Note that Wim is talking about WSLE (Working Set List Entry) not how many pages of memory your process occupies.
You need one WSLE for each page but can have unused WSLE.

Re: wsinc default value

Lawrence Czlapinski — Fri, 04 Nov 2005 15:32:12 GMT

Wim, I would run a WORKSET.COM. Then I would recommend setting explict pql values for the processes if you have the available free memory. You should be able to explicitly set much higher PQL_WSDEF and PQL_WSQUOTA for these processes. Maybe set PQL_WSQUOTA to 600000. If any of these processes share any global pages it will reduce your pagefaulting a lot.
Lawrence

Re: wsinc default value

Andy Bustamante — Fri, 04 Nov 2005 17:24:09 GMT

I've used Lawrence's solution to deal with DBAs who didn't feel the need to monitor quotas. It has the advantage of using minimal resources. I also set a min_PQL_MWSEXTENT.

In troubleshooting peformance issues, I've also had HP recommend increasing WSINC up to 16000. This is for an application where most of the system memory is taken up by global sections and does not reflect the traditional "user" other sites see.

Andy

Re: wsinc default value

Wim Van den Wyngaert — Mon, 07 Nov 2005 11:16:32 GMT

Lawrence,

What do you mean with workset.com ?

There is no way to modify the application creating the process.

Wim

Re: wsinc default value

Lawrence Czlapinski — Mon, 07 Nov 2005 14:49:01 GMT

Wim,
1. You didn't specify how the processes are started. If you are creating subprocesses or detached processes, then
you can:
$run (process)/WORKING_SET=wsdefault_value/MAXIMUM_WORKSET=wsquota_value/EXTENT=wsextent_value.
If you are running images, then you're right you can't set these values.
2. Attached is workset.com as workset.txt. It just gives you a measurement of pages in working set, page faults, etc.
Lawrence

Re: wsinc default value

faris_3 — Thu, 17 Nov 2005 05:31:42 GMT

$!
$! WORKING_SET.COM - Command file to display working set information.
$! Requires 'WORLD' privilege to display information
$! on processes other than your own.
$!
$! the next symbol is used to insert quotes into command strings
$! because of the way DCL processes quotes, you can't have a
$! trailing comment after the quotes on the next line.
$!
$ quote = """
$!
$ pid = "" ! initialize to blank
$ context = "" ! initialize to blank
$!
$! Define a format control string which will be used with
$! F$FAO to output the information. The width of the
$! string will be set according to the width of the
$! display terminal (the image name is truncated, if needed).
$!
$ IF F$GETDVI ("SYS$OUTPUT", "DEVBUFSIZ") .LE. 80
$ THEN
$ ctrlstring = "!AS!15AS!5AS!7(6SL)!7SL !10AS"
$ ELSE
$ ctrlstring = "!AS!15AS!5AS!7(6SL)!7SL !AS"
$ ENDIF
$!
$! Check to see if this procedure was invoked with the PID of
$! one specific process to check. If it was, use that PID. If
$! not, the procedure will scan for all PIDs where there is
$! sufficient privilege to fetch the information.
$!
$ IF p1 .NES. "" THEN pid = p1
$!
$! write out a header.
$!
$ WRITE sys$output -
" Working Set Information"
$ WRITE sys$output ""
$ WRITE sys$output -
" WS WS WS WS PPG GPG Pages Page"
$ WRITE sys$output -
"Username Processname State Extnt Quota Deflt Size in WS Faults Image"
$ WRITE sys$output ""
$!
$! Begin collecting information.
$!
$ som = 0
$ collect_loop:
$!
$ IF P1 .EQS. "" THEN pid = F$PID (context) ! get this process' PID
$ IF pid .EQS. "" THEN goto fin ! if blank, no more to
$! ! check, or no privilege
$ pid = quote + pid + quote ! enclose in quotes
$!
$ username = F$GETJPI ('pid, "USERNAME") ! retrieve proc. info.
$!
$ IF username .EQS. "" THEN GOTO collect_loop ! if blank, no priv.; try
$! ! next PID
$ processname = F$GETJPI ('pid, "PRCNAM")
$ imagename = F$GETJPI ('pid, "IMAGNAME")
$ imagename = F$PARSE (imagename,,,"NAME") ! separate name from filespec
$ state = F$GETJPI ('pid, "STATE")
$ wsdefault = F$GETJPI ('pid, "DFWSCNT")
$ wsquota = F$GETJPI ('pid, "WSQUOTA")
$ wsextent = F$GETJPI ('pid, "WSEXTENT")
$ wssize = F$GETJPI ('pid, "WSSIZE")
$ globalpages = F$GETJPI ('pid, "GPGCNT")
$ processpages = F$GETJPI ('pid, "PPGCNT")
$ pagefaults = F$GETJPI ('pid, "PAGEFLTS")
$!
$ pages = globalpages + processpages ! add pages together
$ som = som + (pages / 16 )
$!
$! format the information into a text string
$!
$ text = F$FAO (ctrlstring, -
username, processname, state, wsextent, wsquota, wsdefault, wssize, -
processpages, globalpages, pages, pagefaults, imagename)
$!
$ WRITE sys$output text ! display information
$!
$ IF p1 .NES. "" THEN EXIT ! if not invoked for a
$! ! specific PID, we're done.
$ GOTO collect_loop ! repeat for next PID
$fin:

Re: wsinc default value

John Gillings — Thu, 17 Nov 2005 21:07:07 GMT

Wim,

SYSGEN Defaults must support ALL possible OpenVMS systems, from a tiny ancient Alpha workstation up to a fully loaded GS1280. They must therefore be conservative and pessimistic.

I think the minimum supported memory configuration these days is 64MB, which is 8192 pages, so increasing the default WSINC to 24000 is NOT a good idea! 2400 seems quite reasonable to me.

In the case you've described, yes, increasing WSINC may be a reasonable thing to do. That's what AUTOGEN is for. To modify defaults to suit your specific circumstances.

I'd also look at creating your processes with explicit quotas, rather than relying on default PQLs.

(IMHO depending on PQL_D or PQL_Ms is a bug. All processes should have explicit quota list).

Expecting a process to fault it's way up past WSQUOTA from PQL_D values for WSDEFAULT is doing it the hard way. If you know in advance what the processes memory requirements will be, TELL OpenVMS up front, rather than complaining that it takes too long for OpenVMS to work out what you already know.

On the other hand, if the system has plenty of memory, it may not make a huge difference in overall performance, as the pages will probably just cycle between process working set, free list and modified page list.

Re: wsinc default value

Wim Van den Wyngaert — Sun, 20 Nov 2005 06:27:24 GMT

Since nobody found any side effects I will give it a try. I already modified it on a 256 MB AS500 without any problems.

John : I whished it were my processes. But modifying something like that in an old application is not easy over here. It's not only for avoiding the PF but to avoid the alarm that is given.

Wim

Re: wsinc default value

Wim Van den Wyngaert — Tue, 22 Nov 2005 04:09:02 GMT

I increased wsinc from 2400 to 24000 on 1 GS160 node. The program has a va size of 735.000 pages on startup (=45Kp).

WSSET sizes (in Kp or 16000 pages) :
Before : 6:55 6 Kp, 8:05 14Kp, 10:00 18Kp
After : 6:55 10 Kp, 8:00 20Kp, 8:40 32Kp. No further increase.

So, the working set grew more rapidly but still to slow. There are less pagefaults but still too many (still alarms).

I will try with 100.000 ...

Wim

Re: wsinc default value

Wim Van den Wyngaert — Mon, 09 Jan 2006 07:56:46 GMT

Seems I forgot to update this one.

The problem is solved with 100.000 pages in wsinc.
No problems seen yet. No alarms anymore of high faulting.

Wim