Did some testing of pagefile usage under 7.3

[Wim Van den Wyngaert]

Not a question, just a report.

I wrote a program that mallocs 200 MB of memory and makes it dirty by filling it with '0'.

I ran it on a AS500 with 256 MB memory and a pagefile of 62469 pages (=1.000.000 pages = 512 MB). I ran it 1 at the time and when it succeeded it stayed alive with the memory allocated. Then I started the next.

Each time the commited pagefile usage as seen with sh mem/fi increased with 200 MB. The free space also decreased with 200 MB but only after physical memory was exhausted.

Before the 4th run, only 972 pages were free. The malloc of 200 MB succeeded and the making dirty started.

The system went into hang for about 30 minutes and not 1 error message.

I investigated with AMDS and found all processes trying to do something in state FPG (waiting for free pages).

After 30 minutes, the workstation was still not dead but it did operations very very very slowly. Control T took about 15 minutes to complete.

I killed 1 process with AMDS and after 30 seconds of high swapper activity the station was running again. But not normal. Several processes were regularly in MWAIT and after 15 minutes DECW was still not reacting. I killed a 2nd process (also one with 200 MB allocated) and only then normal activity was restored.

Nothing special in operator log file.

I find it strange that killing 1 process didn't solve the memory problem.

I have no "playstation" in 6.2 but I a quick test revealed the same behaviour under 6.2. But the system reacted less dead that 7.3.

Wim

[Antoniov.]

Hi Wim,
very interesting!
You have 256+512 Mb of virtual memory.
After 3 running process, each use 200 Mb you allocated 600 of 750 Mb; in effect you see 972 page free and this may be normal.
You run 4th process and it try to allocate more memory than it is avaiable.

It's very strange your process recevice successfull (dirty memrory) allocation !!
It's more strange and dangerous killing 1.st process can't free memory.
How many time did you wait for after 1.st killing? May be system can resume normal activity after 1 minutes? I remember on some old vax in similar situation, machine resumed normal activity after about 1 minute.

Antonio Vigliotti

[Marc Van den Broeck]

Wim,

as far as i recall it has always been that way (for at least 20 years).
But i used to see (opcom?) messages on the console such as:
Free page file size low!
Free page file size critical!

and then the system entered a 'hung' state from which you had to reboot or you had to free memory by killing/ending processes and being very patient.

Rgds
Marc

[Wim Van den Wyngaert]

Further testing.

Started a program when pagefile was nearly full. Before the malloc, I created a 2nd pagefile. The dirty memory was written to both pagefiles. Idem but pagefile added between malloc and making memory dirty : same reaction. Excellent.

Remark : a reboot is now needed to remove a pagefile since all processes directly start allocating pages in it.

Remark 2 : unbelievable how long it takes to kill processes when memory is unavailable (RWMPB, page writer busy).

Wim

[Wim Van den Wyngaert]

Antonio,

Killing a process takes at least 5 minutes on my alphastation 500. Remember that 200 MB of pagefile must be erased but that still doesn't explain why it takes that long. Maybe because at the same time it has to reoganize memory assigned to other processes ?

Marc,

Not a single message ...

Wim

[Wim Van den Wyngaert]

For those that are interrested in how Linux reacts : http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=855513 somewhere at the end.

Wim

[Antoniov.]

Wim,
don't forget your memory is also used by cache and XFC works differently by previous vms version (like V6.2). How cache can interfere with pagefile?

Antonio Vigliotti

[Wim Van den Wyngaert]

Tried to add a 2nd pagefile while the pagefile was full and a process was trying to put dirt in the pages.

Command didn't complete after 10 minutes. I killed (with AMDS) a smaller user process that consumed 10000 pages (of 512). Took 2 minutes to remove the process. Pagefile still not added.

Killed a process holding 4000 pages. Removed after 30 seconds but pagefile still not added. Killed a 3rd with 4000 pages. Pagefile finally added. And used. Excellent except for the time to do it.

Wim

[Uwe Zessin]

Antonio,
a process can request memory against its own pagefile quota. I have never heard that the system checks against what is still free in the pagefile(s).

It's called overcommitment. Else, you would have to create enough pagefile space for _each_ process' pagefile quota.

[Wim Van den Wyngaert]

While system was back normal : removing a process that had 200 MB dirty pages took just a second (a short disk activity was noticed).

While system had exactly 0 pages free but no dirty program was busy : any activity caused lots of mwaits (rwmpb) and the session no longer was usable. Had to delete 7 smaller processes before going back in operation. And still processes in mwait.

Put my prio on 20 while pagefile was full. Reacted faster. I stopped a process and it was directly gone (??? why that fast this time ??).

Wim