Showing results for 
Search instead for 
Do you mean 

Used Swap keeps increasing

SOLVED
Go to Solution
Regular Advisor

Used Swap keeps increasing

Hi All

I have a gs1280 running Compaq Tru64 UNIX V5.1B (Rev. 2650)

The specs are as follows.
8 x 1150MHz
8GB memory
page size 8 gb

The problem is that the Used swap pages size keeps increasing and does not go down.

It then increases to the point that the system then crashes. I have two other gs1280 with the same os and revision level and they do not do the same thing.

How do i find what is causing this?
vmstat - t shows that advfs is increasing each day by about 41645792. Any help?

At the moment i have to monitor and then schedule down time to reboot cluster when size gets to 70 %

HELPPPPPP.
12 REPLIES
Honored Contributor

Re: Used Swap keeps increasing

If you run the command swapon -s and see a lot of reserved swap space, you could change the swap allocation method changing the vm_swap_eager kernel parameter as follow:

vm:
vm_swap_eager = 0

Create a file called vm.stanza that contais the values described above, then run:

sysconfigdb -m -f vm.stanza

You will have to reboot.

Then, when you run swapon -s, should see something like this:

swapon -s
Swap partition /dev/disk/dsk1c:
Allocated space: 4445750 pages (33.92GB)
In-use space: 305 pages ( 0%)
Free space: 4445445 pages ( 99%)


Total swap allocation:
Allocated space: 4445750 pages (33.92GB)
In-use space: 305 pages ( 0%)
Available space: 4445445 pages ( 99%)


"No reserved space"
Por que hacerlo dificil si es posible hacerlo facil? - Why do it the hard way, when you can do it the easy way?
Regular Advisor

Re: Used Swap keeps increasing

> swapon -s
Swap partition /dev/disk/dsk27b:
Allocated space: 1048576 pages (8.00GB)
In-use space: 326254 pages ( 31%)
Free space: 722322 pages ( 68%)

Swap partition /dev/disk/dsk27d:
Allocated space: 1606147 pages (12.25GB)
In-use space: 326778 pages ( 20%)
Free space: 1279369 pages ( 79%)


Total swap allocation:
Allocated space: 2654723 pages (20.25GB)
In-use space: 653032 pages ( 24%)
Available space: 2001691 pages ( 75%)

The above is the result from the command but i am not sure how changing that paramter is going to help me?
Honored Contributor

Re: Used Swap keeps increasing

It won't help. What is your patch kit version?

dupatch -track -type kit

You should at least be running Patch Kit 4, patch kit 5 is the latest version.

This could be a custom program error that does not releases the memory or uses too much memory. Use ps to find out the memory used by programs.
Por que hacerlo dificil si es posible hacerlo facil? - Why do it the hard way, when you can do it the easy way?
Esteemed Contributor

Re: Used Swap keeps increasing

Wayne,

It sounds like you have a process that keeps growing in virtual size. Maybe this command would help identify it:

# ps -eo pid,vsz,rss,user,cmd|more

Look at the 'vsz' column for a process with a huge size.

Vic
There are 10 kinds of people, one that understands binary and one that doesn't.
Regular Advisor

Re: Used Swap keeps increasing

ps -eo pid,vsz,rss,user,cmd|more
PID VSZ RSS USER CMD
2621440 7.33G 303M root [kernel idle]
2623692 1.30G 2.3M oracle ora_pmon_medp
2623696 1.31G 5.1M oracle ora_dbw0_medp
2623698 1.30G 2.1M oracle ora_lgwr_medp
2623700 1.30G 3.1M oracle ora_ckpt_medp
2623704 1.30G 2.5M oracle ora_smon_medp
2623706 1.30G 1.6M oracle ora_reco_medp
2623708 1.30G 4.0M oracle ora_snp0_medp
2623710 1.30G 4.0M oracle ora_snp1_medp
2623712 1.30G 4.1M oracle ora_snp2_medp
2623714 1.30G 3.5M oracle ora_snp3_medp
2623716 1.30G 4.3M oracle ora_snp4_medp
2623718 1.30G 4.3M oracle ora_snp5_medp
2623722 1.30G 4.3M oracle ora_snp6_medp
2623724 1.30G 4.6M oracle ora_snp7_medp
2623726 1.30G 3.7M oracle ora_snp8_medp
2623730 1.30G 3.9M oracle ora_snp9_medp
2623734 1.30G 4.3M oracle ora_snpa_medp
2623738 1.30G 4.0M oracle ora_snpb_medp
2623742 1.30G 4.1M oracle ora_snpc_medp
2623746 1.30G 4.0M oracle ora_snpd_medp

Here is the command results.
Regular Advisor

Re: Used Swap keeps increasing

Hi
I am on patch kit
- T64KIT0020861-V51BB24-E-20031125 OSF540
- T64KIT0021340-V51BB24-E-20040120 OSF540
- T64KIT0025745-V51BB25-20050531 OSF540
- T64V51BB22AS0002-20030415 OSF540
- T64V51BB22AS0002-20030415 TCR540
- T64V51BB24AS0003-20030929 OSF540
- T64V51BB24AS0003-20030929 TCR540
- T64V51BB25AS0004-20040616 OSF540
- T64V51BB25AS0004-20040616 TCR540
- TCRKIT1000017-V51BB25-20051014 OSF540
- TCRKIT1000017-V51BB25-20051014 TCR5404
Honored Contributor

Re: Used Swap keeps increasing

Try comparing the 'ps -eo pid,vsz,rss,user,cmd|more' output in a span of few seconds (say 30 sec) to see whether any process 'grows' while running.
Respected Contributor

Re: Used Swap keeps increasing

Hi Wayne,

If you configured vfast, that might be the culpritt.
The solution is to install this patch or to install minimum patchkit 4. But it's better to go for patchkit #5 (BL26) which is now already 7 months old.

See patch description hereunder
ECO Name: T64KIT0022663-V51BB24-E-20040615
ECO Kit Approximate Size: 4.08MB
Kit Applies To: HP Tru64 UNIX 5.1B PK3 (BL24)

ECO Kit CHECKSUMS:
/usr/bin/sum results:
36669 4180

/usr/bin/cksum results:
2818409452 4280320

MD5 results:
c722e7a6de730f6c98e7297c0645c87a

SHA1 results:
f156642753bd6b75738f8faebdb07f8eeab17714


ECO KIT SUMMARY:

A dupatch-based, Early Release Patch kit exists for HP Tru64 UNIX 5.1B that
contains solutions to the following problem(s):

A memory leak exists in a routine that vfast uses to merge
extent maps. The routine is not freeing allocated memory in
bucket 1 (32-byte bucket). The problem manifests as an
inordinate amount of memory being used in bucket 1. It
can be identified by issuing the vmstat -M command and
viewing the bytes_in_use column for bucket 1.

This problem only occurs when using vfast. The workaround
is to issue "vfast deactivate dmnName" for all domains
with vfast activated.

The Patch Kit Installation Instructions and the Patch Summary and Release
Notes documents provide patch kit installation and removal instructions
and a summary of each patch. Please read these documents prior to
installing patches on your system.

The patches in this ERP kit will also be available in the next mainstream
patch kit - HP Tru64 UNIX 5.1B Patch Kit 4.
To err is human, but to really faul things up requires a computer
Regular Advisor

Re: Used Swap keeps increasing

Hi We are at patch level 4 already.

There seems to be no evidence of bucket 1 growing dramatically.

Doing comparisons of vm stat -M as well as checking the with the ps command given above.

This has truley got me stumped.
Honored Contributor

Re: Used Swap keeps increasing

You should post the results of the commands when the problem presents. The output of the ps above does not help too much.

Please post two samples, when the system is ok and when you have the problem. Run swapon -s, ps auxw, vmstat 5 5, ipcs -a. Attach the results.
Por que hacerlo dificil si es posible hacerlo facil? - Why do it the hard way, when you can do it the easy way?
Respected Contributor

Re: Used Swap keeps increasing

OK Wayne,
There seems to be a memory leak somewhere.

We must determine first if we have a memory leak in a kernel thread or in a user process.

If you run "vmstat -P" , you can monitor the amount wired pages and the breakdown of the Wired pages.

Run following script for a day or so

while true
do
vmstat -MP >> /var/tmp/memory_log
sleep 1200
done


If you see now increase in the wired pages, then we have a memory leak in kernel.

In that case, the "ps" command and others are of little use.

The best way to proceed then is:

- open a case at your HP Support Centre.
- Force a crash or run dumpsys in order to have something to be analysed.

If the wired pages, are not increasing. Then we have a user process, not releasing its memory.
You can check with "ps aux" and others how the processes VSZ and RSS are growing.

Good Luck
To err is human, but to really faul things up requires a computer
Regular Advisor

Re: Used Swap keeps increasing

no sulution found.
//Add this to "OnDomLoad" event