Swap, psuedo swap and glance

[Stephanie Nicholls]

I have an hpux 11.00 L Class server with 3Gb running 3 oracle 8.1.6 instances with a total SGA size of ~1.1Gb.

This server is currently paging out to device swap heavily.

When I look at Glance App's the total RES MEM used is about 2.5Gb however swapinfo tells me that my total used swap is 4.2Gb (this peaked at 5.8Gb on a busy EOM day!) What am I missing ?

How do I work out what is being swapped out to disk?

In addition async io for oracle seems to have been disabled in the past, I am planning to reenable it however when I did this on the test box the amount of psuedo-swap used increased dramatically, from 30% to 75%, and according to a few threads Ive read it seems if psuedo-swap is 100% full out of memory errors are being encountered. I cant afford to run into a problem like this on the production server. Any thoughts or comments ?

Swapinfo -tam, glance app page and ipcs -mob stats attached.

[Scott Van Kalken]

It seems like the bulk of your work is being performed by people doing stuff on oracle.

I had a similar situation last year (not with oracle) and I shoved another gig of ram in.

I suspect that a combination of things could help.

I know we have an N4000 oops rp7400 that's got about 23 8.1.6 databases on it.

It's got 3GB of ram
shmax is set to 0x40000000 which is about 1.1GB.

While about half of these databases are dinky little things, the others are quite large, with the smallest being 10GB in size.

I would definitely check shmax and related parameters.

Remember that memory on an HP is limited to 4.75GB on HPUX 11.00 due to four quadrant memory architecture.

This would be excessive though.

You may in fact be seeing swapping as a symptom.

If CPU(s) are under heavy load and spending a lot of time context switching then you can see swapping starting to occur. This sounds weird, but I've seen it before.

Setting the fs_async kernel parameter is generally not a good idea unless you're using raw volumes. I've seen this lock up a box to the point of not being able to log in.

If pseudo swap fills up you're in trouble as no new processes will be able to be created (got caught out with this one 12 months ago).

You've also got 92% of your primary swap space used. To me this says that you're under memory pressure.

Scott.

[Patrick Wallek]

It appears that you are definitely under memory pressure. I would seriously consider adding more RAM to the machine. Another 3 or 4GB would probably be a good start.

Check out this page at the Kingston memory site:

http://www.ec.kingston.com/ecom/configurator/modelsinfo.asp?SysID=+8812+&distributor=0&submit=Find

Note that a 2GB kit is just $1,215 US. That is pretty reasonable.

Contrary to what Scott says, 64bit HP-UX can handle much more than the 4.75GB RAM that he mentioned. 32-bit HP-UX on the other hand will not see more than 4GB or RAM.

[steven Burgess_2]

Stephanie

Another train of thought

What have you got your buffer cache set to ?

sysdef | grep bufpages
sysdef | grep nbuf

You can see the running total from glance -t

If you are running dynamic buffer cache , have a look at the values

dbc_max_pct
dbc_min_pct

Setting bufpages and nbuf to 0 and the dbc min and max values only allows a % of memory to be used by buffer cache. With Oracle performing read and writes from the database, depending on how your applications are configured this can also cause issues with high memory usage

HTH

Steve