swap usage in V3

[Davis Paul]

Hi all,
Below is the swapinfo output.

# swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 8192 2634 5558 32% 0 - 1 /dev/vg00/lvol2
dev 16384 2635 13749 16% 0 - 1 /dev/vg00/lvol10
dev 8192 2634 5558 32% 0 - 1 /dev/vg00/lvol11
reserve - 15251 -15251
memory 15555 4296 11259 28%
total 48323 27450 20873 57% - 0 -

This is in V3 system in which pseudo swap is enabled by default. But the above swapinfo out put shows that using swap space. How to instruct my system to use physical memory? This is oracle database system.


Thanks
Davis Paul.

[Don Morris_1]

You _are_ using physical memory. Overusing it, in fact such that the system had to push 7903Mb out to the swap device. (Used on the dev lines is actual disk space written to, not reservation -- it means you really honestly paged memory out).

You need to add more RAM or reduce the load. Reducing the load may be on the user side (did you set the SGA too large? Running too many DB connections?) or may be on the kernel side (what's filecache_min / filecache_max? Are you using direct I/O for the oracle transactions [in which case, the filecache is less important and should be reduced]? Are you using VxFS and have you checked the value of vx_ninode (see such links as http://docs.hp.com/en/5992-3364/apbs02.html )?

Your memory line does indicate not that much of this should be kernel dynamic memory [so the vx_ninode may be a factor, but not enough of one]. Those allocations will be considered locked and hence borrow from memory swap. With only 4296Mb of memory swap consumed, it isn't 7Gb of kernel consumption that's your problem -- so you should check file cache and user space loads first (and likely the latter -- too big of an SGA or too many clients).

[Michael Steele_2]

Hi

It is better to have everything in primary swap and yours is into 24 gb of 2ndary swap. Not good. However, 8GB is a lot of primary swap. You can play around with this formula and see if changing maxswapchunk helps.

Use this formula for to determine primary swap.

MaxSwap=maxswapchunks*1024*swchunk

Let say I increased maxswapchunk to 2048. BTW swchunk is set at 2048 (no need to change this). With that..
(2048*1024*2048)/1024=4194304 KB
ie 4096MB So the max amount of swap I can configure is 4GB.

vxinnode is not the problem here, his RAM is 16 GB. The forumla provided is for 3 GB.

More than likely your application, is this SAP, has set up a huge amount of cache within the SGA and it needs to be reduced.

I would look at the application being uses and see about reducing its cache, sga, shared memory consumption.

What is po in vmstat? Is is above 0? If so then you are certainly paging.

Adding more memory would make this a faster box if you can't reduce your 2ndary swap.

[Patrick Wallek]

>>It is better to have everything in primary swap

My personal opinion --- Baloney!

It does not matter HOW you lay out swap. If you start paging out to any significant degree, performance will SUCK!

To paraphrase A. Clay Stephenson: Worrying about swap layout is like rearranging the deck chair on the Titanic. It's basically pointless.

[Michael Steele_2]

Really? Since when was disk faster than RAM?

[Michael Steele_2]

Now really Patrick why? I always get confused on this arguement with Clay. He use to go against an HP white paper too as well and now you support him.

[Dennis Handly]

>Michael: Really? Since when was disk faster than RAM?

That's exactly Clay's and Patrick's points. You don't want to swap at all. By the time it decides to use primary or secondary swap, it is already orders of magnitude too slow.

[Michael Steele_2]

WELL REREAD THE THREAD - THE SUBJECT IS TWO SECONDARY SWAP LVOLS TOTALLING 24 GB!!

[Patrick Wallek]

>>WELL REREAD THE THREAD - THE SUBJECT IS TWO SECONDARY SWAP LVOLS TOTALLING 24 GB!!

No, the subject is "swap usage in V3."

The main question is: "How to instruct my system to use physical memory?" Which has been answered.

Unless you're reading something different than I am.

[Bill Hassell]

As mentioned before, whether you have 20 separate swap areas or one large one, when it gets used, something could not fit into RAM and processes had to be kicked out into swap space. 20 years ago, when RAM was measured in MB, having fast swap was really important because you couldn't afford to have 1000 MB of RAM and you lived with swapping. Today, the internal disks on most servers are much, much slower than fibre disk arrays so technically, using primary swap on vg00 is not a good idea -- assuming you have to page out a lot. In that case, a small primary area (2-4 GB) with a large LUN from a high speed array would be a better choice. But that's because additional RAM is too expensive for your system and therefore maximizing swap speed is the primary objective.

Of course, disk speeds are meaningless if you are not using swap space except for reservation. You don't want processes to ever be paged out if performance is a important.

Now that said, you might have a database engine that fits nicely into RAM but there are 500 interactive users all around the world. They type some data into the system every few minutes but their programs are idle (waiting to do something) for at least 50% of the workday. In that case, those programs can indeed be paged out and then recalled when necessary. There is a slight delay (a second or two) as each program is brought back but no delay while it is in use.