topic Re: swap usage in V3 in Operating System - HP-UX

swap usage in V3

Davis Paul — Tue, 02 Feb 2010 21:15:50 GMT

Hi all,
Below is the swapinfo output.

# swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 8192 2634 5558 32% 0 - 1 /dev/vg00/lvol2
dev 16384 2635 13749 16% 0 - 1 /dev/vg00/lvol10
dev 8192 2634 5558 32% 0 - 1 /dev/vg00/lvol11
reserve - 15251 -15251
memory 15555 4296 11259 28%
total 48323 27450 20873 57% - 0 -

This is in V3 system in which pseudo swap is enabled by default. But the above swapinfo out put shows that using swap space. How to instruct my system to use physical memory? This is oracle database system.

Thanks
Davis Paul.

Re: swap usage in V3

Don Morris_1 — Tue, 02 Feb 2010 21:56:47 GMT

You _are_ using physical memory. Overusing it, in fact such that the system had to push 7903Mb out to the swap device. (Used on the dev lines is actual disk space written to, not reservation -- it means you really honestly paged memory out).

You need to add more RAM or reduce the load. Reducing the load may be on the user side (did you set the SGA too large? Running too many DB connections?) or may be on the kernel side (what's filecache_min / filecache_max? Are you using direct I/O for the oracle transactions [in which case, the filecache is less important and should be reduced]? Are you using VxFS and have you checked the value of vx_ninode (see such links as http://docs.hp.com/en/5992-3364/apbs02.html )?

Your memory line does indicate not that much of this should be kernel dynamic memory [so the vx_ninode may be a factor, but not enough of one]. Those allocations will be considered locked and hence borrow from memory swap. With only 4296Mb of memory swap consumed, it isn't 7Gb of kernel consumption that's your problem -- so you should check file cache and user space loads first (and likely the latter -- too big of an SGA or too many clients).

Re: swap usage in V3

Michael Steele_2 — Wed, 03 Feb 2010 02:19:07 GMT

Hi

It is better to have everything in primary swap and yours is into 24 gb of 2ndary swap. Not good. However, 8GB is a lot of primary swap. You can play around with this formula and see if changing maxswapchunk helps.

Use this formula for to determine primary swap.

MaxSwap=maxswapchunks*1024*swchunk

Let say I increased maxswapchunk to 2048. BTW swchunk is set at 2048 (no need to change this). With that..
(2048*1024*2048)/1024=4194304 KB
ie 4096MB So the max amount of swap I can configure is 4GB.

vxinnode is not the problem here, his RAM is 16 GB. The forumla provided is for 3 GB.

More than likely your application, is this SAP, has set up a huge amount of cache within the SGA and it needs to be reduced.

I would look at the application being uses and see about reducing its cache, sga, shared memory consumption.

What is po in vmstat? Is is above 0? If so then you are certainly paging.

Adding more memory would make this a faster box if you can't reduce your 2ndary swap.

Re: swap usage in V3

Patrick Wallek — Wed, 03 Feb 2010 02:38:59 GMT

>>It is better to have everything in primary swap

My personal opinion --- Baloney!

It does not matter HOW you lay out swap. If you start paging out to any significant degree, performance will SUCK!

To paraphrase A. Clay Stephenson: Worrying about swap layout is like rearranging the deck chair on the Titanic. It's basically pointless.

Re: swap usage in V3

Michael Steele_2 — Wed, 03 Feb 2010 06:13:18 GMT

Really? Since when was disk faster than RAM?

Re: swap usage in V3

Michael Steele_2 — Wed, 03 Feb 2010 06:42:20 GMT

Now really Patrick why? I always get confused on this arguement with Clay. He use to go against an HP white paper too as well and now you support him.

Re: swap usage in V3

Dennis Handly — Wed, 03 Feb 2010 08:48:05 GMT

>Michael: Really? Since when was disk faster than RAM?

That's exactly Clay's and Patrick's points. You don't want to swap at all. By the time it decides to use primary or secondary swap, it is already orders of magnitude too slow.

Re: swap usage in V3

Michael Steele_2 — Wed, 03 Feb 2010 13:39:01 GMT

WELL REREAD THE THREAD - THE SUBJECT IS TWO SECONDARY SWAP LVOLS TOTALLING 24 GB!!

Re: swap usage in V3

Patrick Wallek — Wed, 03 Feb 2010 14:17:06 GMT

>>WELL REREAD THE THREAD - THE SUBJECT IS TWO SECONDARY SWAP LVOLS TOTALLING 24 GB!!

No, the subject is "swap usage in V3."

The main question is: "How to instruct my system to use physical memory?" Which has been answered.

Unless you're reading something different than I am.

Re: swap usage in V3

Bill Hassell — Wed, 03 Feb 2010 14:59:25 GMT

As mentioned before, whether you have 20 separate swap areas or one large one, when it gets used, something could not fit into RAM and processes had to be kicked out into swap space. 20 years ago, when RAM was measured in MB, having fast swap was really important because you couldn't afford to have 1000 MB of RAM and you lived with swapping. Today, the internal disks on most servers are much, much slower than fibre disk arrays so technically, using primary swap on vg00 is not a good idea -- assuming you have to page out a lot. In that case, a small primary area (2-4 GB) with a large LUN from a high speed array would be a better choice. But that's because additional RAM is too expensive for your system and therefore maximizing swap speed is the primary objective.

Of course, disk speeds are meaningless if you are not using swap space except for reservation. You don't want processes to ever be paged out if performance is a important.

Now that said, you might have a database engine that fits nicely into RAM but there are 500 interactive users all around the world. They type some data into the system every few minutes but their programs are idle (waiting to do something) for at least 50% of the workday. In that case, those programs can indeed be paged out and then recalled when necessary. There is a slight delay (a second or two) as each program is brought back but no delay while it is in use.

Re: swap usage in V3

Michael Steele_2 — Wed, 03 Feb 2010 16:56:58 GMT

Patrick

there is 8gb of primary swap in ram and 16gb of 2ndary swap on disk. You're comment is "..all swap is baloney...".

Don't you want to provide some kind of arguement that puts all the 2ndary swap into the primary swap by buying more ram.

Cuz I don't see that explanation at all.

Re: swap usage in V3

Davis Paul — Wed, 03 Feb 2010 17:00:27 GMT

Hi All,

Thanks for your quick response. My question is not yet answered.

In the swapinfo output above only 32% of physical memory is used.System is using swap even free physical memory is available. I need my system to use maximum physical memory then swap.Is it possible?.how?..

Re: swap usage in V3

Patrick Wallek — Wed, 03 Feb 2010 17:19:57 GMT

>>there is 8gb of primary swap in ram and 16gb of 2ndary swap on disk.

What?!?!?! "swap in ram"? What the heck are you talking about? There is no such thing!

The PRIMARY SWAP space (/dev/vg00/lvol2 - 8GB) is on disk. The other swap space (/dev/vg00/lvol10 and lvol11 - 24 GB total), which is what I assume you term secondary, is also on disk.

All 3 device swap areas above are also set at a priority of '1' so they will all get used equally.

If you are talking about 'pseudo swap' as swap space in RAM, then that is not swap at all. It is a calculation to make the system **think** there is more swap than there actually is.

>>In the swapinfo output above only 32% of physical memory is used.

If you are looking at the 'memory' line of swapinfo, that is incorrect. The memory line indicates how much 'pseudo swap' is used. It is not an indication of how much physical RAM is used.

>>I need my system to use maximum physical memory then swap.Is it possible?.how?..

It will do that all by itself. There is no parameter to set to tell the system to use 75% RAM before swapping or 100% RAM before swapping.

With the snapshot posted above, it is possible that RAM usage decreased so that you are not using 100% RAM, but until the pages that have been swapped out to disk are requested again, they will remain on disk.

Re: swap usage in V3

Michael Steele_2 — Wed, 03 Feb 2010 17:37:37 GMT

dev 8192 2634 5558 32% 0 - 1 /dev/vg00/lvol2

So this isn't swap. Right. That's your explanation, right? lvol2 is not swap? Right?

God has HP support gone downhill. What a joke.

Re: swap usage in V3

Patrick Wallek — Wed, 03 Feb 2010 18:01:03 GMT

>>dev 8192 2634 5558 32% 0 - 1 /dev/vg00/lvol2

>>So this isn't swap. Right. That's your >>explanation, right? lvol2 is not swap? Right?

NO, that is NOT my explanation. If you go back and READ what I wrote, you will see that.

LVOL2 is definitely swap. BUT it is DEVICE swap. It is on DISK. It is NOT in RAM.

I am still VERY CONFUSED by your "swap in RAM" statement. That makes absolutely no sense to me.

>>God has HP support gone downhill. What a joke.

You think I work for HP? You are mistaken!

Re: swap usage in V3

Michael Steele_2 — Wed, 03 Feb 2010 18:33:02 GMT

My personal opinion --- Baloney!

It does not matter HOW you lay out swap.

Did you say all swap is baloney or not?

Re: swap usage in V3

Patrick Wallek — Wed, 03 Feb 2010 19:07:10 GMT

>>My personal opinion --- Baloney!

>>>It does not matter HOW you lay out swap.

>>>>Did you say all swap is baloney or not?

No, I did not say "all swap is baloney". What I said, or tried to say, is that having all swap as PRIMARY SWAP, ie. one large swap volume, is baloney.

You said: "It is better to have everything in primary swap"

That is what I consider baloney.

Re: swap usage in V3

Michael Steele_2 — Wed, 03 Feb 2010 19:14:11 GMT

Just so you understand, Patrick, you need lvol2 to boot up, right? And you can't boot up without swap, right? So I guess its not all baloney is it? You need swap to boot up.

Yes or no?

Re: swap usage in V3

Patrick Wallek — Wed, 03 Feb 2010 19:20:11 GMT

>>you need lvol2 to boot up, right?

Yes

>>And you can't boot up without swap, right?

Yes.

>>So I guess its not all baloney is it? You need swap to boot up.

I never said you do not need swap. I never said to get rid of LVOL2.

Again, you said: ""It is better to have everything in primary swap"

That is the statement I disagree with.

For all I care you **COULD** have 24 GB of primary swap (LVOL2) or you could 1 GB of Primary swap (LVOL2) and 23 GB of secondary swap. I don't care. BUT, worrying about what layout will give the **best performance** when paging out is what I have a problem with.

And you still have not explained your "8gb of primary swap in ram" statement.

Re: swap usage in V3

Michael Steele_2 — Wed, 03 Feb 2010 19:23:47 GMT

It is better to have everything in primary swap and yours is into 24 gb of 2ndary swap. Not good. However, 8GB is a lot of primary swap. You can play around with this formula and see if changing maxswapchunk helps.

Keep it up Patrick.