Operating System - HP-UX
1748136 Members
3479 Online
108758 Solutions
New Discussion юеВ

Re: HPUX 11.31 -> Disabling Pseudo-Swap??

 
SOLVED
Go to solution
Emil Velez
Honored Contributor

Re: HPUX 11.31 -> Disabling Pseudo-Swap??

pseudo swap does 1 thing.. It allows you to have less physical swap than you have memory and it allows you to use all of your memory before you use swap space.

Once all of your physical memory is used then your system starts paging.

Do not pay any attention to the memory line it is not important. The important thing is how much free memory you have. If you get low on free memory get more or shrink your filecache_max.

normally when a process starts it reserves swap in case memory gets low and it needs to page out. Since you have 38 GB of memory but only 20 GB of swap you could only start 20 GB of processes before the OS would say you cannot start a process. Pseudo swap makes the OS think it has more swap. When you fill up memory and need real swap it will use real swap and then you need more memory since you should not be swapping.

That is it.

David P Lavoie
Frequent Advisor

Re: HPUX 11.31 -> Disabling Pseudo-Swap??

swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 10240 0 10240 0% 0 - 1 /dev/vg00/lvol2
dev 26624 0 26624 0% 0 - 1 /dev/vg01/swap
reserve - 525 -525
memory 46720 8864 37856 19%
total 83584 9389 74195 11% - 0 -
Dennis Handly
Acclaimed Contributor

Re: HPUX 11.31 -> Disabling Pseudo-Swap??

>reserve - 525 -525
>memory 46720 8864 37856 19%
>total 83584 9389 74195 11%

This shows you aren't using device swap and almost everything is in memory. You aren't under memory pressure. Can you do this when you have Sybase running?
David P Lavoie
Frequent Advisor

Re: HPUX 11.31 -> Disabling Pseudo-Swap??

reserve - 36843 -36843
memory 46720 7389 39331 16%
total 83584 44236 39348 53% - 0 -

This is when Sybase is running. Still not using swap.
David P Lavoie
Frequent Advisor

Re: HPUX 11.31 -> Disabling Pseudo-Swap??

Forgot to tell you that in this case, the mlock was probably disabled or not very high. When the problem occured, they were trying to lock 37 GB as usual and it failed.
Don Morris_1
Honored Contributor

Re: HPUX 11.31 -> Disabling Pseudo-Swap??

As mentioned above, mlock'd memory requires memory swap on v3. Hence even your idle amount above (37856 or 36.9Gb) is insufficient to mlock 37Gb.

You have only three options:

1) Figure out what else is using memory swap (around 7Gb) and reduce that. This is almost certainly kernel consumption [implicitly mlock'd], which means reducing filecache_min / filecache_max might do it -- or creating temporary memory pressure to cause Garbage Collection to run to try to flush the kernel is about the only real option. mlock is not an unconditional call, so the system will not aggressively garbage collect in the hopes of meeting a large lock request -- instead the request is just failed as in this case.

Another possibility would be to configure large amounts of memory as Ejectable if this is a vPar (or if you have this as several Cells in a nPar). The kernel does not consume Ejectable in most cases in v3 -- so you'd be implicitly limiting the kernel. I wouldn't recommend this from what I've seen though -- if the mlock succeeds, it looks like very little spare room exists on the system, so you'd be risking kernel memory exhaustion [pingable hang].

2) Add more RAM, therefore giving more memory swap / lockable memory. You probably only need a couple Gb here, you're just at the boundary.

3) As mentioned, use gWLM/WLM/PRM or Sybase configuration to reduce the mlock request to something the system can more feasibly handle.
PatRoy
Regular Advisor

Re: HPUX 11.31 -> Disabling Pseudo-Swap??

Hey Don.

This might be obvious to you, but not me... a few questions regarding what you said (sorry if this sounds like Unix 101):

=====
1. "Figure out what else is using memory swap..": how would I figure out what's using swap memory here? I have seen howtos like the following to list memory usage by process.. but not sure how to properly interpret this:

UNIX95=1 ps -efo vsz,sz,pid,args | grep -v grep | sort -rnk 1

which will give me an output like :

VSZ SZ PID COMMAND
93312 22304 29794 /sybase/ASE-15_0/bin/dataserver -d/dev/sybase/rmaster -e/sybase/ASE-15_0/install/pptci.log -c/sybase/ASE-15_0/PPTCI.cfg -M/syba
93216 22024 29803 /sybase/ASE-15_0/bin/dataserver -ONLINE:3,0,0x7440a5a2, c000000100000000, 0x8d24d0000, 0x1159
93216 22024 29802 /sybase/ASE-15_0/bin/dataserver -ONLINE:2,0,0x7440a5a2, c000000100000000, 0x8d24d0000, 0x1159
93216 22024 29801 /sybase/ASE-15_0/bin/dataserver -ONLINE:1,0,0x7440a5a2, c000000100000000, 0x8d24d0000, 0x1159
66256 16564 1884 /opt/perf/bin/midaemon
55360 13768 1689 /opt/wbem/lbin/cimprovagt 0 5 10 root SFMProviderModule
46612 3640 1987 /opt/OV/lbin/perf/coda
14196 3524 1687 cimservermain --executor-socket 5
12236 2389 1711 postgres: sysadmin cer [local] idle

=====
2. "reducing filecache_min / filecache_max...": that's where I have a hard time with tunables. Most often, I rather keep the defaults.. unless some particular software (i.e. oracle, sybase, etc.) will implicitly tell me to set a particular value. I just don't know how one can tell what's an "appropriate" value... I'm no engineer here. Just a real junior unix admin... :(

=====
3. "creating temporary memory pressure to cause Garbage Collection to run to try to flush the kernel...": that would force some kind of garbage collection.. freeing up memory swap consumption? That's what you're saying right? How would I do that?

=====
4. "you're just at the boundary...": I had mentioned the fact that more memory would be necessary to be sure to always be able to lock 37G, but if that's not an option, how can I put numbers on this boundary? 30G? 32? 35.. ? How can I ensure the DBAs that mlocking 32G will always work (considering we're not using the server for anything else). What number should I give them?How can I get this number?

So many thanks! We're learning a lot here as we go and even if my head feels like it's gonna blow.... we appreciate so much all responses.

Patrick
Don Morris_1
Honored Contributor

Re: HPUX 11.31 -> Disabling Pseudo-Swap??

1) pstat is your friend (ps wraps pstat -- but doesn't tell you all the raw API will tell you).

Attached is a program (cc +DD32 -D_PSTAT64 -o object_dump object_dump.c) which will report locked pages consumed for either all processes on the system, a particular PID or a particular GID. That said -- I'd still expect most/all of your memory swap consumption before disk reservation exhaustion to be kernel memory allocations. Non-kernel virtual allocations will always try to reserve from disk/FS swap before memory swap, the kernel always reserves memory swap when it allocates since it is considered memory locked.

2) In this case -- I'm talking about a temporary lowering to force a flush (and release kernel metadata). Set both lower, watch for vhand running (or monitor with Glance/kcusage to see when you get down to the new max) and then you can raise them back. This is just an easy way to kick stuff out of the File Cache. Since the defaults are 5% min, 50% max -- setting max temporarily to min (5%) is likely enough.

3) Create a user application which allocates and mlocks X Gb in order to consume free memory, preferably in small chunks to allow for temporarily low memory swap. I've got something I played around with like that a year or so back -- I'll attach that in another message since multiple attachments isn't allowed here.

And before anyone else asks -- yes, more proactively releasing kernel memory or providing an interface/command to flush kernel caching as much as possible is a known enhancement point.

4) The FREE entry for the memory line from swapinfo -atm is your raw boundary. No single mlock request for a new object will be allowed more than that amount (and 37Gb is just above it). I wouldn't mlock more than 36Gb, personally -- leaving the kernel 1Gb of memory swap for new allocations just in case.

You can't give the DBAs a single raw number unless they/you can agree that the idle state before the DB comes up is relatively consistent. Since you do say this is only used for this purpose but originally you stated that memory swap consumption increases while Sybase is down (causing a problem on restart), what I strongly suspect is that something like a backup or other activity is taking place during the Sybase downtime. Said activity is likely then causing more file I/O, increasing the file cache and the metadata associated with it. If that's the case -- you could work with Sybase to understand if it needs the file cache (raw I/O is often done under the databases) in which case, lowering the filecache tunables either before running Sybase or just in general (if the only other real activity is backup -- that tends to not really need the cache even though it can fill it, read-once/write-once activity and all).

Alternately, you could script a tunable change to flush or use the memory hog to regain a known amount of memory, then release it and start Sybase [with a dedicated box, there's little chance of something snitching what you just released].

A better alternative would be if Sybase can be configured to ask for the 37Gb (fine and dandy) but mlock in smaller increments such that it handles a failure better. (It seems overly optimistic to me to be so "all or nothing" here... certainly mlock'ing 1Gb at a time and just noting that performance might be lowered if it didn't get the last 1 to 2 Gb would make more sense to me...)
Don Morris_1
Honored Contributor

Re: HPUX 11.31 -> Disabling Pseudo-Swap??

Second program (mem_grab.c based on Scott Robertson's memdetails.c) as mentioned. [No points to this response, please].
PatRoy
Regular Advisor

Re: HPUX 11.31 -> Disabling Pseudo-Swap??

Thanks Don! I'll try to assimilate all this info and look more at what you've given me. I'll bring these up with my colleague...

Thanks so much and I will update this thread as we go :)