Swap/Memory/Oracle issue

[joe clark]

I have an N4000 with 4 gig of memory,with a lot of oracle databases on it, memory usage is constantly at 96%, I have 3 swap areas set up, 1 4gig PRI 1 ( allocation set up as strict/contiguous) and 2 2-gig swaps Pri-0, on same drive, (allocation as strict), setup through sam, since these were not set up as contiguous, as I assumed they would be, could this be causing the memory issues. I know I should combine the 2 swaps and setup a 4 gig on another drive.Also has anyone benched marked bypassing the unix buffer cache with oracle. Thanks in advance

[fg_1]

Joe

Your swap configuration is ok the way it is, secondary swap spaces such as your 2 2GB sizes are ok and do not need to be contiguous to the disk, only the primary /dev/vg00/lvol2 has to be by requirements.

If you are running into memory usages in the high 90's, you may want to take a look at your shm*** parameters in the kernel. If you are running multiple databases, did you account for that when calculating the values of these parameters for the kernel.

How many db's on the system?
What version of the db?

[G. Vrijhoeven]

Hi,

I am not an oracle expert but i do know that using multiple swap spaces is not a bad thing. If it is created on multiple disks / controllers.

Gideon

[A. Clay Stephenson]

Hi Joe:

1) The two swap volumes on the same disk should NOT be at the same priority. The head is moving like a madman between the two areas on the same disk. You might experiment a bit and set one of those to 1 (the same as your primary swap) and the other of those areas on the same disk at either 0 or 2. It is a good thing to interleave the swapping (i.e. those volumes with equal priority) but not on the same physical disk.

2) High memory usage is not always a bad thing as long as you are not swapping very much (i.e. low page out rates).

3) If you are using dynamic buffer cache and the vdbc_max_percent is still at the default 50% then lower it to around 8-10%. I actually prefer to run static buffer cache with byfpages set to about 80000 (320MB) for this size box.

4) You can bypass the UNIX buffers by using the OnlineJFS vxfs mount options onvsync=direct,mincache=direct,nodatainlog,delaylog. I find that these tend to be best for data files and indices and the options nodatainlog,delaylog to be best for redo and archive logs. When combined with the reduction in UNIX buffer cache and the increase in the SGA buffering (because the UNIX buffers are bypassed), I've very often seen increases in the 10-20% range and on a few rare occasions even higher. If this is a pure database server, you can reduce the UNIX buffer cache down to 100MB or so.

5) As in all things, measure - that is your only true guide.

Regards, Clay

[James R. Ferguson]

Hi Joe:

First, if you haven't already done so, reduce your kernel 'dbc_max_pct' to 2-5% if you're running Oracle and let Oracle's SGA do the buffering it's designed to do.

Second, move your secondary swap to a physical disk that is *separate* from the primary swap disk. Ideally, make the size of this secondary swap the same as the primary with equal priority. In this fashion you will interleave swap I/O should it occur.

Third, if you have Online JFS (and are using VxFS filesystems for your Oracld database), mount the filesystems with these options for the best performance:

convosync=direct,mincache=direct,delaylog,nodatainlog

Have a look at the man pages for 'mount_vxfs' for an expanation of these options.

These changes should help your overall performance.

Regards!

...JRF...

[Thierry Poels_1]

Hi,

hmmm little to add to what already has been said...
but if you are talking about "a lot of" databases, and possibly a nice number of users, the 4GB of RAM might be way to little. A good database server needs lots of memory (primarily for db_block_buffers), a powerfull CPU (or better several of them). Next to that a bunch of fast disks would be handy too ;-)

regards,
Thierry.

[Magdi KAMAL]

Hi Joe,

I was facing a similar problem with oracle and the conclusion is :

1. If your memory usage is about 96 %, you should look if your system is swapping or not ! If No then your system is using the max if memory that it found; If Yes then you would look to increase your swap area to 3 times or even 4 times the amount of RAM that you have.
And if so, tray to put them on different disk to maximize parallelisme on disks to win time.

db_max_pct : have a default value of 50 % with hp-ux. These kernel param. need to be adjusted when using oracle ( up to Oracle recommandation, it need to be decreased to 10% ).
If you are using other applications which are using huge FLAT ( sequential ) files, you may need to increase this value to optimize the IOs to be done in memory.

Magdi

[joe clark]

This is primary an oracle database(s) system.
The kernel paramaters are as follows, dbc_min=5,dbc_max=10, shmmax= 2415919104, set high for some oracle 9i issues, may be able to lower it to 2 gig, I thought this was just a fence for large processes. Swapmem is turned on. The only other disks I can use for swap is the EMC,s only 2
local drives, being mirrored with 4 gig of swap on them , once I move the other 2 swap areas with a 0 pri, vs primary with a pri of 1.
The system appears to be using secondary. swapinfo -tam = Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 4200 0 4200 0% 0 - 1 /dev/vg00/ns_swap

dev 2000 485 1515 24% 0 - 0 /dev/vg00/lvol10

dev 2000 480 1520 24% 0 - 0 /dev/vg00/lvol11
reserve - 3172 -3172
memory 3151 934 2217 30%
total 11351 5071 6280 45% - 0 -
I thought I read that if swap isn't contingous, mor memory is used by the system.

[Bill Hassell]

Swap issues about being contiguous and having various priorities is only important when you are swapping like crazy...and in that case, you are no longer concerned about performance because it will be pitiful.

Swap space does only one thing: allows more processes and data to be active at the same time than you have memory. The only time this is good is when the processes are interactive and the 'think' time for the processes is measured in seconds to hours. In that case, the interactive processes can be swapped and no one will notice.

A system with 100% memory usage is just fine and will perform quite well. Just use vmstat -s to look at page out (page in includes normal program starts, so ignore it). If page out is zero or single digits, you're not actively swapping so you can safely reduce your swap space and return that disk area to active use as data.

In other words, for performance, swapping is bad so being concerned about the quantity and locations and layout of swap is a waste of time. As mentioned, with 64bit HP-UX, many Gigabytes of RAM will significantly contribute to the performance of the system, provided you adjust Oracle's SGA to take advantage of unlimited shared memory.

[joe clark]

Performance on the system does seem ok, however
when using some of oracle's tools, example enterprise manager, it is locking up, assuming because of not enough memory available, will reseach with vmstat tomorrow as dbadmin runs process.

[A. Clay Stephenson]

Hi Joe:

I rather doubt that Enterprise Manager locking up has anything to do with memory. Last year, one of our DBA's attended the Enterprise Manager class and had to be given a full refund because the thing kept locking-up/crashing in class. Everybody's was doing it; even those on different servers. I would check for the latest patches for OEM before doing anything else. As others have mentioned, 4GB is not a lot for what you are trying to do; if you think that memory is killing you simply add some filesystem swap at low priority
to give yourself some virtual memory headroom. It may not be fast if swapping but it shouldn't hang the program.


Regards, Clay

[Dennis J Robinson]

Your oracle database server should not be allowed to swap.

Have the Oracle dba's downsize their Oracle buffer cache so that the total memory utilization is no more than 80%.

It is OK to configure the multiple swap area's.