topic memory issue in Operating System - HP-UX

memory issue

VWR Admin — Thu, 14 Oct 2004 07:34:09 GMT

We recently migrated from a V2600 (32gb ram) to an RP8420 (32gb ram) and seen our memory utilization go up from around 83% on the V to the max (and start swapping) on the RP8420. We have also added 4gb more memory and chewed into that! We ran oracle databases. We basically keep the same kernel params from the V which was running 11.0. Any ideas or experiences from anyone?
Thanks,

Re: memory issue

RAC_1 — Thu, 14 Oct 2004 07:56:49 GMT

Which OS the new server runs??
What is the oracle version that runs on new server?? Is it same as that of old version??

What are the buffer cache settings? What are the SGA settings on new server??

Anil

Re: memory issue

Kent Ostby — Thu, 14 Oct 2004 08:24:31 GMT

Also, please post a swlist -l product | grep PH listing.

That way we can see what patches you already have loaded and what might apply to the situation.

Re: memory issue

Sridhar Bhaskarla — Thu, 14 Oct 2004 08:26:45 GMT

Hi,

Can you post the last two lines in the 'memory' window (hotkey - m ) in glance?.

Compare it with what you had on 11.0 before.

11i does take up little more memory than 11.0 but not in the order of Gigs.

-Sri

Re: memory issue

A. Clay Stephenson — Thu, 14 Oct 2004 08:29:16 GMT

Generally 11.11 is a bit more memory hungry than 11.0 but the differences are not huge. With large systems, I always use static buffer caches by setting bufpages to a non-zero value. You probably want to limit buffer cache to a maximum of about 1600MB -- and that is quite generous. I would next look carefully at the sizes of the SGA's; any gains that you get from buffering in the SGA's (or the UNIX buffer cache) will be completely swamped by the performance hit you take from swapping. Another possible problem might be large memory-mapped files. You really need to use Glance to find where you memory is being used. Without data this is just a stab in the dark. Are you running the same version of Oracle? You should also look at the growth of process sizes over time. You just might have a memory leak that didn't manifest itself until the 11.11 shared libraries came into play. I assume that your 11.11 boxed is patched to the latest QPK level.

Re: memory issue

Fred Ruffet — Thu, 14 Oct 2004 08:43:31 GMT

I come again with the same hint on this kind of problem :)
Oracle and I/O Cache.

Both OS and Oracle uses a cache for disks IO. Double cache implies memory overload and bad IO rate.

Ways to solve the problem :
. Mount the FS with Oracl DB files with convosync=direct, mincache=direct and nodatainlog options. First two options will bypass the OS IO cache.
. Tune your OS kernel parameters dbc_min_pct and dbc_max_pct. They rule the IO cache size. They default on 50% for max value, so that, Oracle running, your cache will soon use 16GB !!! Reducing values with such a memory will surely help.

Other points like 11i using more memory are also true (but you said you added 4GB RAM).

Regards,

Fred

Re: memory issue

VWR Admin — Thu, 14 Oct 2004 08:50:58 GMT

Yes, you are right about dbc_min/max and for that same reason we have always set bufpages. We currently have it at 50000 which gives us approx 195mb of buffer cache. We are thinking of adding another 4gb ram because when we were at 32gb we experienced about 30% device swap and when we went to 36gb ram we seen about 10% device swap. So maybe another 4gb will eliminate any device swap?

Re: memory issue

Steven E. Protter — Thu, 14 Oct 2004 09:00:10 GMT

Your system needs a complete performance evaluation:

See this doc:

http://www1.itrc.hp.com/service/cki/search.do?category=c0&docType=Security&docType=Patch&docType=EngineerNotes&docType=BugReports&docType=Hardware&docType=ReferenceMaterials&docType=ThirdParty&searchString=UPERFKBAN00000726&search.y=8&search.x=28&mode=id&admit=-682735245+1097762347865+28353475&searchCrit=allwords

Collect some data over time with a script(attached) and your system can be turned into a smoking well tuned machine.

SEP

Re: memory issue

Fred Ruffet — Thu, 14 Oct 2004 09:00:20 GMT

How does it comes to use so much memory ? Once you start system and DB, or after a while, climbing slowly ?

Have you changed OS Version ? (This question have been asked, but I didn't saw answer)

Regards,

Fred

Re: memory issue

Sridhar Bhaskarla — Thu, 14 Oct 2004 09:12:56 GMT

Can you post your 'glance' numbers?. Open 'glance' window and press 'm' to bring up memory window. Then post the last two lines that show something like

Total VM : 243.1mb Sys Mem : 3.99gb User Mem: 285.3mb Phys Mem: 32.0gb
Active VM: 167.3mb Buf Cache: 654.7mb Free Mem: 27.1gb

Above is an RP8400 with 32GB of memory. As you see Sys Mem is only 4 GB. I would like to see how different your output is.

-Sri

Re: memory issue

VWR Admin — Thu, 14 Oct 2004 09:50:43 GMT

We are now running 11.11.

GLANCE MEMORY STATS

total vm: 33.7gb
active: 4.64gb

phys mem: 36gb
user mem: 24gb
sys mem: 4.73gb
free mem: 6.53gb
Buffer Cache:195MB

Re: memory issue

RAC_1 — Thu, 14 Oct 2004 10:02:06 GMT

What many oracle instances you run on this system??
what are the values of shmmax, oracle sga??

Anil

Re: memory issue

Sridhar Bhaskarla — Thu, 14 Oct 2004 10:02:07 GMT

Hi,

I believe things other than the OS also changed. 24GB is used by 'user' processes which is 75%.

You still have 6.5 GB available. So, you are not really bottlenecking on memory. So, the device swap you are seeing may be due to memory mapped files.

So, adding another 4GB is not really going to help as the system is not memory constrained yet.

-Sri

Re: memory issue

VWR Admin — Thu, 14 Oct 2004 14:33:50 GMT

Thta was just a snapshot at that time we did have memory left but now we are SOL. Did find a interesting DOC from Oracle about Large Memory Footprnts and 9i tuning on HPUX in dealing with the virtual memory page size!
Doc id 204510.1