Databases
cancel
Showing results for 
Search instead for 
Did you mean: 

Oracle 9i on HP-UX 11i chews up all the Virtual Memory and crashes.

Jason Young_3
Occasional Advisor

Oracle 9i on HP-UX 11i chews up all the Virtual Memory and crashes.

Hi Everyone,

Just wondering if anyone would have any idea why I am having problems with Oracle 9i. I am running Hp-UX 11i on a K570 9000 server with 3 CPU's and 2.5Gb of RAM. Currently when Oracle is running it progressivly uses more and more virtual RAM until there is nothing left and the server crashes (We have 2Gb of Virtual RAM). We have all the "recommended" patches loaded on the server. Any suggestions would be appreciated.

Regards

Jason
11 REPLIES
Michael Steele_2
Honored Contributor

Re: Oracle 9i on HP-UX 11i chews up all the Virtual Memory and crashes.

This is the classic definition of a memory leak. "...Currently when Oracle is running it progressivly uses more and more virtual RAM until there is nothing left and the server crashes...".

Note these kernel parameters:

bufpages, maxssiz, maxtsiz, nbuf, shmmax

...and post there values here.

Do you have glance installed?

You have to begin a process identification of suspected memory leak culprits. Set up a cron with glance to read the value of "Total VSS' (* total virtual statck size *) for the id'd process and note if it increases over time. Take hourly readings. Interactively, read TVSS this way:

gpm > reports > process list > double click on process > find total VSS.

To redirect into a file use the glance advisor file, then :

glance -adviser_only -syntax ./proc_mem_leak -j30

-or-

gpm >> /tmp/file

Note these files for examples:

/opt/perf/examples/adviser
/opt/perf/paperdocs/gp/C/adviser.pdf
/opt/perf/paperdocs/gp/C/metrics.pdf
Support Fatherhood - Stop Family Law
Michael Steele_2
Honored Contributor

Re: Oracle 9i on HP-UX 11i chews up all the Virtual Memory and crashes.

Bill Douglass
Esteemed Contributor

Re: Oracle 9i on HP-UX 11i chews up all the Virtual Memory and crashes.

Also I would consider increasing your VM size. 2GB of swap for 2.5GB RAM seems too small.
Brian Crabtree
Honored Contributor

Re: Oracle 9i on HP-UX 11i chews up all the Virtual Memory and crashes.

Jason,

Can you go ahead and post your kernel parameters and either init.ora file or run the following on your database:

column name format a20
column value format a80
set linesize 150
set pagesize 1000
select name,value from v$parameter where isdefault = 'FALSE';

Thanks,

Brian
Jeff Schussele
Honored Contributor

Re: Oracle 9i on HP-UX 11i chews up all the Virtual Memory and crashes.

Hi Jason,

I'm not a DBA & am no Oracle expert by any means, but we faced a similar situation when we went to 11i & 9i from 11.0 & 8.1.7.

Here are some of the things we found:

"Based on Oracle Note #204510.1, starting Oracle 9.0.1, Oracle uses the largest virtual memory page size setting allowable by HP-UX for allocating process-private memory. It is designated by the value "L" (largest), and is currently 1 GB on HP-UX 11i. Before Oracle 9.0.1, Oracle uses the default page size. So there is no problem with Oracle 8.1.7 running on HP-UX11i or HP-UX11.0. Also there is no problem with Oracle 9.2.0 running on HP-UX 11.0 since the largest page size is 64K (vps_chatr_ceilling on host1 and host2 are 64K). I think that if you set up vps_chatr_ceilling to be 64K, there is no need to use "chatr +pd oracle" to reset Oracle page size. But you may need to check what is the impact on other components by reseting vps_chatr_ceilling.

By the way, I just found out that "chatr +pd oracle" changes the executable oracle's permission from "-rwsr-s--x" to "-rwxr-x--x" and non-DBA users can not connect to the database without using connect string.

So I think that changing kernel parameter may be a better approach"

AND

"The following is how the Oracle defines the virtual memory sizes:

In Oracle 9.2.0's env_rdbms.mk:
LARGE_PAGE_FLAGS=`if /usr/bin/getconf KERNEL_BITS | $(GREP) "64" > /dev/null ; then echo "-Wl,+pi 64M -Wl,+pd L -Wl,+padtext 16M -Wl,+paddata 1M" ; \
else echo "" ; fi`

In Oracle 8.1.7.4's env_rdbms.mk:
LARGE_PAGE_FLAGS=`if /usr/bin/getconf KERNEL_BITS | $(GREP) "64" > /dev/null ; then echo "-Wl,+pi 16M -Wl,+padtext 1M -Wl,+pd 64M -Wl,+paddata 1M" ; \
else echo "" ; fi`

In Oracle 8.1.7.1's env_rdbms.mk:
LARGE_PAGE_FLAGS=
(Note: no value)

And then the variable LARGE_PAGE_FLAGS is passed to cc compile like
(output from relinking Oracle 9.2.0 binary)
- Linking Oracle
rm -f /oracle/product/9.2.0/rdbms/lib/oracle
cc -Wl,+s -Wl,+n +DA2.0W +DS2.0 -o /oracle/product/9.2.0/rdbms/lib/or
acle -L/oracle/product/9.2.0/rdbms/lib/ -L/oracle/product/9.2.0/lib/ `if /usr/b
in/getconf KERNEL_BITS | grep "64" > /dev/null ; \
then echo "-Wl,+pi 64M -Wl,+pd L -Wl,+padtext 16M -Wl,+paddata 1M" ; \
else echo "" ; fi` -Wl,-PF,linkorderfile -Wl,+Ostaticprediction +O2 +Omu
ltiprocessor +Oentrysched +Onolimit +ESlit +Olibcalls /oracle/product/9.2.0/rdbm
s/lib/opimai.o /oracle/product/9.2.0/rdbms/lib/ssoraed.o
....

Changing this maker file is another option to fix the problem."

We basically discovered that if you don't change either an Oracle default or a default kern parm then Oracle will suck the system dry in no time flat....sound familiar?
So check the MetaLink for 204510.1

HTH,
Jeff

PERSEVERANCE -- Remember, whatever does not kill you only makes you stronger!
Jason Young_3
Occasional Advisor

Re: Oracle 9i on HP-UX 11i chews up all the Virtual Memory and crashes.

Hi Everyone ... Thanks for all the help so far. The Kernal Parms that we are currently running are:

bufpages = 0
nbuf = 0
maxssiz = 8388608
maxtsiz = 67108864
shmmax = 67108864

I will also post the init.ora file up here as well.

Regards

Jason
T G Manikandan
Honored Contributor

Re: Oracle 9i on HP-UX 11i chews up all the Virtual Memory and crashes.

Other than loading the proper patches for the OS and Oracle,

I would look like

1.As nbuf and bufpages=0,
what is the value of buffer cache on the system i.e.
dbc_pct_max & min.

For the memory of 2.5Gb,make sure the max value does not cross 15%
When the static buffer cache is zero the dbc_pct_max and bdc_pct_min act as dynamic buffer cache.

2.With your memory and swap full,
how many dedicated server processes are started.
With this load I would assume that the server is running with the dedicated server model and not the MTS.

If that is a dedicated server model,for each client connection the server starts a seperate process which uses around 30MB on an average.

If you have lot of connections then you need to check with this and plan to whether switch the model from dedicated server to MTS(Multithreaded model)

3.Do also check your amount of lockable memory.
Make sure that some kernel parameters are high and the amount of lockable memory is less on the server.

This information you can get it in the /var/adm/syslog/syslog.log file


Revert
T G Manikandan
Honored Contributor

Re: Oracle 9i on HP-UX 11i chews up all the Virtual Memory and crashes.

Other than loading the proper patches for the OS and Oracle,

I would look like

1.As nbuf and bufpages=0,
what is the value of buffer cache on the system i.e.
dbc_pct_max & min.

For the memory of 2.5Gb,make sure the max value does not cross 15%
When the static buffer cache is zero the dbc_pct_max and bdc_pct_min act as dynamic buffer cache.

2.With your memory and swap full,
how many dedicated server processes are started.
With this load I would assume that the server is running with the dedicated server model and not the MTS.

If that is a dedicated server model,for each client connection the server starts a seperate process which uses around 30MB on an average.

If you have lot of connections then you need to check with this and plan to whether switch the model from dedicated server to MTS(Multithreaded model)

3.Do also check your amount of lockable memory.
Make sure that some kernel parameters are high and the amount of lockable memory is less on the server.

This information you can get it in the /var/adm/syslog/syslog.log file


Revert
T G Manikandan
Honored Contributor

Re: Oracle 9i on HP-UX 11i chews up all the Virtual Memory and crashes.

Other than loading the proper patches for the OS and Oracle,

I would look like

1.As nbuf and bufpages=0,
what is the value of buffer cache on the system i.e.
dbc_pct_max & min.

For the memory of 2.5Gb,make sure the max value does not cross 15%
When the static buffer cache is zero the dbc_pct_max and bdc_pct_min act as dynamic buffer cache.

2.With your memory and swap full,
how many dedicated server processes are started.
With this load I would assume that the server is running with the dedicated server model and not the MTS.

If that is a dedicated server model,for each client connection the server starts a seperate process which uses around 30MB on an average.

If you have lot of connections then you need to check with this and plan to whether switch the model from dedicated server to MTS(Multithreaded model)

3.Do also check your amount of lockable memory.
Make sure that some kernel parameters are high and the amount of lockable memory is less on the server.

This information you can get it in the /var/adm/syslog/syslog.log file


Revert
Steven E. Protter
Exalted Contributor

Re: Oracle 9i on HP-UX 11i chews up all the Virtual Memory and crashes.

HP recommends a little more virtual swap than you have allocated, if virtual ram is swap.

Go to between 4 Gig and 5 Gig of Swap and try again.

Do you have all the required java patches at http://www.hp.com/go/java ?

You have adequate memory, but Oracle doesn't perform well without lots of swap.

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
R. Allan Hicks
Trusted Contributor

Re: Oracle 9i on HP-UX 11i chews up all the Virtual Memory and crashes.

Hi,

My apologies for not following all the links of the previous respondents, so this item may have already been addressed.....

I have a 4 CPU K460 running HP-UX 11.00. I did all the proper patches and so forth like you. I also ran out of memory and died a horrible death.

Turns out there were memory leaks associated with dbsnmp. I got a patch from Oracle and have be fat, dumb and happy since. I'm not sure of the patch number. I'm at 9.2.0.2 so I expect if you get to there, you should be ok unless it is something different from what I've experienced.

Good luck
"Only he who attempts the absurd is capable of achieving the impossible