Out of memory

[mehul_3]

Hi there!
Operating System HP-UX11i.
Physical RAM 3GB.
Database Oracle9.2.0.1.0
Two instanaces are running having SGA memory 1425MB and 290MB respectively.

Out of memory error is getting while application runs on frequently basis.
Attached screen shot is for referance.

Waiting for reply.

Rgds,
Mehul

[Joseph Loo]

hi,

any change of reducing the SGA with 1425MB or getting extra memory as 3GB is not exactly a lot? also, r the kernel parameter configure to accommodate 9.2.0.1?

u also sure have lots of unassigned points:

http://forums1.itrc.hp.com/service/forums/pageList.do?userId=CA1190244&listType=unassigned&forumId=1

regards.

[RAC_1]

Reduce the SGA on one of the instances or increase the shmmax a bit.

Also how is swap performing
swapinfo -mat
Also how cache has been set?

[Jean-Luc Oudart]

Hi

It seems yuo're running out of memory.
RAM 3GB
SGA 1715MB
Then yuo need memory for your PGA (oracle processes). When you have too many of them you run out of mempory !
What is the swap area
please post
dbc_min_pct
dbc_max_pct
swapinfo -tam

You will probably have to review SGA size and other oracle parameters unless you can increase the total RAM

Regards
Jean-Luc

[mehul_3]

I checked a swapinfo but I confused how much swap memory is assigned and used from the following output
# swapinfo -mat
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 6144 2467 3677 40% 0 - 1 /dev/vg00/lvol2
reserve - 3673 -3673
memory 2315 996 1319 43%
total 8459 7136 1323 84% - 0 -

[RAC_1]

do not llok at swapinfo. you have set kernel tuanable swapmem_on to 1. this is pseudo swap and when set kernel calculates swap space as ->swap space configured + 75% of RAM.

you will have to decrease one of SGA or increase shmmax a bit. In any case, you shuold be adding memeory 3GB is not much of it for oracle.

[Jean-Luc Oudart]

From swapinfo -tam
your are using real swap area, therefore you will get poor performance.
Too big a SGA (!) is not any help if you are swapping out

you should run perfstat report to see which arra can be resized (downward) until you get more RAM (you really need more !)

Regards
Jean-Luc

[Yogeeraj_1]

hi,

you should review your init.ora initialisation parameters. As mentioned about, the statspack report can help you in determining which instance requires more memory than the other.

hope this helps too!
kind regards
yogeeraj

[Bill Hassell]

If the Oracle code is only 32bit (not 64bit), then it ALWAYS be difficult to allocate that much SGA. The reason is that a 32bit application is limited to about 1750 megs for shared memory, and the map that allocates this space is shared with all other 32 bit applications. Therefore, it will become fragmented as you stop and restart various programs. 3Gb is way too small to provide reasonable performance with these SGA settings. But increasing RAM to 6-8Gb won't help with the shared memory fragmentation, it will stop the paging out to swap.

Use shminfo program to show you how the shared memory map has become fragmented:

ftp://contrib:9unsupp8@hprc.external.hp.com/sysadmin/programs/shminfo/

There are several ways to fix the out-of-memory errors:

1. Reboot - the memory map will be rebuilt and everything will fit (not a good choice for a production machine).

2. Reduce the SGA sizes - Oracle will slow down as more activity is shifted from RAM to disk.

3. Upgrade Oracle and any middleware to 64bit execytables - now shared memory has a massively large map and fragmentation is longer an issue.

4. Implement memory windows - this involves creating a separate window for each instance of Oracle (and any middleware that uses Oracle SGA). This eliminates fragmentation because SGA is in a private memory space. This will not improve performance, it will just prevent the out of memory errors. You still need to double RAM for best performance.

[Jean-Luc Oudart]

Bill

Oracle9i is 64bits

Regards
Jean-Luc

[Frank de Vries]

Memory

With 3Giga you should easily be
able to configure 2 Oracle instances.
Enough volume in total.

A very simple rough rule of thumb
to start with - and then after you
can always tune a bit.
Is the rule of a third.

- A third for OS
- A third for PGA and application
- A third for SGA

So if you have 2 Oracle instances,
you could use 1Giga between them.

Please do not oversize on the block buffers.
At a certain point giving more block buffers does not speed up things but just wastes memory. It should not be a bottleneck that is
all. The rest of memory should go to library cache, shared pool and java pool.

Also ensure that the sga memory fits as one
contiguous area into the unix memory :
check with ipcs -m.
If not indeed tune the unix kernel accordingly.

Last but not least ensure the Oracle instances can allocate enough metaphores
for keeping track of its internals.
check with ipcs -s and the kernel

[orapic@orasrv2:]/stand/build<>>> grep -i SEMM tune.h
#define SEMMAP (SEMMNI+2)
#define SEMMNI 520
#define SEMMNS 1024
#define SEMMNU 512
#define SEMMSL_OVERRIDE 2048

If you still have problems let us know
what the error is.

regards

[mehul_3]

I have reduced one of SGA size by 700MB, still the problem persists.

Pl, help me...

Rgds,
Mehul

[Frank de Vries]

Okay,
You have run into a specific memory problem
that is very specific.
There is a constraint on the OS level
that we need to tune.

Three obvious supspects:

1) swap - can you give the output of swapinfo
and swapinfo -t
2) The kernel parameters - also post output
of sysdef | grep max , sysdef | grep sh and
sysdef | grep sem
3) Defracted Unix memory - post output of ipcs -a

Regards,
Frank

[Frank de Vries · Accepted solution]

By the way I found this on metalink,
and it does not look very good.
May not be your fault at all.

Solution Description
--------------------

Check hardware memory and reboot the box. After restart the server,
all databases can be startup.


Explanation
-----------

HP-UX error 12: not enough space usually means there is not enough
real memory or swap space. In this case, it seems hardware has some
problem freeing the memory. Apparently reboot the box fix the problem


Personally I can't believe just a reboot
would solve it, but in essence it could
be that we have to clear the memory manually
with the ipcrm -m command.

Let me know how ou get on.

Regards,

[Frank de Vries]

I found another note that could be interesting:

Issue command ulimit -a
You would see
data :
stack :
memory :
vmemory :
From your sys admin, increase these values to
a reasonable ones.
Should solve your problem.

Hope this is the winning ticket :°)

[mehul_3]

Dear all,
Thanks to all who gave a detailed explanation and updating my knowledge.
Problem has been solved after rebooting a server.

Thanks once again.

Rgds,
Mehul

[Bill Hassell]

Just a note about the errno 12 error message. While it is true that the error indicates not ehough memory, this isn't simply all memory is used, it really means that a request was refused by the OS. The reason can be due to not enough virtual memory (which is RAM plus swap space), but it can also be due to kernel fences or limits to prevent runaway programs. It can also be due (as mentioned before) to memory fragmentation within a limited map. IF it happens again, you can easily verify that it is due to lack of virtual memory be adding a large amount of filesystem swap space while the box is running. Use swapon to do this. Add perhaps 2-3 Gb space and see if the problem goes away. And defeinitely get the maxdsiz value fixed as the strange number from kmtune may be part of the problem too.