System Administration
cancel
Showing results for 
Search instead for 
Did you mean: 

Catastrophic error #21015: Out of memory

Nandisha
Occasional Advisor

Catastrophic error #21015: Out of memory

Build of one of oracle (11g) source file gives following error:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Catastrophic error #21015: Out of memory

1 catastrophic error detected in the compilation of "kdli.c".
Compilation terminated.
gmake: *** [kdli.o] Error 4
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I am using following compiler version:
$ ... ansic/bin/cc -V
cc: HP C/aC++ B3910B A.06.14 [Feb 22 2007]

Any idea why this error? Is there any known is reported on that compiler?
15 REPLIES
Steven E. Protter
Exalted Contributor

Re: Catastrophic error #21015: Out of memory

Shalom,

I'd take the error on its merits.

as root, check swapinto -tam
Lets see what the memory situation actually is.

I'd also check the system is patched appropriately, the error could come from not meeting Oracle patch requirements.

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
Bill Hassell
Honored Contributor

Re: Catastrophic error #21015: Out of memory

Building Oracle normally requires enormous amounts of memory (dozens of GB), much more than the running Oracle. Your installation notes from Oracle should indicate how much memory you will need. Note that this does not have to be real memory, it can be mostly swap space. This is temporary for the build process -- use swapon to add an unused lvol while building. After reboot, the lvol will be returned to unused state.


Bill Hassell, sysadmin
Dennis Handly
Acclaimed Contributor

Re: Catastrophic error #21015: Out of memory

If you are building (and working for?) Oracle, you should be contacting HP directly, not asking here. (Unless you know that most of HP is closed for the end of the year. :-)

>Catastrophic error #21015: Out of memory

Since you aren't on A.06.15 or later where you you need to configure maxdsiz_64bit to 4 GB or larger, you need to do what SEP and Bill suggests and look at your swap and maxdsiz.
The latest version of aC++ is A.06.24.

>Bill: Building Oracle normally requires enormous amounts of memory (dozens of GB)

This version of the compiler can only use 1 GB at a time.

>Your installation notes from Oracle should indicate how much memory you will need.

To install Oracle shouldn't take large amounts of swap, only building it.
Nandisha
Occasional Advisor

Re: Catastrophic error #21015: Out of memory

Thanks for the reply.

I know it's shutdown period for hp folks and not available till 3rd of next month :-)

Looks like we have a good memory. See detail here:

$ swapinfo
Kb Kb Kb PCT START/ Kb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 8388608 9900 8378708 0% 0 - 1 /dev/vg00/lvol2
dev 37109760 0 37109760 0% 0 - 5 /dev/vg00/lvol10
dev 8388608 9756 8378852 0% 0 - 1 /dev/vg01/lvol1
dev 32563200 0 32563200 0% 0 - 5 /dev/vg01/lvol3
Dennis Handly
Acclaimed Contributor

Re: Catastrophic error #21015: Out of memory

>$ swapinfo

Please provide "swapinfo -tam" as SEP asked.
Also provide the values of maxdsiz, maxdsiz_64bit, maxssiz and maxssiz_64bit.

What optimization options are you using?
Nandisha
Occasional Advisor

Re: Catastrophic error #21015: Out of memory

> Please provide "swapinfo -tam" as SEP asked.

Kindly see the same below:
$ swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 8192 0 8192 0% 0 - 1 /dev/vg00/lvol2
dev 8192 0 8192 0% 0 - 1 /dev/vg01/lvol1
reserve - 885 -885
memory 16364 4012 12352 25%
total 32748 4897 27851 15% - 0 -



> Also provide the values of maxdsiz, maxdsiz_64bit, maxssiz and maxssiz_64bit.

$ kctune |grep -i maxdsiz
maxdsiz 2952790016 2952790016 Immed
maxdsiz_64bit 17179869184 17179869184 Immed

$ kctune |grep -i maxssiz
maxssiz 134217728 134217728 Immed
maxssiz_64bit 1073741824 1073741824 Immed
pa_maxssiz_32bit 83648512 Default
pa_maxssiz_64bit 536870912 Default


> What optimization options are you using?

+Olibmerrno +Ovolatile=__unordered +Ofast +Olit=all +DSblended +Onopreserved_fpregs +Onorotating_fpregs +Onofailsafe +Odatalayout +FPd +Ofltacc +O2 +Oprofile=use:.../flow.data

env variable 'PROFILE_ON=use' is set.

Should we reduce/disable optimization? If so, is this will not create any unexpected results/probolems?

Note:-
Actually, this issue (Catastrophic error #21015: Out of memory) is start coming after applying one of the oracle patch which is adding fix code changes to xyz.c source file. And then the issue is coming during compilation of xyz.c.
Dennis Handly
Acclaimed Contributor

Re: Catastrophic error #21015: Out of memory

>total 32748 4897 27851 15%

Now, you have plenty of swap.

>maxdsiz 2952790016

This is more than ecom can use.

>maxssiz 134217728

This is way too much. You could recompile by first using:
ulimit -s $(( 40 * 1024 ))

>Should we reduce/disable optimization? If so, is this will not create any unexpected results/problems?

You'll have to do that. Either that or go to the latest, A.06.24, where there is no limit.

Otherwise dropping the optimization level may slow down the application.
If you aren't using the very latest compilers, you should remove +Onofailsafe.

>after applying one of the oracle patch which is adding fix code changes to xyz.c source file. And then the issue is coming during compilation of xyz.c.

If you know the function(s) that changed, then you can either split them up or use:
+O1=function1,function2
Nandisha
Occasional Advisor

Re: Catastrophic error #21015: Out of memory

> ulimit -s $(( 40 * 1024 ))

$ ulimit -s
131072

$ ulimit -s 40960
$
$ ulimit -s
131072

The value 40960 is not taken effect for stack size change. Any idea why is it so?

Oracle 11gR1 is recommended A.06.14. So, I don't think we can upgrade to latest which is A.06.24.

Trying out one of other suggested options i.e., removing 'Onofailsafe' and compiling the source. Will update the results.
Nandisha
Occasional Advisor

Re: Catastrophic error #21015: Out of memory

I am able to build my source without +Onofailsafe option.

Is this okay to use as a workaround? OR is it worth while to try other option? i.e.,
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you know the function(s) that changed, then you can either split them up or use:
+O1=function1,function2
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dennis Handly
Acclaimed Contributor

Re: Catastrophic error #21015: Out of memory

>I am able to build my source without +Onofailsafe option.

Do you get a warning saying the compile is restarting at a lower opt level?
(In future compilers, we will no longer do that if we run out of memory.)

>Is this okay to use as a workaround? OR is it worth while to try other option?

If it drops the opt level of the whole file then you'll probably get better performance if you limit the opt level drop with:
+O1=function1,function2
Nandisha
Occasional Advisor

Re: Catastrophic error #21015: Out of memory

1.
After removing option +Onofailsafe, it builds with folloiwng warning:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Warning #21015-D: Out of memory

Warning #20013-D: Optimization aborted in function unknown. Attempting
compilation unit again but dropping the optimization level to +O1.
Please report this warning and error 21015 to HP. (20013)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

2.
I directly lowering the optimization from +O2 to +O1. With this I am able to build required object file.

3.
Then I tried to include almost all changed procedure to +O1 (+O1=func1,func2, func3, etc) in the compiler line. But, still I got same Catastrophic error #21015. So, I don't think it's too effective.
Dennis Handly
Acclaimed Contributor

Re: Catastrophic error #21015: Out of memory

>After removing option +Onofailsafe, it builds with following warning:

Ok, the whole file is dropped to +O1 and unfortunately it doesn't list a particular function name. Do you have lots of data initialization?

>I directly lowering the optimization from +O2 to +O1.

That's the same as +Ofailsafe.

>3. Then I tried to include almost all changed procedure to +O1 (+O1=func1,func2, func3, etc) in the compiler line.

You may have to include ALL functions and then binary search though them to remove them.

Or break up any large functions.
Javed Khan_1
Valued Contributor

Re: Catastrophic error #21015: Out of memory

also look at the space available in /tmp
Never Give Up
Dennis Handly
Acclaimed Contributor

Re: Catastrophic error #21015: Out of memory

>Javed: also look at the space available in /tmp

Why? It would be illegal for the compiler to use space in /tmp/. It uses /var/tmp/ or $TMPDIR and then very very rarely. Use "cc -v".
error #21015: Out of memory is for swap space or maxdsiz and not disk space.
Nandisha
Occasional Advisor

Re: Catastrophic error #21015: Out of memory

Thanks..Will check this.