- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - HP-UX
- >
- Re: Out of Memory Error
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-15-2008 12:25 PM
тАО06-15-2008 12:25 PM
On one of our HP-UX server, we have got following alerts :
Critical 07:51:22 06/15/08 cntrepa1.na.jnj.com HP VantagePoint Operations OpC Scheduled action NCS_OS_HPUX_CUS_Service_Outage(service_outage_check.sh) for user root failed.
Critical 11:36:12 06/15/08 cntrepa1.na.jnj.com HP OpenView Operations OpC opcle (Logfile Encapsulator) Can't run command /sbin/dmesg - > /var/opt/OV/log/OpC/dmesg.out. (OpC30-127) fork(2) failed; no more memory Not enough space (OpC20-411)
On checking server, I have found that both memory and swap utilization are 100%. There are lots of SAP processes running on the server.
ProcList CPU Rpt Mem Rpt Disk Rpt NextKeys SlctProc Help Exit
B3692A GlancePlus C.03.86.00 15:52:29 cntrepa1 9000/800 Current Avg High
----------------------------------------------------------------------------------------------------------------------------------------------
CPU Util SU U | 25% 26% 29%
Disk Util FVV | 3% 5% 12%
Mem Util S SU UB B |100% 100% 100%
Swap Util U UR R |100% 100% 100%
----------------------------------------------------------------------------------------------------------------------------------------------
PROCESS LIST Users= 2
User CPU Util Cum Disk Thd
Process Name PID PPID Pri Name ( 400% max) CPU IO Rate RSS Cnt
--------------------------------------------------------------------------------
dw.sapRP1_D 7184 21507 154 rp1adm 95.3/77.0 41451.5 3.9/ 1.9 1.18gb 1
dw.sapRP1_D 22651 21507 155 rp1adm 0.7/ 2.6 52499.0 0.1/ 0.7 140.4mb 1
krsd 3267 1 168 root 0.3/ 0.4 57515.0 3.3/ 4.0 120kb 1
vxfsd 40 0 138 root 0.2/ 0.1 19848.2 2.9/ 8.4 1.8mb 23
midaemon 3241 1 -16 root 0.2/ 0.3 39558.2 0.0/ 0.0 22.0mb 3
sh 11735 21741 179 root 0.0/ 0.0 0.0 1.1/ 1.1 212kb 1
dw.sapRP1_D 1847 21507 155 rp1adm 0.0/ 2.0 22876.9 0.0/ 0.5 71.4mb 1
sh 11766 21741 179 root 0.0/ 0.0 0.0 1.1/ 1.1 212kb 1
dw.sapRP1_D 21507 21488 154 rp1adm 0.0/ 0.1 6423.5 0.0/ 0.0 24.8mb 1
dw.sapRP1_D 21685 21507 155 rp1adm 0.0/ 0.1 3377.5 0.0/ 0.0 38.5mb 1
dw.sapRP1_D 21661 21507 155 rp1adm 0.0/ 0.0 141.8 0.0/ 0.0 31.8mb 1
dw.sapRP1_D 21660 21507 155 rp1adm 0.0/ 0.0 325.5 0.0/ 0.0 31.8mb 1
dw.sapRP1_D 15997 21507 155 rp1adm 0.0/ 0.1 303.1 0.0/ 0.0 45.5mb 1
dw.sapRP1_D 8455 21507 155 rp1adm 0.0/ 0.2 2406.5 0.0/ 0.0 65.6mb 1
dw.sapRP1_D 21679 21507 155 rp1adm 0.0/ 0.0 199.5 0.0/ 0.0 34.7mb 1
dw.sapRP1_D 7020 21507 155 rp1adm 0.0/ 0.1 258.1 0.0/ 0.0 63.0mb 1
dw.sapRP1_D 21675 21507 155 rp1adm 0.0/ 0.0 2.1 0.0/ 0.0 31.8mb 1
dw.sapRP1_D 4865 21507 155 rp1adm 0.0/ 0.5 1846.9 0.0/ 0.1 74.8mb 1
dw.sapRP1_D 15161 21507 155 rp1adm 0.0/ 1.0 3612.5 0.0/ 0.1 103.7mb 1
dw.sapRP1_D 21680 21507 155 rp1adm 0.0/ 0.0 9.6 0.0/ 0.0 31.8mb 1
dw.sapRP1_D 21655 21507 155 rp1adm 0.0/ 0.1 6329.9 0.0/ 0.0 31.8mb 1
dw.sapRP1_D 21653 21507 155 rp1adm 0.0/ 0.4 22546.0 0.0/ 0.1 31.9mb 1
Because of this I am not able to execute some of the commands as out of memory error is coming.
Please suggest what need to be done in this case.
Thanks a lot for your help !
Solved! Go to Solution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-15-2008 02:39 PM
тАО06-15-2008 02:39 PM
Re: Out of Memory Error
As of now what i can suggest is ask SAP team to shutdown the application and see if you have some free memory.
If this does not help , reboot seems to be the only option.
ALl the error your pasted is just because there is no memory for anything.
BR,
Kapil
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-15-2008 03:11 PM
тАО06-15-2008 03:11 PM
Re: Out of Memory Error
Best as suggested it to shutdown the SAP application and see if the system is stable.
Cheers
Rajeev
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-15-2008 07:58 PM
тАО06-15-2008 07:58 PM
Re: Out of Memory Error
You may have to add more device swap.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-15-2008 10:07 PM
тАО06-15-2008 10:07 PM
Re: Out of Memory Error
If it is normal for the processes/system consider adding more memory to the system
also add additional swap space, there should be swap space requirements for SAP, SAP should give such recommendations.
As already mentioned no more processes can be created because there is a lack of memory and at the process creation the reservation of memory/swap fails.
Try first with adding more swap space.
Check the buffer cache, may be the kernel tunable is with a high value and it can be lowered in order to free some spaceWhat is the OS version?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-16-2008 01:47 AM
тАО06-16-2008 01:47 AM
Solution>dw.sapRP1_D 7184 21507 154 rp1adm 95.3/77.0 41451.5 3.9/ 1.9 1.18gb 1
What is this process 7184 doing. It is consumong 1.18gb - looks not normal comparing to your other SAP processes. If you can identify the process and if it is a normal user process, perhaps you can ask that user to stop the process. Continues the same method with your other big processes, if you still have problems. In that way, you can save a shutdown of the application.
Adding a device swap might help to solve the issue temporarily but then you will get more pageouts because your mem and swap usage is 100%.
As told just give the output of the swapinfo -tam
Regards.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-16-2008 12:25 PM
тАО06-16-2008 12:25 PM
Re: Out of Memory Error
I have contacted SAP team and they recycled the process which was using 1.8 Gb. Now the memory is still 100 % utilized ans swap is 83%.
B3692A GlancePlus C.03.86.00 16:15:51 cntrepa1 9000/800 Current Avg High
----------------------------------------------------------------------------------------------------------------------------------------------
CPU Util SU | 3% 3% 3%
Disk Util F FV | 5% 5% 5%
Mem Util S SU UB B |100% 100% 100%
Swap Util U UR R | 83% 83% 83%
----------------------------------------------------------------------------------------------------------------------------------------------
GlancePlus Commands Menu
Output of swapinfo -tam is:
# swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 8192 6035 2157 74% 0 - 1 /dev/vg00/lvol2
reserve - 2121 -2121
memory 2962 1066 1896 36%
total 11154 9222 1932 83% - 0 -
Please suggest what needs to be done now as memory is still 100% utilized.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-16-2008 12:29 PM
тАО06-16-2008 12:29 PM
Re: Out of Memory Error
Run vmstat and check the 'po' column. If the value is double digits or greater then you have a fairly serious memory shortage.
# vmstat 5 15
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-16-2008 12:49 PM
тАО06-16-2008 12:49 PM
Re: Out of Memory Error
Thanks alot for this. Here is the output of vmstat command :
# vmstat 5 15
procs memory page faults cpu
r b w avm free re at pi po fr de sr in sy cs us sy id
1 1 0 447584 7083 98 43 14 8 3 0 44 1212 3981 241 4 1 94
1 1 0 447584 6656 804 432 20 0 0 0 0 1136 28552 346 5 7 88
1 0 0 447737 7048 599 241 6 0 0 0 0 1065 23262 246 4 5 92
1 0 0 447737 7048 206 86 2 0 0 0 0 1057 9674 147 0 1 99
1 0 0 209908 7032 67 27 2 0 0 0 0 1046 3372 122 0 0 100
1 0 0 209908 7032 33 15 1 0 0 0 0 1076 3200 123 0 0 99
1 0 0 401410 7032 11 4 0 0 0 0 0 1049 1213 105 0 0 100
1 0 0 401410 6869 21 8 11 0 0 0 0 1064 2886 145 1 2 97
1 0 0 401009 7032 90 12 6 0 0 0 0 1058 2434 142 0 0 100
1 0 0 401009 7032 29 3 2 0 0 0 0 1052 941 105 0 0 99
1 0 0 304623 7032 13 3 1 0 0 0 0 1060 1470 110 0 0 100
1 0 0 304623 7032 4 0 0 0 0 0 0 1036 749 98 1 0 99
1 0 0 492770 7032 5 2 0 0 0 0 0 1042 1169 94 0 0 100
1 0 0 492770 7032 1 0 0 0 0 0 0 1072 836 108 2 0 98
2 0 0 493447 7026 5 2 1 0 0 0 0 1158 1506 145 2 0 98
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО06-16-2008 04:32 PM
тАО06-16-2008 04:32 PM
Re: Out of Memory Error
BUT:
As you add more processes and/or SAP or Oracle are configured to use more RAM and they get busy, performance will be awful as pages have to be moved out to the swap area. If you are not concerned about performance, more swap will be fine. Otherwise, double or triple your RAM size and performance will significantly improve.
Bill Hassell, sysadmin