cannot fork: no swap space

[Tom Sepka]

I have a L3000 running HP-UX 11 with 4GB mem and 2GB swap. The server is used to run 2 oracle 8I DBs (32bit). The PROD db SGA is about 880MB and the UAT db SGA is about 400MB. To allow both BDs to run simalaniuosly I have implemented memory windows. This has worked in the past on our DEV system.

The problem I am having is when ever we startup the UAT db wierd things happen. The most resent is that the ping command won't work. It gives the following error:

ora-prod:/oracle/orauat/8.1.7/dbs> ping ora-apps
/usr/bin/getmemwindow[29]: cannot fork: no swap space
PING ora-apps.westlongbranch.tellium.com: 64 byte packets
64 bytes from 192.168.28.64: icmp_seq=0. time=0. ms
64 bytes from 192.168.28.64: icmp_seq=1. time=0. ms
64 bytes from 192.168.28.64: icmp_seq=2. time=0. ms


If anyone has an idea why this is happening I would greatly appreciate your help.

Thanks,
Tom

[Jeff Machols]

Tom,

It appears you have run out of swap. to confirm this, run

swapinfo -tam

you will get out put like this

Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 8000 0 8000 0% 0 - 1 /dev/vg00/lvol11
dev 256 0 256 0% 0 - 1 /dev/vg00/lvol2
reserve - 5810 -5810
memory 9817 5816 4001 59%
total 18073 11626 6447 64% - 0 -


If you are at 100% you will need to add swap

[Tom Sepka]

According to this I should have plenty of swap available.

# swapinfo -tan
Kb Kb Kb PCT START/ Kb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 4194304 0 4194304 0% 0 - 1 /dev/vg00/lvol2
dev 2097152 0 2097152 0% 0 - 1 /dev/vg00/lvol14
reserve - 3170380 -3170380
total 6291456 3170380 3121076 50% - 0 -

[Sanjay_6]

Hi Tom,

Try these links,

http://us-support.external.hp.com/cki/bin/doc.pl/sid=7badd33201d78d8d00/screen=ckiDisplayDocument?docId=200000047771538

http://us-support.external.hp.com/cki/bin/doc.pl/sid=7badd33201d78d8d00/screen=ckiDisplayDocument?docId=200000056493259

Hope this helps.

Regds

[James R. Ferguson]

Hi Tom:

Swap space must be available in the event that a process *might* need it. If you are memory rich, you could enable pseudo swap by setting the kernel parameter 'swapmem_on' to one. This can be done with SAM but will require a reboot after the kernel is regenerated.

Otherwise, you should setup some secondary device swap. This can easily be done with SAM if you like. Start by choosing a disk other than one where your primary swap resides (vg00) and configure an equal size device swap with a swap priority that is equal to your primary swap. This will cause interleaving of swap I/O.

To monitor swap utilization, use:

# swapinfo -tam

Regards!

...JRF...

[A. Clay Stephenson]

I suspect that you have not enabled swapmem_on=1. In the case where you have more memory than swap (which is ok) you need to either enable pseudoswap or actually add swap. If you know, you are not actually going to swap you can augment your existing swap with filesystem swap at low priority. If you actually need to swap then devicve swap is your best answer.

[Patrick Wallek]

I concur with JRF and Clay, in that you should make the swapmem_on kernel parm 1 so that it is active.

The thing I see though, is that you say in your original post that you have 2GB of swap, but your swapinfo output shows that you have 6GB of swap. I assume the 6GB is accurate since swapinfo is showing that. The next question is how much memory do you actually have? Is it 4GB or is it more?

[Tom Sepka]

Sorry about the swapinfo it was wrong. Below is the correct swapinfo from this server.

Kb Kb Kb PCT START/ Kb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 2097152 0 2097152 0% 0 - 1 /dev/vg00/lvol2
reserve - 1641728 -1641728
total 2097152 1641728 455424 78% - 0 -

[Patrick Wallek · Accepted solution]

OK. That's better. With only 2GB of swap space, and with swapmem_on set to 0 (more than likely), and with 4GB of RAM, any process that starts that tries to reserve swap space will fail if it exceeds that 2GB that you have when it tries to reserve it.

Your 2 solutions are:

Set swapmem_on to 1 to enable RAM to exceed swap space. This does require a reboot.

Increase swap space to 4GB so it is the same size as RAM. This can be done without a reboot.

[John Bolene]

Turn on psudo swap by setting swapmem_on to 1 in the kernel parms. This will rebuild the kernel and force a reboot.

This will allocate phantom swap so that the need for actual swap space will be much lower.

[Tom Sepka]

I increased the swap space to 6GB, but swapinfo says that 0% is used. Is this normal?

[Patrick Wallek]

You typically should see a 0% usage of swap space. But you will probably see a fairly high number in the 'reserve' line of the swapinfo output.

This is because each process reserves a portion of the swap space just in case it has to swap. It doesn't actually use it unless it has to, which should be never because RAM is so cheap. This is the beauty of the swapmem_on kernel parameter. When you set this, it actually reserves swap space in memory. Keep in mind that it does NOT use memory for this though. This kernel parameter allows you to have less swap space than RAM, thus allowing you to free up precious disk space.

[Bill Hassell]

You won't use swap space until RAM has been exhausted. However, in your listing:

ora-prod:/oracle/orauat/8.1.7/dbs> ping ora-apps
/usr/bin/getmemwindow[29]: cannot fork: no swap space


getmemwindow indicates that you are running memory windows, something that allows old 32-bit applications to address an entire 32-bit address space independently from other applications. This is more likely an issue with the way memory has been laid out for memory windows rather than swap space.

Unfortunately, the error message does not indicate whether memory window #0 (the default 32-bit addressing space) is the culprit. Since you started ping without any memory window directive, this is likely the case. ping will run in the default window and it may indeed be due to the default memory window being used up.

I would recommend that each instance of Oracle be run in a non-default memory window so that the #0 window will have lots of room for ordinary commands.

[Tom Sepka]

Bill,

With 4GB memory can I create more than 1 additional memory window?

Currently I have max_mem_window set to 1. Below is output for this configuation.

# memwin_stats -m
Shared Memory:
T ID KEY MODE OWNER GROUP UserKey KernId
m 0 0x411c0626 --rw-rw-rw- root root Global 0
m 1 0x4e0c0002 --rw-rw-rw- root root Global 0
m 2 0x4120042c --rw-rw-rw- root root Global 0
m 22531 0x5f1e6748 --rw-rw---- oraprod dba 1 2
m 70148 0x930df460 --rw-rw---- orauat dba Global 0
# memwin_stats -w
Entry USER_KEY KERN_KEY QUAD2_AVAIL QUAD3_AVAIL PID REFCNT
Memory Windows:
0 Global 0 262144 152940 10516 212
1 Private 1 0 0 0 1
2 1 2 262144 45095 10541 177
# ipcs -ma
IPC status from /dev/kmem as of Tue Dec 11 13:38:29 2001
T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME
Shared Memory:
m 0 0x411c0626 --rw-rw-rw- root root root root 0 348 353 353 8:58:33 8:58:33 8:58:26
m 1 0x4e0c0002 --rw-rw-rw- root root root root 2 31040 353 355 3:05:12 8:58:33 8:58:26
m 2 0x4120042c --rw-rw-rw- root root root root 2 8192 353 355 3:05:12 8:58:26 8:58:26
m 22531 0x5f1e6748 --rw-rw---- oraprod dba oraprod dba 88 889032704 3211 10559 13:38:28 13:38:28 17:14:24
m 70148 0x930df460 --rw-rw---- orauat dba orauat dba 50 447250432 6586 10516 13:38:11 13:38:11 12:34:10

[Bill Hassell]

You can create a dozen memory windows. The memory window does not allocate 3.75Gb per window--a memory window provides a private addressing space for 32bit programs which can extend to 3.75Gb (not all 3.75 is available to programs), and this is kept separate from other memory windows.

The idea is to keep each instance of a large program separate from another copy so multiple copies do not have to share a single 3.75Gb area. Now having only 4Gb is going to be a big performance hit if you have four instances of Oracle that each need 1200 megs of RAM. The instances will swap like crazy and performance will be awful.

If you are going to use memory windows to allow multiple instances of the program to use all addressable memory, you'll need a lot more RAM to have acceptable performance. If you change the programs to use only 500 megs each, then 4 copies should fit just fine but then memory windows would probably not be necessary.