Semaphore Issue

[Himanshu_3]

Hi all,

I would like to take the advise of group members on one of the errors recently recieved on one of the HP-UX 11.11 D330 system.

The error recieved was as below

sem_create(): can't lock

Standard system error call I guess ?

sia: [Fatal System Error; 04/08/2005 14:31:28]
Can't get/create semaphore. (28; No space left on device)

This is an error recieved when we try to shutdown one application on HP-UX system.
It seems the system is not able to free semaphores .
kmtune output is as below

sema 1 - 1
semaem 16384 - 16384
semmap 502 - (SEMMNI+2)
semmni 500 - 500
semmns 1500 - 1500
semmnu 30 - 30
semmsl 2048 Y 2048
semume 30 - 30
semvmx 32767 - 32767

Also ipcs -sbp gives output as below

T ID KEY MODE OWNER GROUP NSEMS
Semaphores:
s 0 0x412007b2 --ra-ra-ra- root root 1
s 1 0x4e100002 --ra-ra-ra- root root 2
s 2 0x41240d47 --ra-ra-ra- root root 2
s 3 0x00446f6e --ra-r--r-- root root 1
s 4 0x00446f6d --ra-r--r-- root root 1
s 5 0x01090522 --ra-r--r-- root root 1
s 6 0x61240626 --ra-ra-ra- root root 1
s 7 0x73240626 --ra-ra-ra- root root 1
s 8 0x70240626 --ra-ra-ra- root root 1
s 9 0x69240626 --ra-ra-ra- root root 1
s 10 0x75240626 --ra-ra-ra- root root 1
s 11 0x63240626 --ra-ra-ra- root root 1
s 12 0x64240626 --ra-ra-ra- root root 1
s 13 0x66240626 --ra-ra-ra- root root 1
s 14 0x6c240626 --ra-ra-ra- root root 1
s 15 0x6d240626 --ra-ra-ra- root root 1
s 16 0x6f240626 --ra-ra-ra- root root 1
s 17 0x491c1cad --ra-r--r-- root root 1
s 18 0x411001f9 --ra-ra-ra- root root 1
s 19 0x00000001 --ra-ra-ra- root root 1
s 5020 0x52240626 --ra-ra-ra- root root 1
s 8021 0xb57e5548 --ra-ra---- oracle dba 204
s 1522 0xb2fe7568 --ra-ra---- oracle dba 54
s 523 0x66052000 --ra-ra-ra- tstdba dba 3
s 524 0x66002053 --ra-ra-ra- tstdba dba 3
s 525 0x65052000 --ra-ra-ra- dcsdba dba 3
s 526 0x65002053 --ra-ra-ra- dcsdba dba 3
s 27 0x65012000 --ra-ra-ra- dcsdba dba 3
s 28 0x65012001 --ra-ra-ra- dcsdba dba 3


Is it possible to free sempahores in any way ?
I would like to go on for increasing the kernel parameter semmns as a last option.

Thanx,
Himanshu

[saju_2]

Hihimanshu

I have seen my application guys doing ipcs.zap to release the semaphores used by the process started by that particular user.

Regards
CS

[Henk Geurts]

hi
sometimes the application still ownes semaphores.
you can clean with the ipcrm command
example
ipcrm -s 28

regards.

[Bill Hassell]

It is virtually impossible to determine if a semaphore can really be removed safely. Programs use these as flags for various purposes and since they are so small, you should increase the quantity (double or triple) the current value in the kernel. Also, if you kill any Oracle or other application programs using kill -9 then semaphores (and shared memory segments) will be left behind, but without useful flags as to whether they can be removed.

[Rita C Workman]

Proper way you can 'kill' semaphores is via the ipcrm command. As Mr. Hassell says, you can't just kill things or even ipcrm kill things and ALWAYS be certain your getting the right application, when you have mutiple Oracle instances running.

The problem is obviously in the application...if your trying to shutdown one application and it won't...then the appl needs to be looked at. Otherwise, when you need to shut IT down you'll have to shut down everything on that box and reboot, cause your memory will never clean up unless you do. With that scenario, mgmt may support forcing your developers/DBAs to review that code.

Rgrds,
Rita

[vinod_25]

hi himashu

The command:

ipcs -s

will display information about active semaphores.

Ipcrm(1) will remove one or more specified messages, semaphore or shared memory identifiers:

ipcrm -s semid

Remove the semaphore identifier semid from the system and destroy the set of semaphores and data structure associated with it.

bye


Vinod

[Henk Geurts]

hi all
it wasn't my intention to suggest to "kill semaphores" at random.
i meant that if an application is shut down in an hard way , there can still be semaphores which are not 'freed'.
for those the ipcrm command can be used.

[Himanshu_3]

Rita,
Do you thing doing a hardware reboot can free the semaphores
and the problem would not be repeated in future. Is it an one
off event ?
In my case, the oracle id is holding the maximum sempahores.
Can their be any impact on oracle if I do a force semaphore
release using one of the commands listed by forum members.

Thanx,
Himanshu

[Bill Hassell · Accepted solution]

If you have enough semaphores for Oracle and all the other applications, then a reboot will fix the problem. However, this means that you will not add any more applications to your system that may need semaphores (or shared memory, etc). As mentioned, if anyone used the kill -9 command for programs that used semaphores, then those semaphores could be released--but only if you know that they are not used by any other program--virtuallu impossible to determine.

You never want to release a semaphore that Oracle is using. A semaphore is much like a stop light for cars. When the semaphore is set, cooperating processes are prevented from making changes to database elements until the semaphore is unset. If you remove the semaphore with ipcrm, the database can be thrown into chaos, possibly requiring hours of checking, or worse, a complete reload from a backup.

Semaphores (and other kernel parameters) are designed to be adjusted to fit the needs of your system. If you don't have enough iinformation to change the kernel parameters, have HP or a consultant help you. It would be unwise to try to live with a shortage of semaphores (or any other kernel parameter)