Too many semaphores!

[Jesus m. Charro]

I'm running on HP-UX 11.0 and I recently installed CiscoWorks2000 products.
Since then, and when I start CiscoWorks daemons, the semaphore table (semmni)
begins to grow until it reaches almost the upper limit (in my case 4096). I
thought it was a bug from the operating system, so I installed PHKL_20901, a
fix that exists for semget().

If I run ipcs -s, all the semaphores uncorrectly allocated have the form:

s133779480 0x00000000 --ra-ra-ra- bin bin

(bin is the user for CiscoWorks processes).

Should I blame Cisco, HP or myself?

Any suggestion would really be appreciated.

Regards,

Jesus.

[Evan Day_1]

It may just be a matter of increasing the semaphore kernel parameters to allow
for additional semaphores. If the programs needs as many as it is allocating,
there's really no way around it.

Looking at the uid/gid on the semaphore (bin/bin in this case) doesn't
necessarily indicate the semaphore was created by the CiscoWorks program, only
that the program that created the semaphore had that uid and gid.

[Jesus m. Charro]

Thanks for your answer, Evan.

The problem is that we have already increased the number of semaphores up to
4096. It doesn't matter how many semaphores we have, CiscoWorks2000 processes
(maybe JRE...) lock almost every semaphore available (4092 is usually the upper
limit).

We have isolated the problem to Cisco processes simply because when the daemons
are stopped, the semaphore table does not grow much more than 100 semaphores,
and when we start the daemons, the number of semaphores begin to grow rapidly.

I will try to patch JRE. Nobody with this same problem?

Regards,

Jesus.

[William Ringler]

Hi, There have been issues with Ciscoworks and the semaphores being used up. Since you have already bumped up the parameters in HP, Check the PRs with Cisco, as this had been documented for solaris based machines as well, and can only assume that Cisco is working on a patch, or new release.

[Alex Glennie]

I've seen reports of this :

there's a bug with Cisco-Works that if it's data is
corrupted it gets into a loop and sucks up all the semaphores...
So no matter how much you increased semmni/semmns
ciscoworks would have taken all the semaphores in its corrupted state.

Bottom Line: CicsoWorks can become corrupt and take up all available
semaphores in a loop, thereby causing other applications such as ITO to
fail with "semget(2) failed - no space left on device"

Check with cisco support .. probably a patch by now

[CHRIS_ANORUO]

Hi,

You can release the semaphores by killing the ones genereated by the cisco works process with this command:

ipcrm -s SID (semaphore ID)
For example: ipcrm -s 133779480

Cheers!

[CHRIS_ANORUO]

Hello Jesus,

Are you still having the same problem?

Chris