Runaway 'registrar' process from resource monitoring

[Bill Barto]

We have a ServiceGuard cluster of two HP servers running 11.0 (not 11i). On the "backup" machine which is not running anything other than standard "background" processes, something is causing 100's of registrar processes to be kicked off (/etc/opt/resmon/lbin/registrar which is started by inetd) and filling the process table.

This happened last week and we were unable to login to the machine because the process table was full. This week we left a terminal logged in so we could see it when it happened.

We saw in http://docs.hp.com/en/B7609-90055/ch01s07.html that a newer version of the resource monitoring tools apparantly fixes a problem that could cause this many registrar processes to start, but the cause in that case was a "heavily stressed" system. This system is basically idle waiting in case ServiceGuard needs to move an application package from the primary machine.

What could cause the excessive number of registrar processes on this unloaded machine?


Thanks in advance for any help someone can provide!

[Rita C Workman]

You probably need a patch for this...but..

... we had a problem like this along time ago here. We run 11.11 with EMC Storage & MC/SG. Not sure about your environment or what effect what we did might do to help.

We found out that it might happen when the lock files get removed from /etc/opt/resmon/lock/persistence.lck or register.lck.

Here are my old notes on what we did to address it here (till we got latest patches)

1. run /etc/opt/resmon/lbmin/monconfig and select kill
2. run /sbin/init.d/diagnostic stop
3. kill p_client (rem out of /etc/inittab temporarily
4. restart-run /sbin/init.d/diagnostic start
5. go back and un-rem out of /etc/inittab and run init q
6. restart-run /etc/opt/resmon/lbin/monconfi
and hit start

Just what we did....
Rgrds,
Rita

[Bill Hassell]

> What could cause the excessive number of registrar processes on this unloaded machine?

A bug in the online diagnostics code. The diags are quite complicated and require frequent updating. Just download the latest version. The registrar process is failing to work correctly and the exact reason is not important. Although 11.0 is obsolete and no longer supported, this does not mean you can forget patching. There are still some patches being released that should be applied.

[Bill Barto]

Thanks for the responses. Turns out a hard disk was on the verge of failing. Once it actually failed and was swapped out (easy to do with MirrorDisk and hotswap hardware) the problems went away.

I think Bill Hassell's comments were probably right (out-of-date online diagnostics patches). We're upgrading the whole system soon to new hardware and 11i, so we'll side-step the patch issue for now.

Thanks again!