Cluster error

[zsujith]

Hi one of the node in a two node cluster got rebooted. In syslog we found following messages.
Jul 8 16:49:27 hp07 vmunix: proc: table is full

Jul 8 16:49:27 hp07 above message repeats 2388 times

Jul 8 16:49:01 hp07 prngd[2180]: Could not fork: Resource temporarily unavailable

Jul 8 16:49:27 hp07 above message repeats 23 times
output of who -b is
reboot system boot Tue Jul 8 16:56

Can any one give the possible reasons and solution to fix this issue.

regards
Sujith

[Rasheed Tamton]

Hi Sujith,

It is discussed many times in the forum. It is normally because of the nproc kernel value.

Please see the thread:
http://forums12.itrc.hp.com/service/forums/questionanswer.do?admit=109447627+1215600049708+28353475&threadId=686615

There ia another one just today:
http://forums12.itrc.hp.com/service/forums/questionanswer.do?threadId=1247779

Regards.

[Rita C Workman · Accepted solution]

It means you could not run or fork another process. You need to increase your nproc parameter.

A good way to monitor for this parm would be using the 'sar' command. Example to check every 3 seconds for the next 20 times:

sar -v 3 20

The output will show proc-sz; inod-sz; file-sz
The first value is what is being used, the second in each column shows what the current parm is set at. You might monitor throughout day and maybe write a short script and cron it to run around the time your box showed the issue. You may find by watching you can determine what the best parm value should be.
Remember - increasing a parm is not always giving you more. You may find it best to increase one, while reducing another will best tune your box. It takes a good bit of research on the parms, and a good bit of watching and tuning.

Rgrds,
Rita

[zsujith]

Thank U