iPlanet 4.1 SP11 problem

[Srinivas_3]

My system Config: OS 11.00 QPK Sep 2003/
L - Class/2 Proc/2GB Memory/ iPlanet 4.1 SP11 and we us https 128bit encryption.

Problem:

Some times I get this error " could not fork new process" and I just restart the web server instance, the problem goes away. But again it repeats after few days. Below is the exact error information.

I tuned the kernel, my current kernel settings are:

STRMSGSZ 65535
dbc_max_pct 20
max_thread_proc 900
maxfiles 1024
maxfiles_lim 2048
maxswapchunks 3000
maxuprc 6000
maxusers 3200
msgmax 32768
msgmnb 65535
msgmni (NPROC)
msgseg (MSGTQL*4)
msgssz 128
msgtql 7564
nflocks 8192
nkthread (((NPROC*5)/3)+16)
nproc (64+3*MAXUSERS)
nstrpty 60
nstrtel (MAXUSERS)
semmns (SEMMNI*2)
semmnu (SEMMNI/2)
semume 64
shmmax 0X08000000
shmmni 964

Exact error:

/43288422, cgieng_start_output reports: could not fork new process (exec() failure [Bad address])
[28/Oct/2003:17:09:10] catastrophe (28106): for host 203.126.126.250 trying to POST /cgi-bin/ocbcro/vpu/fortec
gi/43209036, cgieng_start_output reports: could not fork new process (exec() failure [Bad address])
[28/Oct/2003:17:09:16] catastrophe (28106): for host 203.125.52.137 trying to POST /cgi-bin/ocbcro/vpu/fortecg
i/43746715, cgieng_start_output reports: could not fork new process (exec() failure [Bad address])
[28/Oct/2003:17:09:24] catastrophe (28104): for host 202.42.186.172 trying to POST /cgi-bin/ocbcro/vpu/fortecg
i/43337936, cgieng_start_output reports: could not fork new process (exec() failure [Bad address])
.....................
....... etc...

Please adivce me if anybody had seen this problem before.

Thanks,

Sri

[Elmar P. Kolkman]

Next time before shutting down the web server, take a look at your process list to see if anything is hanging.
It seems it wants to start CGI processes which fail. So it could be, for instance, that your CGI scripts/programs are not finishing, meaning they stay in the process list, which results in to many processes.

Try a 'ps -u | wc -l' where you put the username under which your iplanet processes run and see if the result of this is high (at the moment it won't work, it should be around 6000!)

Another reason could be a memory leak, meaning the iPlanet processes don't free memory allocated during the running. Just today someone posted a script to calculate the memory usage of a users processes.

[Srinivas_3]

Hi Elmar,

Thanks for your reply. Surprisingly system health looks fine when the problem happens. CPU utilisation is very normal, and ps - u |wc -l is less than 50, which is fine I guess. Syslog doesn't have any strange errors either.


Sri