Operating System - HP-UX
1748227 Members
4445 Online
108759 Solutions
New Discussion юеВ

Re: profile: cannot fork: too many processes

 
SOLVED
Go to solution
someoneElse_1
Occasional Advisor

profile: cannot fork: too many processes

/etc/profile: cannot fork: too many processes




I am not able to login as user scott

su - scott

/etc/profile: cannot fork: too many processes
and session killed


kctune | grep maxu
maxuprc 512 512 Immed

ps -fu scott | wc -l
514

lsof -u scott | wc -l
278877
esxha309 root /home/siebel:lsof -u scott | head -10
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
ksh 4283 scott cwd DIR 128,0x2 1024 55832 /siebel/inst/siebsrvr/admin/IFB/LOG
ksh 4283 scott txt REG 64,0x8 538740 12268 /usr/bin/rksh
ksh 4283 scott mem REG 64,0x8 5020468 48 /usr/lib/hpux32/libc.so.1
ksh 4283 scott mem REG 64,0x8 293088 75 /usr/lib/hpux32/libxti.so.1
ksh 4283 scott mem REG 64,0x8 1407208 72 /usr/lib/hpux32/libnsl.so.1
ksh 4283 scott mem REG 64,0x8 76236 51 /usr/lib/hpux32/libdl.so.1
ksh 4283 scott mem REG 64,0x8 1066624 1615 /usr/lib/hpux32/dld.so
ksh 4283 scott mem REG 64,0x8 177040 70 /usr/lib/hpux32/uld.so
ksh 4283 scott 0r FIFO 0xe0000003725eb608 0t0 8772280

esxha309 root /home/siebel:ps -fu scott | wc -l
514


If I kill these scott owned processes it comes again with new pid
some script is creating them
how to find the script which creates them
most of these scott owned process are or -ksh

Please help me to kill these defunct processes . i can not increase maxuprc.


25 REPLIES 25
Dennis Handly
Acclaimed Contributor
Solution

Re: profile: cannot fork: too many processes

Can you show us some example scott process trees:
UNIX95=EXTENDED_PS ps -H -fu scott

>ksh 4283 scott txt REG 64,0x8 538740 12268 /usr/bin/rksh

It seems scott may be using lots of rksh?

>If I kill these scott owned processes it comes again with new pid. some script is creating them

You are trying to logon but don't know what scott is doing?
Anyway, by looking at the hierarchy you may get an idea who is starting them.

>how to find the script which creates them
>most of these scott owned process are or -ksh
>Please help me to kill these defunct processes.

You can remove the zombies by killing the zombie master. You'll have to determine which is more important, killing the master and restarting or letting it continue.
Mel Burslan
Honored Contributor

Re: profile: cannot fork: too many processes

When you kill the process, note the PPID (parent process ID) and see if the newly spawning process has the same PPID. If so, after making sure it is not a system process and/or a critical production application, you can see what it is and kill it if it will not harm anything else.

I have seen some "clever-above-all" developers leaving a script running on their desktop terminal, doing exactly this, who don't know how to properly code a daemon. So, thread carefully but I am sure you can find the mother of all evil in this case and eliminate it.

Good luck
________________________________
UNIX because I majored in cryptology...
Alzhy
Honored Contributor

Re: profile: cannot fork: too many processes

As root:

UNIX95=1 ps -efH|tee scott.txt

And send us scott.txt

Hakuna Matata.
someoneElse_1
Occasional Advisor

Re: profile: cannot fork: too many processes

Please find output of
UNIX95=1 ps -efH|tee scott.txt

Thanks
Alzhy
Honored Contributor

Re: profile: cannot fork: too many processes

Wow... who/what is teh "scott" login? Can you kill all of its processes right now in aid of diagnosis?

for PID in `ps -ef|grep -w scott|awk '{print $2}'`;do
kill -9 $PID
done

Also, can you check if it has a cron entry? (crontab -l scott) and see if it has a script that is to spawn additional nested ksh sessions?
Hakuna Matata.
Bill Hassell
Honored Contributor

Re: profile: cannot fork: too many processes

Based on the most recent attachment, user scott is doing a lot of bad things (ksh starts ksh starts ksh starts ksh....) To find all the scott processes:

UNIX95=on ps -fH -u scott

Then start killing the top level processes. You don't want to increase maxuprc because user scott's activities (or scripts) are broken. This is exactly what maxuprc is supposed to do -- prevent runaway scripts or processes from using up every PID.


Bill Hassell, sysadmin
Mel Burslan
Honored Contributor

Re: profile: cannot fork: too many processes

It looks like, this person is running a script and spawning sub-shells to do things in parallel and these shells do not clean up properly.

Unless you are the Scott person and clean up this mess by instating proper error checking and properly killing processes when they are done doing what they are supposed to do, into the script/executable, your only option is to kill all the processes belonging to this person in one clean swoop, something like this:

ps -ef | grep scott | awk {'print $2'} | xargs kill

if plain kill do not help at the end of the pipe, you can add a "-9" to the command above.

But what I am suggesting is putting a band-aid onto a deep cut wound. The script/executable causing this behavior, needs to be fixed in the source code.



________________________________
UNIX because I majored in cryptology...
someoneElse_1
Occasional Advisor

Re: profile: cannot fork: too many processes

I am not able to find the script.

root /:ps -ef | grep scott | awk {'print $2'} | xargs kill -9
kill: 2426: no such process
kill: 29892: no such process
....
...so on

User has run some bad script by mistake.
I do not get machine rebooted. I need to kill the script causing the issue?
Mel Burslan
Honored Contributor

Re: profile: cannot fork: too many processes

more than likely, killing the parent process is killing all the children processes before your kill gets to them.

run

ps -fu scott

right after you run this command and see if there are any processes leftover. Please post the output here again
________________________________
UNIX because I majored in cryptology...