topic SIGCHLD delivery problem in multithreaded app in Operating System - HP-UX

SIGCHLD delivery problem in multithreaded app

sergejusz — Mon, 22 Dec 2008 09:17:48 GMT

Hi guys,
I'm working on stand-alone web-service (OS=HP-11.23, Compiler=aCC). It is multithreaded application based on boss-workers model (I have one boss thread and N working threads). Web-service receives queries and perform some work. This work include executing external process(es). In order to perform query I need to get result of external process, so I need to wait until it terminates. In the first version of this program (that is working for few years without problems) I used fork-exec-waitpid sequence in worker-thread. Since external process takes some time I loose this time, because worker is blocked at waitpid call. Now, I decided to use this time in better way. I execute fork-exec (saving pid somewhere) in worker thread and after that I immediately return my worker to the pool to get new job. Of course, now I need notification that my external process exited I and I can get the result (file or smth. else).
I created special thread (let's call it TSignalHandler) that is waiting for SIGCHLD. After it catches SIGCHLD, it calls wait() and gets pid (and status) of completed process. pid and status are sent to boss and everything Ok.
Now, the problem. When I test my application on small load everything goes w/o problems. All SIGCHLD arrive. But during heavier load I notice that some SIGCHLD are missing that is, of course, absolutely unacceptable.
Now, few details. I block all signals for all threads.
sigset_t mask;
sigfillset(&mask);
int rc = pthread_sigmask(SIG_BLOCK, &mask, 0);

And for open SIGCHLD delivery only for TSignalHandler:
here is simplified thread function loop:

while(true)
{
int sig_number;
sigset_t signal_set;
sigemptyset(&signal_set);
sigaddset(&signal_set, SIGCHLD);
sigaddset(&signal_set, SIGUSR1);
pthread_sigmask(SIG_BLOCK, &signal_set, 0);
sigwait (&signal_set, &sig_number);

switch(sig_number)
{
case SIGCHLD:
Wait();
break;

case SIGUSR1:
Stop();
break;

default:
break;
}
}

And Wait() function code:

void TSignalHandler::Wait()
{
int status;
int pid = wait(&status);
if(pid > 0)
{
m_dispatcher->PostMessage(new TGenerationStatusMessage(TInt32Int32Bool(pid, (int)time(0), ..smth..)));
}
}

So, only TSignalHandler thread may receive SIGCHLD.
My questions are:
1. Are SIGCHLD 100% delivered to application?
2. May be somebody faces similar problems and can help me ?

Thanks in advance.
Serge Zabinskis
//Source of TSignalHandler is attached

Re: SIGCHLD delivery problem in multithreaded app

Laurent Menase — Mon, 22 Dec 2008 11:29:34 GMT

Hi

I just have 2 recommendations:
1) don't fork in a multithreaded application because the perf cost of fork() in that case is really high.
-> use simple monothreaded coprocesses which you communicate with a pipe. This coprocess started once launch subprocesses
-> use /dev/echo and I_SENDFD/I_RCVFD to share file descriptors and pipes, or unix domain sockets
2) SIGCHLD are all delivered to the application, but if more than one is delivered to the application while not on sigwait, only one is kept.
The best way to avoid it is to use sigaction()since the actions may be queued.

Workaround to the lose of signal:
call an extra waitpid(-1,&status,WNOHANG)
void TSignalHandler::Wait()
{
int status;
int pid;

while ( (pid= waitpid(-1,&status,WNOHANG))>0)
{
if(pid > 0)
{
m_dispatcher->PostMessage(new TGenerationStatusMessage(TInt32Int32Bool(pid, (int)time(0), ..smth..)));
}
}
}

Re: SIGCHLD delivery problem in multithreaded app

sergejusz — Mon, 22 Dec 2008 11:56:07 GMT

Thanks Laurent for your very valuable response.

kind regards
Serge

Re: SIGCHLD delivery problem in multithreaded app

Dennis Handly — Mon, 22 Dec 2008 12:41:16 GMT

>I block all signals for all threads.

Don't block the program error signals, Signal 4,5,10,11, these will cause loops if you get them.

Re: SIGCHLD delivery problem in multithreaded app

sergejusz — Mon, 22 Dec 2008 12:46:43 GMT

Of course Dennis, you are right!

Kind regards
Serge

Re: SIGCHLD delivery problem in multithreaded app

sergejusz — Tue, 23 Dec 2008 07:50:34 GMT