pthread scheduler

[Prashun Gupta]

Dear Friends,

I have been in mid of a porting project to migrate my application from task library to pthread library. I have a requirement where i need some suggestions, that is as follows

1. I have define n number of worker threads in my application. Now lets consider I am getting thread requests(tasks) spontaneously, which is more than the number of worker threads. I need to create a sort of buffer to store such requests. Now I need to push my tasks waiting in the queue to acquire a worker thread as soon as any one of the task given to worker thread completes. Now I can not use my main() to incorporate scheduling this pop/push into buffer. I need another scheduler to take care of my buffer management.

2. Just consider in case I use inbuild pthread scheduler and use more number of worker threads by increasing max_thread_proc, and allocate a seperate thread for each task, Do this is consider as right strategy.
I have seen people attempting to change this variable to even 3000, generally in the cases of websites for http requests.

2b. If 2a sounds good then I need to make a strategy to cleanup a thread resources, and make it available to next coming requests. I just wanted to make sure whether pthread_detach() function will be able to proceed in such conditions.


According to the community, what should be the best data structures and alogorithm to incorporate such issues.

If someone wants to answer any part of the query, he is most welcome

Regards
Prashun

[Adam J Markiewicz]

Hi

I see it like this:

1. You make a queue of incomming requests. Together with this queue you combine a mutex and conditional variable. The push() is designed like this:
{
lock( mutex );
queue.insert( request );
broadcast( cond_var );
unlock( mutex );
}
You make how-many-you-want threads. Threads are waiting for something to come in the queue. Use the code like:

lock( mutex );
while( queue.is_empty() ) {
wait( cond_var, mutex );
}
request = queue.getandunlinkfirstrequest();
unlock( mutex );
return request;

2. If you want to clean up:
first of all mechanism to force threads to quit - add another field that says that your application is quitting. combine it with synchronisation from above:
setting quit:
lock( mutex )
quit = true;
broadcast( cond_var );
unlock( mutex );

and modify pop() to check if it is not quitting situation:

lock( mutex );
while( queue.is_empty() && ! quit ) {
wait( cond_var, mutex );
}
check_if_it_was_quit_or_request();
if( !quit ) {
request = queue.getandunlinkfirstrequest();
}
unlock( mutex );
return something_that_sayz_if_it_was_quit_or_request;

And the last thing: waiting for threads: The way I'd make it is define class with the destructor doing this thing and making statical object of this class. The destructor would be called after return from main()


Hope it helped you somehow.

Adam