- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - OpenVMS
- >
- Re: About migrating application to POSIX THREADS
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-05-2008 04:11 AM
тАО02-05-2008 04:11 AM
About migrating application to POSIX THREADS
We have a application that makes a intensive use of ASTS. There is 80 $QIO pending in 80 decnet mailboxes for 80 processes in machine.
The performance are good, but we want to migrate to POSIX THREADS.
We probe a process with 80 threads listening every thread, every channel, but the performance was the worst. We gave up this way.
In a initial aproximation, we introduce the sincronous part in a new thread. This thread deal with packets collect by AST and when there isn┬┤t work, makes a cond_wait. AST, save packets in memory and instead of wake makes a cond_signal when is necesary.
In my openvms readings, i read that AST code run in the context of first default thread. Perfect, I want to gain with SMP, with one AST runnig and second thread running in paralell.
But we have a problem. Althougth I establised SCHED_OTHER and priority 8 for the default thread, it seems that the default thread ( primer thread ), when is running the AST CODE ( this first thread doesn┬┤t do anything, only perpetuous cond_wait ) his SCHED and priority are FIFO and 33 ( i saw this with debug ), and i can┬┤t fight with it, because the second thread has a sched FIFO and priority is 31, the maxinum that system let me for the thread.
Everybody can make a suggestion ?
Thanks in advance....
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-05-2008 04:59 AM
тАО02-05-2008 04:59 AM
Re: About migrating application to POSIX THREADS
Roberto, Welcome to the HP ITRC Forum for OpenVMS.
Nice job in the 'butter up' depertment.
Just a few question, which will hopefully help to get some clarifications.
>> We have a application that makes a intensive use of ASTS. There is 80 $QIO pending in 80 decnet mailboxes for 80 processes in machine.
So help me understand the purpose and point of the ASTs? If each process is just waiting, then it might was well work synchroneously.
Don't get me wrong, I really like AST's.
I think they are a very powerful way to construct and application, possibly better than threads. But it does not sound like they are really exploited here.
>> We probe a process with 80 threads listening every thread,
Have you also tried a single process with 80 AST's outstanding?
How many CPU's are available?
The power of (kernel) threads is the potential to have more than 1 active CPU for a single process.
How much CPU is needed? More than 1 CPU accross the 80 streams? If it is less than 1 CPU, then a single process juggling 80 threads through ASTs may be optimal.
Or maybe a few processes, each dealing with say 20 - 30 ports.
Or, an old time favourite, a single 'listener' or execution controller with unlimited active waits, which then hands out work parcels to the first free slave task (ACMS paradigm).
>> priority are FIFO and 33 ( i saw this with debug ), and i can├В┬┤t fight with it, because the second thread has a sched FIFO and priority is 31,
I'm afraid I never studied this area, but priorty 33 vs 31 are no problem as long as there is more than 1 processor available for the process. Is there?
Hope this helps some,
Hein van den Heuvel (at gmail dot com)
HvdH Performance Consulting
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-05-2008 05:40 AM
тАО02-05-2008 05:40 AM
Re: About migrating application to POSIX THREADS
Another multiple choice question. What is the purpose of migrating to Posix Threads?
I can think of several possible reasons that you may be considering the migration.
1. Portability. You have plans to migrate off VMS, and are starting to rewrite code without VMS specific features.
2. Familiarity. The current support organization is not familiar with VMS and its features, and is familiar with Posix threads.
3. Performance. Hope for better performance due to multiprocessing, and the belief that Posix threads are more scalable in an SMP environment.
4. Management directive to use "standards".
Is it true that the 80 $QIO all have the same AST completion routine? Is this becoming the point of congestion? If you do a
$ monitor modes/cpu
is one CPU busy while the others are nearly idle? If your concern is performance, but no single CPU is limiting your performance, what are your performance goals?
Also, please tell us about your hardware and software configuration (is this VAX, Alpha, IA64, VMS 5.5-2 7.3-2 8.3 ???) Can any of this change?
I realize this answers none of your questions, but it will help us make a better assessment of your problem and goals.
Jon
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-05-2008 05:44 AM
тАО02-05-2008 05:44 AM
Re: About migrating application to POSIX THREADS
The system has 8 CPUS, and i link with upcalls and kernel_threads enabled.
I├В┬┤m trying to find a way to make the same, but with better performance with pthreads.
I tried with 80 threads, but the performance is very bad, because all threads are very related one another, because all save in the same chain of memory several messages. The latencies are high too. The sincronous part is the consumer thread and write the messages into disk.
And now, I trying another possibility, respect the AST code into pthread process. I use only 2 threads, 1 (default) for AST, and another for sincronous part of the process ( when AST do WAKE and mainline code do the work of write messages into disk ). In this case AST don├В┬┤t make WAKE, do cond_signal, and the sincronous part do cond_wait instead of HIBER in the former case.
Thanks.....
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-05-2008 05:48 AM
тАО02-05-2008 05:48 AM
Re: About migrating application to POSIX THREADS
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-05-2008 05:54 AM
тАО02-05-2008 05:54 AM
Re: About migrating application to POSIX THREADS
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-05-2008 05:56 AM
тАО02-05-2008 05:56 AM
Re: About migrating application to POSIX THREADS
Was this OpenVMS engineering, or some local Unix support engineer?
?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-05-2008 05:59 AM
тАО02-05-2008 05:59 AM
Re: About migrating application to POSIX THREADS
Thanks....
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-05-2008 06:00 AM
тАО02-05-2008 06:00 AM
Re: About migrating application to POSIX THREADS
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО02-05-2008 06:16 AM
тАО02-05-2008 06:16 AM
Re: About migrating application to POSIX THREADS
What is the expected main advantage of having all tasks execute in one process?
Shared memory which can not be implemented as true shared memory? Probably not.
A big potential advantage I can think of is a shared file which each worker has to write to. In the multy process case it would have to be opened shared, with each process its own (rms) buffers.
With multythreading you can open it just once, possibly exclusive, and effectively share the buffers between the streams. They could be RMS streams connecting multiple RABs to the same FAB to be able to have individual worker have their own RMS context, or they might just all use the same handle (FAB+RAB).
To take advantage of this, you would not only have to switch to threads, but change the code to use the same file handles. Was that in place for the test?
This is but one example, but without changes like that you are pretty much only hoping/expecting that the OpenVMS (process) scheduler is noticably slower than the pthread scheduler. For that to be noticable at all, never mind whether it is faster, there would need to be relatively little processing and a lot of context switching.
Also, my original comment on the need for ASTs stands.
Again, I love doign it this way, but there needs to be a reasons. With a 1 on 1 AST -to process mapping one has to wonder why each process does not just call a common subroutine set to set up the listening.
In doing so, you may be able to avoid the context switch after the AST being taken, and the worker process being woken up, and avoid a message from having to be passed.
For now, the value of the listener process is not clear yet, and getting rid of it may be a more interesting way to improve the application.
Finally, me being an RMS consultant, you indicated one of the true work activities is to add data to a file.
While it is valueable to try to get to start doign that as efficiently as possible, has there been a review whether the work itself is being done efficiently? There may be much more 'bang for the buck' to be found in the back-end processing than the front-end scheduling and message passing.
Shared file? Indexed? Buffers tuned?
Hope this helps some,
Hein van den Heuvel (at gmail dot com)
HvdH Performance Consulting