- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - OpenVMS
- >
- Re: how to optimize queue parameters
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО09-09-2010 08:52 AM
тАО09-09-2010 08:52 AM
Re: how to optimize queue parameters
Thanks again for your reply. I posted on my first reply on this thread - unless your asking me to run a different command.
If I understand correctly, your suggesting to split some of the workload for example: 100 jobs process in DKC400 and the other 100 jobs process in DKC200, right?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО09-09-2010 09:05 AM
тАО09-09-2010 09:05 AM
Re: how to optimize queue parameters
Thanks for your reply again. A lot for me to digest and absorb.
Off the bat you mentioned increase UAF record which will make more memory available. Can you please give me a hint on how to do that?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО09-09-2010 10:10 AM
тАО09-09-2010 10:10 AM
Re: how to optimize queue parameters
>>>
Thanks again for your reply. I posted on my first reply on this thread - unless your asking me to run a different command.
<<<
Yeah, my bad. I overlooked your firsrt attachment.
And somehow Hein's and my reply intertwined.
As is to be expected, his answer is much more detailed, especially on RMS related stuff.
I ment more or less the same, but his wording in much better and better detailed.
DO try bto digest it, and ask detailed questions on anything unclear!
hth
Proost.
Have one on me.
jpe
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО09-09-2010 01:48 PM
тАО09-09-2010 01:48 PM
Re: how to optimize queue parameters
Shave off 10% is a pretty good result!
>can this queue be optimized?
The word "optimized" should never appear in any question without being immediately followed bny "for
I'll assume the metric you're interested in is ELAPSED TIME.
Queues don't really have much impact on performance, so they're not really a good place to perform tuning, with the SOLE exception of the JOB_LIMIT. This will control how many jobs can run in parallel.
CONVERT and COPY tend to be I/O bound, so it's possible you can run several in parallel (which you can control with JOB_LIMIT), however, if you have multiple I/O streams from or to the same disks, they may get in each other's way and REDUCE performance over single stream. You can also end up fragmenting files unnecessarily.
Without knowing the hardware configuration, or the sources and destinations of your files, it's not possible to give specific advice, but if you think about it as if it's a road network, you may be able to determine a good sequence of jobs. For example, suppose you have 4 disks, A, B, C, D and you have 4 jobs, two of which are A->B and two C->D. If you run them two at a time you could run the two A->B in parallel, then the C->Ds in parallel. This would maximise contention and probably perform poorly. If you go the other way and run A->B against C->D they don't get in each other's way, so perform better.
(This is simplistic, as it doesn't consider controllers, busses and other I/O infrastructure, but hopefully you get the idea)
The key to performance for this type of job is the exact sequence of operations, to prevent contention. You can't really do that if you just generate a bunch of jobs and dumping on a single queue.
What you may want to consider is to analyze your environment to determine which operations can be done in parallel. Create a separate queue for each "contention group" and give it /JOB_LIMIT=1.
Now divide up your jobs and submit each to the appropriate queue. Now you can run as many jobs as possible in parallel (one in each queue), while minimising contention, and thereby minimising total elapsed time.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО09-10-2010 12:43 AM
тАО09-10-2010 12:43 AM
Re: how to optimize queue parameters
Thank you all for posting and guidance. I now understand of how our maintenance program works and see the loop holes you've indicated and have been actively researching various terminology.
I created new scripts using the old scripts as a basis and applied some of your suggestions and letting her rip as of the moment. I know this is quite a bold attempt but nevertheless I was extremely careful in not modifying various core commands.
I appologize as I realize that I mislabelled this post but I had no clue what was going on, what to look for, and where to start (newbie).
I will return to post how much time was saved.
Cheers.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО09-10-2010 01:11 AM
тАО09-10-2010 01:11 AM
Re: how to optimize queue parameters
We are proud of you!
:-)
Seriously, it sounds like you are picking up terminology and understanding quite fast.
Your next step, if needed, is to review the FDL files used.
Specifically you may want to do monthly/yearly ANALYZE/RMS/FDL and EDIT/FDL/NOINTERACTIVE cycle for those files which are not 'hand tuned' already.
Hint: do the ANAL/RMS outside the maintenance window, possibly after all converts are done, on the OLD files.
Cheers,
Hein
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО09-10-2010 02:44 AM
тАО09-10-2010 02:44 AM
Re: how to optimize queue parameters
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО09-10-2010 03:33 AM
тАО09-10-2010 03:33 AM
Re: how to optimize queue parameters
Careful use of the DCL SYNC can help to coordinate jobs as well.
One side note: I'm not sure if contiguous files are as important these days as in the past. You may want to test whether or not that requirement is necesasry. Depending upon how the individual "bits" of the file are accessed, neing contiguous may not be buying you much in performanc.
Dan
- « Previous
-
- 1
- 2
- Next »