Workload management - many questions

[uri_1]

Hi,

We are currently trying prm and wlm.
I have several questions.
1. I have been told that it is always better to use WLM since it has all the functionality of PRM and more. is this true?
2. It seems that these products are best at managing the CPU time. What about I/O? I couldn't understand how this works - And how can I test if it does.
3. Is it recommended to use goal bases policies? It seems a bit scary to me.... It looks like you have less control over what the system does. Is anybody doing it?
4. I need some recommendations or examples of using prm/wlm in the real world. Say I have several db instances+applications on one machine; how is it best to block them? should I use capping? Say I want a 'hard' reservation of resources for an aplication (that noone else can take even if not used), how is it done? should it be done? Is it recommended to use dynamic policies (sounds complicated to manage...)

What i'd like to know is how prm/wlm is implemented in other places and try to learn from that. Also, I want to know what NOT to do.

Sorry for the length,
hope you can help.

[Tom Geudens · Accepted solution]

Hi,
1. This is true. I belief that on the more recent application CD's, PRM is even no longer present.
At this moment I am still using PRM though. I belief (my opinion) that most sites can be handled with simple PRM definitions. WLM requires a lot more planning with - in most cases - little more benefit.

2. You are right about the CPU-issue (as far as PRM is concerned, I haven't looked at WLM in detail yet). PRM memory management is "destructive" (it actually kills processes when capping is enabled). I haven't got any experience with the IO-part ... and since we are using RAID-arrays I'm probably not going to activate it either.

3. This is a WLM feature ... can't help you there. What is usefull (again, my humble opinion), is that you make Service Level Agreements with your endusers. What do they expect (and how much are they willing to pay for it) ? Note that memory is cheap (so buy lots). CPU and IO are the big bucks and it's your job to make sure they : 1. Know that and 2. Are willing to pay for it in order to have performance.
Use Measureware-groups to pinpoint who is using how many resources ... and press their noses into that data.

4. My Measureware-groups are VERY detailed, my PRM-groups are NOT. In PRM I just distinguish between what is BATCH and what is ONLINE. Exceptions are only made for those applications that actually PAY for a certain amount of resources.
Here's the setup :
BATCH : 20
NETBACKUP : 5
ONLINE : 75
During the day (when online transactions are legio) I do use CPU capping (it is activated by a cronjob that runs prmconfig ... do a man prmconfig for the actual command, you can check in glance if it actually works). Otherwise a single batch transaction (which should be running in nighttime anyway) can kill the performance on the server.
During the night I don't use CPU capping (because there's no online anyway) and let batch run it's course (capping is desactivated by a cronjob again).

Ok, I hope this didn't sound to confused. I'm not at my office right know, but if you're interested I'm willing to post my actual configurations / scripts to make this work.

Regards,
Tom

[melvyn burnard]

Might be worthwhile checking out the manuals at:

http://docs.hp.com/hpux/ha/#Process%20Resource%20Manager
for PRM and
http://resourcemanagement.unixsolutions.hp.com/WaRM/wlm/index.html for WLM

HTH

[uri_1]

Thanx, I have the docs.
tom - you've been helpfull. Why does using raid interfere with using i/o resource management?

apart from that - isn't there anyone who uses wlm?

[Tom Geudens]

Hi,
Well, what these storageboxes (we use ClarIIon and Symmetrix) do inside is a black hole. They are filled to the rafter with cache and firmware, you are not sure whether your LUN's are all different physical disks, and so on, and so on ...

I've learned (again, personal opinion/experience) to leave them pretty much alone. If you have / think you have IO problems with those boxes, get the EMC/HP guys to your site and let them pinpoint the problem.

Regards,
Tom

[Dan Herington]

Hi Uri,

I am in the lab for these products, so I will give you more official answers to the questions and clear up some misconceptions.

1. Yes this is true. As a matter of fact when you purchase WLM, you get PRM bundled for free. Tom's statement that PRM is not on the AR CDs is incorrect, we still sell PRM as a standalone product and PRM is included in the Enterprise OE bundle. WLM is included in the Mission Critical OE bundle. To address Toms statement that most sites can be handled with PRM and that WLM requires a lot more planning, this is partially true and partially not. WLM has a tremendous amount of functionality and we try to tout this because nobody else can touch it. However, there are some fairly simple things that can be done with little more planning than PRM. For example, if you wanted a basic configuration that gives you the ability to prioritize workloads, but only bases entitlements on utilization, this is a very simple configuration to set up. If you want to get full benefit of response-time based control, yes this does require more work. What we see most customers doing is starting easy and then adding the more sophisticated controls over time.

2. PRM (and WLM) contol CPU, Memory and LVM Disk I/O. The CPU controls include shares of the CPUs, Processor Sets, and WLM now can move CPUs between vPars. Toms comment about PRM killing processes was never true. Prior to 11i, hp-ux didn't have in-kernel memory controls so PRM implemented a user space deamon to contol memory. In 11.0, it would attach a debugger to processes that were using too much memory, set a breakpoint, wait until the kernel took the memory back and then released the breakpoint. For most processes this works fine, there are some that you wouldn't want to do this with and you can tell PRM not to stop certain proceses if that is true. Now in 11i the story changes completely. They have implemented some new in-kernel memory controls called Memory Resource Groups. This allows you to allocate a block of memory to each PRM group or WLM workload and a separate memory management subsystem will be created for each group. So if an app in a group attempts to allocate more memory than it is entitled to, it will start to page, even if there is other memory available in other groups. This is all configurable. For I/O, we reshuffle the disk access queues in LVM to provide a share of bandwidth to individual volume groups.

3. I would suggest it actually gives you MORE control. It lets you allocate the resources based on BUSINESS priorities rather than standard unix which tends to be egalitarian. Yes we do have customers doing it - most as I mentioned above, start off with simple configs and add features over time. We have toolkits to monitor the performance of Oracle, SAS, Apache/Java, SNMP and others to provide easier access to performance data, should you be interested. The toolkits are free and you can download them from software.hp.com.

4. WLM comes with lots of example configs and each of the toolkits also has examples. If you are managing Oracle, that is our largest installed base. We have thousands of customers using PRM to manage multiple Oracle instances on the same server.

To address your concerns about complexity, it is all relative. If you attempt to create a performance controlled environment for multiple apps in your first deployment, that is complicated. If you are used to doing lots of performance management and already do this kind of thing every day, it may not be. If not, start simple and add complexity as you get more comfortable with it. I guess what I am saying is don't be afraid of it because it's possible to make it complicated - just don't make it complicated until you are comfortable with it.

I hope this helps. If you want more info, you can post more questions here or contact your HP sales rep or SE and have them contact me.

Regards,
Dan

[Tom Geudens]

Hi,
1. Dan, now that's what I call information ! Thank you for taking the time for correcting me. These misconceptions have been growing in my company for years now. You not only helped Uri, you helped me as well ... you deserve full marks !

2. Uri, in the mean time I'm back at the office. I've attached the /etc/prmconf from our "Management Server" (the most important one). In the crontab these two lines activate/desactivate the CPU capping.
30 07 * * 1-5 /opt/prm/bin/prmconfig -k -s -M CPUCAPON >/dev/null 2>&1
00 18 * * 1-5 /opt/prm/bin/prmconfig -k -s -M CPUCAPOFF >/dev/null 2>&1

Regards,
Tom

[uri_1]

Thanx guys.
I still haven't got full information, but having such good resolts in this short time is great.

go itrc forums.

[Tom Geudens]

Hi,
Could I persuade you to assign points for the answers that didn't get any yet ? I'm a bit ashamed that I got 9 for my answer but Dan didn't get any for his (better) answer.

Thanks !
Tom

P.S. N/A for this answer please.