bound/unbound CPU's

[Mark Grant]

Guys and Gals,

Can any one give any information on what a bound CPU can do that an unbound one can't. I know it's IO related but that's about it.

Alternatively, can anyone envisage a scenario where a particular application would run 50% faster with one CPU in a vPar than it would if you put two in. This is the rather peculiar situation we find ourselves in at the moment.

Any help/assistance much appreciated.

[Pete Randall · Accepted solution]

According to the manual

"A bound CPU is a CPU that is assigned to and handles I/O interrupts for a virtual partition. Every virtual partition must have at least one bound CPU to handle its I/O interrupts.

CPUs that are not assigned to any virtual partition or that are assigned to a virtual partition but do not handle its I/O interrupts are unbound CPUs. Unbound CPUs are sometimes called floater CPUs."

Going by that, the only distinction is the ability to handle I/O interrupts versus the ability to be switched amongst VPARs.

As far as the bizarre behaviour you're seeing, I can't even begin to guess!


Pete

[Ranjith_5]

Hi Mark,

Following is an example where you can see the bound/unbound CPU.

root@server5 in />#vparstatus
[Virtual Partition]
Boot
Virtual Partition Name State Attributes Kernel Path Opts
============================== ===== ========== ========================= =====
server1 Up Dyn,Auto /stand/vmunix
server2 Up Dyn,Auto /stand/vmunix
server3 Up Dyn,Auto /stand/vmunix
server4 Up Dyn,Auto /stand/vmunix
server5 Up Dyn,Auto /stand/vmunix
server6 Up Dyn,Auto /stand/vmunix

[Virtual Partition Resource Summary]
CPU Num Memory (MB)
CPU Bound/ IO # Ranges/
Virtual Partition Name Min/Max Unbound devs Total MB Total MB
============================== ================ ==== ====================
server1 1/ 27 1 7 6 0/ 0 8192
server2 1/ 27 1 3 6 0/ 0 8192
server3 1/ 27 1 3 6 0/ 0 4096
server4 1/ 27 1 5 8 0/ 0 6080
server5 1/ 27 1 3 8 0/ 0 4096
server6 1/ 27 1 5 10 0/ 0 8192


The basic difference is a bound CPU (in numbers) will be the minimum configuration of CPUs at any point of time. Unbound CPUs( in munmbers) can be moved accross the vPARs according to your requriement. In the above example of server1 my unbound CPU no is 1 and bound CPU no is 7 That means I can move my 7 CPUs to another server using

#vparmodify -p server1 -m cpu::1
#vparmodify -p server2 -m cpu::11

Regards,
Syam

[Florian Heigl (new acc)]

Hi,

I'm short on time, but I'll try to outline it...

the bound cpu's are the ones that handle I/O by getting allocated irq's for device adapters.
so, on the one hand these cpu's can't be unconfigured, while a unbound cpu can be removed from the running vpar both in case of failure or reduced computing need.

now for the fun part:
on hp-ux You can't really influence the bindings for applications or device irqs in a way like i.e. the irq affinity filter for Windows allows You - I could easily imagine performance issues in the following case:

You have two cpus, which both handle irq's and application load, and the memory footprint of the functions that go to secondlevel cache is higher than it's capacity.

Applications needing a high amount of cpu will be wandering from cpu to cpu in some cases (syslog messages are produced for that), and so You might experience a lot of cache flushing, which will also lessen the effect of branch prediction etc.

You can influence it a bit, but not too much.
OTOH, I haven't seen that bad an impact yet.

[Pete Randall]

Mark,

Maybe not even applicable, but I did have some performance issues early on in my experience with VPARs. It turned out to be a memory shortage, though I tried to figure out a way to blame it on CPU for a while. I finally managed to run "vmstat 1 5" during one of these intermittent slowdowns and found the "po" column had activity, indicating swapping. Adding some memory to that VPAR has since erased the issue.


Pete

[Mark Grant]

Oops, didn't mean to get a bunny onto this thread just yet, never mind. Thanks for the help guys. What we have is a vpar that originally had one bound and one unbound CPU. In this, we run a job that takes 12 hours. It is very memory intensive though not CPU intensive. I had checked "sar" stats for the period it was running and can't see any paging out. Removing the unbound CPU and re-running the job allows it to complete in six hours. We are now re-running the job with two bound CPU's and the job seems to be heading towards the 12 hour mark again. I am contemplating adding more CPU's to see if I can slow it down some more :) I can not see that this makes much sense so am looking at gut feelings now. My thought is maybe it is possible to write a multi threaded application that expects to be able to handle multiple CPU's itself and that this application is therefore fighting the OS's own CPU scheduling in some way and cache flushing issues sound promising. It is worth noting that everything I can see about this application suggests to me it is a lazy Windows port. Any further thoughts would be gratefully received. All our other applications seem to benefit substantially from throwing in a few extra CPU's from time to time.

[melvyn burnard]

One question/suggestion here is what version of Vpars do you have, and how recently have you checked your patching?

[Mark Grant]

Hi melvyn,

VPARMGR B.11.11.01.02 Virtual Partition Manager

I guess this is the right bit.

I am not too sure where this is patched to right now. We created a shiny new depot of HPUX and the vpar software on our Ignite server. We then built the vpar using an ignite image from a 7410 but having changed the Ignite image CINDEX file to include the core_cfg and apps_cfg that were created from the two depots (make_config).

I am not sure but I assume this would make us fairly up to date but where, I don't know!

[Florian Heigl (new acc)]

If I may ask You - would You like to look into the syslog and look for messages like 'migrating PID', or even look at the cpu column in top output.

also, fire up measureware, the cpu run queues might be interesting. :)

[melvyn burnard]

Mark
That is the version of the Vpar Manager product, whgich is separate from the actual Vpars product
Try swlist -l product | grep -i partition

[Mark Grant]

Melvyn,

This is the ouput I get.

NParProvider B.11.11.01.03.00.03 nPartition Provider
NPartition A.01.02 Enhanced NPartition Commands
PartitionManager B.11.11.02.00.03.03 Partition Manager for HP-UX
VirtualPartition A.03.02.04 HP-UX Virtual Partitions Functionality
vParManager B.11.11.01.02 Virtual Partition Manager for HP-UX

Thanks again for your time.

[Mark Grant]

Florian,

This machine probably has the dullest syslog I've ever seen. Nothing of even the remotest interest in there at all I'm afraid.

With regard to the CPU run queue, it looks pretty much OK.

It looks so odd. If you run glance, top or whatever and have one CPU in the vpar, we get a load average of about 5 and the job finishes in six hours. With two CPU's we get a load average of about 1 and the job takes 12. I'm going to add more CPU's and see if I can slow it down even more!

[Matti_Kurkela]

Is the application multi-threaded? The fact that you can get a load average of 5 with the one-CPU case seems to suggest it is.

Just a thought... How I/O intensive the application is?

It could be that when the application is run with one CPU only, the CPU gets to split its time between running application code and handling the I/O.

But when you add another (unbound) CPU, the application is running full-time on the unbound CPU (generating IO requests faster than in the single-processor case, because this CPU does not need to handle IO interrupts) and also running on the bound CPU between the IO interrupt service (generating still more IO requests).

I'd be interested to see what happens if you add the second CPU to the VPAR as a bound CPU. The increased I/O interrupt processing capacity might help in your case.

If the program is intensively reading or writing from disks, it could also be that with more CPUs the disks get more requests all over the disk surface, so the disk spends relatively more time seeking from block A to block B and less time actually reading/writing stuff. If that is the problem, you probably need to get more disk mechanisms to share the load - that is, RAID0 striping.

[Mark Grant]

Thanks Matti,

Actually, I tried it with two bound CPU's and it was only marginally faster than when we had the one bound CPU and the one unbound CPU. I don't really know what the application does really though it appears to read a few thousand rows from the database and then spend ages processing this data.

It doesn't seem to be particularly heavy on IO and certainly not disk io.

We are currently looking at the database itself to see if there are any issues there. As we run similar database configurations all over the place which all seem to work, it isn't the first place we thought of looking.