vCPUs available for guests from a vmhost

[silusan]

hi...

We have a BL860 i2 blade with the CPU details shown below.
Created a vmhost by assigning the entire resources attached to it...

If we want to create vmguest on it with 4vCPUs and 20% entitlement, how many guests can we build.
So far we have two vmguests on this vmhost.

thank you.

VM Host print_manifest details for CPU:

System Hardware

    Model:              ia64 hp Integrity BL860c i2
    Main Memory:        130965 MB
    Processors:         8
       2 Intel(R)  Itanium(R)  Processor 9340s (1.6 GHz, 20 MB)
               4.79 GT/s QPI, CPU version E0
               8 logical processors (4 per socket)
 OS mode:            64 bit

vmhost:/ # hpvmstatus -s
                [HPVM Server System Resources]

        Processor speed = 1596 Mhz
        Total physical memory = 130965 Mbytes
        Total number of processors = 8
        Available memory = 74211 Mbytes
        Available swap space = 129351 Mbytes
        Maximum vcpus for an HP-UX virtual machine = 8
        Maximum vcpus for a Windows virtual machine = 8
        Maximum vcpus for a Linux virtual machine = 8
        Maximum vcpus for an OpenVMS virtual machine = 8
        Available entitlement for a 1 way virtual machine = 1596 Mhz
        Available entitlement for a 2 way virtual machine = 1596 Mhz
        Available entitlement for a 3 way virtual machine = 1596 Mhz
        Available entitlement for a 4 way virtual machine = 1596 Mhz
        Available entitlement for a 5 way virtual machine = 1276 Mhz
        Available entitlement for a 6 way virtual machine = 1276 Mhz
        Available entitlement for a 7 way virtual machine = 1276 Mhz
        Available entitlement for a 8 way virtual machine = 1276 Mhz
vmhost:/ #

vmhost:/ # hpvmstatus
[Virtual Machines]
Virtual Machine Name VM #  OS Type State     #VCPUs #Devs #Nets Memory  Rmt Host
==================== ===== ======= ========= ====== ===== ===== ======= ========
guest1           1  HPUX    On (OS)                         4     3     1   16 GB        -
guest2           2  HPUX    On (OS)                         4     3     1   16 GB        -
vmhost:/ #

hpvmstatus -P guest1 / guest2 gives this: (20% entitlement)

[Virtual CPU Details]
#vCPUs Entitlement Maximum
====== =========== =======
     4       20.0%  100.0%

[Dave Olker]

If you have 8 CPU cores and you want to create VM guests with 4 cores and 20% entitlement then you should be able to create 10 guests.  Obviously this assumes you have sufficient memory, I/O, and other resources needed by these guests.

Dave

[silusan]

10 vmguests?

ok..I am trying to understand.....

4 vCPUs x 20% = 0.8 physical CPU

I have 8 physical CPUs

0.8 phy CPUs x 10 guests = 8

Is that right?

the blade has 8 logical processors - is it nothin but 8 phy CPUs?

Am I right in intepreting ?

thanks

[Dave Olker]

Yes.  Your bl860c i2 had 8 CPU cores.  You are using 20% entitlements, so each core can support 5 guests.  5*8=40 shares.  Since each VM guest is using 4 "shares" you could have 10 such guests.

Dave

[silusan]

Thank you....

I have another vmhost...

This is built on rx8640 along with one more vmhost(so total two vm hosts) and one rac for vpars.

let us say, one of the vmhosts is "anothervmhost", which has guests and the entitlement is not consistent in all the guests.

Some are 10%, someothers are at 20%, 7% etc

details of "anothervmhost"

ia64 hp server rx8640

System Hardware

    Model:              ia64 hp server rx8640
    Main Memory:        131003 MB
    Processors:         4
    Proccesor(0) Speed: 400 MHz
    Proccesor(1) Speed: 400 MHz
    Proccesor(2) Speed: 400 MHz
    Proccesor(3) Speed: 400 MHz
    OS mode:            64 bit

anothervmhost # hpvmstatus -s
                [HPVM Server System Resources]

        Processor speed = 1598 Mhz
        Total physical memory = 131003 Mbytes
        Total number of processors = 4
        Available memory = 47306 Mbytes
        Available swap space = 14187 Mbytes
        Maximum vcpus for an HP-UX virtual machine = 4
        Maximum vcpus for a Windows virtual machine = 4
        Maximum vcpus for a Linux virtual machine = 4
        Available entitlement for a 1 way virtual machine = 1598 Mhz
        Available entitlement for a 2 way virtual machine = 1598 Mhz
        Available entitlement for a 3 way virtual machine = 1438 Mhz
        Available entitlement for a 4 way virtual machine = 1278 Mhz
anothervmhost #

from print_manifest: 

the processor speed is 400MHz, there are four in number.

whereas in hpvmstatus -s, it is 1598Mhz

In my earlier post the output is from BL860 i2 blade,

with processor speed as 1.6Ghz(print_manifest)

1596 MHz (hpvmstatus -s)

  20% is the entitlement we have been asked by our customer.

(1) So, in this rx8640 vmhost, how much entitlement I should give to match that?

(2) Also, if I give 20% entitlement for each guest, with four CPUs, with four processors....will I be able to create only five vmguests?

[Dave Olker]

       (1) So, in this rx8640 vmhost, how much entitlement I should give to match that?

I'm not clear what you're asking.  Are you asking how much entitlement you need to give a VM guest on an rx8640 with 400MHz CPUs to match a VM guest on a bl860c i2 using 20% entitlements of 1.6GHz CPUs?  There are many variables involved here to give a simple answer.  Entitlements are merely guaranteed minimums, so a VM with 20% entitlements running on an otherwise idle system could use 100% of the vCPUs it has been assigned.  Also, if the VM guest does not need to use 20% of the CPU cores it can use less and other VM guests running on the system can use some of it's CPU time.  All the entitlements mean is - when I get busy, I'm guaranteed to get at least X amount of CPU time on these cores.  So without knowing how busy your VM guests are on the i2 blade I cannot determine how much CPU time you'll need on the rx8640.

       (2) Also, if I give 20% entitlement for each guest, with four CPUs, with four processors....will I be able to create only five vmguests?

If you're ONLY going to create guests with 20% entitlements and 4 vCPUs on then you could create 5 VM guests on a VM host with 4 physical CPUs.  You mentioned earlier that some guests have 10%, others 20%, others 7%.  It's also possible that some of those guests might only use 1 vCPU, or 2, or 3, etc.  In that case, if you're mixing and matching VMs with different amounts of entitlements and different numbers of vCPUs, the number you can run simultaneously will depend largely on what order you try to start them.  As the VM guests start and they are assigned their resources, the scheduler will determine how many vCPUs remain available and how much entitlement remains on those CPUs.  

Dave

[silusan]

Dave, perfect! brilliant. thanks for your answers.

You were right in guessing the intent of my first question and I totally understood the answer you gave - the entitlement factor.

Coming to the second answer... I think I need to understand a bit more about these lines..

"the number you can run simultaneously will depend largely on what order you try to start them.  As the VM guests start and they are assigned their resources, the scheduler will determine how many vCPUs remain available and how much entitlement remains on those CPUs."

Our customer comes to me and tells me he needs two vm guests. I create them with the requested no. of vCPUs and entitlement. Later, he comes again for more vm guests... then how do I know that if my vmhost is capable of more vm guests?

(1) Do I need to go to the hpvmstatus of each guest, calculate no. of vCPUs already assigned, their entitlements and then determine the CPU capacity remaining? hpvmstatus -s shows me the very clearly the amount memory available..in that way....

(2) is there any way I can see the CPUs resources available in a vmhost for the creation of a new vmguest? will that 'scheduler' you were talking about helps me in determining the no. of vCPUs available?

The guests which are ON, will remain ON I beleive... I need to think about the guests which I will be asked to create and if they can be started at all in near future.... I think this will be determined by the sequence in which I satrt the guests..(by their no. of vCPUs assigned)

I did some calculations: I added the vCPUs assigned and their entitlement information here for convenience.

anothervmhost  # hpvmstatus
[Virtual Machines]
Virtual Machine Name VM #  OS Type State     #VCPUs    entitlement maximum
==================== ===== ======= ========= ======  ======
guest1                         1   HPUX    On (OS)        2            10%       100%
guest2                         2  HPUX    On (OS)         4              7%       100%
guest3                         3   HPUX    On (OS)        2             10%      100%
guest4                         4   HPUX    On (OS)        2               7%      100%
guest5                         5   HPUX    On (OS)        4              20%      100%
guest6                         6   HPUX    On (OS)        4              20%      100%
anothervmhost #

calculating the CPUs consumed:

2 x 10% = 0.20 CPU
4 x 7%  =  0.28 CPU
2 x 10% = 0.20 CPU
2 x 7% =   0.14 CPU
4 x 20% = 0.80 CPU
4 x 20% = 0.80 CPU
 -------------------
total         2.42 CPU
 ------------------

so, 1.58 CPU is remaining.

let us consider it as 1.6 CPU, to make calculations easy...
so, I can create four more vm guests with (4 vCPUs x 10%) = 1.60 CPU

Am I right Dave?

That begs another question, the vm host itself to be UP and running...doesn't it need any CPU at all? If I give all the available CPUs to guests, how does the vmhost remain UP and running? if the vmhost needs some CPU to remain UP and running, how do I know how much it needs?

Thank you.

[Dennis Handly]

>if the vmhost needs some CPU to remain UP and running, how do I know how much it needs?

Is it doing anything else but serving VM guests?

Unless you have specific SLAs for each guest, you might be able to assume that it only takes negligible resources.

[Dave Olker]

@silusan wrote:

calculating the CPUs consumed:

2 x 10% = 0.20 CPU
4 x 7%  =  0.28 CPU
2 x 10% = 0.20 CPU
2 x 7% =   0.14 CPU
4 x 20% = 0.80 CPU
4 x 20% = 0.80 CPU
 -------------------
total         2.42 CPU
 ------------------

so, 1.58 CPU is remaining.

let us consider it as 1.6 CPU, to make calculations easy...
so, I can create four more vm guests with (4 vCPUs x 10%) = 1.60 CPU

No, this is not a valid way to calculate remaining vCPU resources.  There are several rules the scheduler applies when assigning vCPUs to physical CPUs.  Among them are:

1. Attempt to use all available physical cores, rather than stack VM guests on a single core and overload it.
2. Ensure that vCPUs of a single VM are not assigned to the same physical core.  In other words, if you have a VM guest with 2 vCPUs, the threads associated with those virtual CPUs will run on different physical CPUs.  So you cannot say "this guest has 2 vCPUs and 20% entitlement so that's roughly 40% of a single CPU".  Instead you need to say "this guest has 2 vCPUs and 20% entitlement so it has the rights to 20% of two separate physical CPUs".
3. The scheduler can move vCPU threads around to different physical CPUs, but it makes every attempt to enforce the best locality possible.  In other words, it tries to locate the vCPU threads in the same locality domain as the memory assigned to the guest so that the memory latency of vCPUs accessing their memory will be minimized.

So again, simply multiplying the number of vCPUs by their entitlement and subtracting from the physical CPUs will not give you an accurate picture of which specific vCPU resources are available at any given time.  Here are a couple slides from one of my "best practices" decks that explain this phenomenon:

If you want to see which physical CPUs are assigned to a specific VM guest at any given time, you can use the "hpvmstatus -C" command. This syntax works on Integrity VM v6.1 and newer systems:

atcuxbl1(/) -> hpvmstatus -C

Virtual Machine Name  VM #    ldom # Total Memory(MB) SPU Ids
===================== ======= ====== ================ ==========
guest1                1       0      128              12,14
guest2                2       0      128              4,6,8,10
guest3                3       0      128              0,2
guest4                4       0      128              8,10
guest5                5       0      128              4,6,12,14
guest6                6       0      128              0,2,8,10

I created the same 6 guests on my VM host as in your example.  My host has 8 physical CPU cores and the scheduler spread these 6 guests across all 8 cores.  It also ensures that each vCPU for each VM guest is assigned to a different physical core. 

@silusan wrote:

That begs another question, the vm host itself to be UP and running...doesn't it need any CPU at all? If I give all the available CPUs to guests, how does the vmhost remain UP and running? if the vmhost needs some CPU to remain UP and running, how do I know how much it needs?

The VM host scheduler will always hold back some CPU resources for managing the VM guests.  You can assign 100% of all the available CPUs to VM guests and have them running without worrying about the VM host not getting the CPU cycles it needs to manage the guests.

Hope this helps,

Dave

[silusan]

Thank you so much for that explaination...

however, I didnt quite understand what to get from that hpvmstatus -C output

I have Integrity VM 04.30 in my vmhost/vmguest

vmhost # hpvmstatus -C
[Guest Cell Local Memory Usage]
                           Cell        MB        MB        MB
Virtual Machine Name VM #  Prefer Interleave  Home Cell Other Cell
==================== ===== ====== ========== ========== ==========
guest1                       1    ilm          0          0       5146
guest2                       9    ilm          0          0       8214
guest3                    12    ilm          0          0       5213
guest4                    16    ilm          0          0       5213
guest5                    41    ilm          0          0      16477
guest6                    21    ilm          0          0       5213
guest7                    39    ilm          0          0      16477
vmhost #

After creating two new VMs

vmhost # hpvmstatus -C
[Guest Cell Local Memory Usage]
                           Cell        MB        MB        MB
Virtual Machine Name VM #  Prefer Interleave  Home Cell Other Cell
==================== ===== ====== ========== ========== ==========
guest1                         1    ilm          0          0       5146
guest2                         9    ilm          0          0       8214
new guest aa           43    ilm          0          0      16404
guest3                       12    ilm          0          0       5213
new guest bb           42    ilm          0          0      16404
guest4                      16    ilm          0          0       5213
guest5                      41    ilm          0          0      16477
guest6                      21    ilm          0          0       5213
guest7                      39    ilm          0          0      16477
vmhost #

[Dave Olker]

Refer back to my previous post where I said the hpvmstatus -C output is enhanced at version 6.1 to show which CPU cores are assigned to the guest.  That feature is not available in v4.3 as evidenced by the output you posted.

Dave