KVM to slice and dice a Large X86 Server

[Alzhy]

I've a 48-way Server (8x6-core X86).
It has 6 Combo-4Gbit/Gigabit Ether HBAs.
I am using KVM within RHEL Advanced Platform.

I carved 3 KVM Guests running RHEL 5.X with 16 VCPUs each and oodles of memory each.

I now want to add Physical HBAs (2 each) to each KVM guest. But it is not allowing me to. I guess because the underlying Host OS running KVM has a lock on it?

My Host OS running KVM resides on local SCSI Raided Disks as do the KVM OS Images.

How do I free up those PCI/IO paths from teh host OS so I can add them into the Guest's Devices?

Or is this not at all possible? And That I have to use the host OS, carve up my storage on those HBAs and pass on the virtual disk to the Guest?

TIA for any guidance and opinion...

[dirk dierickx]

Indeed, you should create separate filesystems with virtual disks on them and add those to your guests.

what you want to do, normally works well in HW partitioning, which is not the same as a software virtualisation solution.

[Alzhy]

But it SHOULD be possible to add Physical Hardware (aka a PCI/PCIe Slot) to a KVM Guest Server OS right? The fact that virt-manager has it is there for a reason.

Anyone ever successfully done this? Any host OS gyration to perhaps free the lock on the HW Paths for the PCI slots being offered to the KVM Guest?

[Alzhy]

KVM under RHEL 5.5 supports it. And it works like a charm and very fast too and "all native" compared to vSPhere.

Presenting raw block devices as KVM guest storage is rather painless too and one can use Device Multipath friendly names.

Really awesome that KVM within RHEL (and we're not even taling about RHEV here yet) is really a good, cost effective way to get the most out of today's multipcore, multi-threaded systems!