Number of VMs per VMFS volume *More of a Poll than looking for an answer*

[julez66]

Just curious about the number of VMs people are putting on their Nimble snapped VMFS volumes.

10? 15? 20? 30? 50?

I've been playing around with the appropriate number in order to get consistent snaps.  And I know this depends a lot on the type of server, the amount of data, etc...  But there's always an average baseline to work from.

I had started out with 5 2TB volumes and about 120 VMs.  Some of the volumes would snap ok, then we'd have one that probably had a more noisy VM than the others and would often fail.

And we were looking at roughly ~24 VMs per volume.  It seemed to be a struggle to find exactly which VM was the problem child so instead I decided to break out my VMFS volumes into 1TB volumes.  I'd prefer to have larger volumes of course like everyone would.

[Nick_Dyer · Accepted solution]

I personally tend not to recommend anything above 20 VMs in a VMFS.... especially if those VMs are heavily utilised. This is not just for snapshot quiecsing but also for disk IO queues & latency / disk locking.

Jason Boche did a great study on the appropriate number of VMs which is available to read here: VAAI and the Unlimited VMs per Datastore Urban Myth. His conclusion is:

• 10 high I/O VMs per datastore


• 15 average I/O VMs per datastore


• 20 low I/O VMs per datastore

[julez66]

Yeah that's normally my rule of thumb, but I also like to try to push the envelope to make sure I'm making the most of what I have.  From an I/O standpoint we really don't see any contention.  To give you an idea our 120 VMs on this side of our company averages barely 1800 IOPs during business hours, so I was hoping to stretch it a little bit more, but started running into snapshots not completing.  So was just looking for what others were finding success with.

[mr_vaughn]

Just another question is how many VMDK's per VM?

I remember reading in the far past that about 5-10 VMDK's per LUN depending on IO

But that has all changed with Nimble and how the visualization and speed of storage has matured.

[julez66]

Now that is a good question...I'm going to have to count that up, I know some have 1 VMDK per, some have 2 or 3.  There are definitely significant size differences between some of them.
Most of them though get away with 40gig or less for a system drive.