LVM Performance vs. # of disks

[Reston Tech Support]

There's a debate going around among the SAs in our group. We have some V-Class servers hooked up to EMC frames, and the question concerns the max # of disks in a volume group to maximize performance.
One guy says his instructor in class said there shouldn't be more than 4 or 5 disks in a volume group or you will suffer performance issues. When I called HP, the CE said the amount of disks in the volume group does not impact its performance and you can have as many as you want.

Does anyone else have an opinion on this?

TIA
FD

[Stefan Farrelly · Accepted solution]

Ive never ever heard of anyone staiting before that over 4/5 disks in a VG slows down performance ! Im astounded someone would make a claim which in my opinion is not proveable. In fact the more disks in a VG - so that you can stripe lvols across them - increases IO performance, not decreases it !

[Wim Rombauts]

Stefan is right.

The better you can spread your data over multiple physical disks, the better your performance.

[harry d brown jr]

I have to side with HP on this one. Creating too many volume groups and too many logical volumes is a management nightmare and should be avoided.

Was it an EMC instructor that said that VG's should only have 4 or 5 disks in it?

[harry d brown jr]

Imagine this: You have a database or some other data-pig that is one terabyte. You have an array of 18gb disks. Using the 4 or 5 disks per VG would force you to create 12 VG's. Was the instructor having a bad acid day?

[Reston Tech Support]

Well Harry, that indeed is the issue. We do have systems with terabytes of space on them. Some of our Oracle (project) volume groups are well in the hundreds of gigabytes. I (and others) like to engineer it so that we have one vg per project, with about 8 lv's per vg. Managment wise, it works out nicely. But when we spread a project among 4 or 5 vgs because of perceived performance gains, it does make managing it somewhat more difficult, which is what raised the issue to begin with.

[Thierry Poels_1]

hi,

maybe there was some confusion: more than 4-5 disks in one SCSI-chain might decrease performance because the SCSI-chain might be overloaded.

regards,
Thierry.

[Hrvoje Dubravica]

Hi,
I also have HP-EMC and HP-XP env. and many
vg with various number of disks in volume groups. I havent seen # of disks per vg influence on performance. Most importen thing is on storage side. You must take disks from
diffrent controler units (XP), disk directors (EMC) to have better perfomance, and use more
then on FC adapter on host side.
:-)

[harry d brown jr]

Reston, stick to your guns, because you are doing the right thing!!!!

[Stefan Farrelly]

After some more thought I came up with the same answer as Thierry has already replied - someone has mixed 4/5 disks per VG with the old performance constraint of 4/5 disks per controller (FWD SCSI type) !

[Uday_S_Ankolekar]

Hello,

There is no restriction as such.. but more disks will improve performance when you spread across your big database. Your friend might have confused with controller and disks.

-USA..

[Reston Tech Support]

Yes, spreading across mulitple SCSI controllers I agree with, and have in fact implemented that on our SCSI based machines, for the reason you all have indicated. And thanks a lot for your detailed responses.

But, as you all have confirmed, across fibre channel cards, especially when managed by EMC load balancing software (Power Path), performance is not hindered by the LVM vs. physical disks.

I'm 1 for 1 today!

[Deshpande Prashant]

HI
Do not restrict the volume group for such less number.
With more disk in VG, chanses are better for higher performance and also you have chanse to strip the lvols.
I have some VGs with 35 disks( 9 gb from EMC frame)

Thanks.
Prashant.

[Tim D Fulford]

Hi

I know this is an old thread but I THINK I know what may have gone on here. The EMC guy probably said something like "4/5 disks can flood the buss". This is actually true as a disk 15,000 rpm disks has a burst transfer rate of 50-75MB/s, thus 4 or 5 could THEORETICALLY flood a 2Gbit/s buss. BUT practically speaking the chances of 4/5 disks doing a burst transfere at the same time are slim. This is why volume groups with 20 thimes this number of disks are possible.

OK fluffy cloud stuuf done from here on in it is mathematics. If you hate maths of are adversed to techish then stop reading now...... Now I assume the audience is down to one or two Quantum physicists or mathematicieans. If so I applologise for the quality of the calculations, it is from memory....

p is the probability of a single disk doing a transfer. it is the ratio of the time it takkes to transfer one IO BLOCK to the total time seeking & transfereing.
q = 1-p
Nd = total number of disks on buss
Also asume all the disks are in one stripped group.
Assume that just over 4 disks wil flood the buss
(p+q)^Nd = binomial expansion of the chances of the disks transfereing data at the same time.

P= probability of flooding (1-P)
Q= probability of not flooding (1-Q)

Q0 = q^Nd
Qn+1 = Qn*p(Nd-n)/(n+1)/q
Sum Qn from 0, 1, 2 .. 4 (say)

so IF there is flooding then only 4 disks worth of IO can go ahead, if not then Nd worth of IO can go ahead...

IOrate for buss = IO for disk * ( P*4 + Q*Nd)

You can code this up & pritty soon you will see that at about 35 - 70 disks worth you start to see buss flooding... I'll attach a more detailed paper tomorrow & Excel sprad for the 1-2 of you who may be interested!!!

Regards

Tim

[Tim D Fulford]

Yer tiz

[Tim D Fulford]

Yer tiz

[Tim D Fulford]

Try again (3rd times a charm)