performance issues on near-online disk-group on EVA4400

[iulianv]

yesterday i noticed some weird behaviour of all virtual disks in our 8 x 1GB FATA disk-group - most of them run Linux (CentOS 5.2 or RHEL 4.7), and "hdparm -t" outputs some incredibly low values, ranging from 2MB/s (?!) to 40MB/s.

the disk-group is in Leveling since the day before yesterday, but i've read that the Leveling process should not impact performance.

is there any other tool that i can use to measure this more accurately? are there any things to look for that can cause this behaviour? i'll wait for the Leveling to finish and i'll test again, but i fear similar results...

there's also a 24-drive (146GB, 15Krpm) group in that storage, which doesn't show this issue

[IBaltay]

Hi,
you can see the eva performance via the evaperf utility, this thread links some
useful documents:
http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1178928

useful details:
http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1239472

[iulianv]

right after posting here i did some research and found out about EVAPerf - all parameters that i thought relevant look good, except for the read and write latencies for the near-online disk-group, which show spikes of hundreds of miliseconds every once in a while

the thing is that the spikes don't happen that often (one or two per minute, as graphically shown by Windows Performance Monitor), while the poor performance of the vdisks seems to be permanent

anyway, the local HP support guys will pay us a visit today to investigate another problem related to the near-online disks, and i'll tell them about the performance issue as well

[iulianv]

everything back to normal after leveling was finished, so leveling does have a severe impact on performance, at least for 8*1GB FATA disk groups... i'm looking forward to see how leveling on the 24*146GB FC disk group behaves :)

[IBaltay]

Leveling is always the background process and it depends on the "production" io. If it is high, then it can take few days, if its lower, it is quicker... It can have the higher priority then the ungroup operation only if you have no spare drive (NONE disk protection e.g. after the disk failure, etc.). In that case the controller tries to end the leveling first... and it is blocking the ungroup...

[sreekanthtm]

The same performance issue will be here also.... As u said, "Levelling will impact the performance seviorly".
One more thing -- Near online disks means, the performance of the disk itself is low. may be 10Krpm diks. So leveling will cause more performance impact in near online disks

[Jorge Cocomess]

Julianv - What was the full command line when using "hdparm -t"??

I've been running "sar" and "iostat" for monitoring and see in the hundreds on some of the SAN luns at my site.

Thanks,

J

[iulianv]

just "hdparm -tT /dev/mapper/mpath0" or "hdparm -tT /dev/mapper/vgname-lvname", for measuring cached reads and buffered reads...

i find it strange that i get significantly higher values if i hdparm an LVM logical volume than the whole multipath device - this is why i said hdparm may not be highly relevant

i'll look into the sysstat utilities - thanks for the hint