MSA 2040 Tiering very low IOPS and awful performance

[oli_4]

something is wrong with the MSA 2040 performance tiering, i can only have about 5k IOPS out of it while i have another SAN P2000 G3 which is providing 16K for the same workload.

Both SAN are FC are connected through HP 8/24 SAN switches. can you help me diagnosis the issue ?as i am running out of ideas

we are using HP DL 360p Gen8 as hyperV host for the virtualised workload.

[HPSDMike]

Hello,

Please consider uploading configuration information so the group can help. Please also consider opening an HPE support case if your device is covered under support. 

To generate text file of the config:

• Using Putty (or other SSH client), open a connection to your MSA and log in. 
• Turn off page display pausing by typing 'set cli-parameters pager off' 
• Turn on output logging from Putty to a text file
• Run the command 'show configuration' 
• Upload the text file to a message on the thread

Note that this file will have information in it about your host names, IP addresses, device name, etc. If you are concerned about sharing that then you can edit out what you are concerened sharing or I can private message you a URL to upload the file to. You can also generate a support log and upload that to the URL I send. Attaching something to this message will allow more people to view and suggest options versus private URL. Note that I will be out of the office EOD today for 9 days. I'll take a look at anything that gets uploaded over the next few hours. 

[oli_4]

kindly private message with a URL.  so i can upload the file

[HPSDMike]

Done, please check private mail. 

[oli_4]

done on the URL given.

I am having issues to opload the file here. it doesnt accept txt file format?

[HPSDMike]

File received

[HPSDMike]

I've received and reviewed the config information. For the benefit of the group, I'll post a summary so far.

• Check your server host BIOS power profile settings to make sure all hosts are set to "Maximum Performance". This is the cause of 90% of the storage "performance issues" I have seen in the past. 
• Looking at the MSA config itself, it looks pretty good. Everything looks healthy and you've done a good job keeping your RAID set sizes identical and to a power of 2. 
• One thing you have done, which is not best practice, is that you have 10 and 15k disks in the same pools and they are different sizes.
• The disks in your 15k vdisks are totally full (as would be expected), the 10K pools have an even amount of space which is good and means they are balancing out
• This will cause all new writes to go only to the larger 10k disks. Reads will be unpredictable. Older data will come from both the 15k and 10k disks, newer data may only come from the 10k disks. While this isn't best practice, I'm not sure it's the cause of your issue. I would lean towards host power profile settings, zoning, or multipathing issues. Please check those items. 

[HPSDMike]

Any update on this situation following the suggestions I provided? 

[oli_4]

thanks mike for the followup, i can confirm the power settings in BIOS are already on maximum performance.

unfortunately i havent made much progress in correcting the issue.

Regarding the mix of 15K and 10K disk, how bad can they impact the performance, I presume any degradation would take it to the lowest speed which is 10K, also i made sure that those disks are not used in the same disk group to avoid any issue. the Auto tiering should be able to differentiate the disk groupd speed difference I guess

one question though, the fabrics and all the the FC components run at 8GB/s, do you think that could be the issue? i just noticed that the benchmarking of MSA 2040 on HPE site is only related to 16GB/s, i would be interested to know what they give on 8GB/S 

[HPSDMike]

A healthy 8GB fabric will do just fine. 16GB is faster (obviously) and overall more efficient but I've seen very heavy workloads do just fine on 8GB (and 4GB for that matter). 

Here is how the MSA defines a "tier"

• Performance=SSD
• Standard=15k or 10k
• Archive=7.2k

You can't change this. Therefore; 10 and 15k drives, in the same pool, will been seen as the same tier. IO's will therefore have the potential to perform at the lowest common denominator. In this case 10k. Also, if you have different RAID types or set sizes in the same tier (within a pool) then IOs would porentially operate at that lowest common denominator as well.  

We'd really need to dig into your "workload" more. How are sure that the workload is identical between your environments? What's your percentage of read/write? Block size? etc? 

Can you supply me with some performance info? Like outputs from:

• show controller-statistics
• show host-port-statistics
• show disk-group-statistics
• show disk-statistics

Also, if you haven't already, you may want to open a support case.

Thanks.