cancel
Showing results for 
Search instead for 
Did you mean: 

MSA1500 A/A

 
Bouzarelos
Occasional Advisor

MSA1500 A/A

We have the following topology

Two Clusters (each with 2 nodes)and 2 standalone servers are connected to MSA1500
Each server has 3 fiber cards.Zoning is configured so that the first card of each server is connected to the first controller through the first switch and the second card to the second controller through the second switch.Third card is for backup purposes and is connected to another switch zoned properly with a tapelibrary.
Recenty we migrated from Active/Passive to Active/Active firmware. However now we are facing some serious performance issues mainly with the backup. (The SQL users still continue to complain as before).I have some questions.

1)Do we need to keep zoning in the fiber switches now with A/A firmware?
2)In the ACU, setting the preferred logical paths to each controller will cause temporal failover to the clusters?
3)We have installed SanSurfer. Should we change the rate settings of the fibercards from Auto to 2GB.Is it going to help? Are there any other parameters we could change there to increase performance.

4)Yesterday we run HPReadData and the results we got were frustrating
Server1 D: 3.69MB/Sec

Server1 Ε: 1,59ΜΒ/Sec

Server2 D: 9.76MB/Sec

Clusternode2 H: 0.43MB/Sec


5) The problem seems to be more intense with the cluster nodes running SQL.Now we need almost double time to backup sql databases.

Any suggestions or ideas would be very helpful.


16 REPLIES 16
Puneet Arora
Occasional Advisor

Re: MSA1500 A/A

Hi,
There are couple of things you should keep in mind, here.
I am assuming that there is no ISL between the 2 switches.
1) Earlier since you had A/P firmware, and had 2 paths from each host to the MSA, neways one path was leading to a passive controller and thus was not seeing any LUNs, therefore only one path was active at a time, now because you have an A/A firmware, both the controllers can see the LUNs, thus you need to have a multipathing software on your Host Machines, eg: MPIO.
MPIO gives you an oppurtunity to do either Load balancing or choosing one of the 2 paths to do I/O
This is where the second point comes in the picture.
2) If you have both the Controllers on the MSA as active, have you enabled Prferred path settings? if you have then that leads to Explicit A/A configuration. which means, that one specific controller will manage some specific set of LUNs.
Now lets assume that if the I/O is happening from a 1st FC-HBA of a Host leading to Controller 1 of the MSA, but the LUN you are trying to acees is managed by Controller 2 (because of the explicit prefered path configuration in ACU), then Controller 1 has to communicate to Controller 2 through the Inter Controller Link and to talk to the LUN. now since Inter Controller Link is a serial communication, thus this could be a reason for the bottleneck.

Changing the the settings in HBA, should not make any difference. I would suggest to disable the Preffered path selection (not SSP-selective storage presentation) in ACU, or , Manage the same using the Multipathing software in such a way that it matches your prefered path selection.
Bouzarelos
Occasional Advisor

Re: MSA1500 A/A

Puneet first of all thanks for the response.
In the ACU we have set manually all the LUNS to be managed from Controller1. Also in the HP DSM Manager we have set as the preferred path Controller1. So i guess that traffic from all LUNS pass through Controller1.
Back to the Backup issue now. As i said all servers have 3 fiber cards 2 of which are connected to MSA1500 through SanSwitch1 and the other one through SanSwitch2 to the TapeLibrary.So data to be backed up go like this
MSA1500 -> SanSwitch1 -> HBA1 -> Server -> HBA3 -> SanSwitch2 -> Tapelibrary.

HBA1=Qlogic 2Gb
HBA3=Adaptec 1Gb

I tried to backup local disk C of Server and the speed was almost double. So I suppose that the bottleneck is between MSA1500, SanSwitch1 and HBA1.
Any suggestions?
Puneet Arora
Occasional Advisor

Re: MSA1500 A/A

So which A/A firmware version did you upgrade to?
Bouzarelos
Occasional Advisor

Re: MSA1500 A/A

We upgraded to MSA1500 A/A 7.00b1500 firmware
Bouzarelos
Occasional Advisor

Re: MSA1500 A/A

We are using SanSurfer to monitor our HBAs.
At the connections options in the HBAs there are three choises.
1)Loop Only
2)Point to Point Only
3)Loop preferred- Otherwise Point to Point

By default the third is chosen.

I read somewhere that when the Fiber Card is connected to a switch first and not directly to the MSA1500 Point to Point Only should be used otherwise performance is degraded. Could that be true and how Loop preferred is working?
Rajiv Kulkarni
Valued Contributor

Re: MSA1500 A/A

Hi,

Point to Point otherwise loop preferred is the default option because it is similar to auto negotiate option.

You may need to select point to point explicitly only in case the auto negotiation fails,

As far as I am aware there would be no performance degradation if auto negotiation is selected provided the neogitation is correct which means if an HBA is rated as 2Gbps and it has negotiated at that speed then there shouldn't be any issues,

Thanks & Regards
Rajiv Kulkarni
Puneet Arora
Occasional Advisor

Re: MSA1500 A/A

I am not sure about the truthness of that staement, but Loop will not be working in this case.
That because, when you connect a FC-HBA to a SAN Switch, it goes through a F Login process, which doesnt happen in case of Loops (LIP happens in that case. This let the FC-HBA know that its a part of point to point connection not loop.
In your case if the 3rd option is choosen then FC-HBA will work in case of Loop as well as Point to Point, depending on what is the HBA is connected to.
Now if you want, you can try moving from 3rd to 2nd, which is Point to Point. See if thats the reason for bottleneck.
Bouzarelos
Occasional Advisor

Re: MSA1500 A/A

It seems that changing the HBA parameters didnt make any difference..

Performance is still very poor. It got worse after migrating to A/A.

I have a question about HBA FC2214 drivers.We are using 9.1.0.13 version and i found 9.1.3.11 version but hesitate to update as we dont know if there is compatibility with MSA1500 A/A.

Server is HP DL585 Windows 2000 SP4 Cluster.

After migrating to A/A HP did not mention anything about updating HBA drivers.
Rajiv Kulkarni
Valued Contributor

Re: MSA1500 A/A

Hi,

Could you login to the MSA 1500 using Serial console (hyperterminal) and type "Show tech_support" at the prompt.

Capture the configuration using Capture Text option in hyperterminal and post the same.

We could then analyse the log file and found that what could be broken.

Thanks & Regards
Rajiv Kulkarni
Bouzarelos
Occasional Advisor

Re: MSA1500 A/A

Rajiv, here is the show tech_support.
Rajiv Kulkarni
Valued Contributor

Re: MSA1500 A/A

Hi,

Appreciate your quick response in sending the logs.

Since the firmware version 7.0 is an active/active one there is one important line to consider in the logs, i.e

"Implicit (automatic load-based) LUN ownership changes are currently DISABLED."

which means that as of now all HOST I/O takes place only through Controller 1.

The other thing which needs to be checked is whether load balancing is enabled on any of the hosts where multipath software is installed. In case of active/active firmware on MSA the recommended option is to disable load balancing on host end and enable implicit configuration on MSA.

Hope this helps..

Also please ensure that the drivers and firmware on all the HBA's are up to date.

Thanks
Rajiv
Rajiv Kulkarni
Valued Contributor

Re: MSA1500 A/A

Hi,

Attached zip file containing PDF's which might be useful.

Thanks & Regards
Rajiv Kulkarni
Bouzarelos
Occasional Advisor

Re: MSA1500 A/A

Rajiv

Load balancing in the side of the hosts is set to none.

In the ACU now we have manually set that all IO goes through right controller as this is what happened when we had Active/Passive Firmware.
However i can't understand why the performance got worse as IO traffic travels the same way. I would expect to be the same and then by setting automatic load balancing performance would be improved.

Almost everyday backup needs about 8 hours to finish instead of 2,5 hours with A/P firmware.

John Kufrovich
Honored Contributor

Re: MSA1500 A/A

Bouzarelos,

If you have the Host(HBA) load balancing set to none, your HBA is using a preferred path for IO. And your MSA is running with implicit failover disabled. There is a high probability of proxy IO happening.

If you want to keep "none" for HBA load balancing. I recommend, enabling (Automatic in ACU) implicit failover. This will allow the MSA controllers to move luns based on your perferred path setting on the HBA.



Bouzarelos
Occasional Advisor

Re: MSA1500 A/A

John thanks for the response

Thats how everything was configured at first. Automatic in the ACU and Load Balance was set to none.

But the problem was still there.

I think HP recommends Load Balance is set to none.

Now HP asked to create a new LUN with stripe size 64k (we use 16k) and make some read/write performance tests.

Could stripe size be such a bottleneck?

I am watching the backup process and i can see it sometimes going like crazy (1GB/MIN) for about 5-10 minutes and then it freezes and falls to 1-2 MB/SEC ~ 60MB/MIN for long time._
Ola Andreas Windingstad
Occasional Visitor

Re: MSA1500 A/A


hi!

Any progress/ solution in this case,, i have a customer with much of the same symptoms..

I also se in this case that its not mentioned that effective cache in the controllers probably gets reduced to 50% when customer uses active/active on msa1500.
this cuz of now the cache shold be mirrored from both controllers....this can in some cases affect performance


cheers
Ola