HPE EVA Storage

hp eva 4400 performance issue with CA replication and MPX110

 
Osama Odeh_1
Regular Advisor

hp eva 4400 performance issue with CA replication and MPX110

when we create DR group by HP command view a critical performance issue appear on HP storage eva4400 (main site) many of vmware servers related to that vdisk lost path
and som times freez for seconds and cannot ping guest servers
we check event log for EVA4400 we found this message loged many times:

Excessive PING response time on the inter site link is preventing acceptable replication throughput: Reducing data exchange resources.

on vmware console we found this event:

Successfully restored access to volume 4a5dbb9e-eba9af52-c359-0022649da44b (server100) following connectivity issues.info

Lost connectivity to storage device naa.600508b4000a54db0000900000400000. Path vmhba0:C0:T0:L3 is down. Affected datastores: "VM_datastore1".error 1/4/2010 4:48:34 PM

Path redundancy to storage device naa.600508b4000a54db0000900000400000 degraded. Path vmhba0:C0:T0:L3 is down. 2 remaining active paths. Affected datastores: "VM_datastore1".warning 1/4/2010 4:48:07 PM
** we update firmware to the latest but the still same problem

more details:

Main Site EVA 4400 : version XCS 9522000
DR Site EVA 4000 : version 6.22
VMware : ESX4

we use HP distance Gateway MPX110 for replication between two site.
WAN speed 5mb distamce (60KM).

thanks in advance for any suggestion .
20 REPLIES 20
Patrick Terlisten
Honored Contributor

Re: hp eva 4400 performance issue with CA replication and MPX110

Hello,

do you use synchronous oder asynchronous replication? What round-trip time do you have on the WAN link?

Best regards,
Patrick
Best regards,
Patrick
Víctor Cespón
Honored Contributor

Re: hp eva 4400 performance issue with CA replication and MPX110

Before creating vdisks for the production vdisks, you should check the status of the replication link. Look in the SAN switches counters, and EVAperf counters.

Check CA implementation guide too...
http://bizsupport2.austin.hp.com/bc/docs/support/SupportManual/c01800459/c01800459.pdf

The most common error is bot to change the needed parameters on the SAN switches (In-onder delivery, port-based routing, increase the buffer credits on the ISL ports...)
Osama Odeh_1
Regular Advisor

Re: hp eva 4400 performance issue with CA replication and MPX110

Hi Patrick
* we try to use synchronous and asynchronous replication when we creating Dr group the same problem appeared on both.
* round-trip time on the WAN link =45-55 ms

Osama Odeh_1
Regular Advisor

Re: hp eva 4400 performance issue with CA replication and MPX110

Hi vcpones
we already applied the port-based routing before:

FCSW1:admin> switchdisable
FCSW1:admin> aptpolicy 1
FCSW1:admin> switchenable

I have many questions:

1- are you recommend to run the follwoing command line on ISL ports.
portcfglongdistance 0 LD 1 69

2-
** also I read the following on brocade: to determine the optimal amount of credits:
Credit=(Round_trip_time+Receiving _port_processing_time)/Frame Transmission _time.
In other words, the optimal number of BB credits depends on three key parameters:
1) round trip time, i.e., the distance
2) frame processing time
3) frame transmission time*

how can we calculate that for our environment and what is the best configuration for that.

3- how to to setup In-onder delivery you mention.


Thanks

IvanForceville
Regular Advisor

Re: hp eva 4400 performance issue with CA replication and MPX110

Apart from the port based routing make sure you set the following parameters

Needs to be set on the SAN switches
IOD (In Order Delivery) "Enabled"
DLS (Dynamic Load Sharing)"Disabled"

Can be done via the GUI (http://ip) of the switch => Switch Admin => Advanced

Or through CLI:
"iodsetâ en â dlsresetâ

Make sure you set these parameters on all switches in the fabric.

By the way... we have the exact same setup exept for the target EVA which is a 4400 but also the MPX110's and the 60km's DR :)
Osama Odeh_1
Regular Advisor

Re: hp eva 4400 performance issue with CA replication and MPX110

Still problem not solved also after we set :
1- IOD (In Order Delivery) "Enabled"
2- DLS (Dynamic Load Sharing)"Disabled

the same performance issue appear when establish the connection

" Lost connectivity to storage device naa.600508b4000a54db0000700000920000. Path
vmhba1:C0:T0:L4 is down. Affected datastores: "VM_datastore1".
error
1/13/2010 1:30:33 PM


Path redundancy to storage device naa.600508b4000a54db0000700000920000 degraded. Path
vmhba1:C0:T1:L4 is down. 3 remaining active paths. Affected datastores: "VM_datastore1".
warning
1/13/2010 1:27:54 PM "

Osama Odeh_1
Regular Advisor

Re: hp eva 4400 performance issue with CA replication and MPX110

Does this performance issue related to the link speed between main site and DR site( 5M) or related to EVA storage or SAN switches.

thanks in ADV
IvanForceville
Regular Advisor

Re: hp eva 4400 performance issue with CA replication and MPX110

Just to have an idea... how big are the Vdisks you replicate? What's the load on the Vdisks replicated.

We've seen some realy strange behaviour of CA with low bandwidhts.

Another thing: how are the MPX's configured
Do you have any compression enabled? What's the scaling factor? Do you use a QOS on the link between the sites? Is the 5MBps guaranteed or is this max bandwidth?
Osama Odeh_1
Regular Advisor

Re: hp eva 4400 performance issue with CA replication and MPX110


thanks IVAn for your questions:

we try with Vdisks 90 gb and 130gb for replicate the same issue.

we try mpx110 without compression enabled and with the same problem.
we try the scaling factor disabled and with 0

about QOS it is disabled between two site because we configure the ports as ISL and

the 5MBps used also for another network and replication for oracle around 300mb data transaction>

again my question if the bandwidth not enough between two site make the source eva very busy????