topic Re: Link aggregation consideration for synchronious Nimble replication in Array Setup and Networking

Link aggregation consideration for synchronious Nimble replication

expert34 — Fri, 26 Aug 2022 01:37:33 GMT

Hello,

we moved one of our Nimble storage to a new location. The 2 sites are connected with 2 10Gbit fibre uplinks on HP FlexFabric 5710 switches on each site.

Is there any recommendation how the LACP should be configured for synchronious replication? For example load-sharing mode?

The configuration of the LACP is the following:

interface Bridge-Aggregation4
port link-type trunk
port trunk permit vlan all
link-aggregation mode dynamic

Bridge-Aggregation4 load-sharing mode:
Layer 2 traffic: packet type-based sharing
Layer 3 traffic: packet type-based sharing

kind regards,

Juergen N.

Re: Link aggregation consideration for synchronious Nimble replication

Mahesh202 — Mon, 22 Aug 2022 10:55:52 GMT

Hi expert34
Hope you are doing well..!

We use device-level MPIO for optimal redundancy and performance for data paths between hosts and storage.
Array NIC ports cannot be teamed, aggregated, or bonded in any way.
The switch ports used to connect to the Nimble Array interfaces cannot be EtherChannelled, aggregated, or bonded.

NIC teaming and host link aggregation are traditionally used in NAS storage, to allow NIC redundancy when connecting to CIFS/NFS shares (where MPIO does not apply).
Host-side NIC teaming will over-ride, and can actively interfere with, initiator-side MPIO behaviors, sometimes causing unpredictable behavior or performance issues. There are no workarounds to mitigate these behaviors when they occur. These issues are a direct result of the design of 802.3ad/802.1AX link aggregates, both passive (non-LACP) and active (LACP).

Hope this helps.!!

Regards
Mahesh.

Re: Link aggregation consideration for synchronious Nimble replication

expert34 — Mon, 22 Aug 2022 14:30:32 GMT

Hello Mahesh,

I'm not talking about the connection between storage and switch (no aggregation used), but between the two core-switches. It was just an idea of mine if this configuration has any impact on production. At the moment everything seems to be fine. We have a latency of <=2ms between the to sites.

kind regards,

Juergen N.

Re: Link aggregation consideration for synchronious Nimble replication

Mahesh202 — Tue, 23 Aug 2022 05:58:53 GMT

Hi expert34

In your case, there is no LACP between Nimble and switches but only between core switches - it is okay to do LACP trunk between switches
That said if link aggregates are to connect switches to each other. This is allowed in an iSCSI environment as long as the link aggregates are built in a manner supported by the switch manufacturer(s). Using a link aggregate between switches, however, can cause it to be difficult to understand what the actual physical path of the traffic through the network is.

As far as HPE Nimble is concerned

1) NIC teaming and link aggregation (including LACP) cannot be applied to Nimble array NIC ports.

2) Sync replica uses a replication-group traffic link. We don't recommend LACP on replication links as well so there is no best setting to suggest for synchronous replication.

Hope this helps.!!

Regards
Mahesh

Re: Link aggregation consideration for synchronious Nimble replication

AishwaryaA1 — Wed, 24 Aug 2022 13:58:16 GMT

We don't recommend to use LACP for iscsi or replication.

When it comes to storage they use LACP for NAS (Not SAN). With SAN the host side multipathing software takes care of evenly distributing traffic . Even for replication, we don't recommend configuring LACP because replication will only use 1 network link.