Query: Alletra MP : configuring even/odd ip address zones

[IMPI-Ced]

Hi,

when reading docs i can find this interresting feature. I plug all even IP on switch 1 and all odd on switch 2. I can see some trafic are going throught inter-switch link.

Then i'd like to activate the ip address zone but found nothing into GUI.

I have the CLI reference guide for 6000/nimble and found some commands but they seems non existant on my MP

Do you have some advices ?

Thanks

[support_s]

System recommended content:

1. CLI Administration Guide for Array OS 6.1.2.x | IP Address Zones in Subnets

Please click on "Thumbs Up/Kudo" icon to give a "Kudo".

Thank you for being a HPE valuable community member.

[IMPI-Ced]

Thanks,

this is exactly the doc i'm talking about : there is some commands to do but none are working, it seems to be a different language on MP

Is there the same for Alletra ?

Thanks

[buzzsubash]

Hello IMPI-Ced ,

You are right, NimbleOS commands are supported only for Gen5 (AF-HF series), Alletra 5k and 6k series arrays and may not be availble on MP block arrays. MP series are a different product line altogether.

[IMPI-Ced]

Hi buzzsubash,

thanks for this clarification.

But then how can i do ? Where can i find the right command ?

Thanks

[buzzsubash]

Please check and let know if this link helps https://support.hpe.com/connect/s/product?language=en_US&kmpmoid=1014740902&tab=manuals

[IMPI-Ced]

Many thanks for that,

i downloaded CLI and UI docs.

I did a search and was unable to find the word "odd" then this doesn't smell good !

In fact it seems there is no topic about "ip address zoning", maybe it now have another name ? I hope they did not stopped this functionnality because i do not see how to be sure we do not go throught interswitch link.

Ced

[Cali]

Hi,

you can find the "How To" for iSCSI Implementation here:

https://support.hpe.com/hpesc/public/docDisplay?docId=sd00002428en_us&page=index.html#hpe-greenlake-for-block-storage-vmware-esxi-implementation-guide

Cali

[IMPI-Ced]

Many thanks for that Cali,

i read that but it was for Alletra 6000, this one should be more adapted !

Please allow me some time to review the doc and tell you

Have a nice day

Cédric

[IMPI-Ced]

Cali,

i read the docs but wasn't able to find something like "odd/even IP address zoning", then how can i avoid trafic to go through interswitch link if not usefull ?

I plugged all odd interfaces to switch 1, all even to switch 2, but i had trafic throught uplink. Then i unplugged uplink and then all went ok, even when i plugged again uplink, but this is not really a solution

All iscsi trafics are using the same subnet (replic has its own subnet), as it is stated possible into documentation

Maybe there is something i am misunderstanding.

Thanks for help

Ced

[Cali]

I think, that's the problem.

It is possible and supported, however if you look in the Doc you see

HPE (and I also) using different Subnet by Server Port.
So usually I have 4 Subnets.
Server "Upper" Way
Server "Down" Way
Replication "Upper" Way
Replication "Down" Way
And most time with iSCSI only Switches i did not interconnect "Upper" and "Down" Switches (same as in FC).
Going this i have fully independend transport ways and no "Bad Frames" can disturb the second way.

Cali

[IMPI-Ced]

Thanks for that !

I never used FC and then i'm not used with, sorry

I'm not sure when you said "Upper" and "down" if you talk about physically (the switch that is that the upper place, the one at the lower) or if you mean a switch managing upstream and the other managing downstream. As you see english isn't my native language

I can see that the Alletra, by itself, uses different ports for upstream and downstream even the way i configured it ! That was something i found weird when i was on my switches looking where traffic was going

By the way do you mean :

   - Each ESX have two vmk, one on subnet 1, the other on subnet 2

   - Each odd port on each Alletra (2) and each nodes (2) on subnet 1

   - Each even port ...... on subnet 2

   - Put vmk1 and odd ports of Alletra on switch 1

   - Put vmk2 and even ports of Alletra on switch 2

No interswitch link, in this case it seeams useless

Here i'm only talking about iSCSI trafic, but same should be done with RCIP using two other subnets with the same odd/even ports rules

With this, each port of the ESX can access each Alletra. Then with two subnets how can i be sure that ESX will use its vmk1 to go to volumes on Alletra1 and vmk2 to go on Alletra 2 ? Because it also can use vmk1 to go to Alletra 2 using subnet1, switch 1, port 0:4:1 of Alletra 2. And then all trafic will go through vmk1, using only one switch, this is a pity as it could be separated and use both switch, both vmk.

And then i do not understand how it is better that one subnet ? Sure ther is something i misunderstood.

As this is difficult to talk about that by text, i did a little drawing, that could be great to know if it's ok and how to managed prefered trafic. I could complete it and i hope it will be useful for other member !

Many thanks to you

edit : it seeams that when you say "upper way" and "down way" this is corresponding to "right" and "left" switch on my drawing nothing to deal with upstream and downstream, forget that

[IMPI-Ced]

Update : today we configured our testing system juste as above (2 iSCSI subnets, 2 RCIP subnets) and obvioulsy it is working

But only vmk2 is used, no load balance with vmk1, both Alletra are accessed using vmk2... this seems quite normal as there is nothing that tells the Alletra/ESX what do to.

Another weird thing is that ports on node 1 are used at the same time that some on node 0... I thought Alletra MP was active/passive. I can see on the switch that there is trafic on both node (often one with RX and no TX and the other with TX and no RX)

Ced

[Cali · Accepted solution]

Have you set Round-Robin for your Volumes?

Please check, default is "Fixed-Path".

https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.storage.doc/GUID-AE95818C-0782-429B-9526-DF12673A63C7.html

Set it as default on Server CLI, as described in the HPE Implementation Guide.

Cali

[IMPI-Ced]

Many thanks for that Cali,

i totally went over this notion, sorry. I only read iSCSI part of the doc, and this is just before, between FC and iSCSI there is a LUN section, talking about round robin.

I did it and now this is ok, all ports of ESX are talking.

Then, to be sure all is ok, i'll reread all the iscsi doc to be sure it did not forgot another important notion !

Please allow me 2 more questions

   - is my diagram above correct ? This is what you was waiting for ?

  - i can see that even Node 1 ports are talking, in fact all ports are now talking... I thought Alletra was active/passive and that Node1 will be use when node 0 failed.

Once again many thanks for your valuable help

Ced

[Cali]

Hi,

yes this is what I mean.
Typical Customers have two Rooms with two switches in every Rack and I Name them Up and Down, by physical position.

About Node (Controller), yes both are Active in 3PAR/Primera/Alletra MP, this is different from Nimble & Alletra 6000 and some others.

Cali

[IMPI-Ced]

Ok, Cali, thanks !

This is important what you're telling me as on another thread i was replied it was active/passive, many thanks !

Now all seems ok, we did many tests and this is great.

When unplugging a cable/fibber or a switch we get a "storage freeze" of about 30-35 s then it goes on again. Is that delay correct. ? I tought it was more 10 seconds.

Be aware that your replies really helped us a lot, always accurate and efficient, you deserve tons of kudo

Ced

[buzzsubash]

For nimble/Alletra 5k-6k arrays recommended HBA timeouts for VMware from VMware Integration Guide

LoginTimeout: 30 seconds (default - 5)
NoopTimeout: 30 seconds (default - 10)
NoopInterval: 30 seconds (default - 15)

I will let Cali answer if it is the same for MP block arrays.

[Cali]

Normally any Path Switching under 60s is no problem.
I recommend the following setting:

esxcli storage nmp satp rule add -o "throttle_sll" -s "VMW_SATP_ALUA" -P "VMW_PSP_RR" -O "policy=latency" -c "tpgs_on" -V "3PARdata" -M "VV" -e "HPE Custom iSCSI/FC/FCoE ALUA Rule"
If I remember correctly, this only applies to datastores that are added afterward.
You may try this also

Disabling Delayed ACK in ESXi
vmkiscsi-tool -W -a delayed_ack=0 -j vmhbaxx (vmhba64?)
vmkiscsi-tool -W vmhba64

or

Log in to the Web Client and select the host.
Right click the host and click Maintenance Mode.
Wait for the enter maintenance mode task to complete.
Navigate to the Configure tab.
Click Storage Adapters.
Click the iSCSI vmhba that you want to modify.
Modify the delayed ACK setting on a discovery address :
Click Targets tab on Adapter Details.
Click Dynamic Discovery.
Click Advanced Settings.
In the Delayed ACK Advanced Parameter options, uncheck inherit from parent and DelayedACK.
Click OK
Reboot the host.

It usually leads to much lower latencies in iSCSI environments.
I see Primera Latency going down from 20ms to 1ms, but did not Test with MP.
P.S. Thank's for KUDOS 

Cali

[IMPI-Ced]

Thanks !

I have a little script that copy files and calculate the time to do that, with your parameters we went from 1.8-2.1 sec to 1.2-1.5 which is a real enhancement.

I'm aware that this is a not a real condition test and it has no real importance but it shows improvment.

We only applied your parameters and did not touch the one buzzsubash gave us (LoginTimeout, NoopTimeout, NoopInterval).

Regards

Ced

[Cali]

Great
Hopefully they are ms and not sec.
Cali

[IMPI-Ced]

Hi Cali

No, i'm talking about sec, this is the time the Alletra takes to copy (with verification) a 400-500 GB files, then this is ok

Ced