issue to move from direct-connected to switch-connected

[jmeshal]

Hello 

we’ve project to add two more nodes to current setup (two nodes as direct-connected)

recently we’ve change the disgn to (switch-connected) connecting to our Juniper 10G switch (Juniper QFX5100-48S-AFI 48-Port SFP/SFP+ 10GbE / 6x 40 GbE QSFP+ Network Switch) with assign siprate vlan (504) so now the setup comes like this:

But it seems to me that there is a problem. I can not identify, through constant observation shows that the datastore of Simplivity01 Host is completely disappear from Simplivitu02 Host 

but vice versa seems works but not test yet

based on that system canon do the vmotion get error like this

Migrate virtual machine

New SMSC Server

The operation is not allowed in the current state.

Migrating the active state of Virtual Machine

System

Vcenter-Simpli.xxxxx.com

10/31/2019 8:54:48 AM

10/31/2019 8:54:48 AM

10/31/2019 8:57:32 AM

Now all VMs are running from one node only Simplivity01 except the OmniStackVC

also I found like this on Simplivity tab  shows

In fact, I can't plam network administrators who are setting up the Juinber Switch as they claim that they have met the basic requirements creating a separate vlan and add it the ports on it.

Any idea, for this 

thanks 

[dhooley]

Hi @jmeshal ,

There is really far too much to go through over this forum, so I would highly recommend getting a support case opened with our support team.

At the moment I can see that you have 3 problems:

1) SVTFS service on SimpliVity02 is in a faulty state and needs investigating.

2) IP failover appears to be failing in your setup, which has resulted in storage access being  lost on SimpliVity02.

- One thing to check on that is your NIC teaming configuration at the Storage vSwitch level. For a Switch based setup, we would recommend having the two 10GB NIC's in an Active/Active setup on both nodes.

3) vMotion issue. I suspect this is being caused by the storage access issues mentioned in point 1.

Get a support ticket opened and this should be a quick and easy fix once we get onto the system.

Regards,

David

[LiamP]

Besides the physical change, the only addition was tagging the port group to reflect the addition of vLANs?

(and changing the physical interface to trunking)

No changes were made to IP ranges of the Storage and Federation networks, correct?

Otherwise, HPe SimplIVity support will need to be engaged.

Please note: you are running on a build which is 3 years old!

[jmeshal]

Just there is one thing that I forgot to mention it, these issues are not all the time but most of the time they happen

means after some time maybe 1 hr simplivity02 come to normal and he can access store simplivity01 and it come to fauilty again

as per your recommend I've make it Active/Active but I didn't feel it changed anything  

for changing We just do the change in the 10GBE switch and assign the ports to vlan with (access mode) not trunk mode ...

no changes made to the IP ranges.

I'm going to investigate with our network team before escalated to hpe support, might need to add something with ports, VLANs etc? 

[LiamP]

"these issues are not all the time but most of the time they happen" - might be a misconfiguration on one of the uplinks or even jumbo frames not passing - worth checking that as well.

By not configuring trunking, implies vLAN tagging is not required or changing IP addresses, means no configuration change required on VMware & HPe SimpliVity.

If vMotion is not working, then ESXi logs should tell you why or test connectivity between hosts using vmkping 

[MikeSeden]

Remember the Physical switches are connected to the ESXi, so when you are setting up the Virtual switches they have to be a Standard switch first talking to the physical switches. Then you can move to a Virtual Dijstributed switch on the ESXi hosts.. You are kind of re-deploying everything on the new networks. Just thinking out loud...

[jmeshal]

I have returned back to (direct connection),, everything comes to normal. and Start working very well, vmotion started working and all the errors disappeared

Currently waiting for network administrators to determine where the problem is at the physical Switch.

Also something comes to my mind,, the problem can be from the cable?

the original cable using in (direct-connected) mode is :

AMPHENOL 3.0M P/N 624380003 SFP+ DAC

I've Replace it with the below if using in (switch-connected) mode:

Dell 3M SFP+ 10GbE Cable DAC-SFP-10G-3M 2CM32

both are the same SFP+ DAC 

[DaveOb]

Anything is possible but the most likely cause is the physical switch settings.

From what you have described above I suspect there may be wider network issues,In a healthy system even if we cant communicate on the federation or storage network, the managment network should still allow the host vms to funtion with decreased funtionality performance degredation.

It may be best to engage  support prior to trying again.

[jmeshal]

.

[jmeshal]

last update from network team no misconfiguration on the 10GBE switch also they change the  jumbo frames from standrd mtu to 9216 . with no effect happen , Now I plan to add another 10GBE Switch, any particular brand or model recommendation someone tried before for this type of project?

thanks

[jmeshal · Accepted solution]

Finally, the problem was solved, it found that the 10GBE switch needs to be reload - ( restart ) after changing the MTU to 9000 and there was a stubbornness by the network administrator that this type of switch (juniber) does not need reboot if you change the MTU, But I insist that restarting compared to Cisco switchs, which needs to restart in case change the MTU.

Thanks to everyone for the great support