SV3200 FOC W2012 lost access to volume Storage Controller fault

[Fred Blum]

On nodeb SC1 went offline after a power down of SC1. NodeB-sc1 is failed over to NodeB-sc2. SV3200 has 2 volumes exported to Cluster. One volume Volume2 is no langer online in FOC W2012.

The system was setup by a third party consultant. Going over the settings and thinking about it, I noticed that MPIO is not set on the target discovered using the VIP. In sessions I still see the failed IP adresses .61 and .62 of the failed NodeB-SC1. 

If Volume2 was connected tru .61 and .62 and MPIO not enabled it has no knowledge of an alternative route to Volume 2. Am I right? Or should it use the VIP adress .100 to access the volume? 

According to HPE there is a known issue with using a VIP on a VS3200 and it is not recommended. According to HPE I should remove it in Storage Pool and  use the Bond0 IP Aadresses in ISCSI Initiator. A reboot of SC1 was recommneded to get the stuck SC1 IP adresses and VIP released. and be reachable with SVCM. The reboot however caused the SC1 to go offline. The VIP is accessable again in SVCM.

What is the known issue with a VIP on the SV3200?

What is the Best Practise for connecting Windows 2012 R2 ISCSI to a VS3200?

TIA,

Fred

[Fred Blum]

In the ISCSI initiator target I see the name of the SV3200 Management Group tomconnect to, not the volumes as on the P4300. Is that normal? Or a choice by the consultant that ran the installation wizard?

[SUBHAJIT KHANBARMAN_1]

I am not sure what LHOS version running in your system but regarding VIP there was known issue with LHOS 13.5 where if we configure iSCSI with VIP and for any reason we have to restart one of the controller where that controller was VIP holder then with controller failover VIP won't failover to other controller. This was the issue but it got fixed in LHOS 13.6. Please find the LHOS 13.6 release notes to get the fix,

https://internal.support.hpe.com/hpsc/doc/public/display?sp4ts.oid=null&docLocale=en_US&docId=emr_na-a00035652en_us

*****************************************************************************************************************************************

Now coming to iSCSI configure, 1st you need to enable MPIO in Windows 2012 system,

"Configure MPIO on Hyper-V Host/s
Before you continue, you must add the MPIO (Multi-Patch I/O) which is built into the Windows Server 2012 R2 operating system. To do this, just open Server Manager on each Hyper-V host and select Manage, then Roles and Features. Complete the Add Roles and Features wizard to install the MPIO feature to your Hyper-V Host/s. After the MPIO feature is installed, configure MPIO by opening Server Manager, Tools, then MPIO. Select the Discover Multi-Paths tab, check the Add support for iSCSI devices and select OK. This will require a REBOOT of the Hyper-V Host server."

You can also refer the below links to get idea of how to do it,

https://dataonsupport.dataonstorage.com/support/solutions/articles/6000014962-how-to-install-mpio-

http://www.howtonetworking.com/msapps/iscsi3.htm

*******************************************************************************************************************************************

Then see the below Video how to configure iSCSI setup with SV3200,

https://www.youtube.com/watch?v=ipq9zP1MOI8

********************************************************************************************************************************************

Also request you to mark the forum as resolved if there is no more outstanding query from your end on this issue.

This will help for everyone who are all following your forum.

Hope this helps!
Regards
Subhajit

If you feel this was helpful please click the KUDOS! thumb below!

***********************************************************************************

[Fred Blum]

Hi Subhajit,

Our LHOS is 13.6.

MPIO is installed and the target was connected using MPIO. Saturday I saw all available connections also the 2 IP adresses belonging to the failed SC1. Still it would not bring Volume2 online and the Volume2 disappeared from the Hyper-V Cluster. Volume1 in the same Manage Group target did not have a problem. 

This morning with the  consultant when we walked in we saw only 1 available connection in ISCSI Initiator on Server1 of a 3 node Cluster. We connected the target again on Server1 and Volume2 appeared as if nothing was wrong. I did the same step saturday (all connections present, Volume2 invisible) and at that time it did not work making me wonder if they had installed MPIO properly. I will go over your links to check.

I still have the problem that SC1 is offline, it gives the following error in the mini-USD port when booting: 

2018.10.29 08:17:47.749:I001:SystemErrorMonitor - Error detected: 0x000000B5 - Communication error with the partner expander

 Waiting for an answer from HPE on that.

4 VM's went down today when VEAAM did a resource discovery with Hyper-V.VMMS giving an id 19050 error related to replication and removing checkpints. We do't use VEAAM Storage snapshots to make backups and the backups are done during the night of the VMs. We know that Native VEAAM integration is not supported. I have contacted VEAAM support also how a VMMS discovery can crash a degraded VS3200.    

Thanks for your information and help sofar,

Fred

[SUBHAJIT KHANBARMAN_1]

Regarding SC1, I would suggest to use recessed button at the back of the Controller and try to make it up. You can also connect serial cable at the back of the controller and from haLO try to make the SC1 up. If recessed buttopn or haLO didn't help then try to physically reset it.

There could be two possibility here, as per the error message it looks like backplane issue which needs replacement. Otherwise SC1 needs replacement.

Regarding MPIO and Volume lost access purely confguration check and validation of the same.

Let everyone know until solution of this so that everyone can learn from this forum.

Hope this helps!
Regards
Subhajit

If you feel this was helpful please click the KUDOS! thumb below!

***********************************************************************************

[Fred Blum]

HI Subhajit,

We tried rebooting, powering down and reseated the SC1. The error response is from the serial connection. 

The SV3200 is a horrible system, I am seeing zero events in the SVCM but yesterday evening our cluster servers where flooding with ISCSPRT errors which caused two of the three node cluster servers to go down. 

We have been running this cluster for years with zero problems on a P4300 4Node SAN.

Now I want this SV3200 removed ASAP and everything back on the P4300 and even that could become problamtic because of seeing flush errors and possible corruption errors on the volumes in Windows 2012.   

[SUBHAJIT KHANBARMAN_1]

From hardware perspective to troubleshoot further, you can go for SC1 replacement and backplane replacement as per the error message that you have shared.

Otherwise HPE already declared SV3200 End Of Life as below,

https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-a00041272en_us

If you want to go for entry level system then MSA2050 is very good option as per my experience.

I would request you continue sharing update on this if you have update from HPE support so that everyone can get the update who are all following this forum. If any point you feel you don't want to continue with this product anymore then kindly post your resolution accoringly and resolve the forum.

Hope this helps!
Regards
Subhajit

If you feel this was helpful please click the KUDOS! thumb below!

***********************************************************************************

[Fred Blum]

 Since October we are still in a dispute with HPE over this issue as the official HPE Support channel can not find the cause. The Dutch HPE office has send a storage engineer.

Basicly what he is saying is that Windows 2008 Server and the StoreVirtual 3200 is not supported. Furthermore that our Windows 2012 servers MPIO seems to respond like the Windows 2008 issue so a update of HPE server firmware and Windows updates is recommended with increasing the switch port bufffers to over 512K by lowering the number of QoS queues.

The issue is that with MPIO and multiple paths connected the through-put drops significanty and the IscisPrt starts flapping.  With only one path connected this is not the case. 

We are in the process of updating the firmware, cluster servers and changing the port buffer size on the switches.

Have others seen these issues with W2008? Can someone direct us to HPE notes regarding these issues?

The runaway process in Windows W2012 Fail Over Cluster might be caused by SMB 3.0 redirected IO causing a network flood.