StoreVirtual Multipathing Extension Module for vSphere 5.5 missing VMFS datastores

[Princes]

I've got it working on a 2 node VSA Cluster v12.5 with ESXi 5.5u3 and it seems stable.

[david11]

I have the exact same setup, let us know how it goes after a few weeks.  LIke most of us have said in our posts it always starts out seemingly stable and then goes south fast for unknown reasons.

[Princes]

For the record, I manually installed the HP_StoreVirtual_Multipathing_Extension_Module_for_Vmware_vSphere_5.5_AT004-10524.vib rather than using Update Manager and the build of ESXi is 5.5u3 Build 3116895. VSA's are v12.5.00.0563.0

[david11]

On my install we used the manual method as well through direct command line as well on each of our esxi hosts.

AT004-10524.vib lefthand driver was used. (same as yours)

HP Lefthand OS 12.5.00. 0563.0 (same as yours)

We ran this under ESXI 5.5 U2 (dont remember which build but we tried to go to U3 hoping it would help with the issues)

Currently at ESXI 5.5U3 build 3142196 (newer than your build)

With the above setup it always ran fine for a couple weeks before it just blows up.  I really hope this does not happen to you but it was the cause for major downtime during the day and a long road of hours to recovery to get them back online to remove the driver with vmware support.

My hardware is as follows:  (Please let me know if yours is different)

DL360p G8 Servers

Array:  HP P4500 G2 Lefthand arrays. Using 1gb Iscsi ports.

Best of luck and please report back in a month if still running ok and be sure to post if you run into same issues we all have as the more people on this thread reporting it = better chance of HP acknowledgement and fix for this problem.

Thanks,

David

[rossiauj]

Hi,

I did the same and manual install did not make a difference. We ran smoothly for weeks/months and then it suddenly bit us in the proverbial behind,

So proceed carefully and be aware of the dog.

Kind regards,

Jos

[Princes]

Possibly of relevance: ESXi Build is HP with 'Support Bundle' (i.e. with the HP drivers, AMS, and other apps). Firmware revision (SPP) on the two HP DL 380 Gen9 Servers is: 2015.06.0-SPP2015060.2015_0605.106.

[david11]

Good to know, I also used HP's esxi builds that were posted on vmware's site for our servers.

Mabye g9 doesnt have the issue but I'm leery, let us know back in a few weeks - 1 month and let us know if its still going strong for you.

Happy thanksgiving all!

[NDBT]

I attempted the manual CLI install of this driver to one of my hosts.  It failed in a spectacular fashion.  After running the install of the driver and rebooting the host all of my datastores disappeared from the host. I had not even enabled the driver on any of the datastores. 

Uninstalling the driver resolved the issue. I did not attempt of the procedures mentioned in this thread to resolve as I won't take a chance with my production environment.  Round Robin satisfies our needs for now.

[david11]

Thanks for adding to the conversation, it appears this driver has horrible issues, still hoping HP will answer about this.

Also note when installing the driver, after a reboot, it auto grabs every volume using the new driver by default and for all newly created datastores going forward, its by design.  This is why uninstalling helped fix it in your case, it probably went back to default vmware pathing.  Make sure however you still set it to VmWare_Round Robin with IO set to 1.  You will see the best performance settings on this when using the HP lefthands from my experience.

[NDBT]

I received this notification today from HP regarding the MEM issues

DocumentID: c04901409

Full text here:

http://h20566.www2.hpe.com/hpsc/doc/public/display?docId=c04901409&hprpt_id=HPGL_ALERTS_1862728&jumpid=em_alerts_us-us_Nov15_xbu_all_all_551862_1862728_OptionsStorage_critical__/

[david11]

So this fixes the problem for good or is just another way to re-establish paths after random failure?  It seems to read the in the direction of a way to fix it after everything crashes. 

I hope they release a permanant fix to the problem.  Also not sure if it applies to most of us as the paths are done through dynamic discovery on my setup using the VIP of the lefthand arrays.

Let me know though if these steps work as a permanant fix and the driver becomes completely stable after doing this.

Thanks,

David

[ICT_Krimpen]

I've had the same problem. This article resolves the issue:

http://h20566.www2.hpe.com/hpsc/doc/public/display?docId=c04901409&hprpt_id=HPGL_ALERTS_1862728&jumpid=em_alerts_us-us_Nov15_xbu_all_all_563196_1862728_OptionsStorage_critical__/

Regards,

Robin John

[david11]

Does it fix it after the problem occurs, or is it a permanant fix after installing the MEM driver?

How long have you been running stable since doing this? 

[SCROYSE]

How long have you been running using the MEM? Is it still functional or have you had complications/downtime as a result? I'm looking at deploying to my environment as it is starting to sound as if the issue is resolved, but I want to hear from those brave enough to test for the rest of us.

[NDBT]

I'm definitely not brave enough to try.

[MartinGriesbach]

Hi,

got the same issue with ESXi 6.0 U1b and all the latest 12.5 Lefthand.

The "VAAI ATS heartbeat" was not the problem. None of the mentioned symptoms.

After rebooting the ESX host the datastore was mounted for a short time and then disconnected.

The tip with the rescan worked for me until the next reboot. Same procedure then ...

If found out that there was not the right SATP selected. PSP was "HP_PSP_LH" but SATP was "VMW_SATP_DEFAULT_AA" not "HP_SATP_LH" as expected.

Then I played around some time with the SATP-Rules and found out that "VMW_SATP_DEFAULT_AA" is default for iSCSI connections. The user rules from the HP MEM seem to have no effect. So I added the following rule (one line):

esxcli storage nmp satp rule add -e "My fxxxing custom rule" -M iSCSIDisk -V LEFTHAND -P HP_PSP_LH -s HP_SATP_LH

After the next reboot of the ESXi host the problems were gone.

Can you please verify this ???

[david11]

Interesting find, please post back if it still works with no issues over the next month in your production enviroment.  I will try it after you confirm that because I'm not brave enough to cause production downtime again.

[Semplify]

Any updates on this?

[BenLoveday]

I'm keen to know if this has been resolved also. I'm in the process of installing the 5.5 MEM driver v12.0.0.62 but I can't find the offline bundle, only the .vib files. Has anyone found these?

Also, there appears to be no real release notes for the MEM drivers considering the file version names keep changing (regardless of the LHOS version). Can anyone comment on this?

Cheers,

Ben Loveday