HPE SimpliVity

Simplivity action runs on the wrong host - ever seen this?

 
elange
Occasional Contributor

Simplivity action runs on the wrong host - ever seen this?

While i have a case open let me ask whether this happened to someone else too.

 

We are in the process of updating our Hosts from 3.7.0.260 to 3.7.10. (btw: Cicos based Hardware, so no version 4 for us). Before you ask why we are so far back, after having a rough time in 2017 with tons of support calls when we started with simplivity well it simply worked pretty good then and went into something like "Dont touch it ... it will just break if you do" so we were happy with it for a good time.  Well trapped because we need to update VMware higher and we are closing in on the end of support next year we had to take the update at some point.

 

Now so far half of the updates went well, the other half in most case we had to have the support fix the identity store after which it mostly worked. Now we have one host which still refuse and the upgrade manager ran, restarted the ovc and ended with a failure to confirm the software version and failed. Today with the support we got that through although its not clear what actually fixed it.

 

And this is where it becomes weird and scary. So i have a cluster with two hosts, lets call them a and b.

B did update after fixing the identity store and was done. A was unable to update even after fixing the identity store. Today we went through the update once more and while the update failed again with being unable to confirm the software version afterwards at the exact time we saw evnts on host B that it can´t update because there is an update or commit waiting. So to me it looks like it ran the update on host a and then asked host b for the software version instead of host a.

 

Next thing:  I did shutdown the OVC on host a via the vcenter web plugin and instead of going down - the OVC on a completely different Node (different cluster, different location, different subnet) went down in our primary datacenter. Now first moment you might think you clicked the wrong host but no. I can reproduce this any time i like. Just made a video for the support.

 

So that kinda scares me a lot... and its not a DNS issue. I checked those Host Entries on all our DNS Servers and also the Support ran the zeus script which came back fine for all dns checks.

 

Anyone seen something like this before?

 

Before anyone from HPE asks ... thats covered  in 5351431579   

3 REPLIES 3
AnkiN
Valued Contributor

Re: Simplivity action runs on the wrong host - ever seen this?

Hi @elange ,

Thanks for using HPE SimpliVity Forum, could you let us know about the SimpliVity plugin version installed for the web-client?

elange
Occasional Contributor

Re: Simplivity action runs on the wrong host - ever seen this?

Hi @AnkiN 

after a misscommunication we temporarely had the one bundled with 3.7.8 release installed. With that being completely disfunctional because we were still on 3.7.0.260 we went back to the bundled version from the 3.7.0.260 release.package.

However that issued also occured once before we updated so it should at least not be caused by the temporary visit to the bundled version from 3.7.8.

 

Currently its 13.1.90

 

Best regards,

elange

AnkiN
Valued Contributor

Re: Simplivity action runs on the wrong host - ever seen this?

Hi @elange ,

Thanks for sharing the details.

If I understood correctly we are at a stage where we have multiple OVC releases in federation and plugin at 13.1.90.

As the issue occurred before the upgrade it would be difficult to find out the exact reason unless the old logs are present.

The issue described is uncommon in nature, we need to check the Virgo logs and the SimpliVity support capture with the timestamp in order to find out which methods were invoked wrt the operations timestamps.

I request you to log a support case so that we can check the faced issue and update you regarding this.