Error removing host from Simplivity Federation

[policeman51]

We have a Simplivity host that we did not need any more so we wiped it, but didn't remove it from the Simplivity Federation first so it's in a faulty state now when you run the svt-federation-show command.

I ran the following command to remove the host, but received the following error.  Does anyone now a way to force the removal of a host:

administrator@vsphere@omnicube-ip193-56:~$ svt-federation-remove --peerip 10.216.193.43 --override
WARNING: If there are VMs present that are not HA compliant,
         this operation will result in data loss.  You must reset the
         OmniStack host to factory defaults before you can redeploy it.
Proceed? (y/n): y

One or more OmniStack host systems is faulty.
This operation will not complete until all OmniStack host systems are healthy, or until you remove all faulty OmniStack host systems from the Federation.
Proceed? (y/n): y
ERROR [114]: Unable to find an OmniCube using the provided information.

[policeman51]

Oh and i also tried removing the host from the VMware Web Client but got the attached error.

[DaveOb · Accepted solution]

It is best practice to remove a node as soon it has been identified that it is no longer needed in the federation leaving nodes in a faulty state for prolonged periods may lead to issues.

The GUI option will only work if the node is in a non fautly state,The GUI would be the preferred method of removing  a node to be decommisioned from  a federation.

If it is the only node in a datacenter/cluster then the cluster or datacenter should be removed using the --cluster / --datacenter flag instead of the --peerip.

When specifiying ----peerip use the ESXI host not the OVC managment IP.

If you are still having issues its probably best to open a support ticket and let them take care of it / guide you further.

[policeman51]

Thanks for your reply.  

The command didn't seem to work.  I got the same 114 error so I've opened a ticket with HP Support.

[policeman51]

The support tech was able to remove the old hosts with a script.

Not sure what magic they did, but it worked.

[Interwebz]

Hi,

I'm facing the same issue in our dev environment.  An Omnistack or OVC was removed incorrectly by mistake and then the ESXi host was removed from the vCenter view.  I'm not able to redeploy the node since it's stuck as faulty/orphan while issuing svt-federation-show command.  svt-federation-remove does not work since it cannot communicate with the ESXi host or the OVC.

Did you by any chance get that script?

Is there a svt/dsv command that could remove the faulty host/OVC from the federation so we can redploy it?

[policeman51]

They didn't share it, but I just asked for it.

I'll  reply with it as soon as I get it.

[Interwebz]

Ahh sweet, you could possibly find it on the OVC that they were connected to if they didn't remove it. :)

[policeman51]

Here's the script.  Good luck.

Info Erased

[DeclanOR]

Hi @policeman51 @Interwebz 

Only experienced HPE SimpliVity support personnell should run any scripts within your SimpliVity environment.

Steps to remove a node, if done in the correct sequence, should never require any additional scripts.

In the event that a node has been uncleanly removed, a support case should be created so that we in support can perform the necessary steps to resolve for you.

I strongly advise refraining from using these scripts unsolicited by a SimpliVity support team member.

Incorrect usage of any script can have serious adverse affects in your environment, and HPE SimpliVity will not be held responsible for any misuse of such scripts.

I strongly advise @Interwebz that you create a support case to allow the SimpliVity support team to assist with the removal.

Thanks,

DeclanOR

[DeclanOR]

@policeman51 @Interwebz 

Apologies guys.

We really appreciate, and love seeing you collaborating and chatting on this forum, but to avoid any possible misuse of the script by others who may come across it, we've removed it from the chat.

Hope you understand, and please keep using the forum and chatting!!

Thanks,

DeclanOR

[policeman51]

Haters... just kidding.

It's cool and I know it's not your fault.  Maybe they should make a more open platform or admins can manage their own environment and make changes when needed instead of having the platform locked down where you need to contact support.

For this instace I understand, hosts should be removed from the federation prior to decommissioning, but it's not the only time where I had to contact support to make changes in the Simplivity environment that I manage.  We didn't get into IT just to call support.  Learning involves researching, it involves failing.  

OK, off my soap box.

[Interwebz]

Hi,

I my case it's a test system and has absolutley no impact on any production systems.  It seems quite counter productive to not let admins clean up their own environments.  The only way this can be cleaned that I know of is to reinstall the vCenter and start from scratch.  That's like using a nuke trying to kill a mosquito.

It would be greatly appreaciated if the script could be PM:ed.

Thanks

[DeclanOR]

Hi @Interwebz @policeman51 

Thanks for the responses.

@Interwebz One of the reasons the script was removed was that we also need to be careful regards versions etc.

I think that the node removal for you should be do-able without running this script. The script in essence is basically just the background task for the dsv-federation-remove task (depending on OVC softweare version you run)

I would like to understand what you have already tried also. Can you let me know the following please:

1) OVC software version

2) Steps already taken.

Once I know this, I will advise on next steps. 

Thanks,

Declan.

[Interwebz]

Hi,

OVC, 3.7.6.156

Tried to remove with the svt-federation-remove command but it does not allow force removal of faulty nodes.  The script does the trick, it has the power parameter that 3.7.6 is missing.

My issues is resolved at least.  I wished that HPE was more helpful as this is a simple thing to do.  It should not require a support call to HPE in my optinion.

[Riik]

Hi,

We're also looking for the same script. Unfortunately we've run into the same issue and there is not a clean way anymore to remove the host from SimpliVity Federation. Also a non-production system here where this happened.

OVC 3.7.8.232

Please assist.

[Toonkraft01]

Hello, 

Actually the command is correct, but need to check the syntaxis and well you get this: 

svt-federation-remove --peerip GUID-off-faulty-node --power.(at this point we have to know
which Node ID to release)

Obtain the GUID (Node ID) you will need to use the command: dsv-cfgdb-get-sync-status
After that sure you have no problems to remove the node.

[Riik]

Hi Edgar,

Thanks for your reply, partial correct. The GUID is indeed required in the peerip to point to the right node.

Unfortunately, like all others it's resolved by using the script provided by an HPE SimpliVity engineer.

We've compared the used script with the svt-federation-remove. The --power argument is a hidden parameter and can be used. However in the svt-federation-remove script the parameter doesn't actually do something, therefor it is not passed through the deeper function. The script is slightly different and passes on this parameter, That is the main difference between both codes.

[Alessandroo]

Hi,

Cause my second node dont see my common datastore after creation...

Try to remove the 2nd node from my vcenter => ok

Factory Reset ok.

Deploy ok and waiting for discovered.

Unable to pass the scan menu..=> Could not retrieve the configuration of existing  HPe Omnistack host, Please try again....

Nothing in the logs...

Do i have to reset the first node too...?

Any idea ? Thanks...

[gustenar]

The --power option was removed from the command starting software 3.7.3 if not mistaken.

svt-federation-remove will work properly if there are no faulty nodes in federation. If you need to remove a node from federation that is faulty, support can assist with the task by loading the script needed for this.

[ShawnRC]

Could really use this scropt if you still have it. Did a "dumby" and shot myself in the foot here.... boss less than happy I think so it would be great if you could share.