arbiter disconnected from OVC

[policeman51]

We are running into an issue.  Running the svt-federation-show command on an OVC we are getting an error that the Arbiter is disconnected from several OVCs.

We checked the Ports as well and port 22122 is open.  I've rebooted the arbiter server as well as one of the OVCs and hosts that showed disconnected, but still it's in a disconnected state.

Any ideas how we can get them connected?

 I should have added.  The Arbiter is 3.7.6 version and the OVCs are version 3.70

We were attempting to upgrade the OVCs to the 3.7.6 version, but then ran the show command and saw the disconnections.

[MikeSeden]

If the problem isn't fixed, I would open a ticket so we can do a remote session and see whats going on. Are you using upgrade manager, or manually upgrading? How many nodes? Are any ready to commit?

[policeman51]

Yes, i opend a case and I'm awaiting a callback.

I was working with HPE support on the upgrade, but they wil not proceed until all OVC are in a connected state.  7 nodes totall.  3 are connected, 4 are reporting disconnected.  Upgrading through the Upgrade Manager.

I just want them in a connected state and seeing what commands/options we can try.

[MikeSeden]

Did you get a case number?

[policeman51]

I did but the were unsuccessful.

- Ran Fedping, no issues with connection to arbiter, although we are not seeing it connecting via port # 22122.

- Restarted arbiter services, issue still ongoing.

- Checked arbiter version for 3.7.0, customer has the wrong version. Re-installed correct version. Issue still ongoing.

- Restarted entire vCenter/arbiter server, issue still ongoing.

- Restarted an OVC, no change, issue still ongoing. 

- Attempted to create a new arbiter on a different IP and assign cluster to this temporary arbiter, then re-assign it back to the true arbiter, that didn't resolve the issue.

[DeclanOR · Accepted solution]

Hi @policeman51

I assume this was done, but don't see it mentioned in your list of steps performed, but were the OVC's ever repointed to this arbiter using the dsv-arbiter-instance-set command?

If not, this would be my first port of call.

You can grep the $SVTLOG directly after repointing to see if there are any error messages visible, or if it is successfully picking up the arbiter.

So, steps would be - On the OVC's showing disconnected:

Elevate to super user:

#sudo su

#source /var/tmp/bin/build/appsetup

Re-point to arbiter:

#dsv-arbiter-instance-set --address <ip address arbiter vm>

Once the command runs:

#grep -i arbiter $SVTLOG

You should see in the last few references in the logs exactly what is happening.

Let me know if this helps.

Cheers,

DeclanOR

[policeman51]

We had a couple of issues going on.

One OVC had to be reset.  

Then they ran the #dsv-arbiter-instance-set --address <ip address arbiter vm> on the other 2 to point them to the correct arbiter and it worked.

Thanks alot for your responses and insite.

[DeclanOR]

@policeman51

Great!

Glad to hear the issue has been resolved.

Appreciate the Kudos! ;)

Thanks, and all the best for now,

DeclanOR