StoreVirtual Storage
1748224 Members
4557 Online
108759 Solutions
New Discussion

Add to an Existing Mgmt Group "stuck" on "Syncing configuration"

 
SOLVED
Go to solution
fnbit
Occasional Advisor

Add to an Existing Mgmt Group "stuck" on "Syncing configuration"

Howdy!

We moved our production storage over to 3PAR (yea!) and I was finally able to get around to repurposing four nodes of P4300G2 over to our DR site last week. Since we were breaking up our existing cluster (discontinuing usage of a pair of P4300G1 nodes), I factory reset everything and updated the G2s to v. 12.5.x, etc. When I got them on-site, I had some "goofiness", and I ended up deleting the cluster and management group and starting over. There were no volumes in use...no biggie, right?

One of the nodes requried us to remove it from the "old" managment group from the console (CMC and CLI weren't working). Once we did that, it finally showed up in CMC under the Available Systems (and not as the "old" mgmt group). Meanwhile, I had already created a new management group with the 3 "good children" nodes and all I had left was to add the 4th node.

It has currently been sitting at "Syncing configuration" during the add to mgmt group process for about 4 hours now. I had not created any volumes or even been able to attempt adding it to the cluster.

These units are currently not under a support contract.

Thoughts???

Thanks!

5 REPLIES 5
fnbit
Occasional Advisor

Re: Add to an Existing Mgmt Group "stuck" on "Syncing configuration"

UPDATE - So...we finally had one HDD on our problematic node report with a Predictive Failure and we were able to replace it, allow the hardware RAID to repair itself and then get the P4300G2 updated to the latest 12.5.x version. However...when attempting to add it to the existing management group, it is still getting "stuck" at "Syncing configuration".

Ugh.

Prior to adding, I rebooted the node one last time and reran all of the hardware tests and everything passed without issue.

 

oikjn
Honored Contributor

Re: Add to an Existing Mgmt Group "stuck" on "Syncing configuration"

definitely sounds like that node is doing some sort of soft-fail where its dying without throwing any major errors.  Usually its the HDD, but it could be the controller or memory or something else.  I don't understand how they manage to let something like that totally bring a cluster/mgt group to its kneese, but it does.  Since these are all "clean" nodes w/ no support, I would suggest you try taking swapping all the memory with another node you know works and then all the HDDs with a different node you know works and then see if the problem moved to a different node.

 

Might first want to check the switch to see if there are Tx/Rx errors on the connected ports.

JazzyB
Occasional Advisor

Re: Add to an Existing Mgmt Group "stuck" on "Syncing configuration"

I've had the same issue with Storevirtual version 12.6 - adding a 4th node to a 3 node cluster. The node is a fresh install with identical config to the existing 3 but trying to add it to the existing management group results in it being stuck on the same message - syncing configuration for ever - i left it for over 24 hours without any success. I'm wondering if its an issue with 12.6? I havent yet resolved it - one of the reasons i was checking these forums.

fnbit
Occasional Advisor

Re: Add to an Existing Mgmt Group "stuck" on "Syncing configuration"

Hmmm...are your nodes also on 12.6 or just the CMC?

For me:

CMC = 12.6.0.162

P4300G2 nodes = 12.5.00.0563.0

fnbit
Occasional Advisor
Solution

Re: Add to an Existing Mgmt Group "stuck" on "Syncing configuration"

Annnd...DANG IT! LOL

So...when we originally setup our DR site where these P4300G2s were relocated, we were using the StoreVirtual VSAs without Jumbo Frames on the infrastructure there.

However... #facepalm

The P4300G2s were coming from our production evironment where Jumbo Frames *were* being used.

Anyhoo, getting everyone on the same page fixed everything.

Thanks!