P4000 cluster node failure - swap did not work

[Andrew Hellyer]

Hi all,

Upon doing some firmware updates etc, one of our nodes failed upon reboot and had to be repaired.

We followed the usual process of setting the node in the CMC as having a storage system problem, then it puts in the marker node which remains in place to prevent a cluster restripe.

We then swapped the newly rebuilt node in for the place holder node.

Now - this was two weeks ago, and all my volumes wont restripe because they still all report the node as being down.

The node got some data copied onto it - but now the entire management group (two clusters) is just 'on hold' as far as restriping/syncing goes.

This all appears to be caused by the node being failed.

The only difference was that the failed node was running san/iq 8.5, and it was rebuilt with 9.0 as we did not have the media to rebuild it to 8.5

Is the san/iq version difference the cause of the cluster to not accept the new node as being the rebuilt node?

Is there a way to 'evict' the old failed node and just let the cluster restripe/rebuild from the existing volumes so we can just add in the node as if we were expanding the cluster out?

I have had HP on this for two weeks - but am getting no replies from them anymore despite requesting a level 3 engineer multiple times now. This is frustrating as it has stopped us from being able to upgrade to 9.5..not to mention many of our volumes are now not protected by network raid.

Thanks,

Andrew