Error Message After Removing Node From Cluster

[DA9]

I am getting the following ERROR message in the Health logs:
[clientInboundChannel-42xxx] Could not update server status, server 1 not found

I have a (4) node cluster where I put one node into maintenance mode and deleted it from the cluster. It needs rebuilding but since removing it from the cluster I am getting this error every five minutes or so. I did run a manual "Refresh Short" and a "Refresh Daily" from the cluster but the error message is still there. The cluster does reflect (3) nodes, "pcs status" commands look correct and the license count is correct - I now have two unused sockets. I can rebuild that node but will I be able to add it back into the cluster? Thanks.

[controles]

Hi,

This usually happens when the node was removed from the cluster, but some internal references are still lingering in Morpheus. Even if the cluster itself looks healthy and commands like pcs status or crm status are clean, Morpheus may still be trying to query a server ID that no longer exists.

A few things worth checking before rebuilding the node:

• Make sure the host is fully removed in Morpheus under Compute > Hosts. Sometimes the node looks gone, but a stale entry is left behind.
• If you have access to the Morpheus API or database, check whether there are any leftover records tied to that node.
• Restarting the Morpheus services can help clear cached state. I have seen this exact error disappear after a restart.

If your plan is to rebuild the node and add it back, that should be fine as long as you reuse the same hostname or FQDN, and the old node entry is completely removed from Morpheus before re‑adding it.

I have run into this a few times after hot removals. Most of the time it turns out to be a state cleanup issue rather than an actual cluster problem.

Hope that helps. Let us know how it goes.

Best regards,

[DA9 · Accepted solution]

I was able to clear the repeating error messge by clearing out the cache on the VME Manager:

"sudo morpheus-ctl restart morpheus-ui morpheus-workers"

This restart made them stop.  Thank you for your help.

[controles]

Of course, we’re happy to help and collaborate.

Best regards,
Carlos