Can ping LeftHand nodes but unable to add via CMC or login directly

[Rakhesh]

Hi,

We have a two node LeftHand setup cluster. They were on OS 8.5 until last week (yes!) when I upgraded them to OS 12.0. The nodes have been fine since then but since today we are unable to view them via CMC.

CMC shows both nodes and failover manager as "not found".

I can ping the nodes. And our ESXi hosts can see the datastores and those are still working.

I can iLO into the nodes (or view the FOM console). Further, I am able to type "start" and get to the "Login" screen. But after that I am unable to do anything - nothing works. I am also able to press Alt+F2, F3, etc and get to the CLIQ console screen. I tried typing commands such as "getSystemInfo" it just times out.

Any suggestions on what I can do? I read some forum posts that suggested restarting the CMC but that hasn't helped. Other posts suggest restarting the FOM and/ or nodes but that seems a bit drastic. It looks more like something on the nodes and FOM is stuck (since the datastores themselves are available) but I don't know what.

Please help.

Thanks,

Rakhesh

[oikjn]

these hardware nodes?  if so, which ones? 

My guess is you are low on memory or maybe the default gateway setting was removed.  Check that default gateway.  I know I ran info that problem going from 8.5 to something as well and everything was fixed by that.

[Rakhesh]

These are P4500 G2s.

How can I check the gateway settings? I can't login to the nodes directly nor can I connect via CMC. Got any command to get this?

[Torsten.]

>> I can't login to the nodes directly

Why not? Forgotten password or no login screen?

[Rakhesh]

>> Why not? Forgotten password or no login screen?

That's the thing! I get the login screen but nothing works.

I can connect to the server (and FOM) via iLO. Then I type "start" (so the keyboard works), after which I get the blue login screen with the words "<Login>" (if I remember correctly).

Now usually I'd press Enter here and enter the username / password, but I can't do that here. No key works so I am just stuck on the screen!

Odd thing though - I checked later in the day and now both servers and FOM are responding and I can login. Nothing in the event entries that I can see via the CMC. Are there any logs I can check on the nodes or FOM to see what could have gone wrong?

Thanks,

Rakhesh

[Rakhesh]

The problem still persists.

Here's something I noticed though. When it goes down there are no alerts but when it comes back up again (as in I can connect via CMC) I get the following alerts:

'Cron' Service Status = 'Stopped'. WARNING (E02020101)

'Cron' Service Status = 'Running'. INFORMATIONAL (E02020100)

The alerts come together but I am guessing that's only because the "Stopped" alerts didn't get sent out until later.

These alerts are usually accompanied with one like

The 'Syslog' Service Status = 'Running'. INFORMATIONAL (E02020100)

-or

The 'HWMonitor' Service Status = 'Running'. INFORMATIONAL (E02020100)

-or

The 'GC Agent' Service Status = 'Running'. INFORMATIONAL (E02020100)

Any suggestions on what could be going on?

Thanks,

Rakhesh