topic Re: ILO Disconnect Emails from Blade enclosures in BladeSystem - General

ILO Disconnect Emails from Blade enclosures

ACHCHGUY — Thu, 27 Jan 2011 22:12:12 GMT

Hi all,

I have 2 c7000 blade enclosures interlinked.
OA firmware on both is 3.00 and ILO firmware is
1.82 on all servers.

The issue is I get emails in our HP SYSTEMS INSIGHT MANAGER saying

"Event Name: System is unreachable
URL: http://chchmg02.**.**********.co.nz:280/mxportal/MxContextLaunch.jsp?systems=ilochchmb02&tool=System%20Page
Event originator: ilochchmb02
Event Severity: Critical
Event received: 28-Jan-2011, 06:10:26
Event description: The current system is no longer reachable from the central management server, hardware status polling has marked this system as not responding"

Now when u sign into the c7000 OA ILO is working Fine.

But we get these errors every Night.

I have tried also hard setting the OA NIC card settings on the c7000 OA to 100MB , full duplex and I have also done this on the cisco STACK.

I am at a loss as too why this is happening.

This does not happen to any other HP Servers we have plugged into the stack.

I have also built another HP Systems insight Mnager on another Server and send alerts from the OA to both of these and I get the same result....still get the disconnects.

Any Ideas

Cheers
Steve

Re: ILO Disconnect Emails from Blade enclosures

Jan Soska — Fri, 28 Jan 2011 06:09:07 GMT

Hello, your firmwares are quite outdated. Upgrade OA and ilos as well and come back then.

Jan

Re: ILO Disconnect Emails from Blade enclosures

ACHCHGUY — Thu, 03 Feb 2011 20:23:02 GMT

I have updated the OA's to 3.21 and all the ILOs to 2.05.

Still have exact same issue across all Blades.

The time it starts is 5:46 pm and continues until 9:21pm so each blade fires an ilodisconnect message about every hour ...then theres silence 2:17am until 6:51 am and again each blade send and email.

I am at a loss........

I would be happy just to filter out these alerts in HP SIM so they do email the customer support desk.

HELP.....

Re: ILO Disconnect Emails from Blade enclosures

Torsten. — Thu, 03 Feb 2011 20:34:33 GMT

>> each blade fires an ilodisconnect message

No, IMHO the SIM server fires the messages. Do you use UDN? You should investigate if something prevents the communication between the SIM server and the blades.

Re: ILO Disconnect Emails from Blade enclosures

Torsten. — Thu, 03 Feb 2011 20:53:03 GMT

There are many other possible reasons.

I would check the OA logs first for any related messages.

Maybe you have some mis-configured scheduled custom collections in SIM (e.g. with wrong credentials)?

The exact message you received could be helpful.

Re: ILO Disconnect Emails from Blade enclosures

ACHCHGUY — Thu, 03 Feb 2011 20:54:45 GMT

Initially I thought that it was a sim issue also.

But I built a new SIM Server and both Servers get the Alerts.

We have large cisco enterprise stacks and have over another 50 HP standard dl servers plugged in and they do not have this issue.

What is UDN? (sorry for silly question).
Cheers
Steve

Re: ILO Disconnect Emails from Blade enclosures

Torsten. — Thu, 03 Feb 2011 21:01:44 GMT

Unreachable device notification, an SMH add-on.

Re: ILO Disconnect Emails from Blade enclosures

ACHCHGUY — Thu, 03 Feb 2011 21:05:07 GMT

Some of these Servers have linux and oracle installed of Vmware esx...not all windows.

so the SMH is not installed...is that what u are referring too.

Re: ILO Disconnect Emails from Blade enclosures

Torsten. — Thu, 03 Feb 2011 21:08:11 GMT

You can include UDN into the SMH of the SIM server. This basically "ping" the clients and send alerts if there is no answer.

Can you read from your alert mail "who" is the sender? I assume it is HP SIM.

Re: ILO Disconnect Emails from Blade enclosures

ACHCHGUY — Thu, 03 Feb 2011 21:08:58 GMT

The entire email is in the first post

Re: ILO Disconnect Emails from Blade enclosures

ACHCHGUY — Thu, 03 Feb 2011 21:12:41 GMT

on the second hp sim Server UDN is disabled.

Re: ILO Disconnect Emails from Blade enclosures

Torsten. — Thu, 03 Feb 2011 21:15:11 GMT

Using UDN could be a cross check if the clients are reachable.

Did you find related messages in OA logs?

I'm still curious about the message text in your alert emails.

Re: ILO Disconnect Emails from Blade enclosures

Torsten. — Thu, 03 Feb 2011 21:19:09 GMT

OK, I read the original post again.

It is from SIM.

"Event description: The current system is no longer reachable from the central management server, hardware status polling has marked this system as not responding"

The SIM server cannot reach the ILO for any reason. I would enable UDN now ...

Re: ILO Disconnect Emails from Blade enclosures

Torsten. — Thu, 03 Feb 2011 21:41:31 GMT

When thinking about this issue, I believe it could really be a credential issue emerging during a scheduled polling. This would explain the timeframe too.

I would take a look into the SIM logs too.

Re: ILO Disconnect Emails from Blade enclosures

ACHCHGUY — Mon, 07 Feb 2011 21:45:50 GMT

I have checked sim logs and OA logs there are no authentication issues.

I am about so set up the UDN on the HP SIM Server in the SMH.

I have tried to find a doc to explain how it works or integrates with sim .

Is there a chance you could explain this or provide a link etc.

I am most interested to know how it works/integrates so emails like the one I am getting dont get sent.

Cheers
steve