topic Re: Reboot and Hardware Status Polling tasks in Server Management - Systems Insight Manager

Reboot and Hardware Status Polling tasks

Mahesh Shah_3 — Thu, 28 Dec 2006 07:57:15 GMT

We have over 1550 servers in HP SIM and schedule reboots every Sunday different time zones. We received ‘System is unreachable’ then ‘System is reachable’ messages. I unchecked both options under Options, Events then ‘Status Change Event Settings’, now we do not receiving any unreachable/reachable event messages, but when server is down or reboot hung or not ping-able we are not receiving any alert message either during the hardware status polling task, is there any way the schedule reboot will not send unreachable/reachable event messages.

Re: Reboot and Hardware Status Polling tasks

Rob Buxton — Thu, 28 Dec 2006 14:20:53 GMT

The server reboots faster than the polling interval. So HPSIM polls, you reboot the server and it's back before the next polling cycle.
By default for servers it's every 5 minutes. More than enough time for a server to reboot.

Re: Reboot and Hardware Status Polling tasks

Mahesh Shah_3 — Thu, 28 Dec 2006 14:46:18 GMT

I changed the Hardware Status polling tasks from 5 to15 minutes; still we are receiving false positive pager/email for servers indicating â System is unreachableâ then â system is reachableâ . I would like to avoid these types of messages so you can have good night sleep.

Re: Reboot and Hardware Status Polling tasks

Rob Buxton — Thu, 28 Dec 2006 15:06:26 GMT

What do you mean false positives?

The polling task will generate an event if it cannot access the server.
It doesn't know it's a scheduled reboot. If you're getting events when servers are actually available then you may have network issues.

There is a time filter on the event set up, you could use that to not check at certain times. But that would be difficult with a generic event handler and different time zones.

Also, you'd need to check where the events are coming from. If you've got an event that triggers a page based on a "Warm Start" or similar then that's independent of the HW Polling.

Re: Reboot and Hardware Status Polling tasks

Mahesh Shah_3 — Thu, 28 Dec 2006 15:37:13 GMT

We know that servers are rebooted on the schedule reboot time, and hardware status polling tasks runs during that time, now server is not ping able during that time, so it send us pager/email â system is unreachableâ , and then 15 minutes later we receive â system is reachableâ message, how can we avoid this of type messages. We like to receive the message only when server is down or has a hardware issues, not during the schedule reboot.

You are correct about time filter it is very cumbersome to configure.

I did check the event message and they are coming from HW status polling tasks.
ex:
'The current system is no longer reachable from the central management server, hardware status polling has marked this system as not responding'.

Re: Reboot and Hardware Status Polling tasks

David Claypool — Thu, 28 Dec 2006 16:02:03 GMT

I'd suggest using Suspend during your maintenance window. You can schedule it as a command using 'mxnode.'

Re: Reboot and Hardware Status Polling tasks

Rob Buxton — Thu, 28 Dec 2006 20:09:13 GMT

You'll always get the events, but you don't need to e-mail, page etc. on all events generated.

System Is Reachable is an "Normal" message, you'd need to check why your event handler task is doing anything with these messages.
I only have event tasks that alert me to Critical or Major events. So I certainly do not get any e-mails when a Server is back.

Re: Reboot and Hardware Status Polling tasks

Mahesh Shah_3 — Fri, 29 Dec 2006 09:45:43 GMT

We donâ t have any problem greeting email/pager for critical server events, it is great tool for monitoring HP ProLiant serverâ s hardware and working just fine for us.

Since we have 1500 servers in HP SIM and they are rebooted different time zone, is there any way not to receive any alerts when server rebooted at weekly schedule time.

You are correct about the â System is reachableâ is â Normalâ message and we are not getting email/pager message since we have configured for critical and major events alert tasks to send pager/email only.

Thanks.

Re: Reboot and Hardware Status Polling tasks

Jimmy Rueedi — Tue, 13 Feb 2007 08:42:34 GMT

Same problem here (not with 1500 server, only 150)

We have several different server groups which are rebooting at different times.
So I configured a timeout of 120 seconds and a retry count of 5.
Our idea was to get a tolerance of 10 minutes for the rebooting issue. (Same way we did in the very old Compaq Insight Manager)

This problem seems not to be new:
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1039565

Any help would kindly be appreciated.

Regards
Jimmy

Re: Reboot and Hardware Status Polling tasks

Mahesh Shah_3 — Tue, 13 Feb 2007 10:41:17 GMT

I made the same changes which you recommended, but we are still receiving the system reachable/unreachable messages during reboot. Did you make the timeout of 120 seconds and retry count of 5 changes into Hardware Status Polling tasks only. Or make changes into Global Protocol setting or System Protocol Settings as well as.

Re: Reboot and Hardware Status Polling tasks

Jimmy Rueedi — Tue, 13 Feb 2007 11:49:43 GMT

we did it only in the status polling task

regards

Jimmy

Re: Reboot and Hardware Status Polling tasks

Jimmy Rueedi — Tue, 27 Feb 2007 11:22:04 GMT

Hi all

I think this could be a solution:
Change the global protocol settings for ICMP Ping to a timeout of 120 seonds and for example 3 retries.

After doing so, you have to change the system protocol setting to those global setting for all systems.

Since re-configuring this way, we never get any "false positive" Alert while rebooting any servers.

The other side of the medal is, that a system outage is alerted firstly after 6 minutes...

So you have to decide if you like doing this globally or only for servers which are periodically rebooting.

Best regards

Jimmy

Re: Reboot and Hardware Status Polling tasks

Mahesh Shah_3 — Wed, 14 Mar 2007 10:27:10 GMT

Assigning points