Systems Insight Manager
cancel
Showing results for 
Search instead for 
Did you mean: 

Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

Pber
Advisor

Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

I've recently upgraded my servers to use 9.10.00.0 of the server agents.  Ever since the upgrade,ALL of my servers are now filling the system event log with Event ID 1182 Informational events indicating that "A Change in the health status of the server has occurred".  This occurs every 2 minutes based on the SNMP polling frequency.  It also sends traps to HPSim causing an Informational change of status on the Aggregate Status Icons as well.  I can turn off the trap handling for the trap

cpqHoMibHealthStatusArrayChangeTrap - 11020 in CPQHOST.MIB in HPSim, but doesn't fix the problem of this alert filling up my system event logs.

 

Any ideas other than downgrading back to version 8.70.00.0 which wasn't an issue.

28 REPLIES
BERT_1
Occasional Advisor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

Hello,

I watch the same effect since updates this weekend, but only on 64bit servers?

32bit servers seem not to have this behaviour.

Any solution or config advices?

 

Pber
Advisor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

Good call.  I didn't look that closely since we only have a handful of machines that are 32bit.  But I stand corrected, it is only x64 machines exhibiting this behaviour.  My Windows 2003 x64 and 2008 R2 machines with the latest 9.1 agents are all doing this.

greg long_4
Occasional Advisor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

I have the same problem, 2008 R2 SP2 64 bit

 

Also we are using images with sysprep

BERT_1
Occasional Advisor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

Had a hp support call and was adviced to downgrade:

Older server models with Windows 2003 are yet only allowed to SP 8.70 !!

I did and got rid of these messages.

With HP-SIM the server according version for the reference support pack version should be looked for.

greg long_4
Occasional Advisor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

I have back reved from HP Service Pack for Proliant 2012.06.0 to PSP 9.0

 

I removed the Version control agent, insight agent and home page and then installed the 9.00 version.

 

I now no longer get the 1182 error, but I would like to see HP acknowledge that there is an issue!!!!!

 

This has cost my company 16 to 20 tech hours troubleshooting and reloading machines as this error does not show itself right away or until you look at event log or configure notification.

greg long_4
Occasional Advisor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

FYI We are on

 

DL380 G5 and DL360 G5 running  2008 R2 with SP2

 

MMetzner
Occasional Visitor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

hello,

 

i have the same problem with many servers proliant dl370 g6 and windows server 2003 r2 64 bit. i downgrade only to hp insight agents 8.70 and now i have no more events, i hope hp will fix this very fast.

Jockeman
Occasional Visitor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

Installed 2012-06 yesterday and I have same problem on DL380 G5 with Windows Server 2003 R2 x64

mbrownnyc
Advisor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

Same problem on Windows 2003 R2 32-bit, ML150 G5.

 

About every one-two minutes the Foundation Agents reports some of the services that are running locally (review the Data section of the event in the event log), then reports that they are unreachable.  This cycle continues indefinitely.

 

Here is some of the data I have pulled from the Event log entry present when I was receieving Email notices stating an snmp trap was received with a Details stating NOT AVAILABLE :

 

#Domain Name Server.domain 53/udp
#Domain Name Server.bootps 67/udp

dhcps
#Bootstrap Protocol Server.bootpc 68/udp

dhcpc
#Bootstrap Protocol Client.tftp 69/udp
#Trivial File Transfer.gopher 70/tcp
.finger 79/tcp
.http 80/tcp

www www-http
#World Wide Web.hosts2-ns 81/tcp
#HOSTS2 NameServer.hosts2-ns 81/udp
#HOSTS2 NameServer.kerberos 88/tcp

krb5 kerberos-sec
#Kerberos.kerberos 88/udp

krb5 kerberos-sec
#Kerberos.hostname 101/tcp

hostnames
#NIC Host Name Server.iso-tsap 102/tcp
#ISO-TSAP Class0.rtelnet 107/tcp
#Remote Telnet Service.pop2 109/tcp

postoffice
#PostOffice Protocol-Version2.pop3 110/tcp
#Post Office Protocol -Version3.sunrpc 111/tcp

rpcbind portmap
#SUN Remote Procedure Call.sunrpc 111/udp

rpcbind portmap
#SUN R...WV
CPQTHRSH.HOSTALRT

 

I do not have many of these services running on the local machine.

 

 

When I received an alert email with no Details listed, the Foundation Agent event log event data in the System Event Log listed four processes that were and are running on the machine.

 

Reviewing: %systemroot%\system32\CpqMgmt\cqmghost\cpqmgmt.chm it appears that the Foundation Agent provides things such as process and service monitoring.  You may be able to disable these two features and the problem may or may not go away (but this is what I'm looking to do).

 

 

After some investigation with process monitor, I see that the log entries correlate with cqmghost.exe (the Foundation Agents service) accessing two keys:

HKLM\SYSTEM\CompaqPermanent\

and

HKLM\System\CompaqHost\ 

 

 

In my monitoring, both of these seem to be accessed only by cqmghost.exe.

 

These appear to be a cache of some sort.

 

I wonder if some sort of issue exists when upgrading directly, without uninstallation first.

 

I will uninstall the HP Insight Management Agents, delete these keys, then reinstall later.

 

 

[update 1]
With the agents uninstalled, and the those keys deleted, the web UI is still populating without issue (since it uses WBEM or SNMP, in my case WBEM) [tested by reviewing sectors read and written on a RAID disk].

 

cp015760.exe is the file to install the component "HP Insight Management Agents for Windows Server 2003/2008" (taken from %systemdrive%\cpqsystem\hp\log\localhost\hpsum_log.txt)

 

In the summary of the previous install, I do see that SUM detected the older version of the component:


Component File Name: cp015760.exe
Component Name: HP Insight Management Agents for Windows Server 2003/2008
Original Version: 8.70.0.0
New Version: 9.10.0.0
Installation Result: Success

 

 

I ran cp015760.exe solo.

 

After install a window with manage agents ("C:\WINDOWS\system32\rundll32.exe" Shell32.dll,Control_RunDLL cpqmgmt.cpl /setup) pops up.  Registry keys that were deleted are added again.

 

I unchecked Enable SNMP sets, noting that polling was set to 2 minutes by default... hmm... since I'm using the WBEM (wmi), and click OK.

 

A 1182 event is logged in the System Event Log.

 

Four minutes pass.  No more 1182 events, because there is no more polling...  Enabling polling causes events to be logged.

 

[Conclusion]

So the problem exists with the SNMP sets that are being polled.  You can access the Management Agents configuration control panel applet by ruhning cpqmgmt.cpl or by accessing "HP Management Agents" from the Control Panel itself.

 

After this select your minimum usage.  Since I figured out, by reviewing the Data logged in the 1182 events in the System Event Log, that there was at least one of the "Active Agents" that were causing a problem.  I would go through trial and error to figure out which one is causing the issue.  You will note that there is also a "Process Monitor" tab present, so this might be how the processes above were listed.

 

 

I began tested removing the Active Agents one-by-one.  Luckily, Host Information (near the top of the list), seemed to be the problem cause.

 

I removed the Host Information agent from SNMP polling, waited 8 minutes, and only the single "OK" status change was logged (at three minutes).

 

I then re-added the Host Information agent, waited 3 minutes, saw the "OK" status change again.  Then waited another 3 minutes, received the first status change event (1182, citing CPQTHRSH.HOSTALRT).   It appeared that some processes were listed in the Event Data.  I then waited another 5 minutes and nothing popped up.

 

The info is sparse, but the issue is clearly related to HOSTALRT, which is found in both events' Data.

 

For the sake of testing, I unchecked

 

[tl;dr]

Through  the "HP Management Agents" control panel applet (cpqmgmt.cpl):

 

1) Try to clear all thresholds on the SNMP Settings tab...

 

2) If that fails, Try to disable Application Exception Traps...

 

3) If that fails, remove the Host Information agent from the Active Agents list in the "HP Management Agents" control panel applet.  This clearly stops some polling from occurring, but I can not figure out which OIDs the Host Information Agent polls, or which data will be missing.

BERT_1
Occasional Advisor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

Hello.

Again my earlier reply:

On a hp support call I got the advice, to respect the max allowed support pack, esp. for older server models and os releases. Newer agents may cause problems on this systems!

Within VCA from Insight Homepage you can see on "reference support pack"  the last supported version.

 

I did so and all is running fine.

 

mbrownnyc
Advisor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

BERT_1:  Is the "reference support pack" not dynamic according to the version available to the VCA from the Version Control Repository Manager, but has nothing to do with the server model number or the OS version?\

 

There is no Version Control Repository Manager server here.

BERT_1
Occasional Advisor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

We manage about 60 phys servers and use a zentral VC repository for hp-sim.

In this constellation the "reference supportpack" seems to be dynamic according to model and installed os version.

Otherwise I give you the advice to take a look on the hp support matrix or check with a call to hp support.

Getting "only informational" entries in system eventlog is just harmless, but malfunctions on controllers are no fun.

MaxSys
Occasional Advisor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

Having this same issue on DL360 G4's and ML110 G7's (Running Server 2008 x64).

 

I'll just try to keep them all on outdated software for a while longer I guess.

Dan DeCoursey
Regular Advisor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

I ran SPP 2012 on windows 2003r2 32 bit  servers  and I too see my logs filled every 2 second with this 1182  event

so its not just 64 bit installs

Hers a pinch to grow an inch
Dan DeCoursey
Regular Advisor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

As I said in the above post  I ran SPP 2012

 

I have checked (as recommendaed in one of these ealrier  posts)  the Reference support pack   it currenlty says  8.70   so when  I selectthat version  after finishin that it goes back to the installed software page  and there are namy itens that  have newer versions listed under "latest version"  areyou saying  ishouls stay clars of those and alwasy just use whats listed in the Support pack Verison column??

Hers a pinch to grow an inch
BERT_1
Occasional Advisor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

HP support says: Check info text of each item if it is according to your server model and os, - or check by a support request.

I am confused for myself what may and what must be done...

Imho i think it is a good plan to update to the allowed level in general and do further (corrective) updates only on verified troubles.

 

MaxSys
Occasional Advisor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

Official Response from HP Support:

This issue is reproduced in our internal lab and has already been escalated to our development team.   We are working on a Customer Advisory to document this issue as it is effecting many customers.  A formal fix has been submitted to the 9.20 version of agents that are slated to be released in August 2012.

The workaround is to reinstall the management agents v 9.0.
If you are not willing to downgrade to 9.0 another workaround would be to disable the HOST INFORMATION sub-agent in the Control Panel applet, which will eliminate the three events from occurring on each polling cycle (default polling cycle for snmp is every 2 min).  If you are also running wbem providers this alternative workaround will not disrupt your hardware monitoring.  However, if you are only monitoring these servers by means of snmp this alternative workaround is not recommended as snmp hardware monitoring depends on the host sub-agent.

 

mbrownnyc
Advisor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

thanks.
matiasfortinero
Occasional Visitor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

Hi, Is there any news about the fix for this issue?

 

Thanks

 

Matias

SwisspostIT
Valued Contributor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

Pber
Advisor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

I just installed 9.2 server agents on serveral 2003 and 2008 x64 systems.  They quickly post a 11020 alert, but seems to be limited to the one alert so far.  hopefully this is fixed.

mbrownnyc
Advisor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

Thanks Pber.  Please us know how it works over time.

Marko_MSI
Occasional Advisor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

 

I am getting this mail every 2 minutes:

 

The system has detected the following event:

 SNMP Trap:       11020

 Date time:          12/05/2012  09:50:10 AM

Computer:         HOST

Source:                Foundation Agents

Type:                    Informational

Category:            (4)

 

 

 

 

I installed Management Agents 9.25 version without server restart. It looks the problem was fixed.

shocko
Honored Contributor

Re: Event ID 1182 Foundation Agent: A Change in the health status of the server has occurred.

Same problem. I can't believe v 9.10 of the agents is still available for download with this problem still present.

If my post was helpful please award me Kudos! or Points :)