Unable to add a DL380 G5 to SIM 5.1

[NJK-Work]

Hello,

I have a DL380 G5 server (Windows 2003 SP2) running with the latest (7.8) agents and firmware that I am unable to add to SIM.

I should add, that I have been using SIM for years and have added litterally 1000's of devices without problem. So I have gone through all the standard things to look for (SNMP settings, etc.) - I am not trying to brag, but just avoid the basic troubleshooting suggestions that I have already done.

I think the problem is with the SNMP stack itself. When I try to send a test SNMP trap from the "HP Management Agents" tool in the Control Panel, it gives me the following error:

"The trap could not be generated.
Please ensure that the host agent is enabled and running." (screen shot of this error is attached).

I have:
1) Uninstalled the HP agents and reinstalled them.
2) Uninstalled SNMP and reinstalled it.
3) Rebooted countless times.
4) Verified all agents are enabled (except Rack and Cluster - which we don't use) in the "HP Management Agents" tool.
5) Verified all the typical SNMP settings such as security and community name

So I am wondering if anyone has ever seen this and knows how to fix it.

Thanks
Nelson

[loooolaaaaa]

Hi,

Are you trying to add a new server to HP SIM 5.1? is the server pingable from the CMS? If yes, then it will be added to database. Make sue you turn off all the firewall in the managed node and run an Automatic or a Manual discovery.

[Rob Buxton]

That seems like an odd disconnect between the agents and snmp.
Out of interest how was the server built? Smart Start plus a W2K3 SP2 install or an SP1 build plus SP2 upgrade? Or maybe Smart Start wasn't used?
As the above poster suggested - firewalls? but I can't see how that would get in between the agent and snmp. I would have thought that would just stop communication to / from snmp - which would affect discovery but should cause the failure you're seeing.

Any other "interesting" applications on the server that might grab snmp? I know we have a server that has Ericcson software and that seems to grab snmp and won't let it go - makes updating the agents a real pain.

[NJK-Work]

Hi - thanks for the responses.

The server was built from an image of Windows 2003 SP1 (using sysprep). The original image was built from SmartStart (but don't recall the version - I think 7.6). We have other servers built from this image with no problems.

The server was later upgraded to SP2 before being put into production.

The server is not behind a FW.

The server was shipped to a vendor for custom software installation prior to being put into production (this is not standard for us; we usually do everything in house). Past experience tells me that when we have sent servers off to a vendor (again very infrequently) they come back in a questionable state.

The software they installed is pretty unique stuff. I am betting it disrupted SNMP somehow - I just can't figure out how to "reset it". I'll shoot an email off to them to see if they "fess up" to anything.

Thanks
Nelson

[Rob Buxton]

I'm with you on that, I'm always suspicious when a server needs to be sent away for a software product to be installed.

It's obviously not a server you can easily rebuild either!

[Rob Buxton]

Just a thought, try using the Options - Manage System Types and add a new device. You can use this to see if snmp can actually reach the device and then what it returns.

If that cannot get access and you've rechecked snmp you could download and try one of the snmpwalk type tools to see if that can get access.
This should at least narrow down the problem a bit.

[NJK-Work]

Hi Rob,

I tried manually adding the server. It says it is adding it, but never does.

Can you provide any suggestions on an SNMPWalker I should try?

Thanks
Nelson

[David Claypool]

Nelson, with Discovery Filters off, even if SNMP can't be accessed on the device, HP SIM will add a device that responds to ping and then classify it as 'unmanaged.'

[NJK-Work]

Hmmm - I guess I did not try with the filters off. I will try that.

Rob and David - I did find this event in the system log everytime we restart the agents:

Component: Host Agent
Error: Unable to acquire file system information for %1
Cause: This error can be caused by a low memory condition. Rebooting the server may correct this error.

Event ID is 2099 and category is "Host".

(The server has 1 GB of total memory, and from what I can tell seems to be OK in terms of usage).

Does this error/event mean anything to either of you two? Or anyone else?

Thanks
Nelson

[NJK-Work]

Hi,

OK - here is what I did:

1) Tried a free SNMP test tool - from what I can tell, SNMP works fine on the box. I compared the results to a known good box and they box return similar results (I believe it uses system uptime as the check). So I think SNMP is workin.

2) I removed device filters and added the server. It added as David suggested, but is still unmanaged (also as David suggested).

3) I ran an identify on it and got this as part of the results log:

Checking for SNMP support on system.
The system responded to a SNMP request, it supports SNMP....

and later...

Running HP ProLiant management agent identification
System does not have the SNMP ProLiant Server (foundation) agents installed.

But I KNOW the agents are installed and running. Something must be corrupt in them. Since I already uninstalled them and resinstalled them I am not sure what else I can try.

Any suggestions?
Thanks
Nelson

[loooolaaaaa]

Browse to the System Management Homepage(http://localhost:2301) and see whether agents are proper data. Also goto control panel -> HP Management Agents and see whether agents are enabled

[NJK-Work]

SMH looks good. It correctly shows the model type in the top right corner and shows all the correct agents along the left of the screen (Integrated Agents, Other Agents, Management Processor). Settings tab show correct SNMP settings and my HP SIM certificate is correctly installed.

The control panel -> HP Management Agents is correct as well. All agents are enabled except Cluster and Rack (neither of which we use).

I have just recently built a new SIM server and began migrating all my devices over to it. This problem server is having the same issues on the new SIM server.

Thanks
Nelson

[WD Snider]

This probably won't fix your problem with adding the server to SIM, but it will take care of the error messages that state "Error: Unable to acquire file system information for %1". I've been having that same error message on a DL380 G5 running Win2k. That error message started occurring after Exchange was installed on the server. Within 1 minute of starting the EXIFS driver, the error message started occurring. Anyway, If you remove the "Host Information" Agent from the Foundation Agents list and restart the services, you won't see the error message any more.

[NJK-Work]

Thanks, but I don't have Exchange on that server.

Nelson

[loooolaaaaa]

If SMH is showing all the data, then HP Management agents are working fine. I guess you need to set SNMP correctly. By defualt SNMP will be pointing to localhost, change it to accept from any host and also set the community strings correctly and then reidentify the server in HPSIM. Hope this helps you

[John Jorgensen]

I have exactly the same problem, but with 4 servers: HP ProLiant BL460c G1 running Windows Server 2003 R2 SP2. I have a total of 16 servers, all HW identical. No problem with 12 servers. More than 100 other servers are all correct listed in HP SIM.
All firmware upgraded: BIOS to version 2007.02.26. iLO FirmWare 1.35. Onboard Administrator v2.02.
The 4 servers, which I am not able to add to my HP SIM 4.1, are all running as terminal servers. When trying to send a test trap, I get the error: â The trap could not be generated.â
I have called HP who advised me to uninstall SP2, but the problem is the same. I am waiting for HP to call me back. If I get a solution, I will post it here.

John Joergensen

[NJK-Work]

Thanks.

BTW - you mentioned SIM 4.1. Did you mean SIM 5.1?

Nelson

[John Jorgensen]

NO, I mean SIM 4.1, we are migrating to version 5.1 in a couple of weeks.
HP is still working on a solution.

[ThomasO]

Hi Nelson

Since Proliant Support Pack 7.90 for Windows 2003 is out now, did you try to install that?

I know there were some issues with VCA in 7.80, however, that should not result in the problems you see.

/Thomas

[NJK-Work]

Thanks - I will give that a try.

Nelson

[John Jorgensen]

I have already upgraded to version 7.90 - same error.
HP advised me to do the same thing as Nelson did first: Uninstall Agents, SNMP â Reboot â Install SNMP and last install the HP Agents again. Then I should put an IP address in the trap destination instead of server name, but I still get the same error: The trap could not be generated.
I have sent new log files to HP today.

John

[NJK-Work]

Well,

As suggested, 7.9 did not fix the problem either.

One thing I noticed is that no matter what I do, I cannot get the "Intel Chipset Support for ProLiant Servers for Windows Server 2003" to update to the latest version. The latest version shows 7.0.0.0, but the installed version is 6.1.0.1008 and it will not let me upgrade it.

Could "Intel Chipset Support for ProLiant Servers for Windows Server 2003" be required for some SNMP type stuff?

Thoughts?
Nelson

[ross stewart_2]

Guys, was there ever a solution to this problem. I've just reinstalled sp2 and it made no difference to the foundation error being reported in the event log. This is on W2k3 server running psp8.1 ie latest greatest. Thanks

[NJK-Work]

No, I never found a resolution to this problem other to manually type in all fields for the "unknown" device and set it so that discover jobs do not overwrite the info I typed in manually.

Nelson

[John Jorgensen]

In my case, the problem is because of using Softgrid. It seems to be related the Softgris Q-drive, which is inaccessible. The problem is escalated to Microsoft (Softgrid) and HP, but still no solution. We have recognized that, even if the device is unknown, are receiving alerts.

John