Can't Access SMH via SIM on ESX 3.01 servers

[Scott Huisman]

We have 2 ESX 3.01 farms of 6 servers each, both farms were built from the same RDP job. All of the servers have the System Management Agents v7.70 installed and on half of the servers the SMH is accessible via SIM. The other half report "There is no System Management Home Page for serverXXX".

As has been suggested in many of the forum posts I have run Identification jobs on the servers, Data collection jobs, deleted the systems out of SIM and re-added them, and reinstalled the management agents package. Nothing seems to work! I should mention that even though the pages aren't accessible through SIM they are accessible from the local server and through https://servername:2381.

I've noticed one difference between the functional systems and the non-functional systems is the available protocols listed under product description. The functional systems have "HTTP:, SMH:2.0, SNMP:1.0, SSH:SSH-2.0-OpenSSH_3.6.1p2"and the non-functional systems only have "SNMP:1.0, SSH:SSH-2.0-OpenSSH_3.6.1p2". The problem appears to be that both the HTTP and SMH:2.0 protocols are missing and that's why it's not working correctly.

Any assistance clearing this up would be greatly appreciated. Thanks in advance!

[Rob Buxton]

If the page is accessible from the HPSIM Server using the IP address as noted above I'd next verify SNMP is set up correctly.

Compare the /etc/snmpd/snmpd.conf files are the same.

[Scott Huisman]

The problem is that the homepage isn't working through SIM. I have compared the snmp.conf files on the working servers to the non-working servers and they are the same. SNMP is working correctly since when I go to the SMH page manually the system is identified correctly and the hardware reports correctly. The only problem seems to be the difference in the available protocols.

[Rob Buxton]

Can you browse to https://servername:2381 from the HPSIM Server itself?

You might want to check the ESX firewall file, but I can't see why that would stop only certain requests;
/etc/vmware/firewall/services.xml

How do the servers show in HPSIM, are they correctly identified? Or do they show "unknown"?

[Scott Huisman]

I can browse to the https://servername:2381 from the HPSIM Server no problem.

I've compared the firewall file between a working system and one that's having this problem and they appear to be identical.

Finally the servers are identifying correctly as dl585 G1's, so SNMP is working properly. Thanks for the suggestions, I appreciate your continued assistance.

[Rob Buxton]

Doesn't make a lot of sense.
I'm not too sure what else could make HPSIM not discover the protocol. Especially after a delete and rediscover from HPSIM.

You might want to try running mxnodesecurity -l from a cmd prompt on the HPSIM Server. You'll need to cd to the bin directory of the HPSIM install.
Just check what HPSIM thinks the SNMP and if you use them wbem settings in use are.
I use wbem here to identify the guests. It's enabled globally but with no credentials. I then enable it specifically for the VM guests plus VM Host servers.
I've also found that the mxnodesecurity command sometimes gives different results from the web page.

[Dana Swanson]

Hello,

I had the same problem with the HP Agents for VMware ESX Server 3.x 7.7.0.

Try adding one managed Guest VM to the ESX server your having this issue with and see if this fixes the problem.

When I was having the same problem a few things were changed on the ESX 3.0.1 server by our ESX admin that I think fixed the issue.

The changes were:

a. The ESX server was added to the farm
b. A test guest VM was added to the server
c. The ESX server was patched with the latest security patches from VMWare

On c I don't know what patches were installed since our ESX admin installed them. If your still having the issue after adding the ESX server to the farm and adding a test VM let me know and I can try to get the patches that were installed to the ESX server.

Dana

[Scott Huisman]

Rob: I tried what you said (running mxnodesecurity -l) and the server is reporting everything correctly. The line that I get back says
"abvmw021.ent.agt.ab.ca snmp TMON-R TMON-C
abvmw021.ent.agt.ab.ca wbem root ********
abvmw021.ent.agt.ab.ca ssh root ******** "

Dana: These trouble servers are already part of a HADRS cluster and have many production VM's on them so if you could let me know what patch it was that your team applied that would be great.

[Dana Swanson]

Hi Scott,

If you search for these patch numbers on the vmware site they should all point to a patch for ESX 3.0.1:

4825991
5095559
5140477
6704314
7302867

This one was a mystery to me as well. I called another admin here running the hp insight agents for ESX 3.0.1 and he had the same type of issue and it just started working one day.

Sorry I can't give you a real specific solution. It just started working here as well.

Dana

[Rob Buxton]

Just guessing now as I cannot see anything obviously wrong.

Check the output from mxnodesecurity for those servers working okay.
Also try an Identification, again against a working and non-working server and see what the differences are.

I don't use SSH for any kind of access, don't know if that would get in the way or not.

As regards the ESX firewall, just check there are no other files in the directory that contains services.xml

[A Shine]

I have the same problem. Was a solution found to this issue?

Thanks

Andrew

[Scott Huisman]

Unfortunately I haven't solved this one yet, but I'm still working on it!

[Rich Purvis]

Well, as Rob pointed out, if you can fire up a browser on the system running HPSIM and point it at the server having problems with https://servername:2381 then HPSIM should be able to recognize it. At the bottom of the login page for SMH you will see a string with the version information. Is it the same version of SMH running on the systems you are having problems with, that is running on the one's that are working?

-Rich

[Rob Buxton]

It still seems like a firewall filter issue to me, either the ESX one or another in the network between the HPSIM Server and the ESX Server.
From the VM Host server enter:
esxcfg-firewall -q
At the bottom there should be two entries like;
Opened ports:
hpim : port 2381 tcp.in
sim-cert : port 280 tcp.out

Also recheck there is only the one file - services.xml located in the directory:
/etc/vmware/firewall
Having additional files in this directory will cause the firewall to misbehave.

[Rich Purvis]

If there is a firewall active how do you get the browser to connect to https://servername:2381? Oh wait . . . hmmm initital discovery/identification of SMH is through 2301. Can you connect using a browser to http://servername:2301 - this is http *not* https. If you can it will eventually push you to 2381 but HPSIm does its initial SMH contact through 2301. Could it be 2301 is blocked?

-Rich

[Rob Buxton]

Rich,
Not sure if that would work for the ESX Server as it doesn't look as though the agent installation adds an entry for that port into the firewall. Looking a bit more closely the query of the firewall should give:
esxcfg-firewall -q | grep 2381
0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:2381
hpim : port 2381 tcp.in

I think this is effectively allowing all tcp access on port 2381.

[Frederic Schrobiltgen]

Hi,

Rob: you are right! It's a firewall issue.

We had the same problem with an ESX 3.0.1 and the hpmgmt-7.8.0-vmware3x.tgz package.

As explained, it was possible to:

- Configure the agent
- access directly the hpsmh page on the server

But it wasn't possible to access the homepage from Insight Manager. The HTTP and SMH protocols weren't added.

Why "was"?

Because I have found a workaround. It's not elegant but it's working ;0)

I have carefully read all your comments and asked to myself: why they didn't disabled the firewall during the inscription of the client?

I done it(service firewall stop), went on Configure -> Configure or repair agents and put all the necessary info to repair the link.

I tested it and now it's working well.

Of course, I have enabled the ESX firewall back.

I'm testing it for an half hour and everything is working.

Now, I hope that this modest contribution will help you.

FRED

[Frederic Schrobiltgen]

Hi,

New update:

I wasn't able to access the homepage through Insight this morning. I received a "No management homepage on server ".

I have:

- disabled the esx fw
- done a repair of the bind
- enabled the fw back on the esx

Everything is working again!

It's a fw issue....

I'm going to test it tomorrow morning. If I receive the same error message, I'll do a wireshark capture and analize it.

I'll keep you informed.

Best regards,

FRED

[Frederic Schrobiltgen]

Hi,

New update:

Finally, I prefer to open the common HP SIM Agent ports on the ESX.

The end of the FW status =

Opened ports:
SNMP : port 161 tcp.in udp.in
WaWServer : port 2301 tcp.in udp.in
SNMP_TRAP : port 162 tcp.out udp.out
hpim : port 2381 tcp.in
sim-cert : port 280 tcp.out
WBEM : port 5988 tcp.in udp.in

These port were referenced in the HPSIM_Security_WP file.

Best regards,

FRED

[Frederic Schrobiltgen]

Hi,

I can now confirm that what I have done yesterday is still working today.

Not enough ports were opened to allow the agent to work properly.

You can resolve this case by doing the following procedure:

Prerequisites: ESX 3.0.1 / hpmgmt-7.8.0-vmware3x.tgz successfully installed / no bind from Insight Manager interface

1/ stop the Firewall service of your ESX through a SSH or the console (service firewall stop)
2/ select the ESX in HP Insight Manager and go to the Configure menu -> Configure or repair agents
3/ do the repair
4/ start the Firewall service of your ESX through a SSH or the console (service firewall start)
5/ refresh IM and verify that the hyperlink is working (from the server list/in the ESX zoom view)
6/ go back to the ESX console (or SSH) and type the following commands:

esxcfg-firewall -o 161,tcp,in,SNMP
esxcfg-firewall -o 161,udp,in,SNMP
esxcfg-firewall -o 2301,tcp,in,WaWServer
esxcfg-firewall -o 2301,udp,in,WaWServer
esxcfg-firewall -o 162,tcp,out,SNMP_TRAP
esxcfg-firewall -o 162,udp,out,SNMP_TRAP
esxcfg-firewall -o 5988,tcp,in,WBEM
esxcfg-firewall -o 5988,udp,in,WBEM

(you can copy these commands into a script if you don't want to type them)

7/ enjoy

Of course, you can directly open the ports before installing the software and avoid stopping the firewall.

It's up to you.

Best regards,

FRED