topic Re: net-snmp on HP-UX 11.31 in Operating System - HP-UX

net-snmp on HP-UX 11.31

SyedHamid — Thu, 18 Apr 2013 12:19:10 GMT

Hi All,

I'm trying to monitor a HP-UX server with an application Orion Solarwinds

Following are the version details

NET-SNMP version: 5.7.2

HP -UX : B.11.31

I have disabled HP-UX's snmp and have installed it from HP recommended depot ( https://h20392.www2.hp.com/portal/swdepot/searchProducts.do )

I'm able to snmpwalk on version 1 via public community on the HP-UX server. However I'm not able to get the server to be snmpwalked via a community from the monitoring system.

I have attached my snmpd.conf file and netstat details.

Any help would be highly appreciated.

Thanks

Syed

Re: net-snmp on HP-UX 11.31

SyedHamid — Thu, 18 Apr 2013 13:49:48 GMT

Also following is my lsof output

lsof -i udp:161

COMMAND PID USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
snmpd   4878 root    5u IPv4 0xe0000003139bbc80      0t0 UDP *:snmp (Idle)
snmpd   4881 root    5u IPv6 0xe00000036f71c980      0t0 UDP *:snmp (Idle)

Re: net-snmp on HP-UX 11.31

Matti_Kurkela — Thu, 18 Apr 2013 15:08:14 GMT

Your lsof output and your previous netstat output don't match:
according to the "agentaddress" setting in the configuration you posted, the net-snmp daemon should be listening in port 1161, as confirmed by netstat. Your ps -ef indicates that the net-snmp daemon is running as PID 2022.

Yet your lsof indicates snmpd is running in port 161 and with PIDs 4878 (for IPv4) and 4881 (for IPv6).

What is going on?

Re: net-snmp on HP-UX 11.31

SyedHamid — Fri, 19 Apr 2013 11:50:37 GMT

I was testing,I have changed my agentaddress to 161. My new snmpd.conf ( relating to my lsof output ) has been uploaded.

Re: net-snmp on HP-UX 11.31

Matti_Kurkela — Fri, 19 Apr 2013 15:20:34 GMT

Is 172.16.5.57 the IP address of your monitoring system? At the moment, it seems to be the only remote host that is authorized to query your net-snmp daemon for anything at all.

If your monitoring system has multiple NICs and/or IP addresses, are you sure its requests to this HP-UX hosts are sent out via the 172.16.5.57 interface?

With your current configuration, queries from 172.16.5.57 will only succeed if community "FusAfiL2IQs" is used, and even then with read access only.

Any other non-local queries will fail.

Re: net-snmp on HP-UX 11.31

SyedHamid — Tue, 23 Apr 2013 08:46:52 GMT

Hi,

True, on the HP-UX server

/opt/iexpress/net-snmp/bin/snmpwalk -v 2c -c FusAfiL2IQs 172.16.184.34 system

and

/opt/iexpress/net-snmp/bin/snmpwalk -v 2c -c FusAfiL2IQs 172.16.5.57 system

Work absolutely fine. However when I try and poll the HP UX server from the Orion Solarwinds applications it fails.

Thanks

Syed

Re: net-snmp on HP-UX 11.31

Matti_Kurkela — Tue, 23 Apr 2013 14:01:52 GMT

So the IP address 172.16.5.57 belongs to the HP-UX server itself? And maybe 172.16.184.34 too?

Then you need to find out the IP address the Orion Solarwinds applications send their SNMP requests from, and add it to a line like this:

com2sec mynetwork  aa.bb.cc.dd          FusAfiL2IQs

Replace aa.bb.cc.dd with the IP address the Orion Solarwinds uses.

Re: net-snmp on HP-UX 11.31

SyedHamid — Fri, 26 Apr 2013 11:23:18 GMT

com2sec mynetwork 172.16.5.57 FusAfiL2IQs

is my entry

172.16.5.57 - Is the Solarwinds server

172.16.184.34 - Is the ip of the HPUX box

Re: net-snmp on HP-UX 11.31

Matti_Kurkela — Sat, 27 Apr 2013 08:37:41 GMT

Try restarting snmpd with full debug logging enabled:

/opt/iexpress/net-snmp/sbin/snmpd -LF7 /tmp/snmp-troubleshooting.log -a -p /var/tmp/net-snmpd -c /opt/iexpress/net-snmp/etc/snmpd.conf

This should produce a log file at /tmp/snmp-troubleshooting.log.

Then send some queries locally, using first a correct community name and then an incorrect one. Both queries should produce messages in the log.

Then send some queries from the Solarwinds server and look at the log again.

I think there are three possibilities:

no new messages in the log = something is blocking the queries from the Solarwinds server from reaching the HP-UX system
the messages say that snmpd is rejecting the queries from the Solarwinds server = most likely the Solarwinds server is not actually using the community name you expect it to use, or
the log file says snmpd is receiving the query and responding to it = something is blocking the response from reaching the Solarwinds server.

Re: net-snmp on HP-UX 11.31

SyedHamid — Mon, 29 Apr 2013 09:17:55 GMT

no new messages in the log = something is blocking the queries from the Solarwinds server from reaching the HP-UX system.

Re: net-snmp on HP-UX 11.31

SyedHamid — Mon, 29 Apr 2013 09:59:13 GMT

Here is the output from the log

Warning: no access control information configured.
(Config search path: /opt/iexpress/net-snmp/etc/snmp:/opt/iexpress/net-snmp/share/snmp:/opt/iexpress/net-snmp/lib/snmp://.snmp)
It's unlikely this agent can serve any useful purpose in this state.
Run "snmpconf -g basic_setup" to help you configure the snmpd.conf file for this agent.

Re: net-snmp on HP-UX 11.31

Matti_Kurkela — Mon, 29 Apr 2013 17:19:27 GMT

>Warning: no access control information configured.

If there is nothing else in the log, then the snmpd is not reading the configuration file at all???

Wait... the command in my previous reply was supposed to be a single long line. If you typed it as two separate commands, that might explain this error.

>(Config search path: /opt/iexpress/net-snmp/etc/snmp:/opt/iexpress/net-snmp/share/snmp:/opt/iexpress/net-snmp/lib/snmp://.snmp)

According to this, you should probably put your snmpd.conf file to /opt/iexpress/net-snmp/etc/snmp/ directory. If the directory does not exist, create it. Then you should not need to explicitly tell snmpd the configuration file location with the -c option.

Re: net-snmp on HP-UX 11.31

SyedHamid — Tue, 07 May 2013 14:40:59 GMT

Hi Matti,

Thank you for your patience, still doesnt work

I have snmp(HP's) running on udp 161 and net-snmp on 1161

HPUX-root-:/#>lsof -i udp:1161
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
snmpd 14558 root 6u IPv4 0xe00000036a2ba100 0t0 UDP *:netsnmp (Idle)

HPUX-root-:/#>lsof -i udp:161
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
snmpdm 14986 root 6u IPv4 0xe00000038ead7080 0t0 UDP *:snmp (Idle)

/opt/iexpress/net-snmp/bin/snmpwalk -m ALL -v 2c -c FusAfiL2IQs 172.16.184.34:1161 .1

Does not work

/opt/iexpress/net-snmp/bin/snmpwalk -m ALL -v 2c -c FusAfiL2IQs 172.16.184.34:161 .1

Works

/opt/iexpress/net-snmp/bin/snmpwalk -m ALL -v 2c -c public localhost:1161 .1

Works

Any idea's

Re: net-snmp on HP-UX 11.31

Matti_Kurkela — Wed, 08 May 2013 07:12:55 GMT

> /opt/iexpress/net-snmp/bin/snmpwalk -m ALL -v 2c -c FusAfiL2IQs 172.16.184.34:1161 .1

> Does not work

What is the error message? Is it "Timeout: No Response from 172.16.184.34:1161" or something else?

If you do this with snmpd debug logging enabled, do any new messages appear in the log? What do they say?

Re: net-snmp on HP-UX 11.31

SyedHamid — Wed, 08 May 2013 08:16:16 GMT

/opt/iexpress/net-snmp/bin/snmpwalk -m ALL -Le -v 2c -c FusAfiL2IQs 172.16.184.34:1161 .1

Timeout: No Response from 172.16.184.34:1161

Log output is ( Verbose Mode)

NET-SNMP version 5.7.2
Received SNMP packet(s) from UDP: [172.16.184.34]:59733->[0.0.0.0]:0
GETNEXT message
    -- ccitt.1
Received SNMP packet(s) from UDP: [172.16.184.34]:59733->[0.0.0.0]:0
GETNEXT message
    -- ccitt.1
Received SNMP packet(s) from UDP: [172.16.184.34]:59733->[0.0.0.0]:0
GETNEXT message
    -- ccitt.1
Received SNMP packet(s) from UDP: [172.16.184.34]:59733->[0.0.0.0]:0
GETNEXT message
    -- ccitt.1
Received SNMP packet(s) from UDP: [172.16.184.34]:59733->[0.0.0.0]:0
GETNEXT message
    -- ccitt.1
Received SNMP packet(s) from UDP: [172.16.184.34]:59733->[0.0.0.0]:0
GETNEXT message
    -- ccitt.1

Re: net-snmp on HP-UX 11.31

SyedHamid — Wed, 08 May 2013 08:37:09 GMT

Ok, Now I have made a minor chane to the snmpd.conf file

com2sec mynetwork 172.16.184.34 FusAfiL2IQs

Where the ip 172.16.184.34 belongs to the HP BOX and now when I do

/opt/iexpress/net-snmp/bin/snmpwalk -m ALL -Le -v 2c -c FusAfiL2IQs 172.16.184.34:1161 .1

it works

following is the log

Received SNMP packet(s) from UDP: [172.16.184.34]:60782->[0.0.0.0]:0
GETNEXT message
    -- ccitt.1
Received SNMP packet(s) from UDP: [172.16.184.34]:60782->[0.0.0.0]:0
GETNEXT message
    -- SNMPv2-MIB::sysDescr.0
Received SNMP packet(s) from UDP: [172.16.184.34]:60782->[0.0.0.0]:0
GETNEXT message
    -- SNMPv2-MIB::sysObjectID.0
Received SNMP packet(s) from UDP: [172.16.184.34]:60782->[0.0.0.0]:0
GETNEXT message
    -- DISMAN-EVENT-MIB::sysUpTimeInstance
Received SNMP packet(s) from UDP: [172.16.184.34]:60782->[0.0.0.0]:0
GETNEXT message
    -- SNMPv2-MIB::sysContact.0
Received SNMP packet(s) from UDP: [172.16.184.34]:60782->[0.0.0.0]:0
GETNEXT message
    -- SNMPv2-MIB::sysName.0
Received SNMP packet(s) from UDP: [172.16.184.34]:60782->[0.0.0.0]:0
GETNEXT message
    -- SNMPv2-MIB::sysLocation.0
Received SNMP packet(s) from UDP: [172.16.184.34]:60782->[0.0.0.0]:0
GETNEXT message
    -- SNMPv2-MIB::sysORLastChange.0
Received SNMP packet(s) from UDP: [172.16.184.34]:60782->[0.0.0.0]:0
GETNEXT message
    -- SNMPv2-MIB::sysORID.1
Received SNMP packet(s) from UDP: [172.16.184.34]:60782->[0.0.0.0]:0
GETNEXT message
    -- SNMPv2-MIB::sysORID.2
Received SNMP packet(s) from UDP: [172.16.184.34]:60782->[0.0.0.0]:0
GETNEXT message
    -- SNMPv2-MIB::sysORID.3
Received SNMP packet(s) from UDP: [172.16.184.34]:60782->[0.0.0.0]:0
GETNEXT message
    -- SNMPv2-MIB::sysORID.4
Received SNMP packet(s) from UDP: [172.16.184.34]:60782->[0.0.0.0]:0
GETNEXT message
    -- SNMPv2-MIB::sysORID.5
Received SNMP packet(s) from UDP: [172.16.184.34]:60782->[0.0.0.0]:0
GETNEXT message
    -- SNMPv2-MIB::sysORID.6

However Solarwinds ( windows server) still fails to snmpwalk the machine to the HP box on 1161 port.

Re: net-snmp on HP-UX 11.31

Matti_Kurkela — Wed, 08 May 2013 21:55:43 GMT

OK.

If I understood correctly, your snmpd.conf file should now have both IP addresses on com2sec lines:

com2sec mynetwork 172.16.5.57       FusAfiL2IQs
com2sec mynetwork 172.16.184.34     FusAfiL2IQs

Now, if you try snmpwalk from the Solarwinds server, with the logging enabled at the HP-UX server, what happens?

Do you see messages like "Received SNMP packet(s) from UDP: [172.16.5.57]:<some port number>->[0.0.0.0]" in the log at the HP-UX server, or not?

If not, it may be a network communication issue in getting the packets from the Solarwinds server to the HP-UX server.

If the messages are appearing in the log and you are using the correct community name at the Solarwinds server, then it is probably another network communication issue: something is preventing the answers of the HP-UX server from getting back to the Solarwinds server.

If you see lines like this:

Connection from UDP: [172.16.5.57]:<some port number>->[0.0.0.0] REFUSED

... then your net-snmpd is receiving the query from the Solarwinds server, but rejecting it because the source IP address is not in the allowed list. The net-snmpd may be compiled with the "libwrap support" feature enabled: in that case, you should add a line like this to your /etc/hosts.allow file (or create the file if it does not exist):

snmpd:  127.0.0.1 172.16.184.34

If the log seems to indicate that net-snmpd is receiving and accepting the query from Solarwinds, or if no query from Solarwinds seems to be received at all, it might be a firewall issue.

SNMP is an UDP-based protocol. With UDP, allowing communication from A to B does not automatically guarantee that replies from B back to A are allowed too. This is because some UDP-based protocols are truly one-directional and won't ever reply. (For example, the syslog remote logging protocol.)

With TCP-based protocols, sending something from A to B always requires that B can send acknowledge messages back to A, so if communication from A to B is allowed, replies from B to A must be automatically allowed too.

Most modern protocols use TCP, so sometimes firewall administrators assume that the TCP behavior is the standard for all protocols, and forget that UDP-based protocols can be different. Some firewalls can automatically allow the replies with UDP protocols too, but that requires that the firewall "knows" how the UDP-based protocol is supposed to behave. A SNMP protocol in standard port 161 might have its responses handled automatically, but using SNMP with a custom port 1161 might need some special settings, or just a separate firewall rule like:

allow UDP packets from port 1161 of 172.16.184.34 to any port of 172.16.5.57

The usual firewall rules are like:

allow <protocol> from any port of <source IP> to port <number> of <destination IP>

so the net-snmp in a custom port might need a rule that works in a reverse sense. Most firewalls allow this, but the firewall administrator might not be able to use his/her "standard template" in defining the rule for net-snmp replies, so mistakes are more likely than usual.

The firewall might be implemented in software too: an IPFilter on the HP-UX system or a Windows firewall on the Solarwinds system might be causing the problem. If a HP-UX system has IPFilter installed and configured, its configuration files should be in /etc/opt/ipf. If that directory does not exist or is empty, you can be reasonably sure you don't have IPFilter active on your system.

Re: net-snmp on HP-UX 11.31

SyedHamid — Thu, 09 May 2013 05:02:42 GMT

Thanks for detailed explanation, really appreciated. I have check with IP filtering

######-root-:/#> ipf -V
ipf: IPFilter is currently disabled

I will talk to security team today and see if there is any rule in place or if required will create one.

Thanks and will keep you posted

Re: net-snmp on HP-UX 11.31

SyedHamid — Thu, 09 May 2013 08:05:22 GMT

Ok, now i have some real data.

When I snmpwalk from the Solarwinds server ( 172.16.5.57) to HP-UX box 172.16.184.34 on SNMPv2 and the defined community, following is what is captured in the log file

NET-SNMP version 5.7.2
Received SNMP packet(s) from UDP: [172.16.184.34]:64617->[0.0.0.0]:0
GET message
    -- SNMPv2-MIB::sysObjectID.0
Received SNMP packet(s) from UDP: [172.16.184.34]:64617->[0.0.0.0]:0
GET message
    -- SNMPv2-MIB::sysObjectID.0
Received SNMP packet(s) from UDP: [172.16.184.34]:64617->[0.0.0.0]:0
GET message
    -- SNMPv2-MIB::sysObjectID.0

The server fails detection on Solarwinds

Re: net-snmp on HP-UX 11.31

SyedHamid — Mon, 13 May 2013 04:16:47 GMT

Hi Matti,

Here is my routing table

Routing tables
Destination           Gateway            Flags Refs Interface Pmtu
127.0.0.1             127.0.0.1          UH    0    lo0       32808
172.16.184.34         172.16.184.34      UH    0    lan0      32808
172.16.184.0          172.16.184.34      U     2    lan0       1500
127.0.0.0             127.0.0.1          U     0    lo0       32808
default               172.16.184.1       UG    0    lan0       1500