topic Re: IMC server started sending out Interface Down traps for non-existent interfaces in IMC

IMC server started sending out Interface Down traps for non-existent interfaces

NeilR — Tue, 16 Jun 2015 22:22:09 GMT

The IMC server has started sending out traps for Interface Down for a range of non-existent interfaces, just increasing the index each time.

The server in IMC shows 24 interfaces, but the index is way past that. Getting 10-20 of these a day. Tried rebooting the server but no change. Wondering why this is happening all of a sudden, as its been a few weeks since last install/update. Any Ideas?

Name Link DOWN
Level Major
OID 1.3.6.1.6.3.1.1.5.2.0
Alarm at 2015-06-15 14:46:59
Alarm Source
hpimc.XXX.net(10.10.X.X) More Alarms...
Type Trap
Alarm Category Interface/Link Status Alarm
Recovery Status Unrecovered
Acknowledgement Status Unacknowledged
Description The interface 185 is DOWN.

Versions:

IMC PLAT 7.1 (E0303P10)

iMC NTA 7.1 (E0301P04)

iMC UAM 7.1 (E0302)

iMC WSM 7.1 (E0303P04)

Re: IMC server started sending out Interface Down traps for non-existent interfaces

LindsayHill — Wed, 17 Jun 2015 00:55:12 GMT

What's generating the traps? The underlying OS, or the IMC application?

Re: IMC server started sending out Interface Down traps for non-existent interfaces

NeilR — Wed, 17 Jun 2015 17:54:18 GMT

Good question -

source is host name, but the Windows SNMP service, while configured to be queried, was not set to send traps anywhere so that would mean the IMC application I'm thinking

Did not see anything in obvious windows events either.

Any other way to tell for sure?

Re: IMC server started sending out Interface Down traps for non-existent interfaces

LindsayHill — Wed, 17 Jun 2015 20:08:43 GMT

Something related to trap forwarding then?

Re: IMC server started sending out Interface Down traps for non-existent interfaces

NeilR — Wed, 17 Jun 2015 21:19:54 GMT

Don't think so as the IMC server itself is the destination for traps from other devices. At least not intentionally forwarded traps onward.

I restarted the SNMP agent for the windows OS that imc is running on. That generated a trap for cold start (or agent restart) with the source being the IMC server. OID 1.3.6.1.6.3.1.1.5.0.0

I added public to the traps tab of the SNMP service with the IP of the IMC server (on that same server). Now I get "Device receives an SNMP message with an erroneous community string" error

Same source - the IMC server - OID 1.3.6.1.6.3.1.1.5.4.0

I used the mib management option to query the snmp agent on the IMC server

1.3.6.1.6.3.1.1.5.4 Description: A linkUp trap signifies that the SNMP entity, acting in an agent role, has detected that the ifOperStatus object for one of its communication links left the down state and transitioned into some other state (but not into the notPresent state). This other state is indicated by the included value of ifOperStatus.

Not the same description as IMC is reporting. Sorry, not much of an expert on snmp....

But the Interface down errors MAY be gone - will need to watch for a while. In the meantime removed the public trap config from above so will see if no longer reporting erroneous string.

Re: IMC server started sending out Interface Down traps for non-existent interfaces

NeilR — Wed, 17 Jun 2015 22:55:12 GMT

OK - no issue any longer. To review:

Alarms for non-existent interfaces on the IMC server started 6/4 and continued until today.

Today -

Set public trap destination to IMC server in SNMP service - was blank - restart service

New Alarm for "Device receives an SNMP message with an erroneous community string" every 17 minutes

Clear public trap destination to IMC server in SNMP service and restart service

Both Alarms gone -

Re: IMC server started sending out Interface Down traps for non-existent interfaces

LindsayHill — Fri, 19 Jun 2015 15:41:52 GMT

Weird. At least it's behaving now.

Re: IMC server started sending out Interface Down traps for non-existent interfaces

NeilR — Fri, 19 Jun 2015 18:33:49 GMT

It's really weird - and still not totally behaving. I have not been able to get a repeatable pattern, so I held off adding more to the thread. Reboots, service restarts, nothing has proven to have repeatable cause/relief

Interface down alarms came back, so I poked at it again. Still not fixed. The SNMP settings in IMC match the WIndows service security tab, and test OK in the configure SNMP settings on the device page. All my widnows servers have the same settings for r/w security. But they are not sending traps. Only on the IMC server.

W/o the sending traps, was getting "Device receives an SNMP message with an erroneous community string" alarm

For sending traps, using the same settings as the switches, but you can't set the destination in IMC - gives an error, even if set in the Windows snmp service

I have another server just started sending the same erronous alarm as this discussion has progressed.

Starting to wonder about the imc state in general - my database auto backup stopped working 3 days ago. Also my routing switch, has several times changed from device role routers to virtual. Some data related issue?

All my modules except UAM are latest. UAM still does not authenticate computer properly despite claim to have fixed this.

Re: IMC server started sending out Interface Down traps for non-existent interfaces

LindsayHill — Sat, 20 Jun 2015 02:38:02 GMT

@NeilR wrote:

For sending traps, using the same settings as the switches, but you can't set the destination in IMC - gives an error, even if set in the Windows snmp service

Setting the trap destination from IMC only works for systems that allow SNMP read/write, and where IMC has the right adapters. It won't work for Windows servers.

Seems like something a bit weird is going on with that system. Hard to say for sure what's happening tho

Re: IMC server started sending out Interface Down traps for non-existent interfaces

NeilR — Mon, 22 Jun 2015 17:18:07 GMT

figured that was the case.

Tried it on a netapp server, which I had previously set trap destination. It now fails.

Defintely starting to wonder if there is some DB or data corruption going on. Its been a few weeks now since I upgraded to E0303P10 but odd things have been building. Unless some issue related to running UAM back at E302. But can't go forward on that until they fix PEAP computer auth, now rumored to be in UAM 303P14. Go back to last main version?

Not sure how to address possible DB issues. Do a restore? don't really want to rebuild from scratch.

Re: IMC server started sending out Interface Down traps for non-existent interfaces

NeilR — Tue, 30 Jun 2015 21:30:31 GMT

So interface DownTraps are now up to interface index 180. The server's list contains 22 interfaces. As nothing I've tried seems to break the behavior, I'm thinking filtering.

So i know i can filter all types of traps for all devices, each device, and specific interface index by the paramater settings button (FIlter on index, description, admin or operate status)

But I really want to block traps above the REAL indexes, and still be able to recieve traps for the actual interfaces I might care about.

The device/index selector can only select actual indices discovered by IMC.

Can I use reg expressions in the paratmer field to filter?

No mention in the docs and no complaints about entering one from IMC, but no obvious indication yet that its working either....especially as not so amazing with regex