Disk Thresholds and Systems Management Homepage

[Mike Drobick]

I am using an ML350 G1 as my SIM 4.2 server. I just upgraded yesterday from HP SIM 4.1 and everything has been working fine.

I set my disk thresholds without a problem and then tested on the SIM server by filling the disk up to about 97% to see what the events looked like. I received both the rising threshold and critical rising threshold, but when I cleaned up the data, I never received the critical falling threshold.

I waited a few hours and tried filling the disk up again. This time, I didn't get any traps. I checked the thresholds from the Management Agent page and it is correct. Currently showing 97% disk space utilized with a critical limit set to 95%.

I did send a Test Trap from the Managment Agent in Control Panel and I received that in HP SIM within a minute or so, so I know that should be working. Besides, the trap is coming from itself. Seemed like the easiest test to perform.

At the same time, my Systems Management Homepage seems to be missing a bunch of stuff. I only have a "Home" tab, a "Settings" tab and a "Logs" tab. The main page shows "No failed/degraded items" and the only other section on this page is for Version Control Agents. I distinctly remember there being an Environment section at some point, so it seems like it's messed up.

Now, I have seen this sparse management homepage on a few servers in the past and I just chalked it up to old agents. However, all of my servers are now running 7.10 agents and this server, being the SIM 4.2 server, is running the new Systems Management Homepage 2.0.0.103.

Anyone have any ideas for my two problems:

1) Disk Threshold traps not being sent.

2) Busted up looking Systems Management Homepage.

Anyone else share the same experience?

Thanks.

Mike

[David Claypool]

When there are elements missing from the SMH, typically it is caused by a service that isn't running. Go make sure that all of the agent services show as started.

Break the Disk Threshold problem up into its components: did the agent detect it, did the agent send it, did HP SIM receive it? Start with the Windows Event Log to see if it was detected. It also is very likely that the above problem is the only cause.

[Mike Drobick]

David - Thanks for the reply. I restarted the agents on that server and everything seemed to come back to life.

As for the disk thresholds, let me follow your direction and break that problem up a little more. Give me a few days to get back with that information.

Thanks again.

Mike

[Mike Drobick]

OK...here's what I'm trying. Let me know if I'm doing something wrong.

I set my disk threshold by doing the following in HP SIM:

Click Configure->Disk Thresholds->Set Disk Thresholds.

I select my targets and click Next.

I set the values: Critical=95; Reset Critical=92; Warning=85; Reset Warning=82; Polling=60

I click Run Now and then I check the status and everything completes successfully, no errors.

Do I understand the values as: 95% and 85% are rising thresholds and 92% and 82% are falling thresholds? Is the 60s polling interval the wait time before HP SIM checks to see if the disk are over the thresholds?

If I understand this correctly, here is what I've found:

In the System Event Log on the server, I never get a falling threshold notification. I have seen rising threshold notifications and in turn, do get those email notifcations.

I have recently tested by filling up a drive on my server. On the Systems Management Homepage, it reads 96% full with the critical threshold set to 95%. However, I've now waited 15 minutes and still haven't received a rising threshold notification in the event viewer.

Can anyone shed some light for me?

Thanks.

Mike

[Ted Jerzewski]

Have you received a resolution to this issue?

[Bryan Swanson_1 · Accepted solution]

Mike,

I too have seen this. IMO, It appears to be a quirk of the SMH applet that sets the thresholds. If you do an snmpwalk of a host before and after setting the disk thresholds you'll see the two limits (warning and critical) but the alarm sample type (cpqMeAlarmSampleType) is set to suppress falling trap (absSuppressFallingTrap) which from cpqhlth.mib reads:

If the sample type is absSupressFallingValue(4), only comparisons with the cpqMeAlarmRisingThreshold are performed.

For the record, here's an snmpwalk of one of my boxes, you'll note that the alarm owner for alarms 1 and 2 is "Insight Web Agent" so they were definitely created by the SMH java applet.

CPQTHRSH-MIB::cpqMeAlarmIndex.1 = INTEGER: 1
CPQTHRSH-MIB::cpqMeAlarmIndex.2 = INTEGER: 2
CPQTHRSH-MIB::cpqMeAlarmIndex.3 = INTEGER: 3
CPQTHRSH-MIB::cpqMeAlarmInterval.1 = INTEGER: 15
CPQTHRSH-MIB::cpqMeAlarmInterval.2 = INTEGER: 15
CPQTHRSH-MIB::cpqMeAlarmInterval.3 = INTEGER: 1
CPQTHRSH-MIB::cpqMeAlarmVariable.1 = OID: CPQHOST-MIB::cpqHoFileSysPercentSpaceUsed.0
CPQTHRSH-MIB::cpqMeAlarmVariable.2 = OID: CPQHOST-MIB::cpqHoFileSysPercentSpaceUsed.0
CPQTHRSH-MIB::cpqMeAlarmVariable.3 = OID: CPQSTDEQ-MIB::cpqSeEisaIntSlotIndex
CPQTHRSH-MIB::cpqMeAlarmSampleType.1 = INTEGER: absSuppressFallingTrap(4)
CPQTHRSH-MIB::cpqMeAlarmSampleType.2 = INTEGER: absSuppressFallingTrap(4)
CPQTHRSH-MIB::cpqMeAlarmSampleType.3 = INTEGER: absoluteValue(1)
CPQTHRSH-MIB::cpqMeAlarmValue.1 = INTEGER: 33
CPQTHRSH-MIB::cpqMeAlarmValue.2 = INTEGER: 33
CPQTHRSH-MIB::cpqMeAlarmValue.3 = INTEGER: 0
CPQTHRSH-MIB::cpqMeAlarmStartupAlarm.1 = INTEGER: risingAlarm(1)
CPQTHRSH-MIB::cpqMeAlarmStartupAlarm.2 = INTEGER: risingAlarm(1)
CPQTHRSH-MIB::cpqMeAlarmStartupAlarm.3 = INTEGER: risingAlarm(1)
CPQTHRSH-MIB::cpqMeAlarmRisingThreshold.1 = INTEGER: 50
CPQTHRSH-MIB::cpqMeAlarmRisingThreshold.2 = INTEGER: 70
CPQTHRSH-MIB::cpqMeAlarmRisingThreshold.3 = INTEGER: 0
CPQTHRSH-MIB::cpqMeAlarmFallingThreshold.1 = INTEGER: 48
CPQTHRSH-MIB::cpqMeAlarmFallingThreshold.2 = INTEGER: 68
CPQTHRSH-MIB::cpqMeAlarmFallingThreshold.3 = INTEGER: 0
CPQTHRSH-MIB::cpqMeAlarmPermanence.1 = INTEGER: permanent(2)
CPQTHRSH-MIB::cpqMeAlarmPermanence.2 = INTEGER: permanent(2)
CPQTHRSH-MIB::cpqMeAlarmPermanence.3 = INTEGER: temporary(1)
CPQTHRSH-MIB::cpqMeAlarmOwner.1 = STRING: "Insight Web Agent"
CPQTHRSH-MIB::cpqMeAlarmOwner.2 = STRING: "Insight Web Agent"
CPQTHRSH-MIB::cpqMeAlarmOwner.3 = ""
CPQTHRSH-MIB::cpqMeAlarmStatus.1 = INTEGER: valid(1)
CPQTHRSH-MIB::cpqMeAlarmStatus.2 = INTEGER: valid(1)
CPQTHRSH-MIB::cpqMeAlarmStatus.3 = INTEGER: invalid(4)
CPQTHRSH-MIB::cpqMeAlarmSeverity.1 = INTEGER: warning(4)
CPQTHRSH-MIB::cpqMeAlarmSeverity.2 = INTEGER: critical(6)
CPQTHRSH-MIB::cpqMeAlarmSeverity.3 = INTEGER: unknown(1)
CPQTHRSH-MIB::cpqMeAlarmExtendedDescription.1 = STRING: "/ on /dev/vg00/root disk space"
CPQTHRSH-MIB::cpqMeAlarmExtendedDescription.2 = STRING: "/ on /dev/vg00/root disk space"
CPQTHRSH-MIB::cpqMeAlarmExtendedDescription.3 = ""


As for a workaround, I'm not really sure. I suppose you could change the alarm sample type, but the falling threshold for the rising alarms is really just there for hysteris protection. Probably the right thing to do would be to create an additional alarm for each monitored filesystem that only triggers when the desired falling threshold(s) is/are passed, but that would take either modification of the PHP (something I'm not up to) or post tweaking from some shell/perl script and let's face it, if you're using a script to make changes why not use the script to set ALL the thresholds in the first place.

FYI, on Linux, I believe the script in question is /opt/hp/hpsmh/data/htdocs/webagent/setthreshold.php.en

regards,
-b