EMS is constantly paging

[Rick Garland]

Hi all:

HPUX 11.11 on rp7410 with 2 hard partitions.

EMS is setup for /system/filesystem/availMb/ and there are numerous entries like this. I am confused, the Notify is set to <100% - I hope all filesystems remain less tha 100%.

Second, this config has been in place for awhile, why now am I getting the pages?


Thanks

[Rick Garland]

Also to note, the value that the EMS is alarming on is in the hundreds/thousands %. How does it calulate these figures?

[Andrew Merritt_2]

Can you give an example of the message you are getting, and what the setting is in SAM?

[Rick Garland]

Here is a couple of entries from the syslog.

---EMS Notification--- Value: "error" for Resource: "/system/filesystem/availMb/save" (Threshold: < "100")

---EMS Notification--- Value: "214.6" for Resource: "/system/filesystem/availMb/save" (Threshold: < "100")

Listed above are the correlating entries for the "/save" filesystem.

In SAM the EMS monitor is saying Notify < 100 for the filesystem. The current value states 214.6 as well.

Where is the Value "214.6" coming from? Why Notify <100? Shouldn't they always be <100?

[Andrew Merritt_2]

It's monitoring amount of freespace in Megabytes, not a percentage.

[Rick Garland]

OK, I can go with that. But the Notify is set for <100, all of the entries are dlearly greater than 100 with some in the thousands

[Andrew Merritt_2 · Accepted solution]

I don't have the answer, but just looking to characterise the problem further:

Are all the availMb notifications you have configured causing these entries in syslog, or only some of them?

Do you have the 'Return' option set in the notifications? I'm wondering if the 'error' entry is detected as a fault, and the second notification (if two is all you get for each notification configured) is caused when it is detected the situation has returned to being within the acceptable limits.

You could check the api.log, client.log and registrar.log files in /etc/opt/resmon/log for errors at the time the events are logged, particularly the 'error' ones.

What version of EMS do you have installed?

[Rick Garland]

Hi Andrew:

In the syslog, all entries of the availMb are listed, twice. 1st time with the "Value:error" and the 2nd time with the "Value:".

I have looked at the logs you mentioned but nothing is being updated in the logs. They are several days old since the last posting to these logs.

The EMS version is A.03.20.01. I will be looking at upgrading to the newer product, ISEE (?) I think its called.

[Andrew Merritt_2]

> In the syslog, all entries of the availMb are listed, twice. 1st time with the "Value:error" and the 2nd time with the "Value:".

Ok; did you select the 'Return' option when you set up the notifications in SAM? You can see this when you view the notifications in SAM.

I think what is happening is that for some reason EMS is failing to get the freespace values, and is then generating the first notification (the "error" one). Then it does get the information, detects that the value is back in the acceptable range (<100) and generates the notification as the Return option is selected. (Of course, this doesn't make sense if you didn't select 'Return')

I don't know what is causing the failure to detect the freespace by fsmond. Are there any disk related problems?

ISEE is not a replacement for EMS, it sits on top of it; however I think installing ISEE will require an upgrade of EMS to A.04.00, but I wouldn't swear to that.

[Andrew Merritt_2]

Have you managed to make any progress with this?

If my suggestion does match the facts, and you haven't found out why fsmond is reporting the 'error' messages, I'd recommend that you open a support call with HP.

[Rick Garland]

The cause of this is pointing to the BCV activity. Seems that during the BCV process the FS freep space is lost in translation somewhere and the resultant value is "error" which causes the EMS fsmond to page. Once it gets its values back, the EMS fsmond is paging again to say "all OK".

Can't be sure this is the cause but it is looking like it. Gonna keep diggin.