- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - Linux
- >
- Re: ICE-Linux mond issues with mdadm
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-04-2009 05:27 PM
тАО11-04-2009 05:27 PM
ICE-Linux mond issues with mdadm
Nov 4 14:56:58 usorl03p307 mdadm: DeviceDisappeared /dev/md0
Nov 4 14:56:58 usorl03p307 mdadm: DeviceDisappeared /dev/md2
Nov 4 14:56:58 usorl03p307 mdadm: DeviceDisappeared /dev/md1
Nov 4 14:56:59 usorl03p307 mdadm: DeviceDisappeared /dev/md0
Stopping mond stops the messages.
/etc/init.d/mond stop
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-05-2009 11:57 AM
тАО11-05-2009 11:57 AM
Re: ICE-Linux mond issues with mdadm
These critical alerts are associated with the "Syslog Alerts" Service, correct?
I'd like to see if I can reproduce this. What version of RH5 do you have installed on your managed nodes (e.g. 32bit or 64bit; update 1 or 2)?
If you're not interested in seeing these mdadm critical alerts you should be able to stop the alerts by modifying the /opt/hptc/nagios/etc/syslogAlertRules file.
Try this and let me know if the alerts stop.
Edit syslogAlertRules (make a backup copy first) and change the mdadm rule to look as follows (i.e. add DeviceDisappeared to the list of mdadm events to ignore).
rule mdadm_errors {
name (! /(NewArray)|(SparesMissing) (DeviceDisappeared)/)
relevance ($subsystem =~ /mdadm/)
format "$timestamp $message"
}
Thanks,
Donna
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-05-2009 01:02 PM
тАО11-05-2009 01:02 PM
Re: ICE-Linux mond issues with mdadm
The RHEL version on the node is RHEL 5.4 x86_64 on BL495G5 blades in C7000 chassis.
Have been working with Mitch on other issues also but not this one.
We are interested in seeing valid mdadm alerts, but these are not valid and start after mond is stared.
I will make your suggested changes and report back.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-05-2009 01:12 PM
тАО11-05-2009 01:12 PM
Re: ICE-Linux mond issues with mdadm
maybe shoudl be: (SparesMissing)|(DeviceDisappeared)/)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-05-2009 01:58 PM
тАО11-05-2009 01:58 PM
Re: ICE-Linux mond issues with mdadm
rule mdadm_errors {
name (! /(NewArray)|(SparesMissing)|(DeviceDisappeared)/)
relevance ($subsystem =~ /mdadm/)
format "$timestamp $message"
}
Donna
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-05-2009 02:03 PM
тАО11-05-2009 02:03 PM
Re: ICE-Linux mond issues with mdadm
Donna
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-06-2009 04:52 AM
тАО11-06-2009 04:52 AM
Re: ICE-Linux mond issues with mdadm
mond -> /opt/hptc/supermon/etc/init.d/mond-setup
with mond stopped there are no more messages generated in /var/log so there is something that ICE-Linus (supermon) is doing that is causing the message to occur in the first place.
Need to find the root cause that is causing the messages.
I can provide you a virtual room connection if it would help.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-06-2009 06:46 AM
тАО11-06-2009 06:46 AM
Re: ICE-Linux mond issues with mdadm
On the CMS, vi /opt/hptc/nagios/etc/nagios_vars.ini. In this file you will see mdadminfo and MDAMDCOLLECTIONPERIOD.
MDADMCOLLECTION is set to 15 minutes which means on the target nodes, supermon will call /opt/hptc/mdadm/sbin/getMdadmEvents every 15 minutes. You can change this collection period to anything you like.
If you log in to one of you target nodes, you can look at /opt/hptc/mdadm/sbin/getMdadmEvents which calls mdadm-handler. mdadm-handler sends all messages returned by /sbin/mdadm to syslog.
We recently fixed an issue in our next IC-Linux release (V6.0) where this script was failing because it was being run as Nagios and not root so I'm wondering if your hitting that issue.
Can you run a test for me? On the target node, (as root) run /opt/hptc/mdadm/sbin/getMdadmEvents and tail /var/log/messages and let me know what you see.
Then login as Nagios (su - nagios) and run getMdadmEvents and let me know what you see in /var/log/messages.
In regards to the DeviceDisappeared event, do you think that /sbin/mdadm is incorrectly reporting this error? Or has the device really disappeared?
One work around I can think of is to modify mdadm-handler to check for the DeviceDisappeared event and not call syslog.
Donna
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-06-2009 10:54 AM
тАО11-06-2009 10:54 AM
Re: ICE-Linux mond issues with mdadm
When ran as nagio, each time the command getMdadmEvents generates:
Nov 6 13:45:53 usorl03p309 mdadm: DeviceDisappeared /dev/md1
Nov 6 13:45:53 usorl03p309 mdadm: DeviceDisappeared /dev/md0
Nov 6 13:45:59 usorl03p309 mdadm: DeviceDisappeared /dev/md2
Nov 6 13:45:59 usorl03p309 mdadm: DeviceDisappeared /dev/md1
Nov 6 13:45:59 usorl03p309 mdadm: DeviceDisappeared /dev/md0
I believe the messages are bogus and the devices are NOT disappearing.
dave
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО11-06-2009 12:15 PM
тАО11-06-2009 12:15 PM
Re: ICE-Linux mond issues with mdadm
I realize that's a lot of questions, but I'm just trying to figure out why mdadm would be reporting the error.