vmunix alerts in syslog.log

[#1]

Hello Experts,

I am getting continues vmunix alerts in syslog.log file of server which looks like as below.

DIAGNOSTIC SYSTEM WARNING:
   The diagnostic logging facility has started receiving excessive
   errors from the I/O subsystem.  I/O error entries will be lost
   until the cause of the excessive I/O logging is corrected.
   If the diaglogd daemon is not active, use the Daemon Startup command
   in stm to start it.
   If the diaglogd daemon is active, use the logtool utility in stm
   to determine which I/O subsystem is logging excessive errors.
DIAGNOSTIC SYSTEM WARNING:
   The diagnostic logging facility is no longer receiving excessive
   errors from the I/O subsystem.  1 I/O error entries were lost.

Any idea how to overcome this repeat error?

Thanks & Regards,

Chirag Parikh

[#2]

You are getting I/O errors from a problematic device.  Look for other entries in syslog.log prior to these messages or run the command 'dmesg'.

You dont give details such as OS version or model of the server, but perhaps there are also entries being logged in the EMS event log.  View the file /var/opt/resmon/log/event.log for any recent entries.

Also, you could search this forum for many other examples of others receiving the same message.

-Bob

[#3]

Here are the missing information from my last post.

Ioscan output -- 

XXXXXX:root> ioscan -fn |grep -i "FCP" |more
fcp 2 0/2/1/0/4/0.201 fcd_fcp CLAIMED INTERFACE FCP Domain
ext_bus 20 0/2/1/0/4/0.201.0.0.0 fcd_vbus CLAIMED INTERFACE FCP Array Interface
ext_bus 21 0/2/1/0/4/0.201.0.255.0 fcd_vbus CLAIMED INTERFACE FCP Device Interface
ext_bus 22 0/2/1/0/4/0.201.1.0.0 fcd_vbus CLAIMED INTERFACE FCP Array Interface
ext_bus 23 0/2/1/0/4/0.201.1.255.0 fcd_vbus CLAIMED INTERFACE FCP Device Interface
ext_bus 24 0/2/1/0/4/0.201.4.0.0 fcd_vbus CLAIMED INTERFACE FCP Array Interface
ext_bus 25 0/2/1/0/4/0.201.4.255.0 fcd_vbus CLAIMED INTERFACE FCP Device Interface
ext_bus 26 0/2/1/0/4/0.201.5.0.0 fcd_vbus CLAIMED INTERFACE FCP Array Interface
ext_bus 27 0/2/1/0/4/0.201.5.255.0 fcd_vbus CLAIMED INTERFACE FCP Device Interface
ext_bus 41 0/2/1/0/4/0.201.14.255.0 fcd_vbus CLAIMED INTERFACE FCP Device Interface
ext_bus 37 0/2/1/0/4/0.201.15.255.0 fcd_vbus CLAIMED INTERFACE FCP Device Interface
fcp 3 0/6/1/0/4/0.101 fcd_fcp CLAIMED INTERFACE FCP Domain
ext_bus 28 0/6/1/0/4/0.101.0.0.0 fcd_vbus CLAIMED INTERFACE FCP Array Interface
ext_bus 29 0/6/1/0/4/0.101.0.255.0 fcd_vbus CLAIMED INTERFACE FCP Device Interface
ext_bus 30 0/6/1/0/4/0.101.1.0.0 fcd_vbus NO_HW INTERFACE FCP Array Interface
ext_bus 31 0/6/1/0/4/0.101.1.255.0 fcd_vbus NO_HW INTERFACE FCP Device Interface
ext_bus 32 0/6/1/0/4/0.101.4.0.0 fcd_vbus CLAIMED INTERFACE FCP Array Interface
ext_bus 33 0/6/1/0/4/0.101.4.255.0 fcd_vbus CLAIMED INTERFACE FCP Device Interface
ext_bus 34 0/6/1/0/4/0.101.5.0.0 fcd_vbus CLAIMED INTERFACE FCP Array Interface
ext_bus 35 0/6/1/0/4/0.101.5.255.0 fcd_vbus CLAIMED INTERFACE FCP Device Interface
ext_bus 40 0/6/1/0/4/0.101.14.255.0 fcd_vbus CLAIMED INTERFACE FCP Device Interface
ext_bus 36 0/6/1/0/4/0.101.15.255.0 fcd_vbus CLAIMED INTERFACE FCP Device Interface

two hardware path is captured in event.log file which cause this repeat alerts, check the no_hw status hardware path from above.

OS version -- B.11.11

Hardware model -- 9000/800/rp4440

[#4]

This looks like 1 path to the storage is down for any reason.

You should check the connection (e.g. fcmsutil ... from host or switchshow from switch). Maybe bad cable, bad port, etc...?