VA7400 Error messages in the syslog and armlog

[Aziz Zouagui]

Greetings,

We have installed a new va7400 with 2 brocade switches with dual controllers and total redundancy, the size is 1TB RAID0/1.
We went with the VA7400 trying to move out of the Autoraid 12H, which was an IO hog.

The performance gain was the difference between night and day, compared to the 12H. We have been running in production for over 3 weeks now.

Recently, we have seen some errors in the syslog and some others in the array logs. Any ideas what these errors are is there any way we can get rid of them.

Thanks.

[Aziz Zouagui]

Sorry, I forgot to include the syslog.log file.

[Aziz Zouagui]

These are the type of errors I see in the armlog when I issued this command.

# armlog -c -s 122000002003 array_id | grep "Event Description"|sort -u
----------- Output -----------------------
Event Description = BACKEND_SCSI_EVENT_EH An error occurred on the disk interface. These may occur during disk hotplug. More d
etailed status is available in the event expanded information.
Event Description = DEV_IO_FAILED_EH A Physical Device on a backend loop failed to successfully complete an IO. This could be t
he result of exhausting retries or by running across a failure that disallows retries such as a media error.
Event Description = FRONTEND_FC_ABTS_EVENT_EH This error code indicates that the Host sent a Fibre Channel ABTS (Abort Sequence)
BLS frame to the abort an IO. The array will log this event for informational and debug purposes only. It does not necessarily in
dicate a problem with the array.
Event Description = FRONTEND_FC_LINK_UP_EH The Front End Fibre Channel port is now up.The topology that was acquired is in the e
xtended log information.
Event Description = FRONTEND_SERVICES_EVENT_EH This error code indicates that a Front End Services (FES) event has occurred and
has been logged. Specific informationabout the event is placed in the extended info area of the log.
Event Description = Lab Instrumentation Only.
Event Description = R1_WRITE_RECOVERED_WITH_REDUNDANCY_EH Redundancy was used by the R1 module in the successful completion of a
write.
Event Description = Rebuild has been started either automatically or by command. Unless there wasa reset during rebuild, at leas
t two more messages follow this message, REBUILD_STARTED_EH and one of REBUILD_CANCEL_STARTED_EH, REBUILD_COMPLETE_EH,NO_SPACE_FOR_R
EBUILD_EH, CANT_ALLOCATE_OPTIMAL_EH,REBUILD_FAILED_EH, or VS_ATTACHED_EH. The VS_ATTACHED_EH messageindicates that a format subsyste
m was issued through the front panel or byhost command, eliminating the need for a rebuild. Ifthere was a resetduring rebuild, you w
ill see the REBUILD_STARTED_EH message repeated.
Event Description = Rebuild has completed successfully. This message follows theREBUILD_STARTED_EH message.
Event Description = The drive returned CHECK CONDITION status with meaningless sense data, or sense data not otherwise handled b
y the PDD's error recovery algorithms. The PDD's retries are exhausted or the I/O is not retryable.
Event Description = WARNING_ACTIVATED_EH One of the warning indicators has been actviated. Check the extended event information
for more information.
Event Description = WARNING_DEACTIVATED_EH One of the warning indicators has been deactivated. Check the extended event informa
tion for more information.
---------------------------------


Some users are reporting that these errors could be linked to the PV timeout is too low, any suggestions would be appreciated.

Thank you and happy holidays.

Aziz.

[Ian Hillier]

You have a bad array controller! And from what I can tell, it's controller 2, but I'm not an expert. You should log a call with HP immediately and send them your logs to be analyzed. I had errors just like this and every couple of hours (randomly) one controller would go out and the array data would be virtually inaccessable until the rebuild finished.

Also, if you haven't upgraded your DISK firware, I would do so soon.

Regards,

John

[Mohanasundaram_1]

Hi Aziz,

I had a similar problem in VA7410. The Backend/frontend errors reported or due to a disk firmware being old.

There is a service note for this and you need to update the Disk (VA7400 array disks) firmware to HP09. Once this was done the backend errors were gone.

The power failed message also indicates that you may have to increase the IO timeout value.

You can check this using pvdisplay.

IO Timeout (Seconds) default

It may look something similar to the above. You can increase the value online using

pvchange -t 120

You can also check the below posting earlier by me:
http://forums.itrc.hp.com/service/forums/questionanswer.do?threadId=244099

Let me know if you need any further help on this.

With regards,
Mohan.

[Dave Thomson]

Hello

Welcome to the hell that is VA7400.

We have been busy for 2 months trying to sort out similar issues with HP. Im actuall waiting on a technician to arrive as i type this mail.

GOOD LUCK

[Dave Thomson]

Hello

Yeeehhhaaawwwww

Its the morning of the 31st, and our VA7400 is still not working correctly.

Aint life grand.

We are getting good support from HP, ill give them that much. They just cant seem to fix the problem.

Happy New Year.

[Aziz Zouagui]

Dave,

What problems are you having with your VA7400 ? are they the same ones I mentioned above.

Thanks

Aziz

[Rajesh G. Ghone]

Hi Aziz,

Same problem i have faced & rectified also.
Steps to follow:
1)Upgrade the Firmware of VA Controller to HP19,Hard Disk to HP09 & DS2405 to HP04.please do this first.
2)If this doesnt help then change the Controller no 2.

Hope this solves ur problem.

Regards,
Rajesh G.