topic Re: Smart Array P840ar error in HPE Apollo Systems

Smart Array P840ar error

liechtjc — Fri, 10 Mar 2023 10:52:04 GMT

Hello,

We have an HP Apollo 4200 server with a Smart Array controller p840ar filled with 24 SAS 12GB 18TB drives.

We use it with 12 SAS drive for almost 1 year without any problem.

We expanded the RAID-6 storage with 12 additional drives.

Users are accessing it using a software copying file (Pomfort Silverstack) from a local drive on a mac computer to this drive using a SMB mount The copy always perform without any problem (no error reported).

BUT we discovered that some folder have simply not been copied and some xxh64 hash are later wrong.

I quickly checked the system running Rocky Linux is healthy and don't show any problem.

I have never seen such a big problem and I'm looking for a procedure to investigate this problem that can happen at multiple level:

RAID
Samba
Application itself when writing on samba share

In the Smart Array Serial Log I find these type of fatal errors:

Drive SN: ***confidential info erased*** CDB=0x1201D0000400 CC Sense Data-- 00: 72 05 24 00 00 00 00 1C 02 06 00 00 CF 00 02 00 03 02 00 01 80 0E 00 00 00 00 00 00 1C: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Fatal [ctrl] PR=0x8145eb30 D016 Op=12 PLErr=02 IopErr=04 S=02 KCQ=5:24:00 Bad CDB: 0x0000: 12 01 D0 00 04 00

I can see this kind of errors on almost all drives of the array. (around 20 per drive)

I undertand these are SCSI sense data errors. But I m not able to really interpret them. Are these only bad blocks?

Can this explain the extrem problems we are experiencing?

More how would you investigate this kind of problems? Is the a script you could use to intensively write on the storage and test the accuracy of the writing?

I know this is not the best forum to ask but how to enable extensive log on samba and what kind of error should I look for?

Thank you!

Jean-Christophe

Re: Smart Array P840ar error

Suman_1978 — Thu, 09 Mar 2023 08:02:22 GMT

Hi,

From HW or FW level, this issue could be with the following:

Hard Drive backplane and/or cables.
Array controller Firmware or card itself.

As its happening on almost all drives, please log a support ticket with HPE and or Software vendor to verify some logs.

Thank You!
I work with HPE but opinions expressed here are mine.
Recent Support Video Releases

Re: Smart Array P840ar error

liechtjc1 — Fri, 10 Mar 2023 05:55:29 GMT

@Suman_1978

I'm adding an element to the reflexion: problem can happen at:

System Memory
RAID
Filesystem
Samba
Application itself when writing on samba share

We had to do a (so far successful) xfs_repair on the storage, all errors happens on files written prior to the repair. So far the repair was sucessful but we didn't save the output of the repair.
Is there a possibility to find the detailed log of this repair (that happened 1 month ago)?
I can imagine that it explains the hash error in some file but can this explain that we loose full folder?

I also read that this can be related to RAM issue in the system? can it be?

Re: Smart Array P840ar error

Suman_1978 — Tue, 14 Mar 2023 06:02:29 GMT

Hi,

Please follow the troubleshootng resources available from HPE.

You may also log a support ticket with HPE to analyze the logs.

Thank You!
I work with HPE but opinions expressed here are mine.
Recent Support Video Releases