iLO 5 Drive Status Alerts - Determine failing drive

[DanU1]

Im getting alerts from iLO 5 about some of my disk drives failing. Its telling me the smart status keeps switching from OK to Other, and then back to OK again. It tells me a number for the drive (21) that I cannot match up elsewhere in iLO. How can I determine which drive to replace?  Thanks for any help!

"ID"

Severity

Class

Description

Last Update

Count

Category

410

Informational

Drive Array

ATA disk drive status is now Other for drive 21

09/30/2021 17:52:38

1

Hardware

409

Informational

Drive Array

ATA disk drive status is now Other for drive 21

09/30/2021 17:46:11

1

Hardware

408

Informational

Drive Array

ATA disk drive status is now Other for drive 21

09/30/2021 17:44:07

1

Hardware

407

Informational

Drive Array

ATA disk drive status is now Other for drive 21

09/30/2021 17:42:02

1

Hardware

406

Informational

Drive Array

ATA disk drive status is now Other for drive 21

09/30/2021 17:30:38

1

Hardware

405

Informational

Drive Array

ATA disk drive status is now Other for drive 21

09/30/2021 17:23:07

1

Hardware

404

Informational

Drive Array

ATA disk drive status is now Other for drive 21

09/30/2021 17:15:55

1

Hardware

403

Informational

Drive Array

ATA disk drive status is now Other for drive 21

09/30/2021 17:09:42

1

Hardware

402

Informational

Drive Array

ATA disk drive status is now Other for drive 21

09/30/2021 17:01:13

1

Hardware

401

Repaired

Drive Array

ATA disk drive status is now Ok for drive 21

09/30/2021 17:50:33

9

Hardware

400

Informational

Drive Array

ATA disk drive status is now Other for drive 21

09/30/2021 16:58:06

1

Hardware

399

Informational

Drive Array

ATA disk drive status is now Other for drive 21

09/30/2021 16:51:53

1

Hardware

398

Informational

Drive Array

ATA disk drive status is now Other for drive 21

09/30/2021 16:47:44

1

Hardware

397

Repaired

Drive Array

ATA disk drive status is now Ok for drive 22

09/30/2021 16:41:31

1

Hardware

396

Informational

Drive Array

ATA disk drive status is now Other for drive 22

09/30/2021 16:40:29

1

Hardware

395

Informational

Drive Array

ATA disk drive status is now Other for drive 21

09/30/2021 16:35:19

1

Hardware

[sudhirsingh]

@DanU1

ILO SNMP trap detecting change is drive state and logging these event.

Please check Bay21 and Bay22 drive status under Storage tab in ILO->System Information.

If drive status is failed/Wearout/SMART error/Predictive Failure then you should replace the faulty drive.

Incase all the logical drive and physical drives are OK then you may ignore these ILO alert it may be false however

suggested to update ILO firmware to latest.

hope this helps!

Regards,

Sudhir

[DanU1]

Hi Sudhir, 

Thanks for your reply and suggestion.  I dont have anything in that screen that references a bay 21 or 22. Here is a screenshot:

Here is the screenshot from the event log, which indicates the problem coming from Drive 21/22

I've updated my iLo firmware to 2.44 from 2.33, will reply if that changes anything. 

Thanks again, please let me know if these screenshots give you any other ideas. 

[sudhirsingh]

@DanU1

Is there is any external Disk Enclosure or Storage array connected to this server ?

Can you provide more information about the NVMe drive installed connected to this server ?

Regards,

Sudhir

[sudhirsingh]

@DanU1

Please do check for any drive errors in OS events , and also SSD drive wearout status in HPE SSA.

Thanks,

Sudhir

[DanU1]

Hi Sudhir, 

The NVMe drive is functioning as the boot device for VMware ESXi. There are 4 SSDs that contain storge of virtual machines, but none of them are in an array nor am I using SSA. I havent seen any OS errors regarding the drives. 

Is there any technical reference regarding this machine to identify where Drive 21 could be coming from?

Thanks for your continued help,

Dan

[sudhirsingh]

@DanU1

You can check front drive bay numbering, e.g. Dl380 Gen10 front 24 drive numbers as Bay1...bay24.
However, It does not seems to be physical drives since you have only 4 drives in the server even they are not managed with HPE SSA.

Please check Virtual Hardware health.
Login to VMware webclient and check Hardware health and storage sensor warning:

Monitor->storage sensor-> list of warnings
and
Select Host->Monitor Tab->Hardware Status->sensor
->Warning
->Event log

Try following two workaround:
1. Update controller firmware to latest.
2. Also check for drive firmware for any critical fixes 
3. Disable the SMX Provider (Login to esx cli then run below command)
   # esxi system wbem provider set -e 0 -n vmw_smx-provider