P440ar & battery: TRIM not supported w/RAID? Battery-backed write cache in HBA mode?

[MLHJ]

Hi,

We recently bought two DL380 Gen9's (768345-425) which include the HP Smart Array P440ar Controller and HP Smart Storage Battery. Yesterday I was adding SSD's to the machines and noticed in Linux that the controller does not support TRIM (in HW RAID mode, I'm using RAID1).

Three questions:
1) Am I correct that in RAID mode this controller does not support TRIM/DISCARD?

2) If I instead use HBA mode instead of RAID (I will then use software raid, Linux MD) can I use TRIM?

3) In HBA mode, does the battery-backed 2Gb write cache work? Or is this part completely bypassed in HBA mode?

Thanks in advance.

[parnassus]

Interesting questions!

I think I can give you a straight answer to question 2: if P440ar's HBA mode "offers" disks control directly to overlying Linux OS (with recent Kernels, since 3.8.x) the mdraid (at least for RAID 1) implementation does support TRIM/Discard (With TRIM implemented through "discard" option in Linux's fstab or via the "fstrim" userspace command manually executed or driven by a dedicated cron job).

This happens on normal workstations with SATA in AHCI mode...if P440ar's HBA feature is transparent enough it should work that way too.

Other two questions are really interesting...

[Torsten.]

Hopefully this helps:

https://www.hpe.com/h20195/v2/GetPDF.aspx/4AA4-8144ENW.pdf

[MLHJ]

@parnassus: Indeed, re question #2 that's what I'd expect as well. I guess I'll know soon enough since I will be doing a clean install. Still don't know about the other questions, though. Question #3 is rather hard to test I think.

@Torsten.: Yes, I've read this PDF as well. It has some interesting information about "Smart path". But.. that's also the only subject it covers. Since it was the only PDF that talked about SSD's w/RAID I was hoping it would contain more information as well. But it doesn't mention TRIM/DISCARD at all - something quite crucial for SSD's. It's unfortunately not a generic "SSDs with HP ...." article.. it doesn't answer any one of the three questions :)

[parnassus]

The document @Torsten. linked doesn't contain the HP SmartSSD Wear Gauge concept which, conversely, the HPE Solid State Drives (SSDs) datasheet does.

It's the only reference of the word "wearing"...related to SSDs.

The HP SmartSSD Wear Gauge, AFAIK, is a Smart Array feature that reports, through a summary, a brief overview of the following counters:

• Total Solid State Drives with Wearout Status
• Total Smart Array Solid State Drives
• Total Non Smart Array Solid State Drives
• Total Solid State SAS Drives
• Total Solid State NVMe Drives
• Total Solid State Drives

and, when running a report, the user can either view a graphic representation of the report with SSD usage and estimated lifetime information...or can generate a report without a graphical display, with the option of saving the report (Reference here).

The HP SmartSSD Wear Gauge is basically a SSD wearing monitoring and reporting tool embedded into Smart Array Controllers.

So the question about TRIM/Discard is still on the air...and it is quite interesting...I've read that (some) HP SSDs are rated for (an endurance of) up to 2.5 random writes of the entire SSD's capacity per day (using the acronym DPWP or Drive Writes Per Day)...continuously for 5 years [*]...so, maybe, they didn't implemented TRIM/Discard mechanism in Hardware...or did they (without advertising it so loudly[**])? It's impossible they ddin't.

[*] Reference: "Breakthrough performance for application workload demands" HP Solution Brief (June 2015) about HP 6G SATA Value Endurance (VE) SSDs <-- the word "wearing" or "trim" is absent...

[**] On the datasheet linked above please note the sentence:

"HPE firmware optimizes our qualified SSD performance, wear leveling, and over-provisioning."

...so it's likely they only don't call it explicitly "TRIM Support" (at OS's filesystem level) because they implemented it at SSD's Firmware level.

[MLHJ]

Strange. I've enabled HBA mode but TRIM/DISCARD isn't working under Linux (Ubuntu 16 LTS)

With "hdparm" I can see the two drives sda & sdb support TRIM (which is correct), but a subsequent "blkdiscard" or "fstrim" fails saying "Operation not supported".

This can't be true. It must be possible to get TRIM support, right?

@parnassus: that's useful to know too, thanks!

[parnassus]

You said you installed Ubuntu 16.04 LTS...what's about disk partitoning (and filesystems) used on your SSDs? did you followed the guided partitioning with the LVM option (and, maybe, mdraid on top of LVM) or did you selected guided partitioning with direct creation of partitions on disk without LVM (eventually with mdraid)?

What the lsblk -D command reports on your sda (or sdb)?

[MLHJ]

No software raid, no LVM, so no layers in-between that may lack TRIM support.

hdparm -I shows:
           *    Data Set Management TRIM supported (limit 8 blocks)

..which means the drive supports TRIM.

While cat /sys/block/sdb/queue/discard_granularity shows 0.

..which means I cannot use TRIM.

root@ubuntu:~# blkdiscard /dev/sdb
blkdiscard: /dev/sdb: BLKDISCARD ioctl failed: Operation not supported

I guess I'll have to physically take the RAID controller out of the loop by rerouting the x4 SATA cables to the onboard controller.

I'm really surprised. I would never have thought with direct access it would not support TRIM commands.

[parnassus]

Probably the device access provided by the Smart Array P440ar isn't so "direct" as you expect even if the HBA setting was involved.

The lsblk -D command should report something like (here, as example, the sda is a Samsung 830 SSD - so non HPE - partitioned using EXT4 filesystems on separated /, /boot and /home mountpoints plus the classic Swap partition):

lsblk -D
NAME   DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
sda           0      512B       2G         0
├─sda1        0      512B       2G         0
├─sda2        0      512B       2G         0
├─sda3        0      512B       2G         0
└─sda4        0      512B       2G         0
sr0           0        0B       0B         0

Note how the DISC-GRAN (Discard Granularity) parameter is non Zero (512) as it is reported - for the same device  - through the discard_granularity setting:

cat /sys/block/sda/queue/discard_granularity
512

I'm not testing this on a HPE ProLiant...that's just a Linux Workstation I have.

[MLHJ]

Yes, exactly, it seems not as direct as claimed. These are Samsung 850's by the way (both on sda and sdb)

root@ubuntu:~# lsblk -D
NAME   DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
sda           0        0B       0B         0
├─sda1        0        0B       0B         0
├─sda2        0        0B       0B         0
└─sda3        0        0B       0B         0
sdb           0        0B       0B         0
sr0           0        0B       0B         0