Operating System - Linux
1748227 Members
4458 Online
108759 Solutions
New Discussion

Re: SmartArray actual storage geometry - Partition alignment issue

 
SOLVED
Go to solution
Pascal BERTON
Advisor

SmartArray actual storage geometry - Partition alignment issue

Guys,

 

I'm currently creating a logical drive on a P812 SmartArray seated inside a DL180g6, that has to bee partitionned in 3 parts. This RAID5 LD is made up of six 2.5'' 1TB SATA drives, for a total of roughly 5TB (4.6TB according to hpcucli).

I'm wondering about the best alignment for the 3 partitions I have to place there, so I switched on my calculator and started some cylinder computations. But searching for the related input parameters, I'm facing something a bit odd :

 

hpaculi pretends my LD is 32 sectors per track :

=> ctrl slot=2 ld 3 show

Smart Array P812 in Slot 2

   array C

      Logical Drive: 3
         Size: 4.5 TB
         Fault Tolerance: RAID 5
         Heads: 255
         Sectors Per Track: 32
         Cylinders: 65535
         Strip Size: 256 KB
         Status: OK
         MultiDomain Status: OK
         Array Accelerator: Enabled
         Parity Initialization Status: Initialization Completed
         Unique Identifier: 600508B1001C4D2006D4A991E8DFA0D9
         Disk Name: /dev/sdc
         Mount Points: None
         Logical Drive Label: A0AE9AD1PAGXQ0BRH3E06U81B4

When fdisk states it's 63 sectors per track (Which I believe is true for the physical drives themselves, but the fact they are behind the SmartArray might have some importance...):

# fdisk -l /dev/sdc

WARNING: GPT (GUID Partition Table) detected on '/dev/sdc'! The util fdisk doesn't support GPT. Use GNU Parted.

Disk /dev/sdc: 5000.9 GB, 5000855773184 bytes
255 heads, 63 sectors/track, 607986 cylinders, total 9767296432 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

So who's right, who's wrong ? Further, I made my computations using both values, and when I tried to create my very first partition, leaving the 1st cylinder free, parted complains that alignment is not the best possible when it is supposed to be aligned on a cylinder boundary.

 

1st try, 63 sectors per track as per fdisk's advice :

(parted) mkpart primary 16065s 2147488874s
Warning: The resulting partition is not properly aligned for best performance.

 

2nd try, 32 sectors per track as per hpacucli's recommendation:

(parted) mkpart primary 8160s 2147483519s
Warning: The resulting partition is not properly aligned for best performance.

 

Has anyone already experimented this kind of stuff with SmartArray cars ? I searched for HP white papers speaking about that but nothing really valuable...

 

Any help appreciated.

 

Regards,

 

Pascal.

 

4 REPLIES 4
Matti_Kurkela
Honored Contributor

Re: SmartArray actual storage geometry - Partition alignment issue

With modern disks, the C/H/S geometry is essentially fictional. The real geometry is more complex than that: for example, the tracks near the outer rim of the disk typically contain more sectors than the inner tracks near the spindle.

 

If your Linux distribution includes the "lsblk" command, then "lsblk -t" might tell you useful information about the alignment requirements of the underlying hardware. (On SSDs, "lsblk -D" might be useful too.)

 

I run "lsblk -t" on a BL460c Gen8 blade; apparently its controller really likes to do things in chunks of 256kiB:

# lsblk -t
NAME                     ALIGNMENT MIN-IO OPT-IO PHY-SEC LOG-SEC ROTA SCHED RQ-SIZE
sda                              0 262144 262144     512     512    1 cfq       128
├─sda1                           0 262144 262144     512     512    1 cfq       128
└─sda2                           0 262144 262144     512     512    1 cfq       128

 

 

 

MK
Pascal BERTON
Advisor

Re: SmartArray actual storage geometry - Partition alignment issue

Hi Matti!

 

Thanks for your reply. Interesting. I had already read that block size had changed up to 4k since 2010 I bielieve (Seems like SATA keep reporting 512B lock size though... At least that's what I see in /sys/block...), but I didn't know about this variable sector/track feature.

My SmartArray device shows the following:

 # lsblk -t /dev/sdc
NAME   ALIGNMENT MIN-IO OPT-IO PHY-SEC LOG-SEC ROTA SCHED
sdc            0    512      0     512     512    1 deadline
`-sdc1         0    512      0     512     512    1 deadline

#

 

On another MD RAID based server, I get :

# lsblk -t /dev/sda
NAME        ALIGNMENT MIN-IO  OPT-IO PHY-SEC LOG-SEC ROTA SCHED
sda                 0    512       0     512     512    1 deadline
|-sda1              0    512       0     512     512    1 deadline
|-sda2              0    512       0     512     512    1 deadline
| `-md0             0    512       0     512     512    1
|-sda3              0    512       0     512     512    1 deadline
| `-md1             0 262144 1048576     512     512    1
|-sda4              0    512       0     512     512    1 deadline
|-sda5              0    512       0     512     512    1 deadline
| `-md3             0 524288 1572864     512     512    1
|   `-md3p1         0 524288 1572864     512     512    1
|-sda6              0    512       0     512     512    1 deadline
| `-md4             0 524288 1572864     512     512    1
|-sda7              0    512       0     512     512    1 deadline
| `-md5             0 524288 1572864     512     512    1
`-sda8              0    512       0     512     512    1 deadline
  `-md6             0 524288 1572864     512     512    1

 

In this case, it is a 4 disks RAID5 with 512kB stripes, so the OPT-IO... Makes sense... In the SmartArray case, it seems it doesn't really care of what type of IO will be sumbitted.

 

Then, what's your advice in terms of alignment ? Does it really matter in the end ?

If I understand what you say, a given alignment will make sense for certain areas of the disks and not for others right ?

May be only a 4k boundary, just to avoid a couple of sequential 512B write causes a read of a whole 4k to then update just this part ?

May be aligning to the RAID stripe size as in the MD RAID case ?

 

Best regards,

 

Pascal.

 

 

Matti_Kurkela
Honored Contributor
Solution

Re: SmartArray actual storage geometry - Partition alignment issue

Trying to use the C/H/S geometry to achieve a particular alignment is tricky and may be futile in some cases. The best you can do at the partitioning level is usually treating the disk as a single long stream of LBA blocks and setting the partition start/end points to exact multiples of whatever is optimal for your disk/SSD/SAN model, RAID controller or stripe size. If you aren't sure, multiples of 4kiB, 16 kiB, 64 kiB or 128 kiB are usually good guesses.

 

With Linux fdisk, that means switching off the DOS compatibility mode, if your fdisk enables it by default, and choosing "sectors" as partition size units. ("fdisk -c=nondos -u=sectors"; newest Linux fdisk versions may do that by default.)

 

This way you can set partition start/end points exactly where your alignment requirements dictate: fdisk won't place artificial obstacles to you by modifying your values to conform to some DOS-compatible (fictional) C/H/S geometry. It will populate the C/H/S fields of the partition table with appropriate LBA placeholder values, and only the LBA of the partition start and the total number of sectors will be meaningful.

 

If your disk is larger than 2 TiB in size, I think the current recommendation is to use a GPT partition table instead of the traditional PC-style partition table if possible. The traditional partition table has a hard design limit at about 2 TiB: at that point, the LBA values become too large to be stored in the 32-bit fields in the traditional partition table.

 

 

The GPT has only LBA values in it, so there is not even any way to specify a particular C/H/S geometry any more. On the other hand, it has 64-bit fields for the LBA values, which should be good enough for a while.

 

If your hardware, OS and partitioning tool are new enough, there are standard ways for the hardware to communicate its alignment requirements to the software. The "lsblk -t" command is part of that.

 

If your hardware has a particularly "coarse" alignment requirement (say, 128kiB or more), it might be hard/wasteful to align the first partition on the system disk optimally. But the first partition is often /boot, which experiences major read operations essentially only at boot time and writes only when kernel or bootloader updates are installed, so you might not care about it being non-optimally aligned.

MK
Pascal BERTON
Advisor

Re: SmartArray actual storage geometry - Partition alignment issue

Hi, Matti !

 

Thanks for these precisions. I have played around with lsblk command, along with a couple of others that the man pages led me to, interesting, and finally adopted the Microsoft alignment convention of 1MB. Since my RAID stripes are 256kB, at least it's aligned on that, and then I believe there's not much more I can do...

 

Thanks for your help anyway, that worthed it ! :)

 

Best regards,

 

Pascal.