P400 - Linux poor copy performance

Dédé — Wed, 17 Dec 2008 08:49:15 GMT

On my RH EL 5.5 I get poor copy performance.

I use a Raid Hardware with an HP P400.

When I use "dd" for my file creation test, all looks fine but a simple 2Go copy file took 2 Mn !

I use 2 disks in RAID 0+1 for / and 4 disks in RAID 5 for /oracle. Find below all tests and config details, what is going wrong, is it my card or the config ?

Thanks four your help

P.S: Disk, controller and driver have been updated.

root@wmdata:~# time sh -c "dd if=/dev/zero of=/oracle/file bs=1024k count=4000;"
4194304000 bytes (4.2 GB) copied, 20.8946 seconds, 201 MB/s
real 0m20.919s
user 0m0.005s
sys 0m9.652s

root@wmdata:~# time sh -c "dd if=/dev/zero of=/root/file bs=1024k count=4000;
4194304000 bytes (4.2 GB) copied, 35.2935 seconds, 119 MB/s <------ Half the speed in RAID 1 ???
real 0m35.309s
user 0m0.004s
sys 0m9.996s

root@wmdata:~# time cp /root/file /root/file2
real 2m13.230s
user 0m0.645s
sys 0m14.726s

root@wmdata:~# time cp /oracle/file /oracle/file2
real 2m0.086s
user 0m0.622s
sys 0m13.945s

Smart Array P400 in Slot 1
Bus Interface: PCI
Slot: 1
Serial Number: P61620F9VV6HE8
Cache Serial Number: PA2270J9SWO1PS
RAID 6 (ADG) Status: Enabled
Controller Status: OK
Chassis Slot:
Hardware Revision: Rev D
Firmware Version: 5.22
Rebuild Priority: Medium
Expand Priority: Medium
Surface Scan Delay: 15 sec
Cache Board Present: True
Cache Status: OK
Accelerator Ratio: 50% Read / 50% Write
Drive Write Cache: Enabled
Total Cache Size: 512 MB
Battery Pack Count: 1
Battery Status: OK
SATA NCQ Supported: True

Array: A
Interface Type: SAS
Unused Space: 0 MB
Status: OK
MultiDomain Status: OK

Logical Drive: 1
Size: 68.3 GB
Fault Tolerance: RAID 1+0
Heads: 255
Sectors Per Track: 32
Cylinders: 17562
Stripe Size: 128 KB
Status: OK
MultiDomain Status: OK
Array Accelerator: Enabled
Unique Identifier: 600508B1001046395656364845380008
Disk Name: /dev/cciss/c0d0
Mount Points: /boot 101 MB
Logical Drive Label: A1693596P61620F9VV6HE816AB

physicaldrive 2I:1:1
Port: 2I
Box: 1
Bay: 1
Active Path:
Port: 2I Box: 1 Bay: 1
Status: OK
Drive Type: Data Drive
Interface Type: SAS
Size: 72 GB
Rotational Speed: 10000
Firmware Revision: HPDC
Serial Number: 3NP2KSVB000098358U8T
Model: HP DG072BB975
PHY Count: 2
PHY Transfer Rate: 3.0GBPS, Unknown
physicaldrive 2I:1:2
Port: 2I
Box: 1
Bay: 2
Active Path:
Port: 2I Box: 1 Bay: 2
Status: OK
Drive Type: Data Drive
Interface Type: SAS
Size: 72 GB
Rotational Speed: 10000
Firmware Revision: HPDC
Serial Number: 3NP2M5VE000098358VAP
Model: HP DG072BB975
PHY Count: 2
PHY Transfer Rate: 3.0GBPS, Unknown

Array: B
Interface Type: SAS
Unused Space: 0 MB
Status: OK
MultiDomain Status: OK

Logical Drive: 2
Size: 410.1 GB
Fault Tolerance: RAID 5
Heads: 255
Sectors Per Track: 32
Cylinders: 65535
Stripe Size: 64 KB
Status: OK
MultiDomain Status: OK
Array Accelerator: Enabled
Parity Initialization Status: Initialization Completed
Unique Identifier: 600508B100104639565636484538000C
Disk Name: /dev/cciss/c0d1
Mount Points: None
Logical Drive Label: A8279DD2P61620F9VV6HE865A1

physicaldrive 1I:1:5
Port: 1I
Box: 1
Bay: 5
Active Path:
Port: 1I Box: 1 Bay: 5
Status: OK
Drive Type: Data Drive
Interface Type: SAS
Size: 146 GB
Rotational Speed: 10000
Firmware Revision: HPDC
Serial Number: 3NM4527A00009821NQYY
Model: HP DG146BB976
PHY Count: 2
PHY Transfer Rate: 3.0GBPS, Unknown
physicaldrive 1I:1:6
Port: 1I
Box: 1
Bay: 6
Active Path:
Port: 1I Box: 1 Bay: 6
Status: OK
Drive Type: Data Drive
Interface Type: SAS
Size: 146 GB
Rotational Speed: 10000
Firmware Revision: HPDC
Serial Number: 3NM5375C000098370FVW
Model: HP DG146BB976
PHY Count: 2
PHY Transfer Rate: 3.0GBPS, Unknown
physicaldrive 1I:1:7
Port: 1I
Box: 1
Bay: 7
Active Path:
Port: 1I Box: 1 Bay: 7
Status: OK
Drive Type: Data Drive
Interface Type: SAS
Size: 146 GB
Rotational Speed: 10000
Firmware Revision: HPDC
Serial Number: 3NM3LR0Q00009816LNR9
Model: HP DG146BB976
PHY Count: 2
PHY Transfer Rate: 3.0GBPS, Unknown
physicaldrive 1I:1:8
Port: 1I
Box: 1
Bay: 8
Active Path:
Port: 1I Box: 1 Bay: 8
Status: OK
Drive Type: Data Drive
Interface Type: SAS
Size: 146 GB
Rotational Speed: 10000
Firmware Revision: HPDC
Serial Number: 3NM4523400009823WS1D
Model: HP DG146BB976
PHY Count: 2
PHY Transfer Rate: 3.0GBPS, Unknown

=>
root@wmdata:~# uname -a
Linux wmdata.infotecnet.com 2.6.18-53.el5PAE #1 SMP Wed Oct 10 16:48:18 EDT 2007
i686 i686 i386 GNU/Linux
root@wmdata:~#

root@wmdata:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
65G 22G 40G 35% /
/dev/mapper/VolGroup01-LogVol03
392G 15G 358G 4% /oracle
/dev/cciss/c0d0p1 99M 15M 80M 16% /boot
tmpfs 2.0G 0 2.0G 0% /dev/shm

Re: P400 - Linux poor copy performance

marsh_1 — Wed, 17 Dec 2008 13:03:47 GMT

hi,

i notice that the stripe sizes are different for the two arrays , the smaller stripe on the raid five will generally speaking allow it to split the data up into smaller pieces and spread it across 4 drives , whereas the 2 disk raid 0 is effectively writing 128kb chunks to 1 drive - if you were to make it a straight raid 0 and decrease stripe size and/or increase the number of disks in the 0 + 1 then you should see a difference in the performance.
restore will be quicker on the 0 + 1 as it currently stands with the smaller drives and larger stripe size due to better positioning.

fwiw

Re: P400 - Linux poor copy performance

TTr — Wed, 17 Dec 2008 13:09:09 GMT

There is nothing wrong with your setup. I too have found out that in many cases RAID5 by far outperforms RAID10 using similar dd tests. I have also read that the new algorithms for RAID5 are more efficient that older ones and take advantage of the larger number of spindles (more combined i/o) in RAID5. If you had a 4 disk RAID10 and a 4 disk RAID5 the results might be different (or not). That's why it is always a good idea to run these tests with all array systems (small and large) before choosing the RAID type and number of disks in it. Of course with internal RAID controllers you don't have much flexibility (limited power and and number of disks). The first two disks are used in RAID1 for the OS wich is not i/o demanding as compared with the rest of the disks which are used for database and application i/o.

Re: P400 - Linux poor copy performance

Dédé — Wed, 17 Dec 2008 16:28:21 GMT

Hello,

I agree with you that performance will be better if I change RAID configuration, but not 2x times faster????

I not understand why a 4Go file copy under windows take 30 sec and my brand new server under Red hat minimum 2 minutes.....

Thanks for your help

topic Re: P400 - Linux poor copy performance in Disk Enclosures

P400 - Linux poor copy performance

Re: P400 - Linux poor copy performance

Re: P400 - Linux poor copy performance

Re: P400 - Linux poor copy performance