iSCSI filesystem creation on Linux - Best Practices - Large Volumes - Many Files? - P4500 G2

[Svavaroe]

We recently purchased HP Lefthand 4500 G2 24TB box and also a DL380 G7 server.

I'm running Centos 6.2 (RHEL 6.2).

The Lefthand box consist of 12 2TB HD's configured in RAID6. That gives me about 14.7TB usable space.

Scenario : iSCSI SAN for Fileserver at Advertising Agency.

Are there any recommendations / best practices on filesystem creation on iSCSI LUN's ? Regarding EXT4 or XFS? Write barriers, mkfs options on creating the filesystem ? Like, inode64, noatime etc....

XFS is preferred tho, as we will be hosting a large volume with >=1millions of files.

My Plan is to create 6-8TB volume at the start.

As for now, the server and or Lefthand box is not connected to UPS, but will be real soon.

I have had really good experience with XFS over the last 10 years with large volumes but not so, if not at all any experience with EXT3/4 on large volumes.

Any recommendations and or help would be much appreciated.

Thanks allot.

Best regards,

Svavar Reykjavik - Iceland

[David_Tocker]

As you are running a single node with Centos you are not likely to run into many of the issues that other people run into with more elaborate scenarios. The P4000's have heaps of power behind them as long as the link is optimised - make sure you run flow control on all devices in the chain (nic, switch, p4000) Jumbo frames do not make a big difference - I normally turn it off as it does not appear to make any difference - the only time i can measure a difference is with bad switches that can not handle jumbo+flow control at the same time (a lot of them) You need to keep in mind that iSCSI is very efficient - the 1gb link will cap your throughput, but the access times will be better than you may expect. We normally enable ALB, with a single node however it may be worth investigating LACP as long as you have enough NICs in your host - at the end of the day however it is probably not that big of a issue, dont stress about it too much. If your switch is not overly clever it would be best to avoid it. You are not going to run into quorum issues ever with a single node.

I would forget the rest of the File system issues - pick the FS you are comfortable with and run with it.

Just treat it like a big high IO, medium throughput hard-drive (somewhat like an SSD?) - there is no normally no advantage to creating multiple LUNs (for logs etc) in this situation either, as it is all going down the same pipe.

Good luck - if it is too slow, remember that you can go to 10gbe - and with a single host all you will need is a nic, and the 10gbe option for the p4000 and a patch lead so it works out pretty cheap for a good increase in throughtput (but not a huge increase in IOPS)

[Svavaroe]

Thanks allot for the reply. :)

[oikjn]

contact support. mission critical data on a single VSA node?  Thats scarry by itself.  Would you put mission critical data on a single HDD?  if this was NR10 then this is less likely to be an issue.