stripe size?

[Chris Campbell_1]

I have a mail server that I am setting up right now. I am using the qmail mta instead of sendmail. If you are familiar with qmail, you know that it has the ability to store messages in the maildir format, which results in lots of small files. I am using this format and it looks like the typical e-mail is between 1K and 5K. I am going to put the directory that contains all the users' maildirs on an external disk storage cabinet, an HP-6000 if I recall correctly. The 6000 has 5 disks in it. I am wondering how to calculate the optimal settings. Should I even worry about disk striping? Will it make any difference with files this small? I'm not sure what the acceptable stripe sizes are.

[Carlos Fernandez Riera]

Are you planning mirror lvols for this?

Mirror lvols dont support stripes. You might use extend based strip mirror.

If you are not planning mirrored lvols you can use strip, but the smallest strip size is 4K so i think you will not take adventage ussing files of 1 to 5K size, because the read will be done on the same disk.

[Stefan Farrelly]

Yes, for a mail server you definitely should bother to stripe it (use lvcreate -D). Set the stripe size to 4k if you reall have such small average email sizes, but if you are going to have heavy volume up it to 64k or more. All HP's emailed servers are striped - its absolutely necessary as mail servers are almost always completely i/o bound.

[Tim Malnati · Accepted solution]

A lot depends on how busy you believe your mail server will be. Although creating stripes will not usually benefit individual mail messages, it could significantly improve the i/o performance of the mail server as a whole. The striping will help distribute this i/o load, without it the load will tend to follow the population of the individual drives (the first drive fills up, then the next, etc). Extent striping is probably your best bet though, where it allows full mirroring of your volumes. In a mail server, data is usually changing too quickly to depend on backups alone.

What really concerns me is your use of 6000 cabinets. These have not been manufactured in many years now which makes me wonder how much life really remains on the drives inside. Support is still available (and it's expensive), but any problems you encounter will mean some significant down time (it takes a while to replace a drive and means fully removing the cabinet to do so). 6000's came in both SE and FWD models and most of the drives themselves are the old 5400 rpm spindles. If throughput is an issue, 6000's are not the best choice. I would seriously look at replacing this with some newer technology. New drives will certainly be more reliable, have much better throughput, are usually hot-swappable, and are usually covered under a five year warrenty. In other words, the support costs for these old drives (out of warrenty by now) will usually pay for the new equipment in short order.

[Patrick Wallek]

Stefans post is a bit misleading. If you use the -D option to lvcreate, then you can NOT use the -I or -i options. The -D option tells lvcreate to do 'extent based' striping, thus you can't set the block size to stripe with on the command line. The way to set the "stripe size" with the -D option would be to modify the PE Size when you create the VG.

With the -D option, the logical volume will be striped across all disks in a the physical volume group (PVG) of that VG. When you use -D you must also use the '-s g' switch as well, whether you are mirroring or not.

When an lvcreate is done with the -D option, the LVOL will be created by creating the first extent on the first disk in the PVG, the second extent on the second disk in the PVG, the third extent on the third disk, etc. When it gets to the last disk in the PVG it will start back at the first disk again and keep looping through all the disks until it finishes creating the LVOL.

The advantage to using this extent based striping is that you can actually mirror and stripe in this fashion. If you use normal disk striping, HP does not allow you to do mirroring.

In short, I would keep the PE Size at 4K, use extent based striping and go from there.


I also agree with one of the previous posters as well about the HP6000 cabinets. I've got a BUNCH (40 or so) sitting around not being used full of 2GB FWD drives. No one wants them. I highly recommend going away from those. They are indeed a pain in the A** to work on.

[Chris Campbell_1]

Well let me explain a few things. First of all this is a startup company whose budget is VERY tight right now. I was able to spend only $1500 on the equipment. I got a used HP E45 server (very old too, yes I know) and disk array for about $800. I am considering replacing the disks with new 9GB FWD ones when the company gets more money, which shouldn't be too long. I think this server will be able to handle the load pretty well for the forseeable future. Right now it's going to handle about 15 users. That number may grow to 50 in the next year. So my main concern is failure of the disks. To take care of that I'll mirror and do frequent backups. I just checked and the disks are Seagate Barracuda 4 7200RPM FWD drives. So the performance may be a little better than we were thinking.

As far as e-mail size goes, that's just my estimate. If someone has a better estimate of average e-mail size I'm certainly interested to hear it. Now that I think about it, striping could likely do some good since we are using IMAP and may have multiple users access the machine at the same time. Either way, I think with striping and 7200 RPM Wide-SCSI drives the machine won't break a sweat at least in the next year or so.

[Chris Campbell_1]

I forgot to ask what would you guys recommend as a replacement for a FWD 6000 storage cabinet? I'll read over your responses more over some pizza. Thanks guys!!!

[Patrick Wallek]

Replace recommendations for the HP 6000s:

Hmmmmm.......My first thought is an HP Jamaica with at least 9GB drives in it. A Jamaice has 8 drive bays, dual SCSI buses, dual power supplies and dual fans. I have them with 9 GB drives and 18 GB drives. A Jamaica will set you back a few bucks though.

My next thought would be external drives. Since it is an e-mail server I would be tempted to get more smaller drives so you can stripe across more drives and hopefully increase your throughput. I would definitely recommend that you make sure you get FWD (Fast-Wide Differential) drives. I would also recommend that you have multiple SCSI buses so you don't have all your traffic traveling over one bus.

Mirroring is still a must, as are good backups.

Good luck.

[Bruce Regittko_1]

Hey Patrick,

I could probably take a few of those 6000's off your hands for a reasonable amount as I have a need for external drives for HP's LVM course. Size/support/appearence are not an issue. Email me at bruce.regittko@stratech.com if you are interested.

[Tim Malnati]

You have not mentioned some more particulars that are somewhat important here. What are the size of the drives in your 6000 cabinet and what do you have for internal drives in your E45? Also, is your 6000 cabinet the rackmount model (5 internal bays) or the deskside model (7 bays)?

If all the drives are 2gb you are going to run into problems trying to make everything fit and still have redundancy. If you have 4gb spindles instead, you can reasonably make it work as is.

First, you reasonably need at least 4gb for the operating system volume vg00. The E45 has room for two SE scsi drives. If you only have a 2gb drive in there, you will either need to add another drive or swap it out with a 4gb drive. You want to mirror this though where it is every bit as important as the email data area. Your best bet is to use some storage in the 6000 for the vg00 mirror. Some purists may jump on this thread and exclaim that you don't want to mirror SE and FWD technologies. Fact of the matter is that it works just fine in practice and is better than attempting to mirror two internal 4gb drives on the same SE bus.

If your 6000 has 4gb spindles, you use one of them for the vg00 mirror and have the other four spindles for your mail data. I suggest you set this up with a two-way stripe and and mirror it with the remaining two drives. 8gb of mail storage should be plenty.

If you only have 2gb drives in the 6000 you have problems. You would need at least two of the drives to mirror vg00, leaving 3 spindles for mail data. The best you could hope for is a 3gb of mirrored storage. This should be plenty for 15 users, but would require extensive administration by the 'email police' when you start to grow to your 50 user projection. If your 6000 has 2gb spindles, my best suggestion would be to purchase another 6000 cabinet with 2gb FWD drives and a separate controller for it if your E45 has the bus room. You could probably get away with swapping out a 2gb drive with a 4gb for your vg00 mirror, but I would guess that the entire unit on the used market is about the same price.

By now you've probably noticed that I mention mirroring a lot more than striping. These are old drives that you have here! In my mind mirroring redundancy is far more important in your situation.

You've probably also noticed that I have not mentioned the idea of replacing these drives with something else either. The bottom line is that your budget doesn't allow it for now. Looking at things, you would have been far better off with a D class box instead.