Operating System - Linux
1748158 Members
4129 Online
108758 Solutions
New Discussion юеВ

best pratices for creating a linux file server

 
SOLVED
Go to solution
iinfi1
Super Advisor

best pratices for creating a linux file server

i v been asked to create a linux file server at a client place.
they mite give me a SAN or a machine with 3-4GB RAM and 0.5-1TB disk space to create configure a file server.
please let me know whether what i intend to do is the industry best practice.
CASE 1:
if i am given machine with 0.5-1TB disk space this is what i intend to do.
1)create a partition for /boot 100M and LV for / 10G,swap 6G,/tmp 750M.
2)create a LVM with the remaining disk space and leave around 200G space free or as required.
3)format this LVM with ext3 and assign necessary permissions to the files.

CASE 2:
if i am given a SAN, the logical volume will exist in the network storage. (i have not seen SAN before though i v used openfiler before)
in this case i think there can be two ways.
1) Create a LVM in the SAN and create ISCSI targets of the partition and map it to the Linux file server with a ISCSI initiator.
2) Create a LVM in the SAN and partition it with ext3 and share the partition with NFS. mount this NFS on the file server.

In case two which is the method i should follow?
does what i intend to do make sense or is it too noobish
10 REPLIES 10
Ivan Ferreira
Honored Contributor
Solution

Re: best pratices for creating a linux file server

I think that 10G is large for /, you could create on of 5 GB and then create a separated /var to avoid filling the / with logs and spool information.

Your swap probably could be 1.5 x RAM.

I would start by creating regular size file systems and leave space unused in the root volume group. In that way, if you need more space on existing volumes, or new volumes, you can just extend or create new ones with the available space.

Respect NFS or iSCSI, i saw a report recently about oracle running on both. The performance is very similar. In that case, I would use NFS because is very simple.
Por que hacerlo dificil si es posible hacerlo facil? - Why do it the hard way, when you can do it the easy way?
Steven E. Protter
Exalted Contributor

Re: best pratices for creating a linux file server

Shalom,

Because I tend to use servers for a particularly long time, I use /boot at 200 MB to avoid an OS re-installation.

/boot can not be part of LVM.

No other issues with Case 1.

Case 2.
Do not configure to boot off the SAN. It adds a layer of complexity you don't need.

In general, I build the OS without SAN connection and then incorporate SAN storage in its own volume group(new).

Case 1 modified is my preference with noted provisions above.

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
Nuwan Alwis
Valued Contributor

Re: best pratices for creating a linux file server

Hi Iinfi,
You already got what you wanted so i may just summarize how i do it.

1)Yes, Minimum /boot is 100MB for Linux but at least give it around 200MB as a space to be use for a kernel upgrade. And /boot must not be a part of LVM.

2)Using LVM on other partitions is a good option because it gives the flexibility of expanding partitions.

3)make /var as a separate partition out of the / partition. The var partition tend to grow during the live run because of the growing log files. Keeping /var out from the / pratition will avoid your log files eating up the root space.

4)Allocate sufficient SWAP.

5)Keep your shared directory on a separate partition eg: /Shared for ease of manage eg:you can implement and manage quota easily.

6)Use ip-tables and selnux for additional security.

About your Case 2:
Dont worry if its a i-scssi target because it will be like another hdd added your system. you have to format the new disk and add to your file system.
I would have implement my /SharedDocs folder on the SAN because it gives me more flexibility because it doesn't depend on the host and also data protection levels(RAID) on the SANs.

Good Luck..!

iinfi1
Super Advisor

Re: best pratices for creating a linux file server

hi Ivan Steven and Nuwan,
pity i can give you only 10 points. these replies were worth 100. :)

thanks a lot for your replies.

i have another query, slightly offtopic!
assume i have two independent machines to be used as storage boxes.
Both have 3 HDDs with 100GB.
If i do a RAID5 on each of the boxes, i will get 200GB of each box.

if i map the drives of these two storage boxes to a linux machine, i will have two block devices.

/dev/sda1 200GB
/dev/sda2 200GB

if i need a single 400GB chunk then i can either do a software RAID 0 or create a LVM (pvcreate, vgcreate,lvcreate).
which one would be a better bet?

Steven E. Protter
Exalted Contributor

Re: best pratices for creating a linux file server

Shalom again,

See my comments inline.

assume i have two independent machines to be used as storage boxes.
Both have 3 HDDs with 100GB.

>>>
Assuming a minimal OS install, the maximum storage setup, even with Raid 0 on any one of these boxes is a little less than 300 GB
<<<

If i do a RAID5 on each of the boxes, i will get 200GB of each box.

>>>
With only three actual disks, the data will be striped, but its not really raid 5 as the loss of any one disk will disable much of the storage. Anyway, you get 200 GB per box, perhaps a bit more if you take every bit of space besides the /boot area and configure it that way.
>>>


if i map the drives of these two storage boxes to a linux machine, i will have two block devices.

/dev/sda1 200GB
/dev/sda2 200GB

if i need a single 400GB chunk then i can either do a software RAID 0 or create a LVM (pvcreate, vgcreate,lvcreate).
which one would be a better bet?
>>>
I'm interested how two distinct systems can provide the storage and have it show up as sda1 and sda2. Some kind of scsci connection by cable. You say SAN, but its these two distinct boxes. So I'm not sure how you are going to present this way. iscsi, Need to read up on the specifics there I guess.

Anyway, I would go with LVM because if you later add disks to these systems, you will be able to vgextend and then lvextend the single storage point. Always go LVM for maximum flexibility.

You are in this scenario going to lose a lot of storage and possibly have a file system that can not be mounted with the lost of a single disk.
>>>>

Good Luck,

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
iinfi1
Super Advisor

Re: best pratices for creating a linux file server

thank you steven.
i have not worked in production systems or live environments much.
i am just thinking too much may be.
thanks for your help.
Steven E. Protter
Exalted Contributor

Re: best pratices for creating a linux file server

You are doing the right thing. You are asking questions.
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
Ivan Ferreira
Honored Contributor

Re: best pratices for creating a linux file server

I will share with you some important information:


> On Fri, Aug 7, 2009 at 6:48 AM, Jia Rao wrote:
>> Hi all,
>>
>> I used to host the disk images of my xen VMs in a nfs server and am considering move to iscsi for performance purpose.
>> Here is the problem I encountered:
>>
>> For iscsi, there are two ways to export the virtual disk to the physical machine hosting the VMs.
>>
>> 1. Export the virtual disk (at the target side , it is either an img file or a lvm) as a physical device,
>> e.g sdc, then boot the VMs using "phy:/dev/sdc".
>>
>> 2. Export the partition containing the virtual disks (in this case, the virtual disks are img files) to
>> each physical machine as a physical device, and then on each physical machine I mount the new
>> device to the file system. In this way, the img files are accessible from each physical machine
>> (similar as nfs), and the VMs are booted using tapdisk "tap:aio/PATH_TO_IMGS_FILES".
>>
>> I prefer the second approach because I need tapdisk (each virtual disk is a process in host machines)
>> to control the IO priority among VMs.
>>
>> However, there is a problem when I share the LUN containing all the VM img files among multiple hosts.
>> It seems that any modifications to the LUN (by writing some data to folder that mounted LUN ) is not
>> immediate observable at other hosts sharing the LUN (In the case of nfs, the changes are immediate
>> synchronized at all the nfs clients). The changes are only observable when I umount the LUN and remount
>> it on other physical hosts.
>>
>> I searched the Internet, it seems that iscsi is not intended for sharing a single LUN between multiple hosts.
>> Is it true or ,I need some specific configuration of the target or initiator to make the changes immediately
>> synchronized at multiple initiator?
>>
>> Thanks in advance,
>> Jia
>>



2009/8/7 Fran├Г┬зois Delpierre :
> Hi,
>
> Quite funny. We are using Xen over iSCSI for years, and we are turning to
> Netapp NAS to migrate to NFS for the following reasons :
> - Easier setup, no matter how many xen hosts you have.
> - Avoid dealing with Cluster Filesystem. (We use OCFS2 quite successfully
> anyway.)
> - Some 3 years ago, we were using nbd + drbd + clvm2 + fenced + ... to have
> direct lvm in your VM works, but it's too complicated and too complex to
> maintain, especially if you have an issue or more than 2 dom0.
> - According to the bench I received, there is no difference in performances
> between NFS and iSCSI. There are some differences in some cases due to the
> different levels of cache.
>
> Regards,
>
> Fran├Г┬зois.
Por que hacerlo dificil si es posible hacerlo facil? - Why do it the hard way, when you can do it the easy way?
iinfi1
Super Advisor

Re: best pratices for creating a linux file server

thank you steven and ivan agian.
i am using openfiler for my testing as we dont have a storage box @ our premises.

i tried two diff. scenarios.
case 1:
create iscsi targets on the storage and map the targets to the linux file server using iscsi initiator and mount the filesystems using UUID of the partitions. i dont see a problem here. the disks act as a disk on the linux fileserver itself and i am able to set the user and group permissions on the folders.
case 2:
if i format the drives on the openfiler using ext3 and share the drives to the linux machine using NFS, then i cannot set any permissions on the nfs mounted drives. it would not as the drives are originally on another machine.
in case 2, my imagination is that i should add the openfiler itself to the domain and should not bring in another linux machine into the picture.

So all in all, if there is a storage like SAN available i should use create iSCSI targets on the storage and format it on the linux fileserver.