Why have multiple partitions / file systems for root ( vg00)?

[Geoff Wild]

We are in a debate at work over this (okay, more like an arguement). The "Sun" guys think you should have a "flat" file system of just /. No need to break it up into smaller partitions.

What do you guys think?

Please provide reasons for splitting it up.

For instance, "On a given device, Determination of which partition is causing high IO can be done in Glance, most easily on the IO by Logical Volume screen." This won't work with a "flat" file system.

Also note: disk space isn't an issue - typically most systems now come with 18GB or 36GB drives.

[Pete Randall]

It's really a question of flexibility. With a flat / file system, when it fills up, you're out of luck. With multiple lvols, you're better able to deal with file system growth.

Pete

[James R. Ferguson]

Hi Geoff:

The isolation of the operating system files into multiple filesystems is a protective measure.

The idea is to let something like the root ("/") filesystem fill to capacity (because you mis-spelled '/dev/rmt/om' [note the letter, not the number]), while retaining space in other mountpoints like '/tmp' and '/var' for the operating system to continue to work.

Regards!

...JRF...

[Stefan Farrelly]

On systems with not much diskspace we do create them with only 3 lvols (/, /stand and swap) - it then merges all free space into one lvol (/) which is easier to administer then.

Its a question of whats easier to administer.

If you have enough diskspace then with /usr, /opt, /home, /tmp, /var etc. its easier to administer because only certain types of files go into each one (or not - as in the case of opt or usr unless someone installs something new). So as space grows you can easily find out who/what. Eg. if you have a /home then if some moron user fills it up it wont screw (or possibly crash) the system (as it could do if only a / lvol), same for /tmp and /var.

So its not only easier to administer and to find out who is using up the diskspace but much safer too to have multiple lvols.

[Dirk Wiedemann]

Hello,

I recommend to split the filesystems to various lvols. Even though it's sometimes inconvenient if a filesystem fills out while in other filesystems is enough free space. But afaik if you create a flat filesystem of just / and then it fills up your system is completely halted.
Have a look at this thread:
http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0x390b6049dbb6d611abdb0090277a778c,00.html

regards
Dirk

[Leif Halvarsson_2]

Hi
Hi
I have never liked the idea to divide the OS installation in several small logical volumes (/ /stand /var /opt etc.)as default when installing HP-UX. I usually create one small hfs volume for /stand (as this is necessary) and one large (4GB) for / (rest of the OS). Then I create logical volumes for data, users, applications etc. as needed.

[A. Clay Stephenson]

One of the most compelling reasons for spiting the filesystems is that it is much easier to administer and also you are able to spot potential problems more readily. For example, on a well-behaved system, / should be all but static in size; if all the filesystems are combined then you are not going to able to see this very easily. My reccomendation is to split the filesystems and leave a large fraction of it unallocated so that you can increase any needed LVOL's on an as-needed basis. I tend to create an oversized / and /stand partition - generally about twice the default sizes and a small (256MB or so) primary swap. The remaining areas can be grown as needed.

[Geoff Wild]

Just want to add that you must validate your reasons.

IE: "Splitting it up is easier to administrate" - how? with say a 18GB / - why would it be easier to administrate if you split it up? You would never have to "grow" /tmp - because it would take a lot to fill 18 GB - whereas if /tmp was a 256MB partition - you may have to grow it every now and then - therefore, and increase in the administrative cost to support that box.

If you prefer flat - don't just say "I have never liked the idea to divide the OS installation in several small logical volumes " - tell me why.

[S.K. Chan]

I've seen both kinds of setup. Here our workstations are all setup in just 3 partitions in the root FS, (/,/stand and swap) whereas for the servers the root is setup with separate partitions (/usr,/opt,/tmp and /var). Like other have mentioned "ease of administration and trouble shooting" is probably the main reason for wanting to split them. There is only one catch though, you got to have ONlineJFS to be able to increase these FS size on-the-fly, otherwise it's going to be impossible to umount say /var in a multi-user mode. Having split partitions a sudden file system full event can be quickly isolated and deal with, for example "/tmp" full. If you have a single / partition, this might not be apparent to you unless you have some monitoring process that tells you / has suddenly increase in size by so much. We have decided not to do split partitions on the user workstations because those do not have OnlineJFS.

[Leif Halvarsson_2]

Hi
One simple reason is that the filesystem becoms unnecessary complex. Another is you can use your disk space better. I have often had problems with one of this small filesystems become full (for example when installing patches or OS extensions) but there is plenty of space on other filesystems.

[Chris Wilshaw]

Having recently recovered some corrupt volumes on a system, my opinion is that the better option is to use the multiple filesystem layout.

If you have a single filesystem, and it gets corrupted due to a system failure, you're straight into a pretty serious recovery situation. With multiple filesystems, you've got a fair chance that you'll still be able to access most or all of the commands on the system to carry out repair work on those areas that are corrupt.

You could of course argue that it was possible for all the filesystems to become corrupt at the same time, but that's not too likely.

[Martin Johnson]

Multiple lvols do add a little more maintenance for the SA, but it does reduce system downtime when problems occur.

If you have only one file system and it becomes corrupted, you will have recover the entire system. For me, that's about 4 to 6 hours of downtime. With multiple file systems, a corruption has a chance of occurring in a non critical file system. In this case, there is no system downtime. An application may be unavailable during recovery, but the recovery should take a lot less than 4 hours.

Run away applications that fill up a file system can crash the system if you have only one file system. / and /var are 2 areas that cause problems when they fill up. The last time I had /var fill up, I had to reboot the system to clear things up. I make it a point to give each application there own file system. That way, if they fill things up, they only hurt themselves. (I have a file system checker monitor in ITO to give early warning that an application is about to fill its file system).

To summarize, system downtime costs money and aggravates customers. Using multiple file systems reduces potential system downtime.

HTH
Marty

[Anil C. Sedha]

Hi Geoff,

There are issues where you can keep going over this question. There are very good advantages of splitting into logical volumes the root partition.

1) It is easy to keep and identify every subdirectory that you maintain on your system. e.g. when you partition your /stand, you know that only system specific files are in there.

2) /var will contain only logs and other normal configuration data. It is easy to identify how much capacity it is using. Sometimes, your system get's too slow and you can identify what's causing issues. There might be a open file taking a lot of disk space. You can identify easily which partition is using what space and is increasing drastically or not. Even though you have a big disk size, you need to analyze issues and here it helps.

3) Your /application directories can be used seperately to identify through bdf how many seperate applications are there. By this i mean independent applications.

4) We can get to know if the system crash directory has enough space or not by looking at bdf instead of finding disk space for each directory. like /var/adm/crash.

5) You can mount or unmount filesystems, depending on how you would like to mount them. It is easier for tracking and management of all such filesystems if they are configured as seperate logical volumes instead of a flat / filesystem.

6) You can use maximum I/O utilization by having some free disk space which will not be used and i/o use will be less spanning through the whole disks. The free disk space can be utilized as and when necessary. This would help specifically when multiple disks are in question.

These are just a few issues..

Hope this helps to prove your point

Regards,
Anil

[Christopher Caldwell]

We use multiple partitions (lvols) for

1) system protection - filesystems like /tmp and /var are designed to grow without bound due to logging. If the filesystems are in the root partition unexpected growth will cause unexpected downtime (minimally an ungraceful reboot (TOC) to clear space).

2) different classes of service - for instance, normally everything on our root drive is _required_ to be mirrored (generally /, /var, /opt, /usr, etc.). On our main sw depot server, we separate /var/spool/sw, and we don't mirror it. It's large and static - there's not an issue with loss of data (it's taped and can be easily rebuilt). We do the same thing with some archive log partitions.

3) Performance - if you're running a big mail system for instance, I'd advise you to put /var/mail and /var/spool/mqueue, all on different spindles (different lvols, striped).

4) Expansion - let's say you've got a 9G internal drive (ok, so it's old) - you're on today's 11.x which wants nearly that size drive. I've had systems with /opt and /var on one disk, and /usr, / and /tmp on the boot drive - all set up due to expansion after the system was built.

Use of these techniques are system and situation specific. YMMV.

[Ted Ellis_2]

and don't forget using multiple file systems can be useful in protecting your user or development areas.... some users may utilize (through errors or poor judgement) a lot of space... if you isolate them to a separate /home, or even separate /home/user... then you can protect other developers or users from unwanted interruption because some cowboy wanted to play with something :-)

[Martin Johnson]

By isolating the applications from the OS, gives you the option to do a cold install rather than an upgrade, which is faster and has a lot less problems.

HTH
Marty

[Keith Clark · Accepted solution]

Hi Geoff,

We use both HP-UX and Solaris. The reason the "Sun Guys" like one big partition is for ease of upgrades and the lack of tools to expand root slices. Unlike HP, Sun has not integrated volume management and VxFS into their kernel, so you cannot easily increase the size of the root slices. The volume management is supposedly integrated into Solaris 9, but /, /usr, /var, /opt, etc must be UFS and cannot be increased online. It also has traditionaly made upgrades much easier. Back in the days of 2 & 4 G drives moving to a new release of Solaris was much easier with one big slice, since you did not have to worry about runnng out of space on indidvidual slices.

With OnlineJFS, the only filesystem you cannot increase online is /stand (HFS), which makes it practical to leave everything in seperate lvols. I think everyone else has indicated all of the reasons why they (and I) prefer the multiple lvols, just make sure you invest in OnlineJFS if you need to maximize uptime.

I think if the "Sun Guys" had the flexibility of using a Volume Manager and OnlineJFS they too would choose multiple lvols.

[Shannon Petry]

Well, seein as how Im a Sun guy, I'll have to say that most real Sun guys dont believe in a flat file system either.
The default layout for Sun OS is to have swap, data, and OS segregated since Solaris 2.5.1. In Solaris 7,8,9 they are defaulting to a more LVM structure minus logging for UFS.

Three reasons for splitting up disks (either LVM or partitioning in solaris).

First is security.
As someone else mentioned, if a user types in tar cvf /dev/tape, then the OS disk can fill up causing the system to fail. This same thing occurs with core files, temp files, etc... Watch how fast users hound you when the system will not let them log in and hangs for 10 minutes before disconnecting them because the OS disk is full and they can not see it.

Second is Control/?Mmanagement
First, I like to control how much goes to what. I.E. /tmp is generally 512MB on a CAD/FEA server, but only 256MB on others. So I can control how much space goes to each area, without risk or worry.
If my data drives /1a, /2a, etc.. get full I can add to them with other disks. I can not do the same with the OS as it will negatively impact performance, so have to end up splicing in mounts and this gets ugly to manage.

This brings me to the last part.. performance.
If is use different partitions/volumes, I can tune each filesystem for it's job. I.E. SunUFS/VxFS for critical files systems (/opt /usr, etc...), but why use this for /tmp or fea scratch areas?
S???cratch directories for databases and FEA can use huge block sizes to improve performance, where in user data disks I would loose 5-10% of my space if I used large blocks.

Old school SunOS used to teach the flat file system method (so did HP-UX back in 9.0X), but the servers for the last several years are shipping with Veritas which is much more powerfull than HP's on-line JFS. Both systems allow you to push/pull file systems and make changes on the fly to disks, volume groups, and logical volumes without much more than a bit of knowledge and a few mouse clicks.

Best of luck in your descision ;)
Shannon

[Trever Furnish]

You may be interested in the "Filesystem Hierarchy Standard", here:

http://www.pathname.com/fhs/

...which says, among many other things:

* Disk errors that corrupt data on the root filesystem are a greater problem than errors on any other partition. A small root filesystem is less prone to corruption as the result of a system crash.

[harry d brown jr]

Geoff,

I haven't read anyone else's responses, so that my response would not be tainted.

First of all, sun people are idiots, I mean, look at their archaic operating system, the damn thing doesn't have a logical volume manager! They were the LAST to adopt linux! Hell, IBM beat them to it by YEARS! They rely TOO much on third party dealers, most that I wouldn???t trust to get me a cup of coffee, let alone a napkin for my bagel!

And to clear anything up, I am a certified (probably crazy) solaris admin. Plus I moved to the DARK side this afternoon: I just installed AIX 5L on an IBM power-e 650. OK, now back to the argument:

Any admin will tell you to break up root for MANY reasons:

(1) So that some ROGUE application doesn???t fill up the OPERATING SYSTEM filesystem???s causing the WHOLE server to CRASH. If you got everything under ???/???, then you risk this EVERY SINGLE DAY. Like I want that! We have over 600 unix servers (400-500 HP 9000???s, ~100 solaris servers, & 1 AIX box), you think we need a headache like that?
(2) So you can BALANCE filesystem???s to other disks if necessary, for offloading disk contention.
(3) Backup???s, one so you can backup the operating system without backing up the data/applications of the server, vice-versa.
(4) Searching, like when using ???find??? with the ???-xdev??? option.
(5) Security! if you have sensitive data on a MOUNTED filesystem you can control WHEN that FILESYSTEM is mounted during the boot process, try that with a FLAT filesystem.

SUN???s problem is their limited eyesight. They can???t see that they will be out of business within a few years if they don???t get their act together. Their system management tool is just plain stupid. Their lack of a LVM is crazy. Their lack of a clear vision doesn???t help. Trust me, when I get time to rotate the sun web servers out, I will! I helped put them there, I can take them out!


live free or die
harry

[Mihails Nikitins]

Even if you have large disks, there is a possibility to fill them up, e.g. by accidental extensive logging. So separate /var partrition is always the safest solution.

Just additional 2 cents.

BR,
Mihails

[Heiner E. Lennackers]

I think it depends on the use of your computer. If it is a server with a very static installationen (like database-server) you can split it up for the earlier stated reasons. If it is a very dynamic workstation (CAD/CAE station) i would use only /stand (hfs), swap and /, and maybe /var as an extra partition to prevent spooling files file up your /-filesystem.

We use on our workstations to following setup:
1. disk: vg00
/stand: 84m
swap
(add. swap if high memory)
/ 3gb
/var 2gb
/export/hostname (rest, for local installed applications, all other application are on a binary server)

2. disk: vg01
/export/hostname/u (userdata)

Heiner

[Shannon Petry]

Harry,

I am a Sun person, and NOT and idiot. Use a bit of caution in what you state here please. While my workstation support is "mostly" HP-UX, I have to support a large variety. I have trained extensively in SunOS and use Exclusively Sun servers for cost and performance reasons.

I support SunOS, HP-UX, AIX and Irix. Does this mean I'm bad? No. Applications may require different OS's for different reasons.
I.E. CATIA runs better on an IBM RS/6000 than any other machine. Hands down, no doubt.
Alias runs best on a SGI machine.
My Sun servers have almost 0 downtime, require almost no maintenance, and were 60% the cost of the HP equivalent servers.

Sun was last to adopt linux? Har Har, HP's PA Risc still does not support Linux. HP supports Linux on their PC platforms, but NOT on the PA-Risc systems. Sun's adoption of Linux was public after IBM's, but SGI and HP still have no "official " stance on Linux for their RISC chips.

As for Sun being behind the times, you really should work in the field before you talk. Sun adopted Gnome, GTK, and Netscape 6 long before HP, has had VOLD management and better file system support than any other commercial UNIX, multi-media support in CDE, StarOffice, PC Cards to allow windows to run in an Xwindow on your Sun, and the most current CDE in the market.
Sun also ships veritas with any server which is much better than HP-UX's Online JFS (That's why HP made Veritas an option this year).
All of this is, and has been true for more than 3 years.
I have submitted letters and suggestions over and over to HP because we loose HP workstation support each year to Solaris not because of a better RISC chip, but because the workstations are more user friendly, users can mount and unmount CD's without SUDO or root permissions, play music CD's within CDE, open word documents in StarOffice, etc.. etc.. etc...
I will admit that Sun does not have the easiest tools for managing the OS, but their stance is something like this:
Giving nice pretty tools to users to administer their Sun's would be nice, but without the knowledge to go behind administration a GUI is useless.
Does GM give away a nice toolkit and chiltons manual with every new car they sell? NOPE! Because the tools give you enough to be as damaging as helpfull if you dont know what your doing.

SO enough ranting from me. Next time you go to spew your thoughts choose your words more carefully!

Shannon

[Jeff Schussele]

Shannon,

Harry was referring to Sun as a company - not Sun admins. That was very clear to me.

I really think you need to thicken your skin a little and quit taking things so personally.

As far as I'm concerned, anyone here can say whatever they want - that includes you as well as Harry.

You both made good points, but YOU'RE the one who took it to a personal level. Let's try to keep the discussion on a professional level, please.

Rgds,
Jeff

[Geoff Wild]

Shannon,

Just so you know, you can run linux on HP PA Risc:

http://www.parisc-linux.org/

Rgds...Geoff