Root disk layout

[jmb]

There is discussion among my work group about the pros and cons of merging most of the filesystems on root into /, leaving /var and /tmp. We are dealing with 3 hardware platforms, HP, IBM and Sun, and logically could have a different plan for each.

I looked for white papers, or Best Practices from HP on this subject, but didn't find any. I know they ship their systems with a pretty generic OS, carved up into 7 filesystems.

Relevant info: We're using 2 internal disks, at least 72 G, mirrored, with Online JFS, running on LVM. User files will be placed on the SAN. There's a ton of space on these root disks to use how we want.

So here are the questions:

1) Any advantage to combining HP LVM filesystems into 2 or 3?

2) Are there some fs that are always good to set up with a bunch of extra space, or others that are left pretty full?

3) Given the very large memory built into new systems, what is the "rule of thumb" for device swap size?

Thanks!

[Michael Tully]

For starters /stand must be separate, as it must be a HFS filesystem. So there goes part of theory. Other than making system look alike, why not have a separate SOE for each platform. That's what we do and it works wonderfully.

1) I see no advantage, as / /stand and primary swap must have contiguous and strict extents.

2) The additional space can be added to filesystems like /usr and /opt as these don't need to be contiguous and strict. You can always cut an ignite tape to / /stand and primary swap. Do yourself a favour and have online JFS on production servers, one day it may save your bacon.

3) The old rule of thumb for swap used to be 2 x memory but these days given the amount of RAM on some systems it depends largely on the application and databases being used.

[Sridhar Bhaskarla]

Hi,

1. I prefer to have multiple logical volumes as maintenance is easier. Particularly I would keep / filesystem seperate. The chances to review the size in future will be more if you add /opt and /usr into it and that may be a problem as / filesystem has to be contiguous. My suggestion would be /, /usr, /opt/, /tmp, /var, /etc/lvmconf, (if you have too many VGs and disks),/etc/cmcluster (If it is serviceguarded) and /usr/local.

2. May be you can keep your / at 1 GB just for the sake of "core" dumps. In fact, 500 is more than enough as nothing should grow in / filesystem. You can give some cushion to /var and /opt as they tend to increase.

3. On large memory systems, enable swapmem_on kernel parameter. That will account 75% for swap. It's a good idea to add device swap equal to the size of memory otherwise you may not be able to use all of your memory. With pseudoswap, you don't need to worry about old rule of "twice the memory" swap configuration. pseudoswap plays part only in the calculations and nothing else.

4. Plan for dump. If you have plenty of space left in your vg00, create a seperate logical volume exclusively for dump. With that you can disable savecrash at the bootup to save time during a system crash. You can always run savecrash manually once the system is up.

-Sri

[Ashwani Kashyap]

1. / , /stand and primary swap because they need contiguous extents with strict allocation . I like the way HP ships their OS with 7-8 file systems . If you combine everything into a few , and god forbid there is file system corruption , you might not be able to run fsck , depending on where /usr/bin is . Also it helps when you are booted in single user or lvm maintenance mode .

2.I would always give extra space to /var ,/opt and /usr file systems because they tend to grow . I also prefer to create extra file systems that require huge amount of space like /var/tmp , /var/adm/crash , /var/opt/perf etc.

3.The rule of thumb is 1.5 to 2 times the amount of memory . But given the large amount of memory , it really depends on the application . If its a memory eater/leaker then even with large amount of memory . it will swap .

[Bill Hassell · Accepted solution]

1. Combining filesystems INCREASES the effort for sysadmins to manage disk space for many applications. / is quite specialized and in HP-UX, never grows as it is static in nature. Now, a lot of sysadmins will disagree saying that 150-250 megs is not nearly enough, that they need 2000 to 6000 megs for /. The answer is: there are only 3 directories that are not mountpoints in /: /etc /dev /sbin (everyone is running to check their / directory right now...)

The problem is with undisciplined software and applications that assume / is their playground rather than asking the sysadmin where it belongs. The SysV.4 filesystem layout defines /opt as the location for applications, /usr for the opsystem (with legacy exceptions covering /usr/local and /usr/contrib) and /tmp is for opsystem (NOT users) storage.

So / is (must be) restricted and kept separate. If it fills up, the whole system may go down. /opt is somewhat static and grows only as applications are added. Logs are kept in /var/opt/. And /home--there is no easy answer. On a production server, /home is minimal as users don't really login or get to a shell prompt. In other environments, /home must be carefully managed (read: quotas enforced).

2. /var is the most critical in HP-UX. It is used by kernel processes as well as user processes and always grows without bounds. /var/adm is full of logfiles and always needs monitoring. /var/adm/sw is where backup copies of previous files are stored when patches are installed. This can be etremely large but must be managed with the cleanup utility (NEVER remove files from /var/adm/sw or you'll never be able to install patches again).

Additionally, /var/mail, /var/spool and /var/adm/crash can grow to thousands of megs in short periods--it all depends on what your server is doing.

Which leads to the conclusion that MORE, not less partitions are a better choice for managing some servers. A separate partition for /var/mail and/or /var/spool may be needed to prevent problem users from crashing the system by reprinting a large job 10-20 times. (if /var/spool fills, spooling stops but the system runs. If /var/spool is part of /var, then the whole system goes down.

Putting everything into a couple of giant partitions guarentees that a runaway job or email storm will crash the system. Of course, you can fix this by disabling email and spooling. As far as other opsystems, the design principles apply in the same way. If these are single user workstations with no remote users, then combining should be fine.

2. There are static and dynamic filesystems. Static filesystems are / and /stand..they should not grow. Growth indicates a sysadmin problem. /usr is a bit less static...it will grow a bit when libraries are added and programs are stored in /usr/contrib and/usr/local. NOTE: /usr/local has the wrong permissions--change all /usr/local directories to 755. But some freeware tends to grab a lot of /usr/local.

Dynamic mountpoints are /var, /tmp and /home. /tmp is difficult to cleanup simply because there is so much old code still using /tmp for storage. The correct location is /var/tmp. As has always been the case for Unix systems, cron jobs must watch and trim these mountpoints regularly.

As far as sizes, here's a starting point that works for 9Gb disks and higher:

/ = 250megs
/stand = 150megs
/usr = 2000megs
/opt = 2000megs
/tmp = 500megs
/var = 4000megs
/home = 100 to 2000megs

3. There is no rule of thumb today. Years ago (generations in computer years) the rule was 2xRAM to 5xRAM. If large RAM means 1000megs, then 2xRAM may be necessary for typical applications but for Oracle or SAP, you may need 10x to 20xRAM for installation, then 4x to 8xRAM to run. If large RAM means 8Gb or larger, then device swap can be just a few Gb since it will seldom be used. NOTE: Unlike other flavors of Unix, swap is handled quite differently depending on swapmem_on in the kernel.

Rather than try and figure out what you need, start with 2-4Gb of swap, then monitor swapinfo for actual usage. Large memory systems will use virtually no swap space (memory-mapped files are the exception).

--

While it may look useful to standardize across different opsystems, it usually means some things will be non-standard or awkward in each system, leading to more work rather than less. And within HP-UX, filesystem sizes will be very different for an Oracle system, an Apache web server or an email server.

[jmb]

Great. I appreciate the time and thought given to your answers. I will definitely be assigning points; I need to get some time to review all of your responses.

Thanks again!

[Geoff Wild]

Don't go with a flat filesystem - here's a thread I posted some time ago:

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=19149

Rgds...Geoff