FileSystem size - how big is too big ?

[S.Rider]

I'm working on a new account and there's multiple file systems defined over 100 GB in size. Alot are NOT oracle filesystems and they have millions of files in them with a sub-directory structure that doesn't really split them up logically.
It's cause various management and backup headaches.I'm wondering if any of your sites have any rules like "unbix filesystems can only be up to ? GB, and that's it".

[Alex Lavrov.]

I think it depends on how many free inodes you have. And of course the space in your storage device.

[A. Clay Stephenson]

Generally, I'm much more concerned about the number of files per directory than the total number of files or the size of the filesystem. Large directories can be a significant performance hit especially if the directories need to be written often.

Because the granularity of backup objects for many commercial backup packages is the filesystem (at least for parallel operations), large filesystem restores typically can take long periods of time.
The downside of breaking things into many filesystems is that it's easier to miss one for backup. My approach is to backup everything except what I explicitly exclude.

[Bill Hassell]

It isn't the size of the filesystem. You could have a filesystem with 250Gb of data, but only 10 (very large) files. In that case, backup and filesystem performance will be just fine. But change this to a 1Gb filesystem with 15 million files and both filesystem as well as backup performance will plumment. So the answer is: Unix filesystems can be extremely large, but patience must be a virtue if there are millions of files.

Almost every design I've seen with thousands to millions of files is a hack to avoid using a real database design. Performance of the underlying application may be OK but backup and system maintenance may be enormously slow. Just try a find /bigdir on those millions of files (you can go to lunch while you're waiting..)