Questions on large filesystems

[sreeram n]

Admin,
One of my application team is requesting for a couple of large filesystems of 2TB size.
Do you think a 2TB filesystem cause any performance issue?. If you can give some explanation, that will be grateful.


OS version is 11.23
Hardware Vpar on a SD64 superdome

Thanks

Sree

[James R. Ferguson]

Hi Sree:

Size isn't what's going to matter to performance so much as how you organize your filesystem and how you use it.

The more files in a filesystem the more likely you are to see performance "problems" when you search (find) files. Segregating files into hierarchical directories and limiting searches to smaller branches helps greatly.

Large filesystems with large files are entirely appropriate.

Regards!

...JRF...

[Ganesan R]

Hi Sreeram,

There is no straight answer for this. From filesystem point of view there won't be any performance issues.

But if you look at application point of view then you need to consides block size, mount options, etc depending upon the application requirement.

I hope this document may useful for you.

http://docs.hp.com/en/5576/JFS_Tuning.pdf

[sreeram n]

Thanks for your valuable inputs, James and Ganesan.

Sreeram

[Bill Hassell]

> Do you think a 2TB filesystem cause any performance issue?

Not at all -- until you add files. In other words, the very large empty directory works like a 20 MB filesystem. As mentioned, if the team stores a bunch of 100 GB files, still no issue. But if the team stores 50 million in the filesystem with no subdirectories, then yes, standard commands will appear to be crippled. Now it's not because of the big directory, it is because of the huge number of files. For instance:

- ls on the filesystem may take several minutes to display everything, ll will be significantly worse (and useless since people can't really look at a stream of filenames for several minutes and learn anything)

- "line too long" messages will be common when users use "*" when trying to find filenames.

- novice users that use the awful "find / -name myfile" command will complain that it takes a much longer time to complete.

But a few dozen directories and files will be no impact at all.

[sreeram n]

Thanks Bill for the calrification.

Sreeram

[sreeram n]

I got the required info