NFS filesystem limits

[Marc Bohnert]

I have a customer that is using a NetApp F840 filer. A filesystem will be created on the NetApp and exported of size yet to be determined. I will be mounting the exported filesystem on a HPUX 11.0 server (acting as the client). Can anyone tell me if there is a filesystem size limitation from the HPUX 11.0 (client) perspective and if so where I can find this information. For example if it is possible to create a 4TB filesystem on the NetApp device would I be able to mount it on the HPUX box. NFS documentation I have reviewd does not indicate whether this is possible.

Thanks

[Elena Leontieva]

Marc,

On NetApps support site (you need to have a login)
https://now.netapp.com/eservice/UsrPage.jsp

I found the following info:

The filer does not limit the size of a file system. The maximum size of the file system on the filer is restricted only by the amount of disk space installed. For individual files, the maximum size is 4 GB. (Many NFS clients restrict the file size to 2 GB.)

HTH, Elena.

[Sundar_7]

I dont have a clue how NetApp filter works. But to talk about NFS in 11.0, you have NFS v3 available in 11.0 that can handle large files. The version of NFS that is used to interact with NFS server is decided based on the NFS version the server is running.

[Marc Bohnert]

But how big of a filesystem can the NFS client address?

[Sundar_7]

The largest read and write file size supported by HP-UX on NFS
version 3 is 4,398,046,511,103 bytes

This implies NFS version 3 client can address 4 TB filesystem.

[Dave Olker]

Hello Marc,

Before I get into NFS limits on HP-UX, let me start with local file and filesystem limits. I realize you're using a NetApp NFS server, but this information is for the benefit of other forum readers who may be using an HP-UX server as well.

Below is the *current* officially supported maximum sizes for local files and filesystems. I say "current" because we are looking to expand our supported filesystem sizes for 11i v2 beyond the current supported size of 8TB. In case you're not familiar with our OS numbering schemes, "11i v1" is another name for HP-UX 11.11 and "11i v2" is another name for HP-UX 11.23.


HP-UX JFS Disk Layout File Filesystem
Release Version Version Size Size
---------------------------------------------------------------
11.00 Version 3.1 Version 2 2 GB 128 GB
*11.00 Version 3.1 Version 3** 1 TB 1 TB

11.00 Version 3.3 Version 2 2 GB 128 GB
11.00 Version 3.3 Version 3** 1 TB 1 TB
11.00 Version 3.3 Version 4 1 TB 1 TB

11i v1 Version 3.3 Version 2 2 GB 128 GB
11i v1 Version 3.3 Version 3 2 TB 2 TB
11i v1 Version 3.3 Version 4** 2 TB 2 TB

11i v1 Version 3.5 Version 4 2 TB 2 TB

11i v2 Version 3.5 Version 5** 2 TB 8 TB
---------------------------------------------------------------
* PHKL_22719 is needed to avoid mount problems if extending
or creating file systems beyond 128 GB.

** default disk layouts for particular HP-UX Releases.


Again, these are the maximum sizes of the files/filesystems supported for local filesystems housed on HP-UX NFS *servers*.

It is also very important to distinguish between the limits on *files* vs. *filesystems* as these limits are very different.

As far as NFS clients go, the above listed *file* size limitations would apply to HP-UX NFS clients as well. In other words, an 11.0 NFS Version 3 client would max out at accessing 1TB files and an 11i NFS Version 3 client would max out at accessing 2TB files. There are architectural reasons, which have nothing to do with the NFS protocol itself, why these limits are imposed on supported file sizes.

Does this mean you couldn't write past 1TB from an 11.0 client or 2TB from an 11i client - no. The protocol would not stop you. However, there are other limitations outside of NFS that could potentially break if you attempt to use an offset past 1TB on 11.0 or 2TB on 11i. For this reason, the largest *files* we support accessing from an 11.0 client is 1TB and 2TB from 11i.

Now (finally ;) to your specific question - maximum *filesystem* sizes from an HP-UX NFS Version 3 client. I personally don't know of a stated maximum filesystem size that our NFS Version 3 clients are limited to. I also can't think of a reason why there would be a limit, outside of the architectural limits of the NFS Version 3 protocol specification (which is 8 exabytes).

Have you actually seen some documentation that states HP-UX NFS V3 clients are limited to accessing 2TB filesystems, or is this something you were told by the Response Center?

An HP-UX NFS Version 3 client should be able to work with filesystems at least as large as 8TB, since that is what we support for local filesystems on HP-UX NFS servers today. I cannot think of a reason why one of our clients wouldn't work with a filesystem larger than 8TB. I've personally mounted a 12TB filesystem from an HP-UX 11i NFS client and accessed the files without a problem.

As I stated at the top of this message, we're working on increasing our support for larger local filesystems. VxFS 3.5 disk layout 5 has a theoretical maximum filesystem size of 32TB. Once we announce support for 32TB VxFS local filesystems on our NFS servers, I can't think of a reason why one of our clients would have a problem working with an exported 32TB filesystem.

- Bottom line -
I believe our NFS V3 clients should work with filesystems larger than 2TB without a problem. I cannot think of a reason why there would be a filesystem size limit aside from those stated in the PV3 specification. That's not to say that there isn't an architectural limitation somewhere in our NFS V3 client code, but if there is one I don't know about it.

HP-UX 11i v2 (i.e. 11.23) servers currently support filesystems as large as 8TB and will hopefully soon support filesystems as large as 32TB. I don't know of a reason why one of our NFS clients would have a problem working with one of these 8TB-32TB filesystems.

If your NetApp NFS server provides filesystems larger than 8TB or 32TB, I would be very interested in hearing of any problems an HP-UX NFS client has in accessing those exported filesystems.

Regards,

Dave

[Geoff Wild]

Network File System
The theoretical maximum sizes of files that are accessible through Network File System (NFS) Version 2 and Version 3 are as follows:

NFS Version 2 - (2 GB - 1 byte)

NFS Version 3 - 18 exabytes (2^64 - 1)

This page:
http://h30097.www3.hp.com/docs/base_doc/DOCUMENTATION/V40F_HTML/AQTLMETE/DOCU_010.HTM
states that TRU64 has a limit of 512GB for NFS V3...

With HP-UX 11.0 - max filesystem is 1TB.


Rgds...Geoff

[Dave Olker]

Hi Geoff,

You're correct about the 18 exabyte limit for PV3 offsets - I mistakenly wrote 8 exabytes. I apologize.

Regarding 11.0, yes the maximum *local* filesystem size is 1TB for VxFS filesystems. However, that doesn't mean that an HP-UX 11.0 system wouldn't be able to mount a filesystem larger than 1TB via NFS.

I've personally mounted a 12TB filesystem using to my HP-UX 11i v1 (i.e. 11.11) client and accessed files without a problem. I realize 11.11 is not the same as 11.0, but 11.11 is limited to 2TB filesystems from a VxFS perspective and that didn't stop it from accessing a 12TB filesystem via NFS.

Again, I don't know of a reason why an HP-UX NFS client would be limited from mounting a large *filesystem*. Accessing *files* larger than 1TB (11.0) and 2TB (11i v1) is a different matter, but again that has nothing to do with the size of the filesystem that large file resides in.

Regards,

Dave

[Marc Bohnert]

Thanks for all the information. I know of no reason why I wouldn't be able to mount a 2TB plus filesystem and the information everyone has supplied is further confirmation.

Thanks to all.

[Dave Olker]

Hi Marc,

Glad the information was helpful.

If you have time, please post the results of your customer's tests, either positive or negative. I'd be very interested in any problems they encounter using an HP-UX NFS client to access a large filesystem, even if they aren't using an HP-UX server.

Regards,

Dave