NFS file size limit

[Tom Danzig]

Fellow Admins,

I have a curious problem. Some developers here want to test ftp'ing a 20GB file (please don't ask me why!) to one of our Unix servers from a mainframe. Due to current disk space limitations, the only way I can accommodate the space request is via an NFS mount from another server.

The test ftp crapped out at 2GB (2147481936). The file system is vxfs with largefiles enabled. I verified with prealloc that a 3GB file can be created locally. The same preaaloc fails on the NFS client (prealloc: File too large). Sizes under 2.1 GB work OK.

I was under the impresion that NFS Version3 supports files larger than 2GB. This is a 11.00 system and I have verified via mount that the file system is mounted with Version3:

/tmp/ftptest on server1:/test/ftptest defaults,NFSv3 on Thu Apr 17 13:31:27 2003

Any thoughts? My forum and knowledge base search came up empty.

Thanks,
Tom

[Pete Randall]

Tom,

Can't say as I've tried it recently but I know it *used* to fail, but I can't find any reference to restrictions now. I would have to comment though, that this should take days if you can get it to work over NFS.


Pete

[Stuart Abramson_2]

nfs version 3 is supposed to allow files up to 128 GB.

Are both servers running nfs 3?
Are both servers funning 64 bit OS?
Check large file size set on both sides for all LVs:
fstyp -v /dev/vgXX/lvolNN
f_flag: 16 => largefiles

[Stuart Abramson_2]

What utility did you (or they) use to copy the files? Some utilities don't support large files, like:

tar, cpio
mail
shell script files (i.e. - ksh -x large_file)
vi
executables (loader)
print files

I don't think cp or rcp supports large files. "gtar"' supports copy of large files.

BTW: You should find out why your user wants to copy large files. For example, if one of the reasons is a TIMING TEST, you just invalidated the test. NFS file systems are slower than local file systems.

[Patrick Wallek]

What use are you creating the file with? Check the 'ulimit -a' for that user and make sure that 'file(blocks)' is unlimited.

[Tom Danzig]

As I stated in the original post, the file was being created via ftp. I was testing NFS file creation from the client via prealloc. I'm not aware of any file size limit with ftp.

Based on the fact that the prealloc failed to create a 2.2GB+ file on the NFS mount, I am assuming it's an NFS issue with large files. prealloc with < 2GB file sizes works fine so it's not a permission issue. The prealloc command fails with "File to large".

[Bill Hassell]

NFS cannot handle files larger than 2Gb UNLESS you have the patches installed for NFS version 3 *AND* the other computer also has NFS version 3 capability enabled. ftp does not have any limitations as it never deals with the entire file, just individual reecords. NFS is a complete filesystem so both ends must be able to handle the addressing limitations. NFS (unlike ftp) may be asked to randomly write records over a large area, thus the limitations.

[Tom Danzig]

Thanks for the info Bill.

client1># mount
/tmp/ftptest on server1:/test/ftptest defaults,NFSv3 on Thu Apr 17 13:31:27 2003

client1># rpcinfo -u server1 nfs
program 100003 version 2 ready and waiting
program 100003 version 3 ready and waiting

I am assuming based on the above that I'm good on both sides as far as NFS Version3 goes.

I'll seach aroung for NFS patches as related to 2GB file limits. Other thoughts/info welcome.

[Tom Danzig]

FYI, I found this doc on NFS and largefiles and had high hopes. Alas, I have verified all my configuration as per the doc and the dang thing STILL won't work! I'm at a loss.

http://www1.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000063210623

[Bill Douglass · Accepted solution]

I just tested this on a pair of N-class boxes, and was successful in dumping 10GB of file across an NFS mount.

HP-UX 11.0

Server: /archive on /dev/vgm006/lvol1 delaylog,nodatainlog on Mon Mar 3 15:39:12 2003

Client: /archive on qe2n1:/archive defaults,NFSv3 on Fri Apr 18 15:33:15 2003

I did patch the client recently to the March 2003 Quality Pak. Is your client system up-to-date on patches?

[Tom Danzig]

Bill - I appreciate you taking the time to run the tests. It was not a patch issue.

After re-running exportfs -a on the server, the issue was fixed. My assumption is that the file system may have been originally exported _before_ largefiles was enabled. The only explaination I can come up with anyway.

Problem solved. Thanks to all who posted replies.