UNIX size limit for sort utility, 32 bit applications

[David Fosgate_2]

Is there a limit to the size of a file that can be sorted using the Unix sort utility? On a recent test using a very large file our 32 bit application appears to have hit a limit. The Unix file system had plenty of space, yet the sort ended in error. The sorted file size was 2,147,483,647. If there is a 2 gig limit to the Unix sort utility, does anyone know another solution being offered to overcome this limitation.

Thanks in advance for your thoughts

Dave

[Dan Hetzel · Accepted solution]

Hi David,

Here is a link to the HP-UX Large Files white Paper:
http://docs.hp.com/hpux/onlinedocs/os/lgfiles4.pdf

I don't know about a 2Gb limit for sort but some old programs like tar and cpio DO NOT support files > 2Gb.

Does your filesystem support Large Files ?
To test it out
# prealloc mebigfile 2200000000

if you get "File too large" then it wasn't mounted with largefiles option

Best regards,

Dan

[Alex Glennie]

I could find no info on file size limit either wrt sort .... i don't think it's relevant due to the way sort works.

whats the exact error you get, what params do you have set which are 2 GB ?

[Sandor Horvath_2]

Hi !

Fist check your filesystem can handle more then 2 GB files. You need create this filesystem with -o largefiles options.....


Saa

[David Fosgate_2]

Hi All,

Thanks for your information, the problem is due to the size limitation. I tried the prealloc mebigfile 2200000000 command and received the File too large response. A call to the support line for trhe cognos application confirms this limitation.

Best Regards,

Dave

[Bruce Regittko_1]

Hi,

A quick way to see if your filesystem supports largefiles is to look at the output of the fstyp(1m) command with the -v option. If the f_flag value is 0, largefiles are not supported. If it is 16, largefiles are supported. (Note that this does not say if the filesystem was mounted with the -o largefiles option.)

If the filesystem in question is vxfs, version 3.0 or higher, then you can use fsadm(1m) with the -o largefiles option to change the largefiles compatibility bit. You would then have to unmount the filesystem and remount it with the mount command's -o largefiles option.

--Bruce