Can't have files greater than 2Gb in HPUX 11i

[Simon R Wootton]

Running HPUX 11i on a RP7410 system. Trying to dump a Progress database onto a filesystem that has 25Gb available but the process stops when the dump reaches 2Gb, the message states that the maximum unix filesize has been reached. In SAM, the allow large files parameter is ticked and in /etc/fstab, the lines against the filesystems read 'largefiles'.

Really need to enable files larger than 2Gb in this filesystem urgently, any help will be rewarded.

Thankyou in advance for any help,

Regards
Simon

[Todd McDaniel_1]

you must enable largefiles on the filesystem

fsadm -F vxfs -o largefiles

then edit your /etc/fsadm and add the entry to the line of your mountpoint.

[Todd McDaniel_1]

err... edit the /etc/fstab file and add to the line after the "vxfs" entry...

vxfs largefiles 0 2

[Simon R Wootton]

Todd

Thanks for your help but I think that your suggestion has already been done (or has it ?).

/etc/fstab already has the largefiles entry as opposed to nolargefiles.

Help !!

[Stuart Abramson_2]

There are some commands that just WON'T do large files at all no matter what:

9. Backup utilities that work with large files:

dd
fbackup/frecover

10. UNIX utilities that won't work with large files:

tar, cpio
mail
shell script files (i.e. - ksh -x large_file)
vi
executables (loader)
print files
gzip

11. HP-UX 11.0 will completely support large files.

but not the utilities listed above.

12. All this is explained in:

/usr/share/doc/lg_files.txt

13. Oracle 7.X.X won't support large files.
Oracle 8 will.

14. You can download "gtar" from:

http://hpux.cs.utah.edu/hppd/hpux/Gnu/tar-1.13.25/

but I had trouble with it.

15. You need a new "gzip" to support large files

You can get it at:

http://hpux.cs.utah.edu/
http://public.www.planetmirror.com/pub/hpux/Gnu/gzip-1.3.3/
http://hpux.asknet.de/hppd/hpux/Gnu/gzip-1.3.3

You stated above that "..the process stops when the dump reaches 2 GB..". What command are you using to do the "dump". Maybe you need to change commands.

[Bill Hassell]

Check that you can create a largefile on that filesystem using prealloc as in:

prealloc /bigdir/bigfile 3000000000

which will create a 3Gb file called bigfile. If that works, then your backup is suspect. Are you using an 11.0 version of dump or is this from some other opsystem (not HP-UX)? Or is the backup in tar or cpio or ???

[Simon R Wootton]

The command is run from a progress environment running on hpux 11i. The error is a unix error but my filesystems look ok for >2Gb ????

The routine dumps the contents of a progress database to a different file on a different filesystem, so no specific unix commands used I believe.

Any idea's ??

Thanks Simon

[Simon R Wootton]

Bill

Thanks for post. I ran the prealloc command and it worked fine, created a 3Gb file in the filesystem where I'm trying to dump the data into.

Any idea's ?????

Simon

[Emilio Brusa]

Simon
In SAM
disk
file system
there is a option allow large files

you need check for files more larger than 2gb

E.

[Bill Hassell]

So largefiles works fine on the target directory and it is the Progress program that is failing. Are you using the same version of Progress on both machines? Is the error message explicit enough to spell out the exact path being used to restore the files? Does Progress come in a 64bit version? If so, the defaults for a 64bit program is to allow for large files by changing lseek() calls to lseek64() calls. Perhaps the problem machine only has a 32bit version of Progress?

[Todd McDaniel_1]

I thought this was on a single server? in the OP, Simon doesnt mention 2 servers, only different filesystems. A few things.


One thought is: do you have the largefiles set on the filesystem your Progress DB lives on? Im sure you do, just checking.

Is there any parameter in Progress that restricts the file size to <2GB? maybe that is hanging it.


Lastly, doesn't the /tmp directory or /var/tmp get called in here somewhere for your Progress DB? What size are they or is that an issue?

[Dave Johnson_1]

To contradict Stuart, I have created several files with TAR and gzip that are 5Gb and larger with no problems or special handling.

[A. Clay Stephenson]

You may be able to get around the 2GB limitation of the application by using a named pipe.

1) Create a fifo, e.g. mkfifo -m 660 /u01/mypipe

2) Start a process to read from the fifo and write to a conventional file. e.g. cat < /u01/mypipe > /u02/myexport &


3) Use /u01/mypipe as your output file for the dump. When the dump has finished, the contents of your dump will be in /u02/myexport and, assuming /u02 is largefiles enabled, will not be restricted by the 2GB limit because cat is largefiles enabled. Moreover, because your dump application is no longer writing to a regular file, the 2GB limit doesn't come into play there either.

[Bill Hassell]

tar (as supplied with standard HP-UX) cannot save files larger than 2Gb. While there is a patch that changes this limit to 8Gb, the resultant tar tape cannot be used with another version of tar. And the limit for tar will likely never be increased for that exact reason (compatibility). Now someone may have loaded gtar on your system and linked it to /usr/bin/tar so it appears to handle large files, but a gtar tape with large files will be unusable on any system that does not have gtar.

If the dump is indeed a Progress process, then check with Progress support about largefile capabilities. The database may allow for a large file but someone may have forgotten to patch the export portion.

[Dave Johnson_1]

Bill,
I am running HP-UX 11.00 (64-bit) from the Dec 2000 CD, installed Feb 2001 with some OS patches over the years, none specific for tar. I just completed the following test and was able to create a tar file >2GB:
(uxsvrdev) /u07/app/oracle/ofa/shared
# tar -cvf /u06/tar.backup.test ofas.00[123]
a ofas.001 4194296 blocks
a ofas.002 4194296 blocks
a ofas.003 4194296 blocks

(uxsvrdev) /u07/app/oracle/ofa/shared
# what `which tar`
/usr/bin/tar:
$Revision: 82.4.1.3 $
PATCH_11_00: tar.o 99/09/02

(uxsvrdev) /u07/app/oracle/ofa/shared
# ll /u06/tar.backup.test
-rw-r--r-- 1 root sys 6442444800 Dec 29 15:48 /u06/tar.backup.test

(uxsvrdev) /u07/app/oracle/ofa/shared
#

-Dave

[I.Delic]

Simon,

witch file system do you have ?
Did you check i-nodes on this file system.
bdf -i i think to check your i-nodes tabel

Can you do the dump of your databse to an another files system ?


I hoop this help


succes


Idriz

[Bill Hassell]

The 11.0 version of tar you have is 8Gb capable. So it will save your 6Gb file but not a 9Gb (or 25Gb) file. And the tar backup cannot be restored on a 10.20 system (or most other Unix flavors with standard tar programs). You can use prealloc to create a 9Gb file and try tar on it.

[Simon R Wootton]

Thanks to everyone for their help and input. After much reading and testing, it seems that it is a Progress issue and not unix as originally thought. We are now dumping the database using a modified version of the progress code.

Now where's that mince pie ..... !

[Dave Johnson_1]

Bill,
Actually the man page says "tar does not support the archival of files larger than 2GB". It says nothing about the size of the created archive file or tape.

I was able to create an archive file of 12GB using 6 2GB files. I was then able to extract from it file number 5 using a DG computer running DG-UX using the standard tar program shipped with the computer.

The restriction you talk about is when you try to tar a file that is >2GB or >8GB possibably along with other smaller files.

Hope this clears that up,
-Dave