System Administration
cancel
Showing results for 
Search instead for 
Did you mean: 

GNU cpio 2.6 does not write files larger than 2 GB (>2GB)

 
Highlighted

GNU cpio 2.6 does not write files larger than 2 GB (>2GB)

We ran into the known limits of cpio with a data base file >2GB.
A nice message triggered us and we found an alternative in GNU cpio 2.6 which "should" support larger files.
Using the 11.00 version from http://www.hpux.ws/merijn/downloads.html the messages were gone and everything seemed ok.

Now we need to restore the big file and the only thing we get is a correct directory entry (owner, timestamp) but the file is 0 bytes !!

Investigation showed that only the header was written to the cpio file but not the data.
Tests with only this file seem to confirm that a unrestorable cpio file is created without any warning messages.

Looks like I am not happy.....

Regards,
Andre
7 REPLIES 7
Highlighted
Exalted Contributor

Re: GNU cpio 2.6 does not write files larger than 2 GB (>2GB)

Shalom Andre,

I know gnu tar goes up to 8 GB.

HP has patched their version of tar to go to 8 GB as well.

I mean your file is useless but check the patch database and see if HP has a patch for cpio.

http://itrc.hp.com
Patch and firmware link
click hpux
cpio 8
Should get you down to a single patch for your OS.

There may be a bug in Merijn's version. When he gets back from holiday we'll let him know.

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
Highlighted

Re: GNU cpio 2.6 does not write files larger than 2 GB (>2GB)

Hi Steven,

I already looked for the patch because you mentioned it in
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1019693&admit=-682735245+1156932354438+28353475
:-)
but I have not been able to find it.
:-(
Even a generic search for only "cpio" returns
PHCO_24168, PHCO_25641, PHCO_25418
but none of them mention >2GB
Highlighted
Honored Contributor

Re: GNU cpio 2.6 does not write files larger than 2 GB (>2GB)

Any chance the filesystem you are going to doesn't have largefile support turned on?
# fsadm -F vxfs /var
largefiles

I don't think this is your issue, as I though at least it would grow to 2 gigs then stop. That maybe different with cpio.

Highlighted

Re: GNU cpio 2.6 does not write files larger than 2 GB (>2GB)

Hi Dave,

Yes, that way my HURRAY moment but unfortunately that was not the case.
There is an attachment with this issue to show the diffent cases.

Regards,
Andre
Highlighted
Honored Contributor

Re: GNU cpio 2.6 does not write files larger than 2 GB (>2GB)

I would strongly recommend against any 'classic' Unix tools for large backups today. These tools were designed when the entire disk was just a few hundred megs and all of them (pax, cpio, tar, dump) cannot backup large files. Now 'enhanced' versions may add a small increment (ie, 8Gb) but what to do about 10Gb or 100Gb files?

The answer is to use fbackup which is not only much faster than any of the classic tools but has no limits on file sizes (handles anything the OS supports). fbackup also has numerous reliability features as well as hooks to support tape changers. fbackup is proprietary to HP-UX so as a data interchange tool, it will not work. But using a non-standard tar won't work either until the sender and the receiver have compatible versions. And still you'll have limitations.


Bill Hassell, sysadmin
Highlighted
Acclaimed Contributor

Re: GNU cpio 2.6 does not write files larger than 2 GB (>2GB)

I am not aware of any enhancements to cpio that accomodate files larger than 2GiB --- although I have not examined the Gnu source recently. The cpio ASCII header uses an 11 octal digit field for the filesize so, in theory, a maximum (unsigned) size of 8GiB COULD be represented although that doesn't mean that it is. If your backups do not have to be portable across OS's then fbacker/frecover is the least evil "free" choice. It will handler largefiles with ease and will significantly outperform tar and cpio. If you need portability and can live within the 8GiB limit then the Gnu version of tar has been around a long time and is known to support 8GiB files.

Error recovery of tar and cpio backups is tenuous at best so if this were me I would look at a more robust backup solution like Data Protector. DP allows you to completely automate the backup process and the "point-and-click" restores are hard to beat.
If it ain't broke, I can fix that.
Highlighted

Re: GNU cpio 2.6 does not write files larger than 2 GB (>2GB)

I totally agree, but unfortunately people tend to keep old applications running for ages even after they are announced to be obsolete.
Unfortunately this is not a good time to talk to our customer about replacing it with a newer product.
If we still have this customer.....

Regards,
Andre