topic Re: big-files gets from FTP, but the md5sum is not the same as server in Operating System - HP-UX

big-files gets from FTP, but the md5sum is not the same as server

JengShian — Sat, 25 Jun 2005 08:03:49 GMT

Dear sir,
I get big-files(big than 9GB) form FTP Server, but it cant open correct.
FTP get method is binary, and the permission is correct.
Then i check the file's md5sum, it is not the same as server.
How to solve this problem??

My OS is HPUX 1100

thx!!

Re: big-files gets from FTP, but the md5sum is not the same as server

Matthew_50 — Sat, 25 Jun 2005 08:11:34 GMT

Is size correct ?

Re: big-files gets from FTP, but the md5sum is not the same as server

D Block 2 — Sat, 25 Jun 2005 08:15:47 GMT

Jeng-Shian,

did you check the file length in bytes ?
$ ls -l file

does this length equal length from ftp-server ?

you can also run command:
$ cksum filename

how much free space do you have on your client - the local system ?
$ bdf

does your local file-system support 'largefiles' ?

$ grep largefile /etc/fstab

example: do you use /tmp filesystem
for the local directory location during the FTP.. verify that this filesystem supports largefiles.

$ grep /tmp /etc/fstab
/dev/vg00/lvol4 /tmp vxfs delaylog 0 2

# fsadm /dev/vg00/rlvol4
largefiles
#

last, check you size limits:

$ ulimit

good luck,
Tom

Re: big-files gets from FTP, but the md5sum is not the same as server

Mel Burslan — Sat, 25 Jun 2005 08:29:58 GMT

if your source system and the destination systems are running different operating systems, or even if different versions of the same OS, there might be a disk block size mismatch, in which case, the file at the destination may be a few (or few hundred) bytes bigger.

Re: big-files gets from FTP, but the md5sum is not the same as server

JengShian — Sat, 25 Jun 2005 09:06:12 GMT

Dear all,
the file size is the same as server.
and the local system's free space is larger than the file.

# ulimit
unlimited

# cksum file
=> md5sum is different to server, but the
file size is the same.

If the OS version or platform is different, the md5sum maybe different?
(the OS platform is difference between client and server)

About the "largefiles", I guess my user's filesystem is support "largefiles", I will check it later.
By the way, if the filesystem doesnt support "largefiles", can i save the file larger than 2GB in this filesystem?

thx to all!!

Re: big-files gets from FTP, but the md5sum is not the same as server

D Block 2 — Sat, 25 Jun 2005 11:58:02 GMT

you can also try to create a large 6g file in your local file-system:

prealloc /mnt/san/db/prod/test.file 6000000000

Re: big-files gets from FTP, but the md5sum is not the same as server

A. Clay Stephenson — Sat, 25 Jun 2005 16:16:29 GMT

The md5sum should be platform indepedent. I suspect you would have better success if you use split to break the file into more manageable pieces and then cat them togeter after the transfer. You could get a checksum for each of the smaller pieces so that if there are any problems, you would only need to resend a few pieces.

Re: big-files gets from FTP, but the md5sum is not the same as server

Eknath — Sat, 25 Jun 2005 20:07:05 GMT

Hi Jeng,

How about using rcp or remsh if both systems are unix.

Cheers!!!
eknath

Re: big-files gets from FTP, but the md5sum is not the same as server

Devender Khatana — Mon, 27 Jun 2005 03:14:43 GMT

Hi,

md5sum is independant of the OS platform & should always match for the certaintity of the consisistency of the file. I have also noticed similar problems where my md5sum deferred for 5-6 times and on using that file I got file corruption error at defferent stages every time. Finally after this many attempts I got md5sum right & corruption error was gone. I think ftp is not the safer option to copy this much large volume files. Other methods therefor should be preferred if available.

Large file option on a mounted file system can be set online by

#fsadm -F vxfs -o largefiles /dev/vgxx/lvolyy

HTH,
Devender

Re: big-files gets from FTP, but the md5sum is not the same as server

rick jones — Mon, 27 Jun 2005 12:26:10 GMT

What sort of NICs are being used for the transfer?

There should not be a need to retransfer the file several times to get the md5sum match.

Might also check if the number of TCP checksum errors is increasing during the transfer - while those are _caught_ problems and not then the source of the md5sum diffs it would be indicitive of something going on.

I'd also make sure you are up on the latest FTP, Transport, Filesystem and NIC driver patches for 11.0.