topic Re: copy/ftp issue in Operating System - OpenVMS

copy/ftp issue

Tru64Unix — Thu, 12 Mar 2009 06:42:06 GMT

Hi all,

I have two Alpha server(ES40&GS1280) running OpenVMS8.3.I copy some files from GS1280 to ES40 using copy/ftp in a script.
The script run successfully.But I find one of the files is broken,other files are OK.
All files are .COM file and no more than 30K.TCPIP version is 5.6ECO-2.If I do not run script and only run copy/ftp command,everything is OK.
Does anyone give some comment?

Re: copy/ftp issue

marsh_1 — Thu, 12 Mar 2009 07:23:02 GMT

hi,

any recent changes ? network errors ? is it reproducable - different files affected or the same ones. any error messages ? also posting up the procedure involved may help everyone here to help you much faster.

fwiw

Re: copy/ftp issue

Wim Van den Wyngaert — Thu, 12 Mar 2009 08:31:40 GMT

Could you post the output of the copy/ftp, the return status and a prove that the files are "borken" ?
Make sure that sys$output and sys$error are not directed elsewhere or post it too.

WIm

Re: copy/ftp issue

Tru64Unix — Thu, 12 Mar 2009 13:06:10 GMT

No any changes.I run this script about twice one week, only this time encounter the problem.

source file:
============
$ dir WAIT_FOR_FILES.COM;1/full

Directory DSA101:[TOFF00.DEV.CODE.DCL]

WAIT_FOR_FILES.COM;1 File ID: (69373,11,0)
Size: 21/111 Owner: [NGTS_DEV]
Created: 22-JUN-2007 00:52:00.95
Revised: 11-FEB-2009 11:45:37.73 (6)
Expires:
Backup: 17-FEB-2009 17:55:35.44
Effective:
Recording:
Accessed:
Attributes:
Modified:
Linkcount: 1
File organization: Sequential
Shelved state: Online
Caching attribute: Writethrough
File attributes: Allocation: 111, Extend: 0, Global buffer count: 0, No version limit
Record format: Variable length, maximum 0 bytes, longest 80 bytes
Record attributes: Carriage return carriage control
RMS attributes: None
Journaling enabled: None
File protection: System:RWED, Owner:RWED, Group:, World:
Access Cntrl List: None
Client attributes: None

Total of 1 file, 21/111 blocks.
$
============================================
Target file:
============
$ dir WAIT_FOR_FILES.COM;6/full

Directory TOOLS$:[DELIVERY.SYSTEM_INTEG.DCL]

WAIT_FOR_FILES.COM;6 File ID: (3185,105,0)
Size: 10KB/152KB Owner: [NGTS_99]
Created: 9-MAR-2009 11:00:10.52
Revised: 9-MAR-2009 11:00:10.52 (5)
Expires:
Backup:
Effective:
Recording:
Accessed:
Attributes:
Modified:
Linkcount: 1
File organization: Sequential
Shelved state: Online
Caching attribute: Writethrough
File attributes: Allocation: 304, Extend: 0, Global buffer count: 0
No version limit
Record format: Variable length, maximum 0 bytes, longest 80 bytes
Record attributes: Carriage return carriage control
RMS attributes: None
Journaling enabled: None
File protection: System:RWED, Owner:RWED, Group:, World:
Access Cntrl List: None
Client attributes: None

Total of 1 file, 10KB/152KB
$

Re: copy/ftp issue

Hein van den Heuvel — Thu, 12 Mar 2009 13:17:03 GMT

So help us understand.

What is "the problem" (last reply)

What is "broken" in "I find one of the files is broken,"

How do you know? What is broken about it?
Is there an error? Tell us about it!
Better still. Copy & Paste!

The DIR/FULL looks fine and similar except that one process was set to report in Kilobytes instead of the traditional blocks which will make serious analysis just a little more difficult.

Hein.

Re: copy/ftp issue

Wim Van den Wyngaert — Thu, 12 Mar 2009 13:19:16 GMT

And the log of the copy/ftp ?

Wim

Re: copy/ftp issue

Robert Gezelter — Thu, 12 Mar 2009 13:40:08 GMT

Tru64Unix,

Hein's point is indeed well taken. High water enabled is indeed the default (verified) on 6.2.

- Bob Gezelter, http://www.rlgsc.com

Re: copy/ftp issue

Robert Gezelter — Thu, 12 Mar 2009 13:49:12 GMT

To all,

My apologies, somehow my previous item ended up in the wrong thread.

The moderators may consider this a request to delete both this and my previous entry.

- Bob Gezelter

Re: copy/ftp issue

Hoff — Thu, 12 Mar 2009 14:43:40 GMT

If all we have here is "is broken", well, the set of things that can be broken here is infinite. Specific symptoms or DCL commands or error messages or diagnostics or exact commands and such are a vital part of resolving something that is broken.

Some suggestions around how to pose a question:

http://www.catb.org/~esr/faqs/smart-questions.html

Asking questions is as much a learned skill as answering them, in my experience. And both are worth cultivating.

Re: copy/ftp issue

Peter Zeiszler — Thu, 12 Mar 2009 20:43:15 GMT

have you also thought of scp to move/copy files? I have been getting spoiled using that to move files around.

Re: copy/ftp issue

Tru64Unix — Thu, 12 Mar 2009 23:38:04 GMT

Hi all,
Thanks a lot.
"Broken" means contents after line 167 are missing and leave there blank.

Source file:
===============
$ REAL_ISIX = F$EXTRACT(0, 5,ISIX) --->line 167
$ @TOFF$DCL:GET_CLUSTER "I:''REAL_ISIX'"
......
Target file:
==============
$ REAL_ISIX = F$EXTRACT(0, 5,ISIX) ---->line 167
$ @TOFF$DCL:GET_CLUSTER "

Re: copy/ftp issue

Hein van den Heuvel — Thu, 12 Mar 2009 23:46:50 GMT

>> Broken" means contents after line 167 are missing and leave there blank

Ok. Thanks. Just in case you have other problems in the future please ask yourself how on earth any reader to have guessed that?!

Now line 167 is probably an effect, not a cause. The cause is more likely to be the BLOCK + OFFSET for the NEXT line.
Try DUMP/RECORD=(STAR=166,COUNT=5) on both systems. There will be an 'RFA' which is BLOCK+BYTE.
Now use the BLOCK for $DUMP/BLOCK=(START=x,COUNT=1) /OUT=x.txt

If you can not figure out the contents, then attach good + bad dump to a future reply as a .TXT.

If you enjoy those puzzles add a /HEADER to the DUMP and check for the EXACT EOF BLOCK and BYTE and such.

OR...

Just ZIP up ALL the files in one container (optionally with -V ), transfer and unpack.
So much easier!

And uh, get rid of that good gor nothing KB reporting. It's just confusing for problems like this. I know, 1KB = 2 blocks, but why
make it hard on yourself and us?

Cheers,
Hein

Re: copy/ftp issue

Richard J Maher — Fri, 13 Mar 2009 02:23:47 GMT

Hi,

Did you get an FTP log with some sort of dodgy status reported in it?

FWIW, I would copy over all the "real" files and once that's done then copy over the zero-length setinel file copy_complete.flag (or whatever)

The target platform then just waits for the presence of the sentinel file to tell it that the other files have been successfully copied.

'Cos polling is anathema to me I use the attached mechanism for determining when the sentinel has arrived (or timeout and report problems).

Cheers Richard Maher

Re: copy/ftp issue

Richard J Maher — Fri, 13 Mar 2009 02:37:13 GMT

Hi again,

Obviously won't help you if the destination node is not VMS but attached is the code that I've used to tell me when a file arrives in a directory on VMS.

FWIW, there is/was a product (that I have nothing to do with) that I think is was called something like Connect Direct (Got LiveConnect in my head at the moment so I could be wrong) It used to check such transfers by polling I believe?

Cheers Richard Maher

PS. Just a shame that in 2009 FTP is still the middleware of NO-choice for VMS customers :-( If only they were able to *integrate* their rich herritage of data and business rules, resident on their VMS boxes, with their *nix and Windows platforms then they wouldn't have to rely on dodgy data replication.

"Real-time integrated access to VMS Applications, Business-Rules, and Data" - Now there's a thought!

Re: copy/ftp issue

Craig A — Fri, 13 Mar 2009 09:40:21 GMT

Another thing to bear in mind is that when donig a PUT (or a get for that matter) don't forget that at any point in the actual transfer it may terminate prematurely (system crash, power cutage, device full, diskquota exceeded, network glitch, etc.. which will leave an incomplete file on a system that may (just may) have an automated process in place to detect the file and process it.

For that reason, I was always do the PUT to a .TMP file first and then do a RENAME as part of the script.

Eg:

put craig.dat craig.tmp
rename craig.tmp craig.dat

That way if the PUT fails the RENAME doesn't occur. (Assuming ERROR_LEVEL is set to SUCCESS)

Craig