topic Re: SCP or SFTP to copy binary files? in Operating System - OpenVMS

SCP or SFTP to copy binary files?

Thomas A. Williams — Thu, 11 May 2006 07:53:18 GMT

I've been trying to copy some pcsi product files over (pull) from a remote node. Running into some problems.

They're on a remote node NODE2, the local node is NODE1.

These are the names of the 4 files:

DEC-AXPVMS-VMS732_F11X-V0500--4.PCSI;1
DEC-AXPVMS-VMS732_PCSI-V0300--4.PCSI;1
DEC-AXPVMS-VMS732_SYS-V1000--4.PCSI;1
DEC-AXPVMS-VMS732_UPDATE-V0600--4.PCSI;1

They reside on this directory:

NODE2::SYS$SYSDEVICE:[USER1.PATCH_200605]

and the local target directory is:

NODE1::SYS$SYSDEVICE:[SYSTEM.SOFTWARE.PATCHES.VMS732]

(1)

Using scp, I couldnt' seem to get the files to be recognized. I'm using the following syntax

scp user1@node2:[.PATCH_200605]* []

and I keep getting this error:

fcr_parse_raw: /SYS$SYSDEVICE/system/software/patches/vms732 (src): no such file (server msg: 'syserr: no such file or directory, fi
le: /SYS$SYSDEVICE/system/software/patches/vms732')

The /system/software/patches/vms732 is the path of the desination.

Should I be quoting the source? I tried it but to no avail...

(2)

using sftp, I was able to pull the files over, and I used the "binary" command prior to pulling them with the mget command. But the files were corrupt when I tried to run prod install. Any ideas why?

Thanks in advance.

Re: SCP or SFTP to copy binary files?

Dave Laurier — Thu, 11 May 2006 08:07:46 GMT

I believe that for SFTP to work, the files should be in the following format:

- Stream_LF for text files
- Fixed length 512 bytes for binary files

Normally the PCSI files are in fixed length 512 bytes format so this should not give any problems.

Perhaps you should try to transfer the compressed PCSI archives or put them in a ZIP archive.

Re: SCP or SFTP to copy binary files?

Thomas A. Williams — Thu, 11 May 2006 08:45:01 GMT

Thanks for the tip.

#*&()#% auditors are making us convert to SSH and we will have to disable DECnet at some point. Whole project just makes life more difficult for us... Oh well....

Any ideas on number 1?

Re: SCP or SFTP to copy binary files?

Richard Whalen — Thu, 11 May 2006 08:51:07 GMT

on (1) - I haven't used TCP/IP Services' SCP, but the protocol that SCP uses requires a Unix-like format for file specifications. I don't know if TCP/IP Services added any code to try to convert a VMS style file specification to a Unix-like one.

Another possibility is that you need to add the qualifier saying that the destination needs to be a directory as you are expecting multiple source files.

Re: SCP or SFTP to copy binary files?

Dave Laurier — Thu, 11 May 2006 08:51:25 GMT

What happens if you try the wildcard to be *.* or *.*;* or *.PCSI?

Re: SCP or SFTP to copy binary files?

Thomas A. Williams — Thu, 11 May 2006 09:01:17 GMT

a combination of the previous 2 replies worked:

scp -d user1@node1:patch_200605/*.* .

But alas, problem 2 exists for this too - the files are corrupt.

Re: SCP or SFTP to copy binary files?

Thomas A. Williams — Thu, 11 May 2006 09:05:14 GMT

>Normally the PCSI files are in fixed length 512 bytes format so this should not give any problems.

They're fixed, but 8192 byte length. I guess thats why it doesn't work.

Re: SCP or SFTP to copy binary files?

Volker Halle — Thu, 11 May 2006 09:05:36 GMT

Thomnas,

can you reset the file attributes with SET FILE/ATTR=(...) to be the same as the original files ? Does PRODUCT INSTALL then recognize the files ?

Volker.

Re: SCP or SFTP to copy binary files?

Thomas A. Williams — Thu, 11 May 2006 10:07:24 GMT

>can you reset the file attributes with SET FILE/ATTR=(...) to be the same as the original files ? Does PRODUCT INSTALL then recognize the files ?

Yes, I guess we can, or we can use an FDL to reset the file... But sheeesh - talk about having to jump through hoops to satisfy the auditors....

Re: SCP or SFTP to copy binary files?

Ian Miller. — Thu, 11 May 2006 10:17:22 GMT

welcome to audit hell :-)

I find zipping a file then transfering using binary with scp works best. A current version of zip will preserve the VMS file attributes.

Re: SCP or SFTP to copy binary files?

Thomas A. Williams — Thu, 11 May 2006 10:31:48 GMT

>welcome to audit hell :-)

ummm... Thanks? ;-)

>I find zipping a file then transfering using binary with scp works best. A current version of zip will preserve the VMS file attributes.

Good idea. Maybe we can invent some sort of wrapper script that would automatically do that for all files...

Re: SCP or SFTP to copy binary files?

Thomas A. Williams — Thu, 11 May 2006 10:32:56 GMT

One other thing - the actual file transfer is _painfully_ slow compared to DECnet copy - I guess because of all the encryption/decryption that has to occurr...

Re: SCP or SFTP to copy binary files?

Ian Miller. — Thu, 11 May 2006 10:39:55 GMT

Note for the future that .PCSI$COMPRESSED files are 512 byte fixed length records and do copy sucessfully.

I noticed on VMS Alpha V8.2 TCPIP V5.5 - ECO 1
scp option -p Preserve file attributes and timestamps.

I wonder if it works.

Re: SCP or SFTP to copy binary files?

Doug Phillips — Thu, 11 May 2006 11:38:44 GMT

>>Yes, I guess we can, or we can use an FDL to reset the file... <<

Don't use convert/fdl here. Use set file/attr instead. Convert will actually change the contents of the file --- and you probably don't want to do that.

As Ian says, Zip is probably the best way.

Re: SCP or SFTP to copy binary files?

Thomas A. Williams — Thu, 11 May 2006 11:39:40 GMT

> noticed on VMS Alpha V8.2 TCPIP V5.5 - ECO 1
scp option -p Preserve file attributes and timestamps.

>I wonder if it works.

It's also available on 7.3-2 ECO5 but no joy, didn't work...

Re: SCP or SFTP to copy binary files?

Thomas A. Williams — Thu, 11 May 2006 11:53:50 GMT

>Don't use convert/fdl here. Use set file/attr instead. Convert will actually change the contents of the file --- and you probably don't want to do that.

Regarding that, here is analyze/RMS of both file (before and after)

Before:
File Organization: sequential
Record Format: fixed
Record Attributes:
Maximum Record Size: 8192
Longest Record: 8192

After:
File Organization: sequential
Record Format: stream-LF
Record Attributes: carriage-return
Maximum Record Size: 0
Longest Record: 32767

I was able to set the record format and record attributes using:

set file/attr=RAT:NONE
set file/attr=RFM:FIX

but when I tried to set the block size like so:

set file/attr=BKS:8192

I get this error:

%SET-E-SYNTAX, error parsing '8192'

and

set file/attr=BKS:8

Succeeds but doesn't change these 2 lines:

Maximum Record Size: 0
Longest Record: 32767

What am I doing wrong?

Re: SCP or SFTP to copy binary files?

Jim_McKinney — Thu, 11 May 2006 12:14:19 GMT

$ set file/attr=(lrl:8192,mrs:8192) filename

Re: SCP or SFTP to copy binary files?

Steven Schweda — Thu, 11 May 2006 12:17:22 GMT

When in doubt on SET FILE /ATTRIBUTES, it
pays to find an example like
RESET_BACKUP_SAVESET_FILE_ATTRIBUTES.COM.

Re: SCP or SFTP to copy binary files?

Thomas A. Williams — Thu, 11 May 2006 12:20:04 GMT

OK - good advice. I got this as a result of your suggestions:

File Organization: sequential
Record Format: stream-LF
Record Attributes: carriage-return
Maximum Record Size: 8192
Longest Record: 8192
Blocks Allocated: 1900, Default Extend Size: 0
End-of-File VBN: 1889, Offset: %X'0000'
File Monitoring: disabled
Global Buffer Count: 0
*** VBN 1887: Last stream record does not contain a delimiter.
Unrecoverable error encountered in structure of file.

Whats up with that error?

(At least the record size looks good....)

Re: SCP or SFTP to copy binary files?

Volker Halle — Thu, 11 May 2006 12:28:34 GMT

Thomas,

the file has still STREAM-LF and carriage-control:

$ SET FILE/ATTR=(RFM:FIX,RAT:NONE)

Volker.