topic Re: SCP/SFTP with binary mode in Operating System - OpenVMS

SCP/SFTP with binary mode

Chand_1 — Mon, 08 Oct 2007 11:19:33 GMT

Hi,

I am generating a file with line terminator (Carriage return followed by Line Feed). When I ftp the file from VAX to Unix using SCP/SFTP, one blank line is coming after each record. Looks like the multinet secured ftp is doing some translation to file during ftp process.

Is anybody can help me how can I ftp the file without any problems using SCP/SFTP.

The normal ftp with binary mode is working fine for my files but not secured ftp.

Thanks,
Chand

Re: SCP/SFTP with binary mode

Hein van den Heuvel — Mon, 08 Oct 2007 11:53:38 GMT

It could be a bug in SCP/SFTP, or it could be that the file attributes do not match the contents.

Do a DIR/FULL on a sample file, and compare the record attributes with the actual contents.
($DUMP/BLOC=COUN=1 sample.dat)

hth,
Hein

Re: SCP/SFTP with binary mode

Chand_1 — Mon, 08 Oct 2007 13:18:12 GMT

Here is the output of DIR/FULL & DUMP.

Do you see anything wrong with this file?

INVBE84310_4D_A.TXT;1 File ID: (15573,131,0)
Size: 2/18 Owner: [ZAINABDINC]
Created: 3-OCT-2007 09:18:16.06
Revised: 3-OCT-2007 09:18:16.08 (1)
Expires:
Backup: 3-OCT-2007 22:36:39.86
Effective:
Recording:
File organization: Sequential
Shelved state: Online
Caching attribute: Writethrough
File attributes: Allocation: 18, Extend: 0, Global buffer count: 0
No version limit
Record format: VFC, 2 byte header, maximum 0 bytes, longest 159 bytes
Record attributes: Print file carriage control
RMS attributes: None
Journaling enabled: None
File protection: System:RWED, Owner:RWED, Group:, World:

Dump of file INV on 8-OCT-2007 13:12:15.42
BE84310_4D_A.TXT;1
File ID (15573,131,0) End of file block 2 / Allocated 18

Virtual block number 1 (00000001), 512 (0200) bytes

444F4F46 204E4F53 59547C48 8D010042 B...H|TYSON FOOD 000000
31303030 307C3138 34363233 337C527C |R|3326481|00001 000010
7C207C46 467C5050 7C33357C 527C207C | |R|53|PP|FF| | 000020
38343632 33337C7C 20202020 7C597C20 |Y| ||332648 000030
38343632 33337C52 8D01001A 0A0D7C31 1|......R|332648 000040
0A0D3537 30313031 7C4F507C 20207C31 1| |PO|101075.. 000050
20207C31 38343632 33337C52 8D010017 ....R|3326481| 000060
7C4C8D01 0089000A 0D5A5A5A 7C4D427C |BM|ZZZ.......L| 000070
4944204E 4F535954 7C313834 36323333 3326481|TYSON DI 000080
45544E45 43204E4F 49545542 49525453 STRIBUTION CENTE 000090
4349524C 4F205453 45572032 30347C52 R|402 WEST OLRIC 0000A0
35373237 7C52417C 53524547 4F527C48 H|ROGERS|AR|7275 0000B0
7C303030 307C3031 39303730 30327C36 6|20070910|0000| 0000C0
55507C30 3030307C 30313930 37303032 20070910|0000|PU 0000D0
2020207C 207C207C 207C3830 207C317C |1| 08| | | | 0000E0
36323333 7C7C7C7C 7C202020 20202020 |||||3326 0000F0
36323333 7C528D01 0018000A 0D313834 481.......R|3326 000100
0A0D5A5A 5A7C4D42 7C317C20 7C313834 481| |1|BM|ZZZ.. 000110
337C317C 35373031 30317C43 8D01006E n...C|101075|1|3 000120
43467C39 35373331 30363431 35323830 0825146013759|FC 000130
3030307C 54534220 4E4B4320 4C524720 GRL CKN BST|000 000140
7C34322E 31303030 307C3635 30303030 000056|00001.24| 000150
437C207C 207C3030 2E303034 31303030 0001400.00| | |C 000160
35327C34 342E3936 30303030 307C5341 AS|0000069.44|25 000170
0A0D3030 2E353230 30303030 7C363431 146|0000025.00.. 000180
53557C31 38343632 33337C4C 8D0100A1 Â¡...L|3326481|US 000190
53414745 56205341 4C204356 53444620 FDSVC LAS VEGAS 0001A0
49565245 53444F4F 46205355 7C4D3320 3M|US FOODSERVI 0001B0
38363120 53414745 56205341 4C204543 CE LAS VEGAS 168 0001C0
41474556 2053414C 7C454843 20572035 5 W CHE|LAS VEGA 0001D0
30373030 327C3431 3139387C 564E7C53 S|NV|89114|20070 0001E0
31393037 3030327C 30303030 7C333139 913|0000|2007091 0001F0

Re: SCP/SFTP with binary mode

Richard Whalen — Mon, 08 Oct 2007 16:32:54 GMT

Yes, MultiNet SCP & SFTP convert VMS record oriented files to stream-lf as part of the transfer when doing default binary transfers. If your file has line feeds in it, it could end up with blank lines. The conversion can be controlled with the logical MULTINET_SFTP_TRANSLATE_VMS_FILE_TYPES. See below for how to define the bit mask values for this.

MULTINET_SFTP_TRANSLATE_VMS_FILE_TYPES
This is a bit mask that determines which VMS file types should be translated when not operating in
VMS mode.
â ¢ Bit 0 (1) = FIXED
â ¢ Bit 1 (2) = VARIABLE
â ¢ Bit 2 (4) = VFC
The values are:
â ¢ 0 (zero) = NONE
â ¢ 7 = ALL
Note that this logical affects SCP2 as well as the server, as SCP2 has the server built into it for
handling local file access. If this logical is not defined, the value 7 will be used.

Re: SCP/SFTP with binary mode

Hein van den Heuvel — Mon, 08 Oct 2007 18:13:17 GMT

Great info.
Looks like the input file is terribly messed up.

Did you try typing it on OpenVMS?
It should show extra lines there as well.

You can NOT transfer this file in binary mode.
That does NOT make any sense at all, and any attempt to do so which appears to work is due to software trying to be 'helpful' when it should not (IMHO).

Look at those bits!

See that "8D010042" in the first line?
in binary mode they should be transferred (IMHO) but will be meaningless on any target other than OpenVMS.

The 0042 it the binary represention of the record length (50 bytes). The 8D01 is printer formating info telling VMS this is a simple record with a CRLF following it.
But it also includes a CRLF: The 0A0D in the 5th line: 0A0D7C31.

Want worse news?
The record at line 0000060 starts with 8D010017, so that's and 'odd' length of 23.
The result of this is an 00 alignment byte between the last 0A linefeed databyte, and the 0089 record length for the next record in the next line : 0089000A

Is the receiving system really ready to handle this binary?

So.... do you really want binary transfer, or are we dealig with straight ascii after all?

Please try again WITHOUT adding the CRLF to the records and ASCII mode.

You may want to just create STREAM_LF or STREAM (CRLF) files on VMS.

KISS!

Just use

$CREATE/FDL=SYS$INPUT transfer.dat
RECORD; FORMAT STREAM_LF
$OPEN/APPEND dat transfer.dat
$LOOP:
:
... blah
:
$WRITE/SYMB dat record
$GOTO LOOP

Enjoy!
Hein van den Heuvel

Re: SCP/SFTP with binary mode

Chand_1 — Tue, 09 Oct 2007 10:12:31 GMT

Looks like the below logic should resolve the problem. I am still testing and I will let you know the results ASAP.

$CREATE/FDL=SYS$INPUT transfer.dat
RECORD; FORMAT STREAM_LF
$OPEN/APPEND dat transfer.dat
$LOOP:
:
... blah
:
$WRITE/SYMB dat record
$GOTO LOOP

Re: SCP/SFTP with binary mode

Chand_1 — Fri, 12 Oct 2007 13:39:41 GMT

Hi,

That logic works fine for me.

Thank you very much.

Regards,
Chand

Re: SCP/SFTP with binary mode

Chand_1 — Fri, 12 Oct 2007 13:42:32 GMT

I have got the solution for the problem.

Thank you very much