ftp removes trailing white space

[Jim Krol]

We are trying to ftp from an IBM mainframe to a hpux 11.11 system. No transmission problems, the file gets here fine. The problem is that the file consists of fixed lenth records from the mainframe. When the record ends in spaces, they are truncated. This is a "bad" thing. The file will be an oracle extended table, which needs to have fixed length records. I have tried various site commands with no success. Has anyone seen this before? Is there an easy fix?

Thanks,
Jim Krol
james.krol@boeing.com

[A. Clay Stephenson]

I suspect what is going on is two different problems. EBCDIC to ASCII conversion and FTP. I would FTP first in binary mode --- that should be an absolutely exact copy but if the IBM native encoding is EBCDIC will appear to be gibberish. You should then be able to use dd to do the EBCDIC to ASCII conversion but if the data contains binary data then that is doomed.

[Zeev Schultz]

What if you compress the file with tar/gzip?
And uncompress in hp-ux?

[Shannon Petry]

FTP to my knowledge does not support whitespace in file names.

If you need this record to match, you have 2 choices.

1. rename the files via script, batch, interactive.

2. change the whitespace to underscores or hyphens.

Sincerely,
Shannon

[A. Clay Stephenson]

You might also have a look at the "fold" command to correct the record lengths after the fact. Man fold for details.

[Bill Hassell]

The problem is that Unix does not define file types like commercial opsystems. ftp has only two modes: binary and ASCII. In binary mode, you'll get exactly what the mainframe is sending. That can be as you described, a fixed record length, and the result in HP-UX should be the same result. But there are dozens of file types in the IBM mainframe so there may also be record separators or even index markers.

In ASCII, the mainframe reads the record, then sends that record to the client (HP-UX) and ftpd then adds the EOL character at the end of the mainframe's definition of the record. And therein lies the problem. ftp doesn't know anything about trailing spaces so the file was probably 'adjusted' at the mainframe.

If BINARY does not produce the right file image in ftp then transfer the file in ASCII and convert the file to fixed record length using dd and cbs=80 conv=block,sync options. Or you can use sed to extend the record length. Unix does not make file conversion very easy.

[William Wong_2]

As previously stated ftp does not support whitespaces. If it's possible you might want to create an ASCII version of the file on the mainframe and then FTP that file. Unfortunately for you the HP supplied ftp server does not support EBCDIC otherwise all you would need to do is:

ftp hpserver.com
username
password
bin
quote type e
put mainframe.dataset hp.dataset
quit

Here's a REXX macro you can use to convert EBCDIC to ASCII on the mainframe and then you
can send the file:

EDIT MACRO TO CONVERT TO ASCII
/* REXX */
/* */
/* This edit macro is designed to convert EBCDIC data to ASCII */
/* */
ADDRESS ISREDIT
'MACRO'
'(start) = LINENUM .ZF'
'(endit) = LINENUM .ZL'
DO point=start UNTIL point>=endit
'(line) = LINE' point
address linkmvs "C2ASCII line"
'LINE' point '= (line)'
END
"LOCATE 1"
EXIT 1

And the supporting assembler routine C2ASCII
The following routine requires no special linkage but it will need to be available in the TSO STEPLIB concatenation.

******************************************************************
* C2ASCII: CONVERT DATA TO ASCII
******************************************************************
C2ASCII AMODE 31
C2ASCII RMODE ANY
C2ASCII CSECT
BAKR 14,??
LR 12,15
USING C2ASCII,12
L 1,??(1)
LH 5,??(1) * GET THE LENGTH OF THE PARAMETER

LA 4,2(1)
XLATE (4),(5),TO=A
PR
END

[Dave La Mar]

Ted/Jim -
Interesting that this came up today. I just encountered a problem on our latest version of TCPIP for VSE similar to what you describe.
For your instance, our shop routinely pads records with spaces (40) and we got away from the problem you state.

Best regards,

DL

[Jim Krol]

I believe the problem has been solved. There is a site parameter "nostrip" that will not truncate spaces at the end of a line. Our mainframes use a TCP stack from CA that has this parameter. I am not sure if the IBM stack has this also. Thanks to all that replied. There were a couple of interesting solutions, I will try to remember for the next time we have data translation issues.

Thanks to all,
Jim Krol
james.krol@boeing.com

[okcunix]

I've seen this before at my site, which is DOD. We routinely ftp files from legacy MVS systems to HP servers for ORACLE insertion. As you suspect, it is an issue with the TCP stack they are using. There is a "native" IBM stack, and various COTS stacks. The primary COTS package in use here is "OpenLink" stack, which seems to send records in the fixed length.

[W.C. Epperson]

For anyone who looks up this thread in future and doesn't have the "nostrip" solution:

I've encountered this problem any number of times when transferring files between heterogenous systems. The solution that has always worked infallibly is to append a character to the end of the record at the sending end and strip it off on the receiving end. Piece of cake to add it and strip it off with utilities on most systems.