Text files EOL problem

[Ernesto Silva]

Hi,
first off all, I'm a real newbie in OpenVMS, I came here from the Linux environment, so please be patient :)

The alpha box is running OpenVMS 7.2.1

I'm trying to generate the Freeware 7 cdroms but I'm having problems with the classical EOL differences between systems.

After I downloaded the 3 .zip files (freeware70_?.zip) I unzipped them and I got 3 .bck file.
After a while I figured out how to "unpack" them generating all the subdirectories and files.

I've tried to run the necessary .com scripts but they assume they are mounted on a cdrom unit, so I zipped the files, I transfered them via binary ftp to a Linux box, unzipped and burn 3 cdroms.

I don't know where the problem is, but as a result I 'see' the files (in Linux and OpenVMS) with an EOL represented by a single character (decimal ascii 10), so the OpenVMS scripts doesn't work.

I've tried various zip, unzip, Linux, Windows, OpenVMS combinations but I can't solve the problem.

My OpenVMS does not have a CD Writer unit.

Can you help me please?

Best regards,
Ernesto Silva.

[Steven Schweda]

> [...] they assume they are mounted on a
> cdrom unit, [...]

Could you describe the actual problem here?
There's probably a simpler, better solution
than making CD-ROMs.

However, if you insist:

If the files looked right before you used Zip
and UnZip on them, and bad afterward, then
the problem may be in how you did the
Zip-UnZip.

If the files looked good before you made the
CD-R* discs, and bad afterward, then the
problem may be in how you made the CD-R*
discs, or how you mounted the CD-R* discs on
the VMS system.

Because you didn't supply any of the details
on these steps, it's hard to say just what
went wrong, or what else you could do about
it.

[Hoff]

You don't need to use the DCL command procedures (the "scripts"). Seriously. The menu system was and is not the strongest part of the Freeware. It's also not necessary, as the contents can be accessed directly.

The [FREEWARE] directory and the [000000] root directory have files containing all the abstract listings and all the files. Look around in these two directory areas for *.TXT text files, and read the contents. In particular, [FREEWARE]FREEWARE_README.TXT can be one starting point.

The ASCII 10 is a linefeed. In all likelyhood, there's a CR-LF pair here, as this is one of the normal sequential file formats found on OpenVMS. If so, the files have not been transfered correctly, or have been transfered through a foreign system and have had the attributes removed. Various of the OpenVMS sequential file formats are simply not transportable to other platforms. You're seeing what happens.

As for an approach toward avoiding this case and burning the disks, there are very detailed instructions around the OpenVMS disk structures and on burning disks for use with OpenVMS at the following site: http://64.223.189.234/node/28

In the LabsNotes document you'll find there, you'll see how to use LD or such to master the disk on OpenVMS, then to transfer the bits over to Linux or Windows. Or -- with an OpenVMS-local recording tool -- how to burn disks on OpenVMS.

Stephen Hoffman
HoffmanLabs

[Ernesto Silva]

Hoff,
I'll read the files thoroughly and give the link a try to get the cd's working.

I'll be back in a day or two.

Regards

[Ernesto Silva]

Steven:
if I zip and unzip the files (for testing only) in the OpenVMS there's no problem.

When I zip the files in the OpenVMS, transport them (via binary ftp) to Linux and unzip them there I lost the correct EOL, I only have a newline. I couldn't get the files with the rigth EOL in Linux.

I've tried the same on Windows and when I open the files with the Notepad I see 2 characters but they seem unreadable because the file is in one line only.

I believe my error is supousing the binary transfer doesn't change anything. Or perhaps the unzip process (linux or windows) unpacks the file according to the host operating system.

Regards

[Hein van den Heuvel]

>> I've tried the same on Windows and when I open the files with the Notepad I see 2 characters but they seem unreadable because the file is in one line only.


With 'the files' do you mean zip files or contents of a zip file, perhaps some C source code or a .COM file?

What do you want to do with those files on the PC? Read / Print?

That contents was likely a VARIABLE LENGTH record file on VMS. Each record 16-bit aligned, preceeded by a 16 bit length word, and no record terminator characted woudl be present. That's more or less unuseabel on anything but a VMS system.

If you just want to look at the sources, then I would urge you to use the ONLINE Freeware.

That tends to do the right things on a per file basis, after making sure it does not try to execute .COM file, but just open them with an editor. :-).

I find that the CRIMSON EDITOR does a better job than most to visualize text files which came from VMS: http://www.crimsoneditor.com/

Alos, below you'll find a bit of Perl I wrote to 'decode' OpenVMS Variable length files on a windoze or Unix box should they have accidently landed there.

If you need more help, then be sure to give a specific example.

Regards,
Hein.


------------------- vms_variable_to_text.pl -------
binmode STDIN;
while (read STDIN,$length_word,2) {
# avoid using "S" or "V". just do the math.
($length,$null) = unpack ("CC",$length_word);
$length += $null*256;
last if ($length > 32767);
$read = read STDIN,$line,$length;
print "$line\n";
read STDIN,$null,1 if ($length & 1);
}

[Hoff]

>>>
When I zip the files in the OpenVMS, transport them (via binary ftp) to Linux and unzip them there I lost the correct EOL, I only have a newline. I couldn't get the files with the rigth EOL in Linux.

I've tried the same on Windows and when I open the files with the Notepad I see 2 characters but they seem unreadable because the file is in one line only.

I believe my error is supousing the binary transfer doesn't change anything. Or perhaps the unzip process (linux or windows) unpacks the file according to the host operating system.
<<<


This isn't so much an issue of binary or text transfer so much as an issue of low-level file formats. Had the input formats on OpenVMS been stream-LF -- a format which exists on both platforms -- we would not be having these discussions.

Some FTP tools and some NFS tools will attempt to convert file formats on the fly.

To test this (and I'm typing from memory here, so there may be a syntax error or two in the following), try the following:

$ copy nla0: x.txt ! get a null file
$ set file/attr=rfm=stmlf x.txt ! set stream-lf
$ edit x.txt ! and fill in some text
$ copy/ftp/ascii x.txt x.y.z"usr pwd"::x.txt

And over on the x.y.z host:

vi x.txt

And let us know if it is readable.

Having a local CD-R/RW or DVD+R/RW is often the easiest and cheapest approach, assuming you have OpenVMS Alpha V7.3-1 or later or V8.3 or later, and the particular AlphaStation or AlphaServer has an IDE bus. (If you issue SHOW DEVICE DQ and see a pile of device information and not a no-device-found error of some sort, you very likely have an IDE bus, and can retrofit a recording drive from HP or from another vendor.)

Stephen Hoffman
HoffmanLabs

[Steven Schweda]

> [...] to Linux and unzip them there I lost
> the correct EOL, I only have a newline.

On Linux, that _is_ the right EOL.

> perhaps the unzip process (linux or
> windows) unpacks the file according to the
> host operating system.

Most likely, but it may also depend on which
unzip program you're using, and how you're
using it. Around here ("UnZip 5.52 of 28
February 2005, by Info-ZIP"):

unzip -h
[...]
-a auto-convert any text files
[...]

"unzip -v" output might also mention
"VMS_TEXT_CONV", which would suggest that it
may be able to convert VMS variable-length
record files to some locally popular format.
(Which is not to say that you actually had
any files in that format.)

I'm lost. Did you ever describe what, back
at the beginning, exactly, you tried to do,
and what, exactly, happened when you tried to
do it?