Problem with results -trying to OPEN/READ and then WRITE output file

[Cathy_20]

I need to modify a text file that has been created and "formatted" via Datatrieve. I am using a DCL command procedure to OPEN/READ the file, parse for what needs changed and write the "corrected" line (and all other lines) to the output file. Blank lines and form feeds do NOT get written properly to the output file. So, all the original page numbering is out of whack, etc. I stripped out all logic and just looped to read/write each line and STILL had the problem of these things dropping out. A "DIFF" comparing the beginning and ending file came out as ZERO differences. ???? However, a DUMP in HEX shows that bytes are different from one file to the next. I am sure that there is an easy solution, but have not found it. I do not have the option of correcting the Datatrieve output as it is created, so must deal with the text file as it is. Any help greatly appreciated. Thanks, Cathy

[Ian Miller.]

can you post the result of DIR/FULL for the file before and after processing as I suspec the RMS attributes may be changing.

[Martin P.J. Zinser]

Hello,

I agree with Mike, most probably the file from
DTR is a VFC file

(i.e. something like this in DIR
Record format: VFC, 2 byte header) the first two bytes contain formatting information (like formfeeds ;-)

A very stupid workaround (I am sure there are much better solutions!):

zip the original file on your VMS system,
unzip using -a (like ascii) and this will put all the lines in explicitly.

Greetings, Martin

[Antoniov.]

Yes,
sure your trouble is different file format.
Before open output file you could create a empty file withe same structure of original.
Look at this example:
$! Create a FDL file of characteristic
$ ANALYZE/RMS/FDL /OUT=TMP.FDL
$! Now I create a empty file
$ CREATE /FDL=TMP.FDL
$! Now open files (look for output file!)
$ OPEN/READ SRC
$ OPEN/APPEND TGT
$! Follws my standard code ...

H.T.H.
Antoniov

[Hein van den Heuvel]

Hmm, I've done this without problems many years ago. Datatrieve (used to!?) create a normal VMS Variable length record file with hard formfeeds (char 12) and expanded tabs (no char 8). See procedure below. My initial suspision goes to your code.

I agree with the others that the DCL standard VFC output might be thwarting the result. Instead of OPEN/WRITE try:
$CREATE 'output'
$OPEN/APPEND output 'output'
(or CREATE/FDL=NL:)

Hmm... is the report to be edited caught form a batchjob output log, or is it a proper 'report on' in Datatrieve?

Depending on the exact edits to be performed, you may want to consider an in place update! Use OPEN/READ/WRITE, $READ through the file until you find an interesting line, edit, $WRITE/UPDATE (keeping the record length exactly the same! Pad with nulls or spaces as needed). Next.
See second example with cute hack (IMH0) to 'patch' out the leading and trailing datatrieve formfeeds with minimal IO/work.

You mention DUMP showing differences, but DIFF not? Would that difference be a seemingly 2 byte longer record, mostly with prefix "8D01" ? That would be a typical VFC for a normal print line, no extra linefeeds, no formfeeds. One of the few (only!) tool giving you access to those bytes is CONVER/FIXED_CONTROL (and private program specifying RAB$L_RHB to point to a filed sized FAB$B_FSZ)

--- removing leading an trailing formfeed with new file ----

$open/read inp 'input'
$create 'output'
$open/append out 'output'
$ read/end=done inp a-rec ! Dump first ff
$ read/end=done inp a-rec ! Start one-off
$loop:
$ read/end=done inp b-rec
$ write/symbol out a-rec
$ read/end=done inp a-rec
$ write/symbol out b-rec
$goto loop
$done:
$ close out
$ close inp


---- removing leading and trailing formfeeds 'in place' ------

$eof = f$file(p1,"EOF")
$ffb = f$file(p1,"FFB")
$IF ffb .LE. 2 ! FF record spanning two blocks?
$ THEN ffb = ffb + 512
$ eof = eof - 1
$ENDIF
$ffb = ffb - 4 ! Two for length word, two more for rounded up data byte
$SET FILE/ATTR=(ebk:'eof',ffb='ffb') 'p1
$OPEN/READ/WRITE file 'p1
$READ file ff
$ff[0,8]=0
$WRITE/UPDATE file ff
$CLOSE file

[Hein van den Heuvel]

btw... If the prior replies did not yet allow you to solve your problem, then I would recommend to post a reply with a partial dump in the text of the reply.
More importantly, attach the real dumps and maybe some DIR/FULL output as a text attachment or anal/rms (overkill?!) as the forum software mangles the spacing otherwise.

You may also want to describe the edit you are trying to do, in case some reader here has a better solution.

You know details of the failure. Why not share that instead of letting us guess!

Regards,
Hein.

[John Gillings]

Cathy,

As others have suggested, I'm sure it's the VFC information that's causing you trouble.

Hein is the master at hacking RMS files, so his advice is correct and very slick. However, if you're confused by it all, here's a more pedestrian approach. I'm going to use CONVERT/FIXED to move the VFC header into the data stream. You can then operate on the resulting variable length file (making sure you leave the leading 2 bytes alone!). We can then use CONVERT/FIXED again to convert the file back to VFC.

input is "in.dat" output is "out.dat"

$ CONV/FIXED in.dat WORK.DAT/FDL=SYS$INPUT
file
sequential
record
format variable
$ OPEN/READ WORK WORK.DAT
$
$! note - this FDL is unnecessary, just
$! keeping things explicit...
$ CREATE/FDL=SYS$INPUT WORK1.DAT
file
sequential
record
format variable
$ OPEN/APPEND WORK1 WORK1.DAT
$
$! your code to process records from
$! WORK.DAT and write them to WORK1.DAT
$
$ CLOSE WORK
$ CLOSE WORK1
$ CONV/FIXED WORK1.DAT out.dat/FDL=SYS$INPUT
file
sequential
record
format vfc
$ EXIT

[Cathy_20]

Thanks everyone.

I recognize the file format gets changed from start to finish and this seems to be my problem. I am attaching the DIR/FULL of the original file and the target file that was created by my command procedure. In addition, I will attach a portion of the DUMP results for each.

I am also attaching my "basic" command procedure. The logic for changing specific lines has been removed while troubleshoting this formatting thing. I am using ANALYZE/RMS/FDL in an attempt to keep the file format intact - but this does not seem to work.

Hein: I need to keep the blank lines and all form feeds intact when going from the source file to the target file. Perhaps you thought that I need to get rid of them, but I need to keep them.... the problem is that they are being removed as my command procedure runs - or maybe I misunderstood your reply? How do I keep these things intact?

John: I have not had the chance to work it through - but did try your suggestion using convert. However,I must have been doing something wrong as I received an error.

Martin: I do not think that ZIP is installed on this system... but if that worked - that would be great.



I have included a portion of the DUMP files of the before and after. The blank lines that seem to be dropping are after the line "ALL CHECKS SELECTED", after the line that has dashes (-------) all across the page, after the line that says "Check Total", and I am also loosing the form feed between pages.

Thanks again to everyone.

Cathy

[Hein van den Heuvel]

Record format: VFC, 2 byte header, maximum 132 bytes, longest 132 bytes
Record attributes: Print file carriage control

Ah! Now I understand:
Datatrieve changed its report tool (most likely in V6 I am wispered by hte current maintainer) to use PRN files. See: FAB$B_RAT = http://h71000.www7.hp.com/doc/731FINAL/4523/4523pro_006.html#rms_record_attribute_field

I woudl urge you to check out ($HELP!) ECHANGE /NETWORK. Maybe, just maybe you can use its /TRANSFER=CONVERT=CARRIAGE

Or... you need a combination of Antiniovs reply and John's, or an adaption to John's adding "carriage_control print" in the record section of the output fdl.

Or... my inline update would also work... if you can keep the record lengths in check.

I know you did not want to remove form-feeds (in fact, that was the problem!), but my examples addressed a different question where to object was in fact to remove the leading and trailing formfeed.

Or... I supposed you could also write a relatively simple (c) program to deal with the 'fixed headers' that these special files have.


Or... you could dive into the Datatrieve documentation to figure out a way (if there is one) to revert to the old reports... but you probably can not change that part of the problem anyway.


I'm sure this helps :-),
Good luck,

Hein,

[Cathy_20]

Me again... I still have not had much success.

I tried to add "carriage_control print" to the record section of the FDL and received an incompatibility message. Maybe that type of carriage control is permitted with the other qualifiers????

In any case, at this point, the easiest thing to do is to parse for a "Page" number and cause a formfeed to occur so I can get the pages breaking properly. I can also parse for keywords to cause all the necessary line breaks. So... now the big qestion is - What's the syntax to insert a FF in a WRITE statement? Sorry for such a silly question - but, it's been years since I code in DCL......

And again, thanks to all of you for helping out.

Cathy