topic Re: How to reformat a fixed length file using scripting? in Operating System - Linux

How to reformat a fixed length file using scripting?

Melvin Thong — Mon, 19 Jun 2006 21:26:27 GMT

Dear all,

I face difficulties to code a script to do a file processing. I was given a file with fixed length records.

My task now is to grep the line record that has the first character with the number 5 and need to substring the line starting from position 12 with a string length of 6 (12,6) to indicate the transaction date.

Then the transaction date will be used to append at the end of the subsequent line of records that has the first character with number 6.

The script has to detect for the change of transaction date in between the records. If there is another line starts with number 5 appear, then the same process has to be done to substring starts from position 12 and length 6.

Then use the new transaction date to append to end of the line of the subsequent records that start with number 6.

The new record format has to be output into a new file.

I have attached a sample file for your understanding of how the file format look like.

Is there a way where awk programming or normal shell script can accomplish this task?

I hope the Unix gurus here can advise me. I will appreciate any help on this matter.

Thank you.

Regards,
Mel

Re: How to reformat a fixed length file using scripting?

A. Clay Stephenson — Mon, 19 Jun 2006 22:06:08 GMT

It's a bit difficult to tell from your file because I don't know if the records are actual text records terminated with a LF, CR, or a CR/LF pair or if they are indeed simply fixed-length records as you seem to imply. I don't know if your attachment is an accurate translation of the data or if you have passed it thru a filter or selected a particular output format. It would be helpful if you indicated if this is actual raw data and what is the record length.

If the records are truly-fixed length w/o a normal record terminator then tools like grep, sed, or awk aren't going to very useful. Your best bet will be Perl because it can easily process fixed-length records as well as normal line-oriented text. Another option is to use dd to unblock the input records and attach a LF to each of these so that the more conventional tools (awk, grep, sed) can then be used.

Re: How to reformat a fixed length file using scripting?

Mike Stroyan — Tue, 20 Jun 2006 17:34:25 GMT

If the file really is divided into lines like your attachment then you could use this-

awk '/^5/{date=substr($0,12,6)}/^6/{sub("$",date)}//{print}' input > output

Each time it sees a line with a leading 5 it remembers that as date. Each time it sees a line with a leading 6 it appends the current value of date.

Re: How to reformat a fixed length file using scripting?

Melvin Thong — Wed, 21 Jun 2006 06:15:30 GMT

I have tried to code using perl. It worked!

Thanks to Clay and Mike for your replies.