topic Re: reformat the flatfile in unix using cut command in Operating System - HP-UX

reformat the flatfile in unix using cut command

sasikala — Sun, 27 Jan 2008 14:46:00 GMT

Hi ,

I have a flatfile called a_M having 2 records in it the below content wherein the single record is scattered into 2 lines.

a_M
====
0001 13 3 HP Product and Pricing System ^M
0001 16 5 HP Product ^M

I need to reformat it and create a new file called b_M.$timestamp.

For this I have created a unix shell script with the below commands. But inthe output file i am getting some junk values. It looks if there is a space in the line the cut command does not take the proper column value.

Could anyone please help me out on this as it is very urgent.

#!/bin/ksh

for i in `cat a_M`
do
b=`echo $i | cut -c1-4`
c=`echo $i | cut -c17-50`

echo "FAM;A;"$b";";"$c >> b_M
done

timestamp=$( date +%d%m%y%H%M%S )
mv b_M b_M."$timestamp"
#END OF PROG

The ideal output should be
=============================
FAM;A;0001;HP Product and Pricing System
FAM;A;0001;HP Product

Thanks
Sasi

Re: reformat the flatfile in unix using cut command

James R. Ferguson — Sun, 27 Jan 2008 15:24:01 GMT

Hi Sasi:

You have several problems. Your 'for' loop will execute for every "word" in the file --- not what you want. Instead you want to read a line at a time. You could do this:

#!/usr/bin/sh
while read i
do
b=`echo $i | cut -c1-4`
c=`echo $i | cut -c11-50`
echo "FAM;A;"$b";"$c
done < a_M > b_M

Notice that the input is read from the file named "a_M" into the variable "i" one line at a time. Any output of the 'while' loop is redirected to the file named "b_M".

You show a "^M" in your input file. This is a carriage-return character and means that your file came from a Windows platform. UNIX doesn't end lines with the carriage-return plus newline characters; only with a newline. You can strip these by doing:

# dos2ux a_M > a_M.fixed

Regards!

...JRF...

Re: reformat the flatfile in unix using cut command

Dennis Handly — Mon, 28 Jan 2008 04:31:31 GMT

I just about never use cut(1) since it is so limited. You may want to consider using awk(1).

Taking the comments made by JRF about dos2ux(1):

timestamp=$(date +%d%m%y%H%M%S)
dos2ux a_M | awk -v OFS=";" '
{
num = substr($0, 1, 4)
name = substr($0, 11, 40)
print "FAM;A", num, name
} ' > b_M.$timestamp

>echo "FAM;A;"$b";";"$c >> b_M

You shouldn't "stutter" your quoting, do it in one go:
echo "FAM;A;$b;$c"

Re: reformat the flatfile in unix using cut command

sasikala — Mon, 28 Jan 2008 05:16:57 GMT

Thank you JRF and Dennis,

Could you please tell me if i need to code this awk command in unix shell script?

And moreover could you please tell me what does OFS mean?

does substr($0,11,40) give 11th character to 40th character of each line in the input file?

Could you please clarify these.

Thanks
Sasi

Re: reformat the flatfile in unix using cut command

Dennis Handly — Mon, 28 Jan 2008 09:51:03 GMT

>Could you please tell me if I need to code this awk command in unix shell script?

You might as well, that was my intention.

>And moreover could you please tell me what does OFS mean?

Output Field Separator, that supplies the ";".

>does substr($0,11,40) give 11th character to 40th character of each line in the input file?

No, it starts at 11 for 40 chars. Indexing starts at 1. (Your original question said 17 but JRF had 11.)
http://docs.hp.com/en/B2355-60130/awk.1.html

Re: reformat the flatfile in unix using cut command

sasikala — Mon, 28 Jan 2008 09:58:07 GMT

Thanks Dennis,

In unix itself I could achieve this as below. The space was considered when I use "$i" in echo command. Previously I didnt use "" for $i. Hence the problem. Thank you for your help.

#!/usr/bin/ksh
while read i
do
b=`echo "$i" | cut -c1-4`
c=`echo "$i" | cut -c11-50`
echo "FAM;A;"$b";"$c
done < a_M > b_M

Thanks
Sasi