Shell script to concatenate columns from different files into one

[hp_user_1]

Hi there,

I have five files and each file has one column of data defined in it and all files have the same number of rows. I need to create a new file and add all five columns in such a way that they appear side by side in five columns. Can someone help me create a POSIX shell script to perform this.

Thanks

[Mark McDonald_2 · Accepted solution]

Shouldn't be too hard.

you could read each file in to an array, then echo these out in to a new file.

There is probably a neater way though.

How many lines are we talking about?

[hp_user_1]

About 30 lines....

[Mark McDonald_2]

ok, try something like

#!/usr/bin/ksh
#populate an array for file 1
count=1
cat file1 | while read LINE
do
FILE1[$count]=$LINE
increment count
done

#repeat for each file

then loop through each position in the arrays and cat in to a new file:
count=1
echo FILE1[$count] FILE2[$count] FILE3[$count] .... >> NEWFILE
increment count
etc

[Hein van den Heuvel]

Why not use the 'paste' command?

Hein.


NAME
paste - merge same lines of several files or subsequent lines of one file

SYNOPSIS
paste file1 file2 ...
paste -d list file1 file2 ...
paste -s [-d list ] file1 file2 ...

DESCRIPTION
In the first two forms, paste concatenates corresponding lines of the given input files file1, file2, etc. It treats each file as a column or columns in a table and pastes them together horizontally (parallel merging).

[Hein van den Heuvel]

Now if somehow you wanted to 'do something with those columns' then you need control that 'paste' does not offer.

I would use a PERL script for that.
For example:

------------------- my_paste.pl --------
use strict;
use warnings;
my ($done,@fh);
while (my $file = shift) { # Open all files
open $fh[@fh],"<$file" or die "Could not open file $file\n$!";
}

while (1) { # forever ?
my $line; # clean line every iteration
for (@fh) { # each file handle
$line .= <$_> or $done=1; # append chunck with newlines
}
last if $done; # done ?
$line =~ s/\n\b/\t/g; # replace newline followed by word with tab
print $line;
}
-------------------------------------

[Arturo Galbiati]

Hi,
paste file1 file2 file3 file4 file5 >newfile

This should work for you.
HTH,
Art

[Shahul]

Hi,

If I am right in understanding your requirement, I think "paste" should work with the help of "awk".

If your column seperator is "space", and you want 2nd column of file1 and 1st column of file2. The following will do,

#paste -d " " file1 file2 |awk '{print $2, $3}'

Try "paste -d " " file1 file2, then you understand which filed you need to filter.

Good luck
Shahul

[Laurent Menase]

in pure shell

exec 3
while read -u 3 f1
do
read -u 4 f2
read -u 5 f3
read -u 6 f4
read -u 7 f5
printf "%-15s %-15s %-15s %-15s %-15s\n" $f1 $f2 $f3 $f4 $f4
done

read -u 4 f2 && read -u 5 f3 && read -u 6 f4 && read -u 7 f5 && echo we still have some lines in the other files

[OldSchool]

According to the original query, 5 files each w/ once column of data....

Hein had the simplest answer, use "paste", as thats precisely what its made for....

[hp_user_1]

thanks