topic Re: Shell script use to combine several files into one files in Operating System - HP-UX

Shell script use to combine several files into one files

wfalai — Tue, 14 Jan 2003 08:10:18 GMT

Dear professionals,

I want to combine serveral text files into one textfiles. e.g.

Severals Files in a directory:
AAA00001
AAA00002
AAA00003
AAA00004
BDD00001
BDD00002
BDD00003
BDD00004
BDD00005
.....
...
..

These files will be combined into:
AAA00000
BDD00000
....
..
.

Where the content of AAA00000 equal to the combination of AAA00001, AAA00002, AAA00003, AAA00004.

Pls advise

Thanks a lot.

Re: Shell script use to combine several files into one files

Systeemingenieurs Infoc — Tue, 14 Jan 2003 08:31:25 GMT

ls|cut -c-3|
while read fname
do
cat ${fname}* >${fname}00000
done

something like that

Re: Shell script use to combine several files into one files

Systeemingenieurs Infoc — Tue, 14 Jan 2003 08:35:43 GMT

should've been : ls |cut -c-3|sort -u|...

Re: Shell script use to combine several files into one files

wfalai — Tue, 14 Jan 2003 09:09:34 GMT

Hi,

If I need to count back from the last position, say, 00001 and do a cut there.

The remain text become AAA .

How can I do .?

Thanks a lot

Re: Shell script use to combine several files into one files

Justo Exposito — Tue, 14 Jan 2003 11:00:33 GMT

Hi,
Try this:
for i in $(ls | cut -c1-3)
do
cat ${i}* >> ${i}00000
done

Regards,

Justo.

Re: Shell script use to combine several files into one files

Justo Exposito — Tue, 14 Jan 2003 11:13:49 GMT

Hi,
Sorry one correction:
for i in $(ls | cut -c1-3 | sort -u)
do
cat ${i}* >> ${i}00000
done

Regards,

Justo.

Re: Shell script use to combine several files into one files

wfalai — Tue, 14 Jan 2003 15:58:46 GMT

Thanks all..

I should say that:

"AAA" is the prefix of the filename. However, it is a randomly prefix pattern. So, say, it may be Dedss00001, Dedss00002.....etc., KKKAA00001, KKKAA00002...etc...

Because many many files with different length there and I may not be possible to use " cut -c1-3 " apply to all of the files. So, how can I do so that the script can find how long of the prefix ?

that is, cut -cx-y , where y is the last position of the filename and x is the 6th counting from the last position y.

Thanks ...

Re: Shell script use to combine several files into one files

Carlos Fernandez Riera — Tue, 14 Jan 2003 16:13:29 GMT

something like ?

ls -1 *[0-9][0-9][0-9][0-9][0-9][0-9] | awk ' { print " cat" ,substr($1,1,length($1-6)) "[0-9][0-9][0-9][0-9][0-9][0-9] > " substr($1,1,length($1-6)) "0" }'

Re: Shell script use to combine several files into one files

Justo Exposito — Tue, 14 Jan 2003 16:17:17 GMT

Hi,

This is the solution if the last part is always six characters:

for i in $(ls | sed 's/......$//' | sort -u)
do
cat ${i}* >> ${i}00000
done

Regards,

Justo.

Re: Shell script use to combine several files into one files

Carlos Fernandez Riera — Tue, 14 Jan 2003 16:20:47 GMT

ls -1 *[0-9][0-9][0-9][0-9][0-9][0-9] | awk ' { print " cat" ,substr($1,1,length($1-6)) "[0-9][0-9][0-9][0-9][0-9][0-9] > " substr($1,1,length($1-6)) "0" }'
| sort -u > file_command

sh file_command

Re: Shell script use to combine several files into one files

James R. Ferguson — Tue, 14 Jan 2003 16:22:38 GMT

Hi:

Given a filename you can use the shell's parameter substitution to get the basename and the length of that token.

Given:

# NAME=/var/tmp/filename

# BN=${F##*/}
# echo $BN
#...returns "filename" or the 'basename'

# L=${#BN}
# echo $L
#...returns the length or here, eight (8)...

Regards!

...JRF...

L=${#F} will

Re: Shell script use to combine several files into one files

Jean-Louis Phelix — Tue, 14 Jan 2003 16:42:46 GMT

Hi,

Fix your OLD suffix (one of the current files ie 000001) and NEW suffix (ie 000000) and try this one :

#!/usr/bin/sh
OLDSUF=001
NEWSUF=000
for var in *$OLDSUF
do
PREF=$(echo $var | sed 's/'$OLDSUF'$//')
cat ${PREF}* > ${PREF}${NEWSUF}
done

Regards

Re: Shell script use to combine several files into one files

H.Merijn Brand (procura — Tue, 14 Jan 2003 16:50:24 GMT

# perl -e'local$/;for(<* >){/^(\w+)\d+$/ or next;open$x,"<$_";open$y,">>${1}00000";print$y <$x>}'

unchecked, untested. If taking this to production, add tests to the open calls.

Enjoy, have FUN! H.Merijn

Re: Shell script use to combine several files into one files

H.Merijn Brand (procura — Tue, 14 Jan 2003 16:50:55 GMT

# perl -e'local$/;for(<* >){/^([A-Z]+)\d+$/i or next;open$x,"<$_";open$y,">>${1}00000";print$y <$x>}'

unchecked, untested. If taking this to production, add tests to the open calls.

Enjoy, have FUN! H.Merijn

Re: Shell script use to combine several files into one files

H.Merijn Brand (procura — Tue, 14 Jan 2003 16:51:57 GMT

Ignore the first. An Opera-7 glitch. sorry. N/A

Re: Shell script use to combine several files into one files

Rodney Hills — Tue, 14 Jan 2003 17:22:00 GMT

Here is a perl script that takes in account to NOT process the XXX00000 file as part of the input. It also has the benefit of not having to call external processes to copy each file.

HTH

-- Rod Hills

open(LIS,"ls -1|");
$prv=""
while() {
chomp;
$fil=$_;
next if substr($fil,3,5) eq "00000" ;# ignore result file
$cur=substr($fil,0,3);
if ($prv ne $cur) {
close OUT if $prv;
open(OUT,">${cur}00000");
$prv=$cur;
}
open(INP,"<$fil");
while() { print OUT $_; }
close INP;
}

Re: Shell script use to combine several files into one files

H.Merijn Brand (procura — Tue, 14 Jan 2003 19:35:16 GMT

Good catch Rodney!

# perl -e'local$/;for(<* >){/^([A-Z]+)(\d+)$/i or next;$2>0 or next;open$x,"<$_";open$y,">>${1}00000";print$y <$x>}'

Re: Shell script use to combine several files into one files

wfalai — Wed, 15 Jan 2003 02:24:00 GMT

Really thanks a lot.

I did the script by reference your examples. Here:

for i in (`ls -1 *[0-9][0-9][0-9][0-9][0-9][0-9]|sort -u)
do
cat ${i}* >> ${i}00000
done

Happy scripting!