Reassembling files broken-up by "split"

[Danny Fang]

Hi,

May I know what is the utility in UNIX which allows me to reassemble all files that were broken-up in chunks by the "split" utility?

I had broken-up the original file using "split", with the following options:
# split -b 1024 testFile.log
which had generated the x** files (xaa, xab, etc...)

The purpose of this question is solely educational, as I would like to know the techniques of reassembling the x**, should the original(testFile.log) go missing or got deleted by accident.

Thanks in advance.
Danny

[Patrick Wallek · Accepted solution]

You could use cat in a script.

for FILE in x*
do
cat $FILE >> newtestFile.log
done

[Eknath]

Hi,

You can use following command for reassembling
#cat xaa xab xac xad ... >testFile1.log

Enjoy!!!
Eknath

[Muthukumar_5]

split command will split the file based on size or line into number files. Files naming will be in a sorted order. To recover it you can do as,

> newfile
for file in `ls xa*`
do

cat $file >>

done

hth.

[Danny Fang]

Hi,
I tried the method below as suggested in the replies to this thread:
#!/bin/sh
for i in `ls x*`
do
echo $i
cat $i >> myFile
done

However, the "diff" operation between the original file and newly concatenated file seem to differ (see output below):
[clownfish] /users/leawc> diff myFile template.tar
Binary files myFile and template.tar differ


I performed a split on a "tar" file and I'm now attempting to reassemble them.

I had also used the same script above for concatenating ordinary text files which were broken into smaller chunks, and found that they differ too (See output below - note that the original file is xml_creator.log, and the myFile is the concatenated files):

[clownfish] /users/leawc/TEMP3> diff xml_creator.log myFile
80a81,160
> [Wed 11-Aug-2004 16:09:06 SGT] STARTING xml_creator.ksh -source Ericsson STP v1.0 -detail -file loztest.xml -allnc -verbose
> [Wed 11-Aug-2004 16:09:06 SGT] INFO: Exporting attributes...
> [Wed 11-Aug-2004 16:09:06 SGT] INFO: Processing STP attributes...
> [Wed 11-Aug-2004 16:09:06 SGT] INFO: Processing SS7_Link attributes...
> [Wed 11-Aug-2004 16:09:06 SGT] INFO: Processing SS7_LinkSet attributes...
> [Wed 11-Aug-2004 16:09:06 SGT] INFO: Processing Node attributes...
> [Wed 11-Aug-2004 16:09:06 SGT] INFO: Exporting busy hours...
> [Wed 11-Aug-2004 16:09:06 SGT] INFO: Processing SS7Link_Discard_BH...
> [Wed 11-Aug-2004 16:09:06 SGT] INFO: Exporting KPIs...
> [Wed 11-Aug-2004 16:09:06 SGT] INFO: Processing STP.Ericsson.SS7.STP KPIs...
> [Wed 11-Aug-2004 16:09:06 SGT] WARNING: The STP.Ericsson.SS7.STP KPIs are not linked to any raw table columns.
> [Wed 11-Aug-2004 16:09:06 SGT] INFO: Processing SS7_Link.Ericsson.SS7.Link KPIs...
> [Wed 11-Aug-2004 16:09:06 SGT] WARNING: The SS7_Link.Ericsson.SS7.Link KPIs are not linked to any raw table columns.
> [Wed 11-Aug-2004 16:09:06 SGT] INFO: Processing SS7_LinkSet.Ericsson.SS7.LinkSet KPIs...
> [Wed 11-Aug-2004 16:09:06 SGT] WARNING: The SS7_LinkSet.Ericsson.SS7.LinkSet KPIs are not linked to any raw table columns.
> [Wed 11-Aug-2004 16:09:07 SGT] FINISHED xml_creator.ksh
>
>
> [Thu 12-Aug-2004 09:51:22 SGT] STARTING xml_creator.ksh -source Ericsson STP v1.0 -detail -file loztest.xml -allnc -verbose
> [Thu 12-Aug-2004 09:51:22 SGT] INFO: Exporting NC tables...
> [Thu 12-Aug-2004 09:51:22 SGT] INFO: Processing nc_stp table...
> [Thu 12-Aug-2004 09:51:22 SGT] INFO: Processing nc_ss7_link table...
> [Thu 12-Aug-2004 09:51:22 SGT] INFO: Processing nc_ss7_linkset table...
> [Thu 12-Aug-2004 09:51:22 SGT] INFO: Processing nc_node table...
> [Thu 12-Aug-2004 09:51:22 SGT] INFO: Exporting raw tables...
> [Thu 12-Aug-2004 09:51:22 SGT] INFO: Processing ericsson_ss7_link_link_1_raw table...
> [Thu 12-Aug-2004 09:51:23 SGT] INFO: Processing ericsson_ss7_link_link_2_raw table...
> [Thu 12-Aug-2004 09:51:23 SGT] INFO: Processing ericsson_ss7_linkset_linkset_raw table...
> [Thu 12-Aug-2004 09:51:23 SGT] INFO: Processing ericsson_stp_stp_raw table...
> [Thu 12-Aug-2004 09:51:23 SGT] INFO: Exporting attributes...
> [Thu 12-Aug-2004 09:51:23 SGT] INFO: Processing STP attributes...
> [Thu 12-Aug-2004 09:51:23 SGT] INFO: Processing SS7_Link attributes...
> [Thu 12-Aug-2004 09:51:23 SGT] INFO: Processing SS7_LinkSet attributes...
> [Thu 12-Aug-2004 09:51:23 SGT] INFO: Processing Node attributes...
> [Thu 12-Aug-2004 09:51:23 SGT] INFO: Exporting busy hours...
> [Thu 12-Aug-2004 09:51:23 SGT] INFO: Processing SS7Link_Discard_BH...
> [Thu 12-Aug-2004 09:51:23 SGT] INFO: Exporting KPIs...
> [Thu 12-Aug-2004 09:51:23 SGT] INFO: Processing STP.Ericsson.SS7.STP KPIs...
> [Thu 12-Aug-2004 09:51:23 SGT] INFO: Processing SS7_Link.Ericsson.SS7.Link KPIs...
> [Thu 12-Aug-2004 09:51:23 SGT] WARNING: The mapping for the SS7_Link.Ericsson.SS7.Link MSUDISC0_PC KPI is in an unexpected format.
> [Thu 12-Aug-2004 09:51:23 SGT] INFO: Processing SS7_LinkSet.Ericsson.SS7.LinkSet KPIs...
> [Thu 12-Aug-2004 09:51:23 SGT] FINISHED xml_creator.ksh
>
>
> [Thu 12-Aug-2004 10:12:42 SGT] STARTING xml_creator.ksh -package all -detail -file all.xml -verbose
> [Thu 12-Aug-2004 10:12:42 SGT] INFO: Exporting NC tables...
> [Thu 12-Aug-2004 10:12:42 SGT] INFO: Processing nc_stp table...
> [Thu 12-Aug-2004 10:12:42 SGT] INFO: Processing nc_ss7_link table...
> [Thu 12-Aug-2004 10:12:42 SGT] INFO: Processing nc_ss7_linkset table...
> [Thu 12-Aug-2004 10:12:42 SGT] INFO: Processing nc_node table...
> [Thu 12-Aug-2004 10:12:42 SGT] INFO: Exporting raw tables...
> [Thu 12-Aug-2004 10:12:42 SGT] INFO: Processing ericsson_ss7_link_link_1_raw table...
> [Thu 12-Aug-2004 10:12:42 SGT] INFO: Processing ericsson_ss7_link_link_2_raw table...
> [Thu 12-Aug-2004 10:12:43 SGT] INFO: Processing ericsson_ss7_linkset_linkset_raw table...
> [Thu 12-Aug-2004 10:12:43 SGT] INFO: Processing ericsson_stp_stp_raw table...
> [Thu 12-Aug-2004 10:12:43 SGT] INFO: Exporting attributes...
> [Thu 12-Aug-2004 10:12:43 SGT] INFO: Processing STP attributes...
> [Thu 12-Aug-2004 10:12:43 SGT] INFO: Processing SS7_Link attributes...
> [Thu 12-Aug-2004 10:12:43 SGT] INFO: Processing SS7_LinkSet attributes...
> [Thu 12-Aug-2004 10:12:43 SGT] INFO: Processing Node attributes...
> [Thu 12-Aug-2004 10:12:43 SGT] INFO: Exporting busy hours...
> [Thu 12-Aug-2004 10:12:43 SGT] INFO: Processing SS7Link_Discard_BH...
> [Thu 12-Aug-2004 10:12:43 SGT] INFO: Exporting KPIs...
> [Thu 12-Aug-2004 10:12:43 SGT] INFO: Processing STP.Ericsson.SS7.STP KPIs...
> [Thu 12-Aug-2004 10:12:43 SGT] INFO: Processing SS7_Link.Ericsson.SS7.Link KPIs...
> [Thu 12-Aug-2004 10:12:43 SGT] WARNING: The mapping for the SS7_Link.Ericsson.SS7.Link MSUDISC0_PC KPI is in an unexpected format.
> [Thu 12-Aug-2004 10:12:43 SGT] INFO: Processing SS7_LinkSet.Ericsson.SS7.LinkSet KPIs...
> [Thu 12-Aug-2004 10:12:43 SGT] FINISHED xml_creator.ksh
>
>
> [Thu 12-Aug-2004 11:22:19 SGT] STARTING xml_creator.ksh -package all -file all.xml
> [Thu 12-Aug-2004 11:22:19 SGT] INFO: Exporting NC tables...
> [Thu 12-Aug-2004 11:22:19 SGT] INFO: Exporting raw tables...
> [Thu 12-Aug-2004 11:22:19 SGT] INFO: Exporting attributes...
> [Thu 12-Aug-2004 11:22:19 SGT] INFO: Exporting busy hours...
> [Thu 12-Aug-2004 11:22:20 SGT] INFO: Exporting KPIs...
> [Thu 12-Aug-2004 11:22:20 SGT] WARNING: The mapping for the SS7_Link.Ericsson.SS7.Link MSUDISC0_PC KPI is in an unexpected format.
> [Thu 12-Aug-2004 11:22:20 SGT] FINISHED xml_creator.ksh
>
>
Therefore, the filesizes differ for the both files after the concatenation.
[clownfish] /users/leawc/TEMP3> du -sk myFile
12 myFile
[clownfish] /users/leawc/TEMP3> du -sk xml_creator.log
8 xml_creator.log


Could anyone help me out?

Thanks
Danny

[A. Clay Stephenson]

Because the split command automatically produces lexically sorted output file names all you have to do is
cat x* > mybigfile

Mybigfile is then automatically reassembled in the correct order. A much better test for comparing files is cksum or sum. Diff cannot be used on files which contain binary data.

[Stefan Schulz]

Hi,

this sounds like you accidently put one or more splitfiles several times in your newly joined file.

Just use cat x* > newfile as those files are created in the rigth order so globing will use them correctly.

By the way, whens splitting textfiles like your logs i always use split -l for line_count. This results in more readable files as they are split on complete lines only.

Hope this helps.

Regards Stefan

[A. Clay Stephenson]

One possible source of error would be if you already had some pre-existing files in the directory that started with 'x'.

[Stephen Keane]

Another possibility is that, as you are appending

cat $i >> myFile

If you run the command more than once, it won't work the way you might think. You should truncate the file before re-use, or use A. Clay Stephenson's suggestion instead.

[Danny Fang]

I just noticed that I had another file begining with the alphabet "x" - so that's the reason why `cat x* >> file` failed earlier on.

Thanks everyone. Issue resolved.