Combine two text files

[Derek Card]

Hi,

I have two text files that have a starting date in one file and an ending date in the other file.

File "startdates"
1-Jan-1999
1-Jan-1999
2-Jan-1999

File "enddates"
12-Jun-1999
----------
31-Mar-2002

I would like to combine each line of these two files into 1 line
like this:
1-Jan-1999 12-Jun-1999
1-Jan-1999 ----------
2-Jan-1999 31-Mar-2002

The actual file has thousands of entries. I am fairly new to UNIX. Is there a command to easily do this?

TIA, Derek Card

[A. Clay Stephenson · Accepted solution]

Derek, this is about as easy as it gets.

paste startdates enddates > bothdates

Man paste for details.

[Dave Chamberlin]

There is - it is the paste command. Ex paste file1 file2. Do a man paste for options.

[MANOJ SRIVASTAVA]

Hi Derek


Use paste startdates enddates > test to get the result .


Manoj Srivastava

[Ian Kidd_1]

Another option is pr -tm startdates enddates > bothdates.

The only advantage this has over paste is some formatting (although there's no advantage with your example.

But if the lengths of the lines in startdates varies, for example:

10-January-1999
1-March-1999
2-May-1999

then the output of paste would be:
No date has been specified 22-Jan-1999
10-January-1999 12-Jun-1999
1-March-1999 ----------
2-May-1999 31-Mar-2002

(I added 22-Jan-1999 to make the number of items/file equal)

with pr -tm
No date has been specified 22-Jan-1999
10-January-1999 12-Jun-1999
1-March-1999 ----------
2-May-1999 31-Mar-2002

[Ian Kidd_1]

I was afraid that my extra white spaces would vanish in my reply above. The attachment I included should show the correct view (I hope)

[Derek Card]

Hello again,

Thank you for all your replies. I'm afraid that I didn't quite get enough information and now I'm really stuck. These two date files have an additional entry called ID Number. I really need
to combine the Id Number, the start dates, and the end dates into 1 line. I don't even know where to begin.

File "startdates"
1-Jan-1999 1009
1-Jan-1999 1010
2-Jan-1999 1011
5-Jan-1999 1012

File "enddates"
12-Jun-1999 1009
---------- 1010
31-Mar-2002 1011

1009 1-Jan-1999 12-Jun-1999
1010 1-Jan-1999 ----------
1011 2-Jan-1999 31-Mar-2002

If there is no matching ID Number, no output is needed.
I've been looking at awk. Is that what I need to use?

Thanks, Derek Card

[A. Clay Stephenson]

When I saw this post yesterday I thought this was a very hokey data model but now it makes a bit more sense. You could use awk (or Perl) for this but today I think a little sort and join will do the trick.

#!/usr/bin/sh

TDIR=${TMPDIR:-/var/tmp}
PID=${$}
T1=${TDIR}/X${PID}_1.tmp
T2=${TDIR}/X${PID}_2.tmp

sort -k2,2b startdates > ${T1}
sort -k2,2b enddates > ${T2}
join -j 2 ${T1} ${T2} | sort -k1,1n
STAT=${?}
rm -f ${T1} ${T2}
exit ${STAT}

That should be a complete solution. (If my expert typing is okay.)

Man sort, join for details.

Clay

[Steven Mertens]

hi ,

what about :
(you can put the output in a file)

#!/bin/sh

startDateFile=./startdates
endDateFile=./enddates

while read -e startDate idNr
do
while read -e endDate idNr2
do
if [ "${idNr}" = "${idNr2}" ]
then
echo "${idNr} ${startDate} ${endDate}"
fi
done < ${endDateFile}
done < ${startDateFile}

regards,

Steven

[Leif Halvarsson_2]

Hi
There is a (perhaps not so known) command in HP-UX called "join" which is a kind of "relational operator". It matches fields in two files. The files need to be sorted on the matching fields.

In your case the "basic" of your script should look somthing like.

join -j1 3 -j2 1 -o 1.1 2.2 2.1 startfile endfile

Which means it matches field 3 of startfile with field 1 of endfile and output field 1 of startfile and field 2 and 1 of endfile.

Have a look at man join.

[Derek Card]

Thanks!

Clay, I copied and pasted your script and it works but I just tried the script that Leif posted. It works too and is simpler.
I think I will use Leif's.

Thanks again, Derek Card

[A. Clay Stephenson]

Well Derek, I considered the simple join as well BUT I ruled it out for these reasons:

Join requires data sorted in LEXICAL order (not numeric) and while your data files appear to be sorted, I couldn't know for sure.

You probably want the data out in numeric order. That's the reason for the final sort using a -n key otherwise the order might be 1,10,100,1000,2,20,200,2000 rather than 1,2,10,20,1000,2000.

If you KNOW that you incoming data is lexicographically sorted the the 'one-liner' will work. I'm one of those 'better safe than sorry' type guys.

I'm not saying that Leif's solution was in any sense wrong.


Food for thought, Clay

[Derek Card]

Hi Clay,

Okay, you convinced me. I could follow most of your script but I don't understand what TDIR=${TMPDIR:-/var/tmp} is supposed to do. I know you are making temporary files. I commented the rm statement out and the temporary files are created in /var/tmp.

Thanks for your help.

Derek Card

[A. Clay Stephenson]

I could have simply used:
TDIR=/var/tmp
but instead, I used the conditional assignment:
TDIR=${TMPDIR:-/var/tmp}

If the environment variable TMPDIR is defined then it's value is used otherwise /var/tmp is used. TMPDIR is a standard variable used by commands and many programs to define the temp directory name.

Man sh-posix for details.

Regards, Clay