topic Re: Combine two text files in Operating System - HP-UX

Combine two text files

Derek Card — Tue, 16 Jul 2002 20:45:11 GMT

Hi,

I have two text files that have a starting date in one file and an ending date in the other file.

File "startdates"
1-Jan-1999
1-Jan-1999
2-Jan-1999

File "enddates"
12-Jun-1999
----------
31-Mar-2002

I would like to combine each line of these two files into 1 line
like this:
1-Jan-1999 12-Jun-1999
1-Jan-1999 ----------
2-Jan-1999 31-Mar-2002

The actual file has thousands of entries. I am fairly new to UNIX. Is there a command to easily do this?

TIA, Derek Card

Re: Combine two text files

A. Clay Stephenson — Tue, 16 Jul 2002 20:46:20 GMT

Derek, this is about as easy as it gets.

paste startdates enddates > bothdates

Man paste for details.

Re: Combine two text files

Dave Chamberlin — Tue, 16 Jul 2002 20:46:55 GMT

There is - it is the paste command. Ex paste file1 file2. Do a man paste for options.

Re: Combine two text files

MANOJ SRIVASTAVA — Tue, 16 Jul 2002 20:57:11 GMT

Hi Derek

Use paste startdates enddates > test to get the result .

Manoj Srivastava

Re: Combine two text files

Ian Kidd_1 — Tue, 16 Jul 2002 21:36:59 GMT

Another option is pr -tm startdates enddates > bothdates.

The only advantage this has over paste is some formatting (although there's no advantage with your example.

But if the lengths of the lines in startdates varies, for example:

10-January-1999
1-March-1999
2-May-1999

then the output of paste would be:
No date has been specified 22-Jan-1999
10-January-1999 12-Jun-1999
1-March-1999 ----------
2-May-1999 31-Mar-2002

(I added 22-Jan-1999 to make the number of items/file equal)

with pr -tm
No date has been specified 22-Jan-1999
10-January-1999 12-Jun-1999
1-March-1999 ----------
2-May-1999 31-Mar-2002

Re: Combine two text files

Ian Kidd_1 — Tue, 16 Jul 2002 21:41:24 GMT

I was afraid that my extra white spaces would vanish in my reply above. The attachment I included should show the correct view (I hope)

Re: Combine two text files

Derek Card — Wed, 17 Jul 2002 17:14:42 GMT

Hello again,

Thank you for all your replies. I'm afraid that I didn't quite get enough information and now I'm really stuck. These two date files have an additional entry called ID Number. I really need
to combine the Id Number, the start dates, and the end dates into 1 line. I don't even know where to begin.

File "startdates"
1-Jan-1999 1009
1-Jan-1999 1010
2-Jan-1999 1011
5-Jan-1999 1012

File "enddates"
12-Jun-1999 1009
---------- 1010
31-Mar-2002 1011

1009 1-Jan-1999 12-Jun-1999
1010 1-Jan-1999 ----------
1011 2-Jan-1999 31-Mar-2002

If there is no matching ID Number, no output is needed.
I've been looking at awk. Is that what I need to use?

Thanks, Derek Card

Re: Combine two text files

A. Clay Stephenson — Wed, 17 Jul 2002 17:20:11 GMT

When I saw this post yesterday I thought this was a very hokey data model but now it makes a bit more sense. You could use awk (or Perl) for this but today I think a little sort and join will do the trick.

#!/usr/bin/sh

TDIR=${TMPDIR:-/var/tmp}
PID=${$}
T1=${TDIR}/X${PID}_1.tmp
T2=${TDIR}/X${PID}_2.tmp

sort -k2,2b startdates > ${T1}
sort -k2,2b enddates > ${T2}
join -j 2 ${T1} ${T2} | sort -k1,1n
STAT=${?}
rm -f ${T1} ${T2}
exit ${STAT}

That should be a complete solution. (If my expert typing is okay.)

Man sort, join for details.

Clay

Re: Combine two text files

Steven Mertens — Wed, 17 Jul 2002 17:33:56 GMT

hi ,

what about :
(you can put the output in a file)

#!/bin/sh

startDateFile=./startdates
endDateFile=./enddates

while read -e startDate idNr
do
while read -e endDate idNr2
do
if [ "${idNr}" = "${idNr2}" ]
then
echo "${idNr} ${startDate} ${endDate}"
fi
done < ${endDateFile}
done < ${startDateFile}

regards,

Steven

Re: Combine two text files

Leif Halvarsson_2 — Wed, 17 Jul 2002 17:39:46 GMT

Hi
There is a (perhaps not so known) command in HP-UX called "join" which is a kind of "relational operator". It matches fields in two files. The files need to be sorted on the matching fields.

In your case the "basic" of your script should look somthing like.

join -j1 3 -j2 1 -o 1.1 2.2 2.1 startfile endfile

Which means it matches field 3 of startfile with field 1 of endfile and output field 1 of startfile and field 2 and 1 of endfile.

Have a look at man join.

Re: Combine two text files

Derek Card — Wed, 17 Jul 2002 17:44:21 GMT

Thanks!

Clay, I copied and pasted your script and it works but I just tried the script that Leif posted. It works too and is simpler.
I think I will use Leif's.

Thanks again, Derek Card

Re: Combine two text files

A. Clay Stephenson — Wed, 17 Jul 2002 17:53:29 GMT

Well Derek, I considered the simple join as well BUT I ruled it out for these reasons:

Join requires data sorted in LEXICAL order (not numeric) and while your data files appear to be sorted, I couldn't know for sure.

You probably want the data out in numeric order. That's the reason for the final sort using a -n key otherwise the order might be 1,10,100,1000,2,20,200,2000 rather than 1,2,10,20,1000,2000.

If you KNOW that you incoming data is lexicographically sorted the the 'one-liner' will work. I'm one of those 'better safe than sorry' type guys.

I'm not saying that Leif's solution was in any sense wrong.

Food for thought, Clay

Re: Combine two text files

Derek Card — Wed, 17 Jul 2002 18:12:18 GMT

Hi Clay,

Okay, you convinced me. I could follow most of your script but I don't understand what TDIR=${TMPDIR:-/var/tmp} is supposed to do. I know you are making temporary files. I commented the rm statement out and the temporary files are created in /var/tmp.

Thanks for your help.

Derek Card

Re: Combine two text files

A. Clay Stephenson — Wed, 17 Jul 2002 18:16:17 GMT

I could have simply used:
TDIR=/var/tmp
but instead, I used the conditional assignment:
TDIR=${TMPDIR:-/var/tmp}

If the environment variable TMPDIR is defined then it's value is used otherwise /var/tmp is used. TMPDIR is a standard variable used by commands and many programs to define the temp directory name.

Man sh-posix for details.

Regards, Clay