topic Re: sort command in Operating System - HP-UX

sort command

Babak_4 — Mon, 05 Jul 2004 10:41:40 GMT

Hi,

here is my file's content:
$ cat gg
9|4|1|1118|-7.79|607|K7K|6B3|3.309
9|4|1|1117|7.49|724|N1K|1R7|4.981

and here is my sort:
$ sort -n -t'|' -k 1,5 -k 9 gg
9|4|1|1118|-7.79|607|K7K|6B3|3.309
9|4|1|1117|7.49|724|N1K|1R7|4.981

As you see the second line should come first, since its fourth field is less than the first
record's.

But when I remove -k 9 it works fine:
$ sort -n -t'|' -k 1,5 gg
9|4|1|1117|7.49|724|N1K|1R7|4.981
9|4|1|1118|-7.79|607|K7K|6B3|3.309

Am I doing a silly mistake here? or ...

I appreciate your help.

Babak

Re: sort command

Bharat Katkar — Mon, 05 Jul 2004 11:11:31 GMT

Babak,
from man of "sort"
" When there are multiple sort keys, later keys are compared only after all earlier keys compare equal.
Lines that otherwise compare equal are ordered with all bytes significant.
If all the specified keys compare equal, the entire record is used as the final key. "

Hope that addresses your issue.
Regards,

Re: sort command

Babak_4 — Mon, 05 Jul 2004 12:12:25 GMT

Hi Bharat,

thanks for it, but still I'm confeused. the first key -k 1,5 says sort on the first five fields, so the second record should come first,(as it does when -k 9 is removed), or mein understanding is wrong?

Babak

Re: sort command

Bharat Katkar — Mon, 05 Jul 2004 14:38:28 GMT

Hi Babak,
Compare following two Outputs:

1. $ sort -n -t'|' -k 1,9 -k 9 gg
2. $ sort -n -t'|' -k 1,9 gg

I hope both should be same.

Regards,

p.s. I have counted the PIPE also.

Re: sort command

Babak_4 — Mon, 05 Jul 2004 15:04:55 GMT

Sorry, I can't get your point. Anyway I tried yours, and they are not same!
Something that I found is if I remove -n from "sort -n -t'|' -k 1,5 -k 9 gg" it works!

Regards,

Babak

Re: sort command

Babak_4 — Tue, 06 Jul 2004 08:53:56 GMT

I guess sort has a problem with multiple -k. Is it a known issue and is there any patch for it?

Babak

Re: sort command

Muthukumar_5 — Tue, 06 Jul 2004 10:38:24 GMT

Hai,

Bharath gave the key point from sort man page. If we are using multiple key value on the sort command, it will process key 1 and to key n.

>>>>>>>>>>>>>>>>>>>>>>>>>>>
When there are
multiple sort keys, later keys are compared only
after all earlier keys compare equal. Lines that
otherwise compare equal are ordered with all bytes
significant

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

From your example,
sort -n -t'|' -k 1,5 -k 9 gg

It will be processed as like,

sort -n -t'|' -k 1,5 gg | sort -n -t'|' -k 9

The output is also descibing that,
9|4|1|1118|-7.79|607|K7K|6B3|3.309
9|4|1|1117|7.49|724|N1K|1R7|4.981

Look at the ninth field, of 3.309 and 4.981 they are sorted as your requirement.

The contents of your example strings are sorted with key 1,5 fields in the begin and with key 9 in the end.

The key's we are using to sort must be significant.

If you test the example as like,

sort -n -t'|' -k 9 -k 1,5 gg

The above details will be changed,
It must be
9|4|1|1117|7.49|724|N1K|1R7|4.981
9|4|1|1118|-7.79|607|K7K|6B3|3.309

It will not come. sort call is working as stated in the man page.

Regards,
Muthukumar.

Re: sort command

Babak_4 — Tue, 06 Jul 2004 11:51:00 GMT

Hi,

please bear me for more questions:

But what about the fourth field? The second record has less value (1117) than the first record (1118). The fourth field should be considered before the ninth one.

Doesn't "sort -n -t'|' -k 1,5 -k 9 gg" mean that first 5 fields should be considered first and only if they are equal then consider the ninth field?

"sort -n -t'|' -k 1,5 gg | sort -n -t'|' -k 9" is not and shouldn't be as same as "sort -n -t'|' -k 1,5 -k 9 gg"

my intention of "sort -n -t'|' -k 1,5 -k 9 gg" was to sort the file based on the combination of six fields: 1 to 5 and 9.

Please correct me if I'm misunderstanding the man page.

Thanks,

Babak

Re: sort command

Rodney Hills — Tue, 06 Jul 2004 11:51:30 GMT

If you sort on -k 4,5 -k 9 then it works.

If you change your data line #2 position 1 to an "8", then the sort works.

If you seperate the keys out to -k 1,1 -k 2,2 -k 3,3 -k 4,4 -k 5,5 -k 9,9, then it works.

It is acting like the key "range" -k 1,5 is thinks it is a duplicate key field when only the first field position is duplicated.

It looks like a bug to me...

-- Rod Hills

Re: sort command

Babak_4 — Wed, 07 Jul 2004 12:45:25 GMT

It would be very helpful is someone from HP confirm this.

Thanks,

Babak

Re: sort command

Fred Ruffet — Wed, 07 Jul 2004 13:09:21 GMT

I have here syncsort as a replacement for sort on HP server (faster, better, more expensive :).

I have exactly the same sort with this second program. I can't imagine the similar bug on two so used programs.

What I understand in man is that specifying multiple keys of multiple fields will compare second fields of each keys only if first fileds of each keys are equal. And so on for second fields, third...

An exapmle for I say with your example :
add a -k 4 in your sort, and it will do.

regards,

Fred

Re: sort command

John Kittel — Wed, 07 Jul 2004 13:16:14 GMT

I also tried it with a linux system and a true-64 system, and it works behaves exactly the same way on them. So maybe it is not a bug. I too am interested in hearing from some authoritative source if it IS a bug or not.

Re: sort command

Fred Ruffet — Wed, 07 Jul 2004 13:30:59 GMT

On a Sun OS 5.6,

with sort :
$/usr/bin/sort -n -t'|' -k 1,5 -k 9 sort_file
9|4|1|1118|-7.79|607|K7K|6B3|3.309
9|4|1|1117|7.49|724|N1K|1R7|4.981

with syncsort :
$sort -n -t'|' -k 1,5 -k 9 sort_file 9|4|1|1118|-7.79|607|K7K|6B3|3.309
9|4|1|1117|7.49|724|N1K|1R7|4.981

It acts the same again. Confort me in my opinion : it's not a bug, it's a feature :)

Regards,

Fred

Re: sort command

Tim D Fulford — Wed, 07 Jul 2004 13:37:45 GMT

It look weird, but you will ned to EXPLICITY state the order one by one It look like the implied 1,5 does not wotk!!

Try
sort -n -t'|' -k 1,5 -k 9 gg
is the same as
sort -n -t'|' -k 9 -k 1,5 gg

what you want is
sort -n -t'|' -k1 -k2 -k3 -k4 -k5 -k9 gg
will be different to
sort -n -t'|' -k9 -k5 -k4 -k3 -k2 -k1 gg

regards

Tim

Re: sort command

Babak_4 — Wed, 07 Jul 2004 13:42:21 GMT

Fred,

thanks for response. But I tried it with syncsort, it works fine. Can you post your syntax? I want to be sure that syncsort does not behave like unix sort.

Regarding adding -k 4, as Rod said it works if you specify each field separately, but it's not the way that man page says:

...
Restricted Sort Key
-k keydef The keydef argument defines a restricted sort key.The format of this definition is
field_start[type][,field_end[type]]
which defines a key field beginning at field_start and ending at field_end.
...
Multiple -k options are permitted and are significant in command line order. A maximum of 9 -k options can be given.
...

Besides, we've got restriction on number of -k (max. 9).

Thanks,

Babak

Re: sort command

Fred Ruffet — Wed, 07 Jul 2004 13:46:08 GMT

I insist :
Problem is not that 1,5 is not working. Sort is done on first field of each key (1 and 9) then on second field of each key (only 2 because 2nd key do not have other field) and so on. So the sort you ask is equivalent to
-k 1 -k 9 -k 2 -k 3 -k 4 -k 5
whereas what you want is
-k 1,5 -k 4 -k 9

Regards,

Fred

Re: sort command

Fred Ruffet — Wed, 07 Jul 2004 13:51:20 GMT

Babak,

Didn't saw your answer before posting...
OK, but note that, as somebody extract it before from man :
"When there are multiple sort keys, later keys are compared only after all earlier keys compare equal. Lines that otherwise compare equal are ordered with all bytes significant"

syncsort syntax is in my precedent post, the same as sort. I am using sort shell script in syncsort bin directory, and not directly syncsort program.

Regards,

Fred

Re: sort command

Babak_4 — Wed, 07 Jul 2004 14:03:50 GMT

Fred,

let me emphasis that "-k 1,5" means "-k 1 -k 2 -k 3 -4 and -k 5". As I mentioned in my earlier reply if you remove -k 9 from my command it works, meaning the value of the fourth field is considered, whereas with -k 9 it doesn't.

Thanks,

Babak

Re: sort command

Rodney Hills — Wed, 07 Jul 2004 15:15:03 GMT

Babak,

I read from "man sort" that-

sort -k 1,5
and
sort -k 1,1 -k 2,2 -k 3,3 -k 4,4 -k 5,5

should be equivalent, but in my previous tests it was not...

Even though it is behaving the same on a Linux system, I would say that HPUX and Linux might share the same source code for a number of utilities, and thus share some of the same bugs.

Whether you call this a bug or an undocumented feature, it still is frustrating when a command doesn't do what you expect it to do.

my 2 cents.

-- Rod Hills

Re: sort command

Fred Ruffet — Thu, 08 Jul 2004 07:05:20 GMT

IMHO "-k 1,5" is not equivalent to "-k 1 -k 2 -k 3 -k 4 -k 5".

The first one sorts on a key wich is a concatenation of the first 5 fields. For the second, the 5 first fields have the same importance.

That's why I say that in your original sort, 1 and 9 are sorted first, then 2, then 3...

...and that's why removing 9 works the way you think it (1 2 3 4 and 5 are already distincts).

Regards,

Fred