topic Re: sort assistance in Operating System - OpenVMS

sort assistance

Rich Hearn — Tue, 22 May 2007 07:57:24 GMT

I'm sure I'm missing the obvious, but I haven't been able to see it yet. I'm trying to sort a file in numerical order 0000 - 9999 (nlp devs). Presently, 101 is after 1009 (see below), I'd like to be able to have 0-9, 10-99, 100-999,1000-999 in "normal" decimal order - not computer sort order. What am I failing to understand so I can create the correct collate sequence? I've tried:

/collat=(sequence=("0"-"9","10"-"99","100"-"999","1000"-"9998"))

CACHE1::DISK$INFSYS:[RJHEARN]_>sort/key=(pos:11,size:8)/spec=seq5.dat pq_list.txt pq_list.srt
%SORT-F-SPCIVC, invalid collating sequence specification, at line 7

nlp_dev = NLP1
nlp_dev = NLP10
nlp_dev = NLP100
nlp_dev = NLP1000
nlp_dev = NLP1001
nlp_dev = NLP1002
nlp_dev = NLP1003
nlp_dev = NLP1004
nlp_dev = NLP1005
nlp_dev = NLP1006
nlp_dev = NLP1007
nlp_dev = NLP1008
nlp_dev = NLP1009
nlp_dev = NLP101
nlp_dev = NLP1010

Thanks for thoughts,
Rich

Re: sort assistance

Hein van den Heuvel — Tue, 22 May 2007 09:02:31 GMT

I'm afraid you can not get there from here.
VMS Sort is strictly fixed field driven.

You would have to process the data witha program of dcl script to align, sort, reformat.

How much data is there?
It's a trivial one-line excercise in perl.
For example:

perl -e "@lines=<>; print sort{substr($a,13) <=> substr($b,13)} @lines" x.txt

In slow motion:

perl -e " ! Program on command line
@lines=<>; ! Suck input stream into array lines
print !
sort ! Invoke sort on an array, normally comparing the whole array lines as text
{} ! Sort function to use
here:
{substr($a,13) ! Grab character from column 13 onwards from left
<=> ! Compare as integers (versus cmp)
substr($b,13)} ! other value to compare
@lines" ! The array with input
x.txt ! They data

hth,
Hein.

Re: sort assistance

Rich Hearn — Tue, 22 May 2007 09:36:37 GMT

Hein,

Thank you for the response. Tho' it's not the "preferred" answer, I can, at least, move on and sort the file using DCL (I don't have Perl on our systems) instead of continuing to try to get the /collat= statement correct.

Rich

Re: sort assistance

Hein van den Heuvel — Tue, 22 May 2007 10:19:25 GMT

Here is the core of how how I would do it in DCL:

$open/read in 'p1
$tmp = "sys$login:sort.tmp"
$open/write tmp 'tmp
$loop:
$read/end=sort_it in in_record
$out_record = f$extr(0,13,in_record) + f$fao("!5UL",'f$ext(13,99,in_record)')
$write tmp out_record
$goto loop
$sort_it:
$close in
$close tmp
$sort/key=(pos=13,siz=5) 'tmp 'tmp
$open/read tmp 'tmp
$open/write out 'p1
$loop2:
$read/end=done/err=done tmp in_record
$out_record = f$extr(0,13,in_record) + f$edit(f$ext(13,99,in_record),"trim")
$write out out_record
$goto loop2
$done:
$close tmp
$close out
$delete 'tmp';*

Re: sort assistance

Rich Hearn — Tue, 22 May 2007 10:59:07 GMT

Hein,

Thanks for the example - I'll give it "a whirl" and see how it does. Either way, I appreciate your assistance and willingness to help.

Tnx agn,
Rich

Re: sort assistance

Phil.Howell — Tue, 22 May 2007 19:56:17 GMT

SORT/STAT/KEY=(POS:17,SIZ:1)/KEY=(POS:16,SIZ:2)/KEY=(POS:15,SIZ:3)/KEY=(POS:14,SIZ:4)
Phil

Re: sort assistance

Rich Hearn — Tue, 22 May 2007 20:43:48 GMT

Phil,

I never cease to be amazed by what VMS can do - if you know the magic incantation. This is an "almost" - what that means is I'll have to play with it and see what else I can try - thank your for showing me this train of thought! (see below)

Hein,

I had to "tweak" a few thing in the DCL .com file, but I'm still having a problem with the f$fao statement - it gives me **** instead of NLP device #'s, so I'll be playing with that some more too. I expect when I get it working (I'm an optimist :^) I'll post it for others to reference.

Thank you both for your thoughts & time

Rich

nlp_dev = NLP0 pq_num2 = PQE9 Inum = i27664 Ip_Addr = 10.10.55.108 MAC_Addr = 00:0e:7f:37:d3:97 Type = JetDirect 17xÂ
nlp_dev = NLP1 pq_num2 = PQTS Inum = is0061 Ip_Addr = 10.33.49.106 MAC_Addr = 00:00:C9:03:4E:31 Type = NetQue PS 1 Â
nlp_dev = NLP2 pq_num2 = PQPG Inum = i14052 Ip_Addr = 10.98.17.175 MAC_Addr = 00:00:C9:0A:64:24 Type = NetQue PS 2 Â
nlp_dev = NLP4 pq_num2 = PQZG Inum = i11584 Ip_Addr = 10.130.33.88 MAC_Addr = 00:00:C9:09:0C:B2 Type = NetQue PS 2 Â
nlp_dev = NLP5 pq_num2 = PQ74 Inum = i30941 Ip_Addr = 10.65.49.210 MAC_Addr = 00:11:0a:bf:74:50 Type = JetDirect 17xÂ
nlp_dev = NLP6 pq_num2 = PQ76 Inum = i26817 Ip_Addr = 10.194.80.200 MAC_Addr = 00:11:0a:bb:dc:4c Type = JetDirect 17Â
nlp_dev = NLP7 pq_num2 = PQ72 Inum = i17922 Ip_Addr = 10.67.81.46 MAC_Addr = 00:01:e6:5d:de:75 Type = JetDirect 17x Â
nlp_dev = NLP8 pq_num2 = PQAT Inum = i01604 Ip_Addr = 10.10.27.204 MAC_Addr = 00:00:C9:09:85:C3 Type = NetQue PS 2 Â
nlp_dev = NLP9 pq_num2 = PQB6 Inum = i29829 Ip_Addr = 10.194.80.220 MAC_Addr = 00:11:0a:bf:0d:42 Type = JetDirect 17Â
nlp_dev = NLP10 pq_num2 = PQAV Inum = i01665 Ip_Addr = 10.69.33.171 MAC_Addr = 00:01:e6:a5:21:e2 Type = JetDirect 17Â
nlp_dev = NLP20 pq_num2 = PQSJ Inum = i14741 Ip_Addr = 10.161.17.60 MAC_Addr = 00:01:e6:35:a7:73 Type = JetDirect 17Â
nlp_dev = NLP30 pq_num2 = PQBF Inum = i33009 Ip_Addr = 10.194.32.162 MAC_Addr = 00:11:0a:f1:78:89 Type = JetDirect 1Â
nlp_dev = NLP40 pq_num2 = PQXM Inum = i04835 Ip_Addr = 10.193.1.23 MAC_Addr = 00:00:C9:08:9A:47 Type = NetQue PS 2 Â
nlp_dev = NLP50 pq_num2 = PQQL Inum = i13938 Ip_Addr = 10.10.31.125 MAC_Addr = Ã -::::: Type = Que_Name = PQQL Â
nlp_dev = NLP60 pq_num2 = PQ10 Inum = i30908 Ip_Addr = 10.195.32.244 MAC_Addr = 00:11:0a:bf:f7:30 Type = JetDirect 1Â
nlp_dev = NLP70 pq_num2 = PQ27 Inum = i20384 Ip_Addr = 10.10.38.106 MAC_Addr = 00:01:e6:7a:8c:7b Type = JetDirect 17Â
nlp_dev = NLP80 pq_num2 = PQ08 Inum = i21884 Ip_Addr = 10.194.32.213 MAC_Addr = 00:01:e6:75:b5:ed Type = JetDirect 1Â
nlp_dev = NLP90 pq_num2 = PQ50 Inum = i04703 Ip_Addr = 10.7.38.96 MAC_Addr = 00:00:C9:08:99:B2 Type = NetQue PS 2 Â
nlp_dev = NLP11 pq_num2 = PQAV Inum = i01665 Ip_Addr = 10.69.33.171 MAC_Addr = 00:01:e6:a5:21:e2 Type = JetDirect 17Â
nlp_dev = NLP21 pq_num2 = PQOG Inum = i01008 Ip_Addr = 10.194.64.177 MAC_Addr = 00:11:0a:ba:bf:51 Type = JetDirect 1Â
nlp_dev = NLP31 pq_num2 = PQBH Inum = i20817 Ip_Addr = 10.10.55.102 MAC_Addr = 00:01:e6:4d:2a:62 Type = JetDirect 17Â

Re: sort assistance

Jan van den Ende — Tue, 22 May 2007 20:52:20 GMT

Rich,

>>>
but I'm still having a problem with the f$fao statement - it gives me **** instead
<<<

are you sure you took the correct use of apostrophes from the example?
As far as I can see it seems correct. But that error usually impleis some type conversion issue.
If necessary. take the detour of first doing the string extraction into a symbol, and then usinf the value of that symbol in the F$FAO.

hth

Proost.

Have one on me.

jpe

Re: sort assistance

Rich Hearn — Tue, 22 May 2007 21:23:49 GMT

Jan,

I'll have to spend some more time looking into it tomorrow - the 45 minutes I've spent tonight hasn't yielded any luck, so I'll pick it up in the morning

Thanks for the thought though...
Rich

Re: sort assistance

Hein van den Heuvel — Tue, 22 May 2007 22:41:26 GMT

Phil, that sort does not work for me:

$ SORT/STAT/KEY=(POS:17,SIZ:1)/KEY=(POS:16,SIZ:2)/KEY=(POS:15,SIZ:3)/KEY=(POS:14,SIZ:4) tmp.tmp tt:
nlp_dev = NLP1
nlp_dev = NLP2
nlp_dev = NLP10
nlp_dev = NLP11
nlp_dev = NLP114
nlp_dev = NLP105
nlp_dev = NLP2001
nlp_dev = NLP1002
nlp_dev = NLP2002

Rich,
Did my example at least work with the data example you published?
Your real data (next time, attach as text file to original topic?!) does not 'end' after the number but has 'stuff'.
So the DCL needs to extract not 'the rest', but just up to the first space in the tail.

The sorting subroutine in perl gets more challenging also, needing to 'match' against decimals following NPL up to the next word 'break' (space, or EOL).

Perl:
perl -e "@lines=<>; print sort{$a=~/NLP(\d+)\b/; $x=$1; $b=~/NLP(\d+)\b/; $x<=>$1} @lines" tmp.tmp

DCL:

$close/nolog in
$close/nolog out
$close/nolog tmp
$open/read in 'p1
$tmp = "sys$login:sort.tmp"
$out = p2
$if out.eqs."" then out = p1
$open/write tmp 'tmp
$loop:
$read/end=sort_it in in_record
$tail = f$ext(13,999,in_record)
$space = f$loc(" ",tail)
$out_record = f$extr(0,13,in_record) + -
f$fao("!5UL",'f$extr(0,space,tail)') + -
f$extr(space,999,tail)
$write tmp out_record
$goto loop
$sort_it:
$close in
$close tmp
$sort/key=(pos=14,siz=5) 'tmp 'tmp
$open/read tmp 'tmp
$open/write out 'out
$loop2:
$read/end=done/err=done tmp in_record
$out_record = f$extr(0,13,in_record) + -
f$edit(f$ext(13,5,in_record),"trim") + -
f$extr(13+5,999,in_record)
$write out out_record
$goto loop2
$done:
$close tmp
$close out
$delete 'tmp';*

Re: sort assistance

Rich Hearn — Thu, 24 May 2007 07:31:34 GMT

Hein,

If I understand your question, I did have to make a couple of "adjustments" to the original .com you posted. I didn't post the data file because it's ~600 vms blocks in length. I have zip'd it up and attached it if you're interested. I should have done this originally - lesson learned. Your original .com gave me enough to work with, so I was just going to keep working with it until I had it working correctly (for me).

Since I can't see the previous responses once in this forum editor, I can't tell you which of my responses had the output you asked about. I don't remember which response I gave that had a result of the modified original .com file.

Tnx,
Rich

Re: sort assistance

Hein van den Heuvel — Thu, 24 May 2007 09:31:26 GMT

>> I did have to make a couple of "adjustments" to the original .com you posted.

There is a date header line in the date which needs to be excluded (see below)

>>attached it if you're interested.
The script below works on that data file.

>> Since I can't see the previous responses once in this forum editor,

I always grab a second window to reply (^N in exploder) leaving the original, closing the extra after the acknowledgement.

Hein

$ type tmp.com
$close/nolog in
$close/nolog out
$close/nolog tmp
$open/read in 'p1
$tmp = "sys$login:sort.tmp"
$out = p2
$if out.eqs."" then out = p1
$open/write tmp 'tmp
$read in date_record
$loop:
$read/end=sort_it in in_record
$tail = f$ext(13,999,in_record)
$space = f$loc(" ",tail)
$out_record = f$extr(0,13,in_record) + -
f$fao("!5UL",'f$extr(0,space,tail)') + -
f$extr(space,999,tail)
$write tmp out_record
$goto loop
$sort_it:
$close in
$close tmp
$sort/key=(pos=14,siz=5) 'tmp 'tmp
$open/read tmp 'tmp
$open/write out 'out
$write out date_record
$loop2:
$read/end=done/err=done tmp in_record
$out_record = f$extr(0,13,in_record) + -
f$edit(f$ext(13,5,in_record),"trim") + -
f$extr(13+5,999,in_record)
$write out out_record
$goto loop2
$done:
$close tmp
$close out
$!delete 'tmp';*

Re: sort assistance

Rich Hearn — Thu, 24 May 2007 10:37:39 GMT

Hein,

I can only say "Thank You" for your efforts & patience. Yes, the DCL .com does, indeed, work - quite nice. Now, I'll have a look at what *I* wasn't doing correctly so I'll know for future reference.

Thanks, also, for your thoughts of 2 windows - I'll be doing that from now on.

Thank you again for all your help,
Rich