sorting files

[Gyankr]

Hello,

I want to sort below files in a particular order

currently it is like below:
>ls
fancy_LANG_STD_AU_2008-03-05.dat
fancy_LANG_STD_HK_2008-03-06.dat
fancy_LANG_STD_NZ_2008-03-05.dat
fancy_STD_AU_2008-03-05.dat
fancy_STD_HK_2008-03-06.dat
fancy_STD_NZ_2008-03-05.dat




i want to sort it like this

fancy_STD_AU_2008-03-05.dat
fancy_LANG_STD_AU_2008-03-05.dat
fancy_STD_HK_2008-03-06.dat
fancy_LANG_STD_HK_2008-03-06.dat
fancy_STD_NZ_2008-03-05.dat
fancy_LANG_STD_HK_2008-03-06.dat

i.e i want the standard files for a country first ( fancy_STD_HK...) and then its corresponding language files(fancy_STD_LANG_HK_...).

Please let me know how it can be done

Regards,
Gyan

[Dennis Handly]

You'll need to break up the filenames using awk then use sort:
ls | awk -F_ '
{
if ($2 == "LANG") {
# combine fields
country = $4
lang = "STDLANG"
} else {
country = $3
lang = $2
}
print country, lang, $1, $0
}' | sort | awk '{print $4}'

Did you want to also sort by the first "fancy" or the date?

[Aashique]

Hi,
first create a file with all your country code
lets say test1

vi test1
-----------
AU
HK
NZ

then write another sctipt name "test2"

vi test2
--------
set `cat test1`
for i in $*
do
grep $i test|sort -rn>>test3

$ more test3
fancy_STD_AU_2008-03-05.dat
fancy_LANG_STD_AU_2008-03-05.dat
fancy_STD_HK_2008-03-06.dat
fancy_LANG_STD_HK_2008-03-06.dat
fancy_STD_NZ_2008-03-05.dat
fancy_LANG_STD_NZ_2008-03-05.dat


Thanks & Regards

Aashique

[Aashique]

Hi,
i missed to write something.in the "test" file

vi test
----------
fancy_LANG_STD_AU_2008-03-05.dat
fancy_LANG_STD_HK_2008-03-06.dat
fancy_LANG_STD_NZ_2008-03-05.dat
fancy_STD_AU_2008-03-05.dat
fancy_STD_HK_2008-03-06.dat
fancy_STD_NZ_2008-03-05.dat

[Hein van den Heuvel]

At first is seems you can just sort specifying -t_ to make underscore be a field terminator. But the field number also changes.

So you need a sort with a twist.
Here is a perl solution:

$ cat x
fancy_LANG_STD_AU_2008-03-05.dat
fancy_LANG_STD_HK_2008-03-06.dat
fancy_LANG_STD_NZ_2008-03-05.dat
fancy_STD_AU_2008-03-05.dat
fancy_STD_HK_2008-03-06.dat
fancy_STD_NZ_2008-03-05.dat

$ perl -e 'sub twist {$x=@_[0]; $x=~s/_LANG(_STD_..)_/${1}x/; return $x}; print sort {twist($a) cmp twist($b)} <>' x
fancy_STD_AU_2008-03-05.dat
fancy_LANG_STD_AU_2008-03-05.dat
fancy_STD_HK_2008-03-06.dat
fancy_LANG_STD_HK_2008-03-06.dat
fancy_STD_NZ_2008-03-05.dat
fancy_LANG_STD_NZ_2008-03-05.dat

Just to clarify, here is that twist function alone at work:

$ perl -e 'sub twist {$x=@_[0]; $x=~s/_LANG(_STD_..)_/${1}x/; return $x}; foreach (<>) {print twist($_)}' x
fancy_STD_AUx2008-03-05.dat
fancy_STD_HKx2008-03-06.dat
fancy_STD_NZx2008-03-05.dat
fancy_STD_AU_2008-03-05.dat
fancy_STD_HK_2008-03-06.dat
fancy_STD_NZ_2008-03-05.dat

fwiw,
Hein.

[Gyankr]

Hi again,
I am not so familiar with perl.
I am trying something like this
local filelist="$@"
where filelist contains all the list of files passed from someother file.

Now how do i sort it,i tried Dennis' solution
local filelist="$@"
local filesortedlist=`echo $filelist | awk -F_ '
{
if ($2 == "LANG") {
# combine fields
country = $4
lang = "STDLANG"
} else {
country = $3
lang = $2
}
print country, lang, $1, $0
}' | sort | awk '{print $4}'`;

but filesortedlist contains only the first of the sorted file(i.e fancy_STD_AU_2008-03-05.dat).
I want all the sorted files into a variable(filesortedlist).


Regards,
Gyan

[James R. Ferguson]

Hi:

If you want to use Dennis' solution, you began with:

> local filelist="$@"
> local filesortedlist=`echo $filelist | awk -F_ '

This would work if you changed the 'echo' to 'cat' in order to read the file represented by 'filelist'. You don't need to create a separate process with 'cat' to pipe the output to 'awk'. That is wasteful!

Using Dennis's solution and your surrounding shell logic (as written):

#!/usr/bin/sh
local filelist="$@"
local filesortedlist=`awk -F_ '
{
if ($2 == "LANG") {
# combine fields
country = $4
lang = "STDLANG"
} else {
country = $3
lang = $2
}
print country, lang, $1, $0
}' $filelist | sort | awk '{print $4}'` ;
echo ${filesortedlist}

...That is, 'awk' takes the first argument supplied to it as a FILE to be opened and read. The output of the 'awk' process is then piped to a 'sort'.

Perl makes this process-efficient.

Regards!

...JRF...

[Hein van den Heuvel · Accepted solution]

well, this seams a perfect excuse te learn perl to me!

If you do, be sure to think outside the immediate question and ask yourself how the input came to be, and what is going to be done with the output.
You may well find that those steps are in fact better done in perl as well.

Anyway... the awk solution failed because all data line were really words in a string.
So you'd have to split those first.

This calls for a minor variation on Dennis's suggestion:

Too bad hpux does not have an asorti function like 'gawk'

$ cat x
fancy_LANG_STD_AU_2008-03-05.dat
fancy_LANG_STD_HK_2008-03-06.dat
fancy_LANG_STD_NZ_2008-03-05.dat
fancy_STD_AU_2008-03-05.dat
fancy_STD_HK_2008-03-06.dat
fancy_STD_NZ_2008-03-05.dat
$ local filelist=$(cat x)
$ echo $filelist
...
$ echo $filelist | awk '{while (++ifancy_STD_AU_2008-03-05.dat
fancy_LANG_STD_AU_2008-03-05.dat
fancy_STD_HK_2008-03-06.dat
fancy_LANG_STD_HK_2008-03-06.dat
fancy_STD_NZ_2008-03-05.dat
fancy_LANG_STD_NZ_2008-03-05.dat


core:
while (++isplit($i,x,"_") # each field is split into array x by "_"
y=(x[2]=="LANG")?4:3 # is sub-field 2 = LANG?
# then language is in 4, not 3.
print x[y],y,$i} # print language,helper,file ready for sorting

[Gyankr]

Thanks everybody.I guess its time for me to start cramping perl

Regards,
Gyan

[Gyankr]

Hi everybody,

Just when things were going fine,i got a big hiccup, the filelist has more than 100 files and when i use the below code

local flist="$@"
local filelist=`echo $flist | awk '{
while (++i<=NF) {
split($i,x,"_");
y=(x[2]=="LANG")?4:3;
print x[y],y,$i}
}' | sort -t"-" +1 +2 | awk '{print $3}'`;


I am getting an error like

: Input line FANCY_LANG_STD_AU_20 cannot be longer than 3,000 bytes.


I tried something like


local filelist=`echo $flist | fold -w | awk '{.... but the last the sorting of the files fails :(


Regards,
Gyan