UNIX scripting & Storage methods

[u856100]

People,

I am trying to store the occurence of a particular field in a file into something like a vector. I want to store each distinct occurence of the value, and then search the vector to do a count of the number of entries.

for example, I have a load of files that have entries like the following :

|header file|
|1|a|b|c|d|
|1|a|b|c|
|2|a|b|c|d|e|
|1|a|b|
|footer|2|

where the value "2" in the footer file is the distinct number of 'numeric types' that exist in the first pipe delimited field of each row. So if the file was as follows :

|header file|
|1|a|b|c|d|
|2|a|b|c|
|3|a|b|c|d|e|
|1|a|b|
|footer|3|

The footer shows 3 types (hope this makes sense).

what I would like to do is store each numeric type and then, as I progress through the file rows, increment a count each time a new distinct value occurs. I will need this value to stick in a file at the end of the file traversal.

I am not too familiar with UNIX scripting as I am a java programmer (and hence very familiar with vectors, etc), so I don't even know whether vectors exist in posix shell scripting.

thanks in advance people
John

[Bill McNAMARA_1]

it may be useful if you post up what you'd like the result file to look like.

[u856100]

Hi Bill,

Thanks for the reply, I am not too concerned with what the output would be like, just obtaining the value and being able to create this temporary storage area containing distinct values that I can count would be ideal. If it helps, just writing to any file with the distinct number count would be useful.

What I will do for the end result, is overwrite an erroneous value ie.

the wrong file footer (Only two distinct value (1 & 2) but the footer says 3):

|header|
|1|a|b
|1|a|
|2|a|b|c|d|
|footer|3|

would be overwritten using awk, to :

|header|
|1|a|b
|1|a|
|2|a|b|c|d|
|footer|2|

Which, I can manage OK

thanks again
John

[harry d brown jr]

Why not do it in java? perl and awk can also do this.

live free or die
harry

[u856100]

Heelo again Bill,

Could you give me an example of how awk does this.

I have not done it in Java because the function I am trying to create is a smaller part of a much bigger script that uses a korn script. I am just trying to keep things simple and neat.

thanks again
John

[Jean-Louis Phelix · Accepted solution]

Hi,

Try this script :

cat file | awk -F'|' '
/^|header\|$/{
getline
print
getline
nb=0
max=0
while ($2 != "footer")
{
print
if (a[$2] != 1)
{
a[$2]=1
nb++
if (nb > max)
max=nb
}
getline
}
printf("|footer|%d|\n", nb)
for (i=0;i<=max;i++)
a[i]=0
}'

If input file is :

|header|
|1|a|b
|1|a|
|2|a|b|c|d|
|footer|3|
|header|
|1|a|b
|3|a|
|2|a|b|c|d|
|footer|3|

Output file will be :

|header|
|1|a|b
|1|a|
|2|a|b|c|d|
|footer|2|
|header|
|1|a|b
|3|a|
|2|a|b|c|d|
|footer|3|