how to keep lines with certain pattern and delete all the others

[patrick xi]

I have a very huge file which have thousands of lines with same length.
I want to create a new file containing only lines with pattern '01' or '02' at 31 and 32 column.
hope you could help me, thank you for taking care of this message.

[Ronelle van Niekerk]

If you want column 31 to be 01 and column 32 to be 02 then try this:

cat $file | awk '{FS=":"; if ($31 == "01" && $32 == "02") { print $0}}' >> $new_file

replace the : with whatever your Field Seperator is. If it is blanks then leave out the whole FS piece up to the ;

Else, if you want column 31 and 32 to be either 01 or 02 then try this:

cat $file | awk '{FS=":"; if ($31 == "01 || $31 == "02") { if ($32 == "01" || $32 == "02") {print $0}}' >> $new_file

Again, replace the : with your field seperator.

-Ronelle

[Ronelle van Niekerk]

Sorry, that last command should read:

cat $file | awk '{if (($31 == "01" || $31 == "02") && ($32 == "01" || $32 == "02")) {print $0}}' >> $new_file

Much simpler.

[Rodney Hills · Accepted solution]

Probabily the most effecient way would be to use sed, like-

sed -n -e '/^.\{30\}0[12]/p' largefile >only3x

Here is a break down of the regular expression (what's between '/' and the next '/')-

^ matches beginning of string
.\{30\} matches the first 30 anything
0 matches a 0 in column 31
[12] matches either a 1 or 2 in column 32

The -n option prevents sed from outputing the lines unless explicitly stated (ie "p")

You could use "awk" by using the "substr" function (ie substr($0,30,2)=="01"). The default field seperator is blank/tab, but you can look at the whole line with $0.

HTH

-- Rod Hills

[patrick xi]

hi Rodney,
you've solved my problem. The script is just what I need.

and for Ronelle,
sorry, there are no separators between columns. each charactor is one column.

thank you all for taking the time.

[patrick xi]

problem solved, please close it.