Print a specific line of a large text ascii file

[Patrick Chim]

Hi,

Is there any QUICK method in command line to print a specific line of a large ascii file (over 4M rows) ?

Regards,
Patrick

[Steven Sim Kok Leong]

Hi,

1) With the line number:

# cat -n $FILE | cut -c5- | grep "^$LINENO "

2) Without the line number but with the specific contents:

# grep $PATTERN $FILE

To identify the line number after from the grep:

# grep -n $PATTERN $FILE

Hope this helps. Regards.

Steven Sim Kok Leong

[Steven Sim Kok Leong]

Hi,

1) With the line number, you can also do this:

# head -$LINENO $FILE | tail -1

Hope this helps. Regards.

Steven Sim Kok Leong

[Ralph Grothe]

e.g. print line #1325

sed -n '1325p' /your/large/file

e.g. (with a not really big file)

# cat -n /etc/services |sed -n '123p'
123 hacl-hb 5300/tcp # High Availability (HA) Cluster he
artbeat

[Ralph Grothe]

Of course we should kill the old beasts sed, awk etc. in favour of Perl ;-)

# perl -ne 'print if $.==123' /etc/services
hacl-hb 5300/tcp # High Availability (HA) Cluster heartbeat

[Peter Kloetgen]

Hi Patrick,

if like the "old beasts".... Here is a solution with awk:

awk 'NR=1234, NR=1234' input_file

This will only print the line with number 1234.

Allways stay on the bright side of life!

Peter

[Bart Beeren]

Yes there is. Just by usage of more:

more +/pattern file #Starts 2 lines above search string

more +linenumber file #Starts 2 lines above desired linenumber

See for more details/options --> man more

BB

[Peter Kloetgen]

Hi Patrick,

if you need to print only one line with a specific pattern in it:

awk '/your_pattern/, /your_pattern/' input_file

Or what exactly do you want?

Allways stay on the bright side of life!

Peter

[Juan González]

Hi,
don't forget another infamous beast:

$ echo "100p"|ed - my_file

Best regards
Juan

[harry d brown jr]

Patrick,

I noticed no one got the bunny, is there something else you are looking for?

live free or die
harry

[V. V. Ravi Kumar_1]

hi,
to get a specific line either u should know some pattern in the line or line number. but better go by line number because pattern may match with some other lines also.
by line number 23
#cat -n |sed -n '23p'
if u want to print
#cat -n |sed -n '23p'|lp
it goes to default printer.

regds

[H.Merijn Brand (procura]

Patrick, Ralph. '$. == 123' can better/more flexible be written with a less known feature:

# perl -ne '123 .. 256 && print'

to print lines 123 through 256

[Patrick Chim]

Hi Harry,

The reason why no one get the full point is that I have tried all the methods above before I post this question. I only want to seek anyone's help whether they have any other QUICK method to do that as all the methods above take a number of seconds to process.

Anyway, I thank all of the above experts to share their ideas.

Many thanks and regards,
Patrick

[Frank Slootweg]

If you are looking for a "QUICK method" with "quick" meaning a short *execution* time (instead of a *simple* method), then you are looking for the impossible!

For an unstructured ASCII text file, i.e. a file with lines of *variable* length records/lines, you will have to read the file *sequentially* from the beginning to the desired line, i.e. it will be 'fast' for low line numbers and 'slow' for high line numbers.

Just *reading* a file of "4M rows" (How big, i.e. in MBytes, is the file?) will already take several seconds.

[Rodney Hills]

If this is a file that is accessed a number of times, then maybe creating a second file that is an index to the text file would be helpful.

That way if someone asks for a specific line in the text file, you could develop a program that could use the "seek" i/o function to position directly to that spot and begin reading. In this way you would only read through the entire file once to build the index.

I did this once a few years ago, but I can't find the program I developed.

Sometimes there is a trade off in your time developing a solution versuses the payback some other user will get.

Hope that Helps...

-- Rod Hills

[H.Merijn Brand (procura]

Rodney, nice idea. For small files this will - of course - be slower than just to access the file directly, but attached set of (perl) scripts might mimic what you mean


a5:/tmp 138 > pr -t -n3 xx.c
1 #include
2 #define FARTHING 0.25
3
4 int main ()
5 {
6 double farthing = FARTHING;
7 double nutherfarthing = 0.25;
8 printf ("%g\n", farthing);
9 printf ("%g\n", nutherfarthing);
10 return 0;
11 } /* main */
a5:/tmp 139 > perl mk_index.pl xx.c
a5:/tmp 140 > perl ls_index.pl xx.c
Line Offset
------- ----------
1 0
2 19
3 41
4 42
5 54
6 56
7 94
8 128
9 159
10 196
11 210
a5:/tmp 141 > perl rd_index.pl xx.c 4 2 7
int main ()
#define FARTHING 0.25
double nutherfarthing = 0.25;
a5:/tmp 142 >

[Rodney Hills]

Procura,

That's the right idea, but I think the index can get quite large if their are a lot of lines.

I was thinking of an index that had the block number (block being user defined, ie 4096 bytes) as the grouping and index value would be the line number that that blocks begins with. Then you could begin reading there until you reach the desired line number.

eg
block ... line#
0 1
1 359
2 782
3 1202
...

This way if you wanted line 837, you would seek to beginning of block 2 and read until you got to 837.

Since line numbers don't necessarily cross evenly over the fixed block size, if you wanted line 782, you would have to start at block 1 and count up.

-- Rod Hills

[H.Merijn Brand (procura]

All depends on the tradeof speed-disk

Another point in my/your approch is the support for 64bit integers. If files get indexed past 64bit offsetts, my scripts will only work if both perl and DB_File support those large numbers.

Disk size can be minimized by using pack instead of just storing the plain number, which might also be much faster. If this were production, I'd for sure also make the key (line number) a packed integer, rather than the integer itself, causing a better distribution and all keys of the same length.

Whatever, it was just a quick hack to see if it would work. And it does. It's up to the one to use it how to optimize the process to her/his needs. This would also include the question of how static $big_file is, and if it changes, how fast it could/should be re-indexed.

You can even use a database like MySQL or progress to do it :P

[H.Merijn Brand (procura]

Even faster and memory/disk more efficient:

--8<--- mk_index.pl snippet
open FILE, "< $big_file" or die "$big_file: $!";

my ($pos, $line, @idx) = (0, 1);
tie @idx, "DB_File", $index, O_CREATE|O_TRUNCATE|O_RDWR, 0644, $DB_RECNO;
while () {
$idx[$line++] = pack "L!", $pos;
$pos = tell;
}
-->8---

only if you have a more recent DB_File (with $DB_RECNO support) and native 64bit longs.

I'll leave the changes to the other files up to the reader