strip characters from file

[Michael Murphy_2]

Does anyone know of a utility to strip x number of bytes off of the top of a file? The file is binary - the rest of the file needs to be unchanged - there are no "lines", so head/tail do not work - any suggestions?

Mike

[spex]

Hi Mike,

$ dd if=foo of=foo_stripped bs=1 skip=10

will strip the first 10 bytes from foo and save the result to foo_stripped.

PCS

[A. Clay Stephenson]

While this solution,
dd if=foo of=foo_stripped bs=1 skip=10,
will work, it's going to be slow as molasses if the file is anything but tiny in size because the blocksize is so small.

I would break this into 2 pieces; one using the small bs and a second using a larger blocksize.

dd if=infile bs=1 skip=10 count=1014 of=outfile

dd if=infile bs=1k skip=1 >> outfile

This will read the first 1KiB of the file (10 + 1014 bytes) and then switch to a larger blocksize (1Kib) for the remainder of the file.

[spex]

Mike,

Another way:

$ typeset -i SKIP=10
$ tail -c $(( $(cat foo | wc -c)-${SKIP} )) foo > foo_stripped

PCS

[James R. Ferguson]

Hi Mike:

This Perl script will strip the first 10-bytes, updating "inplace" the file specified as an argument to the script:

# cat ./strip
#!/usr/bin/perl
use strict;
use warnings;
my $file = shift or die "Usage: file\n";
open( FH, "+<:raw", $file ) or die "Can't open $file: $!\n";
local $/ = undef;
$_ = ;
seek( FH, 0, 0 );
truncate( FH, 0 );
s/.{10}//s; #...strip 10-bytes...
print FH;
close FH;
1;

...run as:

# ./strip myfile

Regards!

...JRF...

[Hein van den Heuvel]

Spex>> $ dd if=foo of=foo_stripped bs=1 skip=10

A minor improvement might be to use:
dd if=foo of=foo_stripped bs=10 skip=1
... but some dd implementations are not happy with partial blocks at the end.

JRF,
I think that'll work on HPUX, which is the platform of choice here, but will fail on Windoze. Unless in binmode the Windoze the CR-LF line terminators will be presented as single newlines to the script and expanded back out on output. Thus a random binary single byte newline will become two byte.

I would suggest simple copy loop in any language willing to go binary.
In perl that could look like:

#!/usr/bin/perl

use strict;
use warnings;

my $OFFSET = 10;
my $BUFLEN = 8192;
my ($input_file, $output_file, $bytes, $total, $buffer);

$input_file = shift or die "Please provide input and output file name";
$output_file = shift or die "Please provide output file name";
open (IN, "<$input_file") or die "Failed to open $input_file for input";
open (OU, ">$output_file") or die "Failed to open $output_file for output";
binmode IN;
binmode OU;
$bytes = sysread IN,$buffer,$BUFLEN;
$total = ($bytes)? $bytes : 0;
while ($bytes) {
syswrite OU,$buffer,$BUFLEN,$OFFSET;
$bytes = sysread IN,$buffer,$BUFLEN;
if (defined $bytes) {
$total += $bytes;
} else {
print STDERR "Error reading $input_file: $!";
}
}
print STDERR "$total bytes read.";


hth,
Hein.

[Dennis Handly]

>Clay: it's going to be slow as molasses if the file is anything but tiny in size because the blocksize is so small.

Yes, I found that out years ago. I wanted a piece out of the middle. So I used tail and I had to write a ftruncate program.

>PCS: $ typeset -i SKIP=10
$ tail -c $(( $(cat foo | wc -c)-${SKIP} )) foo > foo_stripped

Since tail doesn't work more than 20Kb (fixed in 11.11), you should use tail +:
$ tail -c+$(($SKIP+1)) foo > foo_stripped

Note: There is no reason to use cat above, use:
$(wc -c < foo)

[James R. Ferguson]

Hi (again):

> Hein: JRF, I think that'll work on HPUX, which is the platform of choice here, but will fail on Windoze. Unless in binmode the Windoze...

Yes, I agree, but my understanding (assuming a current Perl version --- and that may be a poor assumption!) is that the specification of the 'raw' layer in the open() is equivalent:

http://perldoc.perl.org/PerlIO.html

Regards!

...JRF...