topic does file truncate affect current file offset in Operating System - HP-UX

does file truncate affect current file offset

Scotty HD — Sat, 12 Jun 2010 11:46:59 GMT

i can truncate a file to increase or decrease the file EOF.
does this change the current file offset?
if it dont then do i have to place the file offset to correct
place in my program ?

Scotty

Re: does file truncate affect current file offset

Matti_Kurkela — Sat, 12 Jun 2010 12:13:33 GMT

You really should have specified which programming/scripting language you're talnking about...

But according to the POSIX standard, the truncate()/ftruncate() system call will not modify the file offset.

It is not mandatory to change the file offset after shortening a file with a truncate operation; but if you don't, you may be creating a sparse file (intentionally or otherwise).

Please see:
http://en.wikipedia.org/wiki/Sparse_file

The Wikipedia page talks about sparse files in Linux, but HP-UX and most other modern Unix-like filesystems will behave the same way.

MK

Re: does file truncate affect current file offset

Scotty HD — Sat, 12 Jun 2010 12:23:38 GMT

Matti Kurkela>>
thanks for reply

i take example to understand better.
My file has eof at 500 blocks.current offset is at 400 blocks.
now i truncate file to 300 blocks.

so according to POSIX standard,
what is current file offset?
next read/write on file will be from what position in file?

Scotty

Re: does file truncate affect current file offset

Dennis Handly — Sat, 12 Jun 2010 12:38:27 GMT

>so according to POSIX standard, what is current file offset?

Unchanged at 400 * blocksize.

>next read/write on file will be from what position in file?

A read should get an error (the extra data shall no longer be available to reads on the file) and a write may leave a hole.

Re: does file truncate affect current file offset

Matti_Kurkela — Sat, 12 Jun 2010 15:12:36 GMT

As Dennis said, the offset will be unchanged, so a read operation without changing the offset will fail.

But if you write 1 block, the write will happen to block 400, and after the write, the offset will be at block 401. The file size will jump from 300 to 401 blocks.

Your file will now consist of 300 blocks of existing data + 100 blocks of zero bytes + 1 block of new data. The "ls -l" will report a file size of (block size * 401 blocks). If another program would read the file from the beginning to the end, the program would get 401 blocks of data from the file.

But if the filesystem supports sparse files, then instead of storing those 100 blocks of zeroes, the filesystem makes a brief note to the metadata of the file: "100 blocks of zeroes at blocks 300-399". Because those blocks don't really exist on the disk, the "du" command will report the file occupies only 301 blocks on the disk.

If you later change the file offset and write something to the 100-block "hole", the filesystem will transparently allocate real disk blocks to cover parts of the hole as necessary. Once the hole is completely filled, the sizes reported by "du" and "ls -l" will again be in agreement.

If the underlying filesystem did not support sparse files, the POSIX semantics would require the OS to automatically write blocks 300-399 to the disk as real blocks of zero bytes.

MK

Re: does file truncate affect current file offset

Scotty HD — Sat, 12 Jun 2010 15:41:30 GMT

Dennis Handly>>, Matti Kurkela>>
Thanks for reply.

i remember writing program before where i saw the file offset getting
set to new EOF after i truncate it. may be i did some program error.
i need to check where i kept that program.

if somebody give me a program it is helpful. i want program to truncate
a file to increase and decrease file length. in both case i want to know
file offset after trucate and also do a read/write after truncate and
check where the data gets read/write to file.

Scotty

Re: does file truncate affect current file offset

Matti_Kurkela — Sat, 12 Jun 2010 19:02:53 GMT

I (and apparently also Dennis) talked about the behaviors and semantics of POSIX-compliant systems. When you program in C on HP-UX or some other Unix-like platform, that's what you usually get.

But if you program in Java, you will have the Java Virtual Machine between your code and the operating system. JVM will hide the semantics of the OS (POSIX, MS-Windows or whatever) and provide another set of semantics that is (supposed to be) uniform across all the OS/hardware platforms Java can run on.

If Java API specifies that a truncate operation must change the file offset to the new EOF, then the JVM could automatically execute "lseek (fd, 0, SEEK_END);" system call (or something equivalent) after each file truncation operation when it runs on a POSIX-compliant platform.

Even a C programmer can see different semantics, if he/she uses the API of some project-specific library instead of the POSIX API. In that case, the semantics should be described in the documentation of that library.

So, unless you tell us at least the name of the programming language you're using, the only answer that can be given without making any guesses about your programming environment is "Please read the documentation of the programming API you're using".

If you want an example program, the choice of the programming language is even more important: if the language of some provided example is not important to you, the time spent in writing it is simply wasted.

MK

Re: does file truncate affect current file offset

James R. Ferguson — Sat, 12 Jun 2010 20:15:11 GMT

Hi Scotty:

Here's a simple Perl script that opens and writes a 500-character file; seek();s backwards 100-characters; and then truncate()s the file to an overall length of 300 before writing one character more. Since no seek() was performed after the truncate(), the position in the file was unchanged.

As Dennis noted, the read() failed after the truncate() and the subsequent write() created a sparse file (one with "holes"). You can see this as follows:

# cat ./mytruncate
#!/usr/bin/perl
use strict;
use warnings;
use Fcntl;
my $fh;
my $file = '/tmp/myfile';

sub tattle {
my @msg = @_;
print "offset=", tell($fh), " @msg\n";
}
sysopen( $fh, $file, O_RDWR | O_CREAT ) or die "Can't open '$file': $!\n";
print $fh 'x' for ( 0 .. 499 ); #...file file with 'x' characters...
tattle("after open and write");
seek( $fh, -100, 2 ); #...back off from eof...
tattle("after seek()");
truncate( $fh, 300 ); #...truncate to offset...
tattle("after truncate()");

if (<$fh>) {
tattle("after successful read");
}
else {
tattle("after an unsuccessful read");
}
print $fh '!'; #...mark with a exclamination point...
tattle("after a write");
1;

# ./mytruncate
offset=500 after open and write
offset=400 after seek()
offset=400 after truncate()
offset=400 after an unsuccessful read
offset=401 after a write

# xd -ta /tmp/myfile
0000000 x x x x x x x x x x x x x x x x
*
0000120 x x x x x x x x x x x x nul nul nul nul
0000130 nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul
*
0000190 !
0000191

As you can see, in the file dump, 0x190 is decimal 400 (zero-relative) and 0x12b is decimal 299 or offset 300, zero-relative. The 'nul' characters are the "holes".

Regards!

...JRF...

Re: does file truncate affect current file offset

Scotty HD — Sun, 13 Jun 2010 05:03:22 GMT

Matti Kurkela>>
sorry i missed your question posted earlier about programming language.
i was referring to C program on HP-UX.
the JVM stuff you told was intresting.

James R. Ferguson>>
thanks so much for program. i will run it on HP-UX system and check.
you have given perl script. c program would also behave similarly, correct ?

Scotty

Re: does file truncate affect current file offset

Dennis Handly — Sun, 13 Jun 2010 06:49:49 GMT

>C program would also behave similarly?

Yes. It is the system calls that matter. You can download and use tusc to check them.

Re: does file truncate affect current file offset

Scotty HD — Sun, 13 Jun 2010 09:01:19 GMT

>> Dennis Handly
# You can download and use tusc to check them.
can u give me link.

if anybody have c program for truncate, pass it to me

Scotty

Re: does file truncate affect current file offset

James R. Ferguson — Sun, 13 Jun 2010 13:02:50 GMT

Hi (again) Scotty:

> you have given perl script. c program would also behave similarly, correct ?

Yes. Perl's guts are written in C and hence use standard system calls. You can see this in the 'lseek(2)' and 'truncate(2)' calls. Look at the HP-UX manpages.

Some of the standard C library functions are re-implemented in Perl's internals but the overall goal is the same. Perl's 'tell(3S)' is roughly equivalent to the C function 'ftell()'.

Regards!

...JRF...

Re: does file truncate affect current file offset

James R. Ferguson — Sun, 13 Jun 2010 16:40:47 GMT

Hi (again) Scotty:

You can download a 'tusc' binary from the PH-UX Porting Centre. Installation is done with 'swinstall' into the '/usr/local/bin' directory:

http://hpux.connect.org.uk/hppd/hpux/Sysadmin/tusc-8.0/

Regards!

...JRF...

Re: does file truncate affect current file offset

Scotty HD — Mon, 14 Jun 2010 12:02:59 GMT

James R. Ferguson>>
thanks much.
i will check it.

is there a link for posix specifications for read/write/truncate/... system calls

Scotty

Re: does file truncate affect current file offset

Dennis Handly — Mon, 14 Jun 2010 12:15:33 GMT

>is there a link for posix specifications for read/write/truncate/... system calls

I'm not sure. But you should trust what's on truncate(2), since Integrity HP-UX 11.31 is Unix 2003 branded.

Re: does file truncate affect current file offset

James R. Ferguson — Mon, 14 Jun 2010 12:47:15 GMT

Hi (again) Scotty:

> is there a link for posix specifications for read/write/truncate/... system calls

I believe you will find what seek (no pun intended) here:

http://www.unix.org/version3/online.html

Regards!

...JRF...