C/C++ API for large files

[Don Burt_2]

An existing C/C++ application now needs to operate on files larger than 2TB, the limit of the C standard lib pointers. What API is available for AdvFS files up to 16TB? If there are several choices, which will be easiest to adopt from the standard lib?

[Hein van den Heuvel · Accepted solution]

A friend in Tru64 engineering (AdvFS) answerred to me:

"
The Tru64 UNIX libc library has always supported huge files and all I/O related libc functions and kernel system calls use a 64 bit argument to handle file offset information. This has been true since the first release of Tru64 UNIX (aka, Digital UNIX, aka Digital Equipment Corporation OSF/1).

I am unaware of any file size limitation in the Tru64 UNIX libc or kernel system calls beyond those specific to the file system itself, at least not for a C application compiled with the Tru64 UNIX C compiler. _IF_ there is a specific C++ limitation (or gcc/g++) it is separate from Tru64 UNIX libc which fully supports 64 bits file offsets and huge files up to and including the 16TB sized files allowed by AdvFS.
And I said _IF_ because I do not know enough about C++, gcc, or g++ to make any statements one way or the other.

The Tru64 UNIX file system offsets use off_t and off_t is declared in /usr/include/sys/types.h as:

typedef long off_t;

and all longs on Tru64 UNIX are 64 bits by default.

I may miss a few function, but lseek(), fseek(), fseeko(), fseek_unlocked(), ftell(), ftello(), pread(), and pwrite() are all long based.
fsetpos() and fgetpos() use fpos_t, which also uses a long to hold the file offset information.
aio_read(), aio_write(), and lio_listio() use struct aiocb which uses an off_t for the file offset.

read(), write(), fread(), fwrite(), printf(), fprintf(), vprintf(), vwprintf(), wprintf(), getchar(), getc(), fgetc(), fgetcw(), getcw(), putchar(), putc(), fputc(), fputwc(), putwc(), gets(), puts(), scanf(), fscanf(), wscanf(), etc... all depend on the kernel "struct file" structure to keep track of the current file offset, and that uses an off_t to keep track of the file offset.

I have most likely missed a few libc or kernel system calls that also do I/O and care about the file offset, but as far as I know all file I/O APIs in Tru64 UNIX are all 64 bit implementations. "

Hth,
Hein.

[Don Burt_2]

Sounds like I have been confused by the application's int references to the long offsets! Problem solved.