topic Query regarding CTRL ftruncate call in Operating System - OpenVMS

Query regarding CTRL ftruncate call

ambika_1 — Fri, 19 Jun 2009 02:58:27 GMT

Hi,
I am new to OpenVMS. I am writing some sample programs to basically change the EOF of a stream file. I am changing the EOF to some large number 1GB using the ftruncate call, but my clients gets a timeout error since ftruncate taking more time to reply. The reason behind this is ftruncate does fill the gap between old EOF to new EOF with zeroes.

In my case , client is just asking to set EOF so that in future it might need that much space, but for more that 1GB files client is timing out. Is there any alternative the change EOF without filling the empty space with zeros.

On UNIX, ftruncate does fill the gap, it instead creates SPARSE file. If any one can help regarding this i would be glad.
If I am posting in wrong forum, please excuse me.

Thanks and Regards,
Ambika

Re: Query regarding CTRL ftruncate call

Steven Schweda — Fri, 19 Jun 2009 03:25:18 GMT

If you're creating the file, then you should
be able to specify one of those VMS-specific
creat()/open()/fopen() options like
"alq = n" (and its friends).

HELP CRTL creat Arguments

You still may need to worry about high-water
marking on the file system, so it may still
not be as quick as you might like.

alp $ show device /full alp$dka0:

Disk ALP$DKA0:, device type IBM-ESXS ST336605LC !#, is online, mounted, file-
oriented device, shareable, served to cluster via MSCP Server, error logging
is enabled.
[...]
Volume Status: ODS-2, subject to mount verification, protected subsystems
enabled, file high-water marking, write-through caching enabled.

If you're not creating the file, then some
RMS-level code may be faster than the CRTL
ftruncate(), but I don't know about that.
It's possible that ftruncate() is slow
because the file's default extend quantity is
small, so it's doing the I/O in little
pieces. ("deq = n" can affect that, as can
a DCL SET RMS_DEFAULT command.)

Re: Query regarding CTRL ftruncate call

ambika_1 — Fri, 19 Jun 2009 03:37:27 GMT

Hi Steven,
Thanks for a quick response. Client wont be knowing the size initially, its trying to set the EOF at the later point of time. So i cant specify it while creating the file.

Thanks and Regards,
Ambika

Re: Query regarding CTRL ftruncate call

Jan van den Ende — Fri, 19 Jun 2009 07:03:35 GMT

Ambika,

Steven already touched the subject:

CHECK the relevant disk for FILE HIGHWATER MARKING. (Which _IS_ the default!!)

If it is set, ANY extending of the file WILL force a full overwrite of ALL new allocation.
A simple
$ SET VOLUME /NOHIGH
will take virtually no time, and THEN the forced overwrite will no longer occur.

_DO_ realise that this means that read access to the file then implies the ability to read what was in that/those location(s) on the drive, so that might mean a violation of your local security. Only you(r site) know if that is considered an issue (it rarely is).

hth

Proost.

Have one on me.

jpe

Re: Query regarding CTRL ftruncate call

Hein van den Heuvel — Fri, 19 Jun 2009 10:27:21 GMT

Sounds like your client really wants to get ALLOCATED blocks, but not change the EOF.
This can easily be done on OpenVMS using the SYS$EXTEND call.
They have to choose between using the ACC_CALLBACK to get access to the FAB that the C-RTL uses, or just call RMS directly.
Attached is an (excessive for your purpose) example of calling extend from C.

Pre-allocation speed is independent of High-Water-Marking. That only come into play when the EOF, is increased.

OpenVMS does not have sparse files.
For some purposes you could use INDEXED files to get some of the behaviour of a sparse file with the requirement that there is a (unique) primary key identifiable within each record.

hth,
Hein.

Re: Query regarding CTRL ftruncate call

Hein van den Heuvel — Fri, 19 Jun 2009 10:29:36 GMT

I had to re-reply, and forgot the attachment.
retry.
Hein.

Re: Query regarding CTRL ftruncate call

John Gillings — Sun, 21 Jun 2009 22:10:20 GMT

Ambika,

> In my case , client is just asking to set
>EOF so that in future it might need that
>much space

As others have pointed out, this is achieved in OpenVMS by (pre)allocating space to the file, without moving the EOF. Using Hein's example code, you should be able to write a routine "fallocate" to do it for you.

That said, is there any real advantage to preallocating space to the file? We know there's a disadvantage (this post!). The OpenVMS file system is quite capable of allocating space when it's required. It's unlikely any real world application would notice or care about the performance of the extend, and with sensible cluster and default extend sizes, fragmentation isn't much of an issue.

Perhaps this part of the code is a throwback to times when file extensions were expensive, or disk space was scarce enough that it made sense to hoard it up front?

Maybe the simplest solution is to remove this section of code and let OpenVMS handle the issue of file extension?

Re: Query regarding CTRL ftruncate call

Steven Schweda — Mon, 22 Jun 2009 01:50:07 GMT

> [...] It's unlikely any real world
> application would notice or care about the
> performance of the extend, [...]

Oh, I might dispute that. The CRTL/RMS
default extend size is so small, that a
zillion small extensions can be noticeably
slower than a single equivalent non-default
(big) extension.

If you have doubts, try using a program like
gzip to expand some big-ish thing with the
default extension quantity, and then with a
bigger, non-default extension quantity. I
claim that the difference is pretty easy to
notice. ("fop=sqo" can help, too, if you
don't break the rules.)

Re: Query regarding CTRL ftruncate call

ambika_1 — Mon, 22 Jun 2009 03:16:14 GMT

Hi All,
Thanks a lot for the quick and detailed responses.
I will modify my client by removing the allocation of space in the earlier stage. As John mentioned, for the further writes, OpenVMS itself will allocate the space when required. Client need not worry about it.

Once again thanks allot for the responses. I will close the post.

Thanks and Regards,
Ambika

Re: Query regarding CTRL ftruncate call

ambika_1 — Mon, 22 Jun 2009 03:18:45 GMT

Thanks for the quick help in analyzing the problem.