Difference between open and fopen

[Masthan]

Hi All,

Can any one explain the exact difference between open and fopen, read and fread, write and fwrite ?

-Masthan

[AwadheshPandey]

http://www.csie.ntu.edu.tw/~ktw/sysprog/sys-prog-prn-5.pdf
http://publib.boulder.ibm.com/infocenter/systems/index.jsp?topic=/com.ibm.aix.basetechref/doc/basetrf1/fread.htm

[Dennis Handly]

What makes you ask that question? Are you trying to decide what to use?
Are you interested in performance? Or portability?
Are you doing seeks?

[Masthan]

I just want to know how they works. And which one is best.

[Dennis Handly]

>I just want to know how they work. And which one is best.

It depends on whether you need the buffering or not. Whether you need to process the data and need fancy fscanf and fprintf or just binary access.

Whether you are using sequential access or random and the size of your requests. If copying a file, just use system calls with Mb sized buffers.

[A. Clay Stephenson]

The 'f' functions work at the FILE abstraction level whereas the open(), read(), and write() functions are actually system calls. The 'f' functions are built on top of the corresponding system calls and provide application level buffering in addition to the UNIX buffer caching that the lower-level system calls generally do.

Essentially the 'f' functions do a lot of the work that you would do to write efficient code at the expense of losing some measure of control.

Originally UNIX allowed 20 open files per process. Later the number of file descriptoes was increased (usually to 64) but many flavors of UNIX still only allowed 20 high-level *FILE's so that the 'f' functions could only be used if the file descriptor were 19 or less. Later this limitation disappeared as well.

[TwoProc]

using the "fopen", "fclose", etc gets you, as A. Clay said a file handle of type "FILE". So, this allows you to use fprintf, fgets, etc. These functions allow you to parse in and out data to/from files as if you were handling text right out of a buffer like all of the "s" type commands like sprintf, and pretty similar to handling screen text like printf.


So, unless you want to write and handle EVERYTHING yourself, use fopen() with the FILE type, otherwise, use open() for a more low level do it yourself approach.

[Laurent Menase]

Hi,
fwrite()/fprintf() are buffered at user level, so if you need to be sure data are flushed from user level buffer to the file at some point, you MUST use fflush().

Same for reading, if you have a random access file which may be changed and read at the same time by multiple programs, you may prefer to avoid the use of user level buffer.

[Masthan]

Hi All,

Thanks for your response. But unfortunate thing is still I am not convionced with your answers. Yes, you all are correct. fopen/fread/fwrite are buffered and open/read/write are unbuffered. What benifit we will get with buffered IO over Unbuffered IO.

When we have open/read/write calls, why should we go for fopen/fread/fwrite calls ?

is fopen/fread/fwrite offering any special benifit over open/read/write ? If so, what it is ?

-Masthan D

[Steven Schweda]

So, now you're asking why using buffers can
be a good thing? (Yikes.)

[TwoProc]

So, despite what we've all said, you're wanting a educational overview of the advantages and purposes of buffering from a practical view.

How about taking a class on data structures, to see what is out there in the world for you. Find out what a circular queue is, also what a link list is, what a hash is, learn some graphing theory, you might even check out chapter two, buffers. Feel free to think beyond what the book says that they are useful for, and perhaps envision what more you can do with them and think of ways to extend their abilities beyond the teachings of the book and the class...

...

Then, go right ahead and doubt that any thing you've learned is of any use, and never agree to use them unless someone somehow convinces you that they are any good.


Really, buffering is used to soften the effects of ineffective delays caused by hardware, algorighthms, heuristics, or analysis, and to provide the ability to do analysis and manipulation of data beyond a single unit of data: in our world, a byte.

Examples of a buffer:
cache in the cpu, on the mother board - to soften the hardware blow of reaching commonly used data from distant ram.

Disk cache in a program or an OS: to soften the blow of reaching for data directly from disk.

In your case, buffering is used to soften the blow of byte-at-a-time I/O and provide extended use of I/O streams capable of data manipulation approaching that of the data types being used by the language (not be confused with true object definition, but coming close in a sense).

[Dennis Handly]

>What benefit we will get with buffered IO over Unbuffered IO?

You need to answer our questions. What are your operations and sizes?

>When we have open/read/write calls, why should we go for fopen/fread/fwrite calls?

Are you happy with the performance? If you already have the former, then tusc will tell you exactly what you are doing.

>is fopen/fread/fwrite offering any special benefit over open/read/write?

If your sizes are trivial, then you want buffering.

[A. Clay Stephenson]

I am very concerned that you are unconvinced so I have the perfect answer for you. Do both. You can use the fdopen() function to associate a FILE with a file descriptor already returned by open(). This way, you can actually do low-level and high-level i/o on the same file descriptor and buffer or not-buffer at the application level as you choose. Ideally you would use your vast abilities to create heuristic algorithms that would allow the application itself to choose which was the more appropriate choice and would reevaluate that decision periodically. You would also have to make sure that your code was smart enough to flush buffers properly when you "shift gears" so that no data are lost. If I were as unconvinced and wise as you are, this is exactly what I would do.

[Laurent Menase]

Mixing buffered and unbuffered action on the same file is really not a good idea if you want deterministic result.

Basically if you make sequential access to a file, and want to get it pieces by pieces, use FILE/f*. It will optimize the syscalls, getting big chuncks.
If you want to make pure random accesses of small chunks, it won't make a big difference.

When you use low level, you can make some more optimized operations, if you take care of the file offset alignement, and size ( multiple of the needed alignement), but this not so easy.

[Masthan]

Hi All,

I am not looking difference in application point of view ..

I want to know how internally open and fopen (and read and fread, write and fwrite) differs.

Am sure, fopen interanlly calls open. and fopen is buffered IO, open is unbuffered IO.

open+wrapper funcionality+buffer=fopen.

I know that fopen/fread/fwrite is better than open/read/write. I want to know what benefit we are getting with fopen/fread/fwrite over open/read/write and HOW WE ARE GETTING THAT BENEFIT ?

-Masthan

[Dennis Handly]

>I am not looking difference in application point of view

This is the only thing that is important, how a particular application performs.

>I know that fopen/fread/fwrite is better than open/read/write.

Not always, it depends on your application.

>I want to know what benefit we are getting with fopen/fread/fwrite over open/read/write and HOW WE ARE GETTING THAT BENEFIT?

As mentioned several times, buffering. A simple experiment would be to have a loop write 1 byte, 1000 times. Then do this with fwrite.
And another test, use read vs fread.

You have not assigned any points yet in a year of questions. This is unmutual. Please read the following:
http://forums1.itrc.hp.com/service/forums/helptips.do?#33

[Laurent Menase]

Yes like already said, the fact fopen is bufferised makes that sequencial access, are more efficient than a program which would read small chunks. In fact bufferisation reduces the number of syscall (open,read,write) made. Every syscall makes a transition from user to kernel and then to user level+ the time to get the read answer...
So for instance:
a loop which fread() 64 bytes at a time 1024times, will call read() of 4k 16 times.
a loop which read() 64 bytes at a time 1024 times will call the read() syscall 1024 times.
Let's take the hypothesis that read() syscall take about 40usec without counting the time waiting for disk data. you will use aout 40msec just because you call read() 1024 times when you could have taken only 64usec.

- the time of 40usec is just an hypothesis.

So if you are making small unalign and sequential access bufferised is better.
If you do pure random, unbuffered can be better.
You can even get more perf with open/read() if your access is on a 8k boundary for a multiple of 8k bytes.

[A. Clay Stephenson]

>>> I am not looking difference in application point of view ..

But all of the differences occur in user space and are, by definition, only meaningful when viewed from the standpoint of the application.

>>> I want to know how internally open and fopen (and read and fread, write and fwrite) differs.

Then I suggest that you examine the source code for Linux or OpenBSD.

>> Am sure, fopen interanlly calls open. and fopen is buffered IO, open is unbuffered IO.

Wrong again! Open() is unbuffered only with respect to user data (the application which you don't care about); it is still buffered by the UNIX buffer cache unless you specifically open() a raw character device (e.g. /dev/rdsk/c1t2d0) rather than a block device (/dev/dsk/c1t2d0) or a fully-cooked file (myfile.dat).