Named pipes and disk I/O

[Ken Sell]

I'm having performance problems with writing
large amounts of data. I'd like to write to
a named pipe, and would expect it to be
very fast, but It appears that named pipes
cause disk I/O. Can anyone elaborate on
how named pipes are implemented, and if there
is anything I can do to increate performance?

Thanks,

..Ken

[TTr]

How exactly are you planning on utilizing the named pipes? What does the current i/o flow looks like?
Named pipes don't necessarily work in all cases. They may help you eliminate some intermediate files that your i/o process creates (save i/o, not make it fast). So without knowing your process we can not tell you how to use them.

[Ken Sell]

Hi,

Thanks for the reply.

The product is a client/server where
the client in running on an HP-UX box.
The client is multi-threaded, and
receives a great deal of data from the
server (Gigabits). It can write the data
to multiple files or named pipes
simultaneously. The data is stream is
continuous. The performance goal is
to get the data from the server as
quickly as possible and write it to
the appropriate file(s) or named pipes.

Without going into the full history of
our performance testing, it appears that
disk I/O is a bottleneck when the end
user wants to use named pipes. After
investigation (i.e. looking at disk I/O
usage via GlancePlus) it appears that
disk I/O is at 100% and the Req Queue is
growing). Note, this is a problem with
file writes as well, but I'm not addressing
that at the moment.

I've tried increasing the size of the
buffer I write, and that helps a little,
but not much.

Thanks,

...Ken

[Dennis Handly]

>I've tried increasing the size of the buffer I write, and that helps a little,

You probably can't. Pipes have a fixed (very small) size buffer. PIPE_BUF 8192

I don't know about disk I/O but they would continually block.