topic Re: Is larger sizes of FIFO from "mkfifo" supported in Operating System - HP-UX

Is larger sizes of FIFO from "mkfifo" supported

dscao — Tue, 31 Jul 2007 09:43:02 GMT

I've raised the issue long before. On the arriving of V3, it seems that the size of FIFO is still fixed at 8k. I have the impression that someone told me that the size of FIFO can be adjusted between 8 ~ 32 k.
Is larger sizes of FIFO supported?
How to change it?

Re: Is larger sizes of FIFO from "mkfifo" supported

Sandman! — Tue, 31 Jul 2007 10:31:35 GMT

I am not sure what you mean by V3...would help if you could clarify? All atomic reads and writes to FIFOs are governed by the PIPE_BUF parameter which is defined in the "/usr/include/limits.h" file and is set to 8Kbytes. Don't think it is a good idea to change that value as all pre-compiled applications are making use of it. However if you want to try (just for grins) then it would be better to first open a support call with HP.

Re: Is larger sizes of FIFO from "mkfifo" supported

dscao — Tue, 31 Jul 2007 19:48:32 GMT

It's not about the size of atomic writes. It is the size of total bytes that can be written into the fifo before a reader must extract some of them.

Re: Is larger sizes of FIFO from "mkfifo" supported

Sandman! — Tue, 31 Jul 2007 20:11:04 GMT

In that case the parameter of interest is PIPE_MAX. See the man page of limits(5) for details. Also check out the "RATIONALE" section of the URL listed below.

# man 5 limits

http://www.opengroup.org/onlinepubs/009695399/functions/write.html

Re: Is larger sizes of FIFO from "mkfifo" supported

Dennis Handly — Wed, 01 Aug 2007 06:24:41 GMT

>Sandman: I am not sure what you mean by V3

I assume dscao means 11i V3, 11.31.

Re: Is larger sizes of FIFO from "mkfifo" supported

dscao — Wed, 01 Aug 2007 08:33:45 GMT

I mean HP-UX 11.31 by V3.
The number of bytes guaranteed to be written atomically is 8K. But here I'm not interested in the atomic writes. I'd like to know how many bytes can be written into a fifo before some bytes must be read out! It is always 8k from 11.00.
has the limit been lifted?
Is it adjustable? how?

Re: Is larger sizes of FIFO from "mkfifo" supported

Dennis Handly — Wed, 01 Aug 2007 20:22:01 GMT

>But here I'm not interested in the atomic writes. I'd like to know how many bytes can be written into a fifo before some bytes must be read out!

That's probably what atomic means in this case. You can only write PIPE_BUF bytes before someone has to read. This is discussed in the RATIONALE that Sandman provided, under "Complete/partial/deferred".

I don't see any kernel tunable for pipe or fifo for 11.31.

Re: Is larger sizes of FIFO from "mkfifo" supported

A. Clay Stephenson — Wed, 01 Aug 2007 20:51:24 GMT

This is always going to be a finite limit so if I were designing an application that could take advantage of a larger than expected PIPE_BUF value, I would do a pathconf(_PC_PIPE_BUF) to ask the box itself and then dynamically allocate buffers and adjust my i/o sizes to match. Nothing in the 11.31 documentation suggests that PIPE_BUF is anything other than a constant value. The beauty of asking the box itself and adjusting accordingly it that should the value be bumped up or made adjustable in a future release, you won't have to do anything to your code.

Re: Is larger sizes of FIFO from "mkfifo" supported

dscao — Thu, 02 Aug 2007 04:13:55 GMT

The application does just this. It restrains itself from sending more than 8K on HP-UX. A reader extracts the message and does the processing. If the sender can not write more, it returns a message to the client and fails the request. The failure rates are unnecessarily higher, since there are enough CPU cycles but this reader is just temporarily slower down!

Re: Is larger sizes of FIFO from "mkfifo" supported

Sandman! — Thu, 02 Aug 2007 09:39:27 GMT

If yoy looked at the URL I posted it says that PIPE_BUF is the upper limit to how many bytes can be written to a pipe. Further writes will error out unless the data is first extracted from that pipe or FIFO. Moreover if you read the limits(5) manpage it says that the PIPE_BUF limit may be greater than the specified value on certain HP-UX systems. Why would you be interested in increasing its value from the default of 8K?

Re: Is larger sizes of FIFO from "mkfifo" supported

dscao — Thu, 02 Aug 2007 22:46:28 GMT

A larger size of "fifo" here means a larger buffer between two processes. Under the heavy load, with CPU usage around 90%, a larger buffer compensates for the temporarily slowdown of the reader. Otherwise the reqests are failed because the messages can not be written into the "fifo"
Messages have very small size, around 200 bytes.
A kernel parameter "streampipes" can be used to raise the size of "pipe", but not "fifo".
AIX has a larger size of "fifo" up to 32k, at least. The app runs more stable, even if the CPU usage is around 98%! On HP-UX, there are many failures of requests, even if the CPU usage is at 90%!
A small margin means a lot in a head to head competition.

Re: Is larger sizes of FIFO from "mkfifo" supported

dscao — Sun, 05 Aug 2007 21:32:08 GMT

It seems that I have to close this thread. This fifo size issue had been raised before HP-UX 11i V2 was released internally. But I'm so disappointed that it is not implemented even in V3! It was very difficult? or was just rebuked by some idealistic experts?

Re: Is larger sizes of FIFO from "mkfifo" supported

dscao — Sun, 05 Aug 2007 21:34:11 GMT

I will raised the issue internally and have some patches made for this, provided that it is not difficult to implement!