Use of named pipe with posix shell

[Eric SAUBIGNAC]

Hi everybody,

i thought that on a named pipe just one reader and one writer are allowed at a time. In fact i can have more than one reader, and more than one writer working together on the same pipe at the same time !!!!!

Was my idea a dream or did i miss something ?

Thanks in advance for your help

[Alan Riggs]

A pipe can be written to and resd from by any user-process with appropriate permissions. You can restrict this with chmod or ACLs.

[harry d brown jr]

Actually I have found that to be a good thing. Are you upset that it allows it?

live free or die

harry

[A. Clay Stephenson]

This is completely normal behavior assuming the permission's bits on the pipe permit it.
In fact this is one means though typically not a good one to synchronize multiple processes.

If you man write 2 it specfically set the rules for writing to a pipe from multiple processes.

[Eric SAUBIGNAC]

No Alan, my problem doesn't concern file access permissions.

I thought (and i still think) that when 2 procces try to write at the same time on the same named pipe, only one perform the write and the second one is delayed.

[Eric SAUBIGNAC]

Clay,

I don't want to use C langage, just redirection of stdout of shell scripts or executables.

[A. Clay Stephenson]

Hi Eric:

I realized you were using the shell but what do you think the shell uses? I wanted you to look at the system call so that you would understand 'what lies beneath'.

If you put together the following test, I think you will agree that you can have 1 writer and 2 readers:

Let's create a FIFO, mknod /tmp/mypipe p


The 'writer x.sh'
#!/usr/bin/sh
I=1
while [ ${I} -le 10 ]
do
echo "From X ${I}"
I=$((${I} + 1))
done
exit 0

Reader 1, y.sh which in turn calls a 2nd
reader z.sh

#!/usr/bin/sh
PID=${$}
echo "Y.SH PID=${PID}"
z.sh < /tmp/mypipe
read X
echo "Y.SH OUT: ${X}"
exit 0

Finally, the 2nd reader, z.sh

#!/usr/bin/sh
PID=${$}
echo "Z.SH PID=${PID}"
read X
echo "Z.SH OUT: ${X}"
exit 0

Now chmod 755 x.sh y.sh z.sh
Let's fire up the writer:
x.sh > /tmp/mypipe &
Now lets fire off the reader:
y.sh < /tmp/mypipe

I think you will see output from 2 reader processes.

Regards, Clay

[harry d brown jr]

Check this out:

http://www.ecst.csuchico.edu/~beej/guide/ipc/fifos.html

If memory serves me right, the writing to a pipe is handled by the os.

[Eric SAUBIGNAC]

well clay, I already did something like that and that's why I am here.

Here is my test

One reader : file reader
# Permament reader
PROC=$$
while :
do {
echo READER $PROC : new session ready
while read a
do echo "READER $PROC : $a"
done
echo READER $PROC : end of session
} < named_pipe
done

One writer : file writer1

sleep 1 # give time to reader
{
echo Beginning of WRITER1
sleep 10 # To see wether or not WRITER2 is delayed
echo End of WRITER1
} >named_pipe

A second writer : file writer2

sleep 2 # give time to writer1
{
echo Beginning of WRITER2
echo End of WRITER2
} >named_pipe

And those scripts executed like this :

umgp(smerep):/tmp>{
./reader &
./writer1 &
./writer2 &
}

[1] 10273
[2] 10274
[3] 10275
READER 10273 : new session ready
READER 10273 : Beginning of WRITER1
READER 10273 : Beginning of WRITER2
READER 10273 : End of WRITER2
READER 10273 : End of WRITER1
READER 10273 : end of session
[3] + Done {;./reader &;./writer1 &;./writer2 &;sleep 15;}
[2] + Done {;./reader &;./writer1 &;./writer2 &;sleep 15;}
umgp(smerep):/tmp>

===> 2 writers at the same time !!!!

Well, i finally think that my idea about how named_pipe are handled by the kernel is simply a mistake ....

Any idea to simply synchronise process ?

[Eric SAUBIGNAC]

Ok harry

that link seems a good reading ... but not today : it's about 22h00 in france !!!

See U 2morrow guys.

[Bill Hassell]

The best way to think about a named pipe is to pretend it is a simple file. Multiple processes can read anbd/or write to a single file if they have permission. For example, you can zero out a logfile while a process is using it. Like any shared file, you'll need to perform file locaking or setup a semaphore of some sort to sync the write/read tasks, just like a simple disk file.

[A. Clay Stephenson]

Okay Eric,

Plan A) Pretty Good but not perfect
If all you need if fairly casual synchronization, create a directory in /var/tmp
to handle the locking. Because the OS must do some synchronization in creating directories this works fairly well. The existence of the directory is used as the flag. You can put any other data like pid of process holding the lock into a file within this directory. The important point is that the directory is the flag andthing in files below it is fluff. When the lock is no longer required, the directory is deleted.

Plan B) Damn good but requires that you know Perl. Use the IPC::Semaphore module. Unlike Plan A these actions are atomic so that when the test is made for a lock the lock is actually created unless the lock test fails.
This is about as good as it gets in a scripting language.

You could actually just use a very small perl script to handle the locking/unlocking while
the grunt work is done within a shell script.

Food for thought, Clay

[Sridhar Bhaskarla]

The reason why a named pipe is "named" is to allow multiple processes to recognize and then use it. In Eric's example, if the "reader" is not much bothered about who is writing into the pipe, there is no problem. Also as commented by Bill, we can just treat the named pipe as a file that is sensitive to permissions.

If "reader" is particular about what's coming out of the pipe, then is the problem. Then it depends on the writer and the corresponding reader to "agree upon" a synchronization mechanism. That's when we start talking about message queues and other good IPC stuff.

-Sri

[Eric SAUBIGNAC]

Late to close this post ... but it is never too late to make well !