Linux pipe(): Reading from a pipe doesn't always unblock writers - c

Question

I have a problem using pipe under Linux. I would like to fill a pipe to make further write's call blocking. An other process should be able to read some characters from the pipe that should allow the other process to write.
The example code:
#include <stdio.h>
#include <errno.h>
#include <unistd.h>
#include <stdlib.h>
int main()
{
int pipefd[2];
int size = 65535;
int total = 0;
// Create the pipe
if(pipe(pipefd) == -1)
{
perror("pipe()");
exit(EXIT_FAILURE);
}
// Fill in (almost full = 65535 (full - 1 byte))
while(total < size)
{
write(pipefd[1], &total, 1);
total++;
}
// Fork
switch(fork())
{
case -1:
perror("fork()");
exit(EXIT_FAILURE);
case 0:
// Close unused read side
close(pipefd[0]);
while(1)
{
// Write only one byte, value not important (here -> total)
int ret = write(pipefd[1], &total, 1);
printf("Write %d bytes\n", ret);
}
default:
// Close unused write side
close(pipefd[1]);
while(1)
{
int nbread;
scanf("%4i", &nbread);
char buf[65535];
// Read number byte asked
int ret = read(pipefd[0], buf, nbread);
printf("Read %d bytes\n", nbread);
}
}
return 0;
}
I don't understand the behavior below. The process write one last time because I didn't fill the pipe completely, normal. But afterwards, the write is blocking (pipe full) and any read should unblock the waiting write call.
test#pc:~$./pipe
Write 1 bytes
4095
Read 4095 bytes
1
Read 1 bytes
Write 1 bytes
Write 1 bytes
Write 1 bytes
Write 1 bytes
Write 1 bytes
Write 1 bytes
...
Instead, the write call is unblocked only after having read 4096 bytes... WHY????
Normally, after a read success of X bytes, there should be X bytes of space available in the pipe and so the write should be able to write up to X bytes, no?
How can I have the behavior "read 1 byte, write 1 byte, etc" instead of "read 1 byte, read 1, read 10, read 2000, ...(until 4096 byte read), write 4096" ?

Answer 1

Why it doesn't work the way you think
So basically what I understand is that your pipe is associated with some kind of linked list of kernel buffers. Processes waiting to write to your pipe are waken up only when one of these buffer is emptied. It happens that in your case these buffers are 4K in size.
See: http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/fs/pipe.c?id=HEAD
Specifically line: 281 Where the test on the buffer size is done and line: 287 where the decision to wake up other processes is done.
The size of the pipe buffer is indeed dependent on the memory page size, see man fcntl
F_SETPIPE_SZ (int; since Linux 2.6.35)
Change the capacity of the pipe referred to by fd to be at least arg bytes.
An unprivileged process can adjust the pipe capacity to any value between
the system page size and the limit defined in /proc/sys/fs/pipe-max-size
(see proc(5)). Attempts to set the pipe capacity below the page size are
silently rounded up to the page size. Attempts by an unprivileged process to
set the pipe capacity above the limit in /proc/sys/fs/pipe-max-size yield
the error EPERM; a privileged process (CAP_SYS_RESOURCE) can override the
limit. When allocating the buffer for the pipe, the kernel may use a
capacity larger than arg, if that is convenient for the implementation. The
F_GETPIPE_SZ operation returns the actual size used. Attempting to set the
pipe capacity smaller than the amount of buffer space currently used to
store data produces the error EBUSY.
How to make it work
The pattern you try to achieve is classical. But it is used the way around. People starts with an empty pipe. Process waiting for an event, does read the empty pipe. Process wanting to signal an event, write a single byte to the pipe.
I think I seen that in Boost.Asio but I'm too lazy to find the correct reference.

Answer 2

Pipe uses 4kB pages for buffer and write is blocked until there is an empty page for write and then do not block until it is full again. It is well described in fjardon's answer. If you would like to use the pipe for signalling you are looking for opposite scenario.
#include <stdio.h>
#include <errno.h>
#include <unistd.h>
#include <stdlib.h>
int main()
{
int pipefd[2];
// Create the pipe
if(pipe(pipefd) == -1)
{
perror("pipe()");
exit(EXIT_FAILURE);
}
// Fork
switch(fork())
{
case -1:
perror("fork()");
exit(EXIT_FAILURE);
case 0:
// Close unused write side
close(pipefd[1]);
while(1)
{
char c;
// Read only one byte
int ret = read(pipefd[0], &c, 1);
printf("Woke up\n", ret);
fflush(stdout);
}
default:
// Close unused read side
close(pipefd[0]);
size_t len = 0;;
char *str = NULL;
while(1)
{
int nbread;
char buf[65535];
while (getline(&str, &len, stdin)) {
if (sscanf(str, "%i", &nbread)) break;
};
// Write number byte asked
int ret = write(pipefd[1], buf, nbread);
printf("Written %d bytes\n", ret);
fflush(stdout);
}
}
return 0;
}