I am writing a parent process, that needs to count events from a group of child processes.
I am going to use pipe() to achieve this.
Can I open a single pipe on the parent, and then fork 4 child processes that will use that same pipe to communicate with the parent, or must I create 4 different pipes? (1 for each child process)
It is important to state that the parent never communicates with the child processes. All it does is: Count and sum up the rate at which the child processes raise events.
Also: In case I can use a shared pipe, what would the atomicity of the messages be. Do I have to keep them one byte long, or can I assume that two 4 byte messages will not get their bytes interpolated?
You can use a single pipe.
You don't need to limit yourself to single-byte events.
man 7 pipe on Linux states:
PIPE_BUF
POSIX.1 says that write(2)s of less than PIPE_BUF bytes must be
atomic: the output data is written to the pipe as a contiguous
sequence. Writes of more than PIPE_BUF bytes may be nonatomic: the
kernel may interleave the data with data written by other processes.
POSIX.1 requires PIPE_BUF to be at least 512 bytes. (On Linux,
PIPE_BUF is 4096 bytes.)
(Related: The description of write in POSIX.)
Another option is to use one datagram unix socket pair created with socketpair instead of a pipe. In this case each write creates a separate datagram and each read returns one datagram only. This way messages can be larger than PIPE_BUF and still be atomic.
Try using named pipe is this example https://www.geeksforgeeks.org/named-pipe-fifo-example-c-program/ Rin the reader and then the written should do the work, you should also think about message quue that is asynchronous way of communicating between process
Related
From my understanding, C pipes are like a special kind of file, where internally, the kernal keep tracks of the openings and closings from each process in a table. see the post here
So in that sense:
Is it possible for 1 single pipe to be connected by multiple processes?
If it is possible, can multiple processes read the same data?
If 2 is possible, will they be reading the same data, or does reading the data "empty" the data?
For example: process 1 writes into pipe, can process 2,3,4 read the data that process 1 wrote?
Yes, multiple processes can read from (or write to) a pipe.
But data isn't duplicated for the processes. Once data has been read from the pipe by one process, it's lost and available only to the process that actually read it.
Conversely, there's no way to distinguish data or from which process it originated if you have multiple processes writing to a single pipe.
1. Is it possible for 1 single pipe to be connected by multiple processes?
Yes.
2. If it is possible, can multiple processes read the same data?
No!
Unix fifos (pipes) can not be used in "single producer, multiple consumer" (spmc) manner; this also holds for Unix Domain Sockets (for most implementations UDS and fifos are implemented by the very same code, with just a few configuration bits differing on creation). Each byte written into a pipe / SOCK_STREAM UDS (or datagram written into a SOCK_DGRAM unix domain socket) can be read from only one single reading end.
However what's perfectly possible is having a "multiple producer, single consumer" fifo, UDS, that is the consumer having open one reading end (and also keeping open the writing end, but not using it¹), multiple producers can send data to the single consumer. For stream oriented pipes there's no strict ordering, so all the bytes sent will get mixed up. But for SOCK_DGRAM UDS socketpairs message boundaries are preserved.
¹: There's a particular pitfall, that if the creating process does not keep open its instance of the writing end, as soon as any one of the producer processes closes one of their writing end, it will tear down the connection for all other processes.
I'm reading about pipe(7)s in Linux and came across the following thing:
POSIX.1 says that write(2)s of less than PIPE_BUF bytes must be
atomic: the output data is written to the pipe as a contiguous
sequence. Writes of more than PIPE_BUF bytes may be nonatomic: the
kernel may interleave the data with data written by other processes.
POSIX.1 requires PIPE_BUF to be at least 512 bytes. (On Linux,
PIPE_BUF is 4096 bytes.)
This is not quite clear. Does POSIX require that all writes less then PIPE_BUF are atomic? Or this is true to pipes created with pipe(int[2], int) only?
The quoted behavior is pipe specific (but applies to all pipes, no matter how they were created (e.g. by pipe, mkfifo+open, etc)).
From the POSIX description of write:
Write requests to a pipe or FIFO shall be handled in the same way as a regular file with the following exceptions:
[...]
Write requests of {PIPE_BUF} bytes or less shall not be interleaved with data from other processes doing writes on the same pipe. Writes of greater than {PIPE_BUF} bytes may have data interleaved, on arbitrary boundaries, with writes by other processes, whether or not the O_NONBLOCK flag of the file status flags is set.
I'm doing IPC using named (FIFO) pipes and I would like to coordinate that program can only write into the pipe when program reading the pipe has read the previously written data out from the pipe. So I would like to block the write until the the pipe is empty. Is this possible?
One option that I though is that write function blocks when the pipe is full. But I would like to do this to much smaller amounts of data than the pipe size in Linux. E.g I would like that program can only write 20 bytes and then it waits until other end has read the data. I think you can not shrink named pipes to be so small. (Minimum size seems to be page file size (4096 bytes)?)
Thanks!
A possible solution is to have the reading process sending a signal to the writing process when it has read some data. You can do this using kill() to send SIGTERM to the writer, since SIGTERM can be catched and handled.
I have my child process counting the frequency of words from a text file. I am using pipe() for IPC. How can the child process return both the word name and the word frequency to the parent process? My source code is in C and I am executing it in a UNIX environment.
Write the two values to one end of the pipe in the child, separated by some delimiter. In the parent, read from the other end of the pipe, and separate the content using the delimiter.
Writes to a pipe up to the size of PIPE_BUF are atomic (included in limits.h), therefore you can easily pack your information into some type of struct, and write that to the pipe in your child process for the parent process to read. For instance, you could setup your struct to look like:
struct message
{
int word_freq;
char word[256];
};
Then simply do a read from your pipe with a buffer that is equal to sizeof(struct message). That being said, keep in mind that it is best to only have either a single reader/writer to the pipe, or you can have multiple writers (because writes are atomic), but again, only a single reader. While multiple readers can be managed with pipes, the fact that reads are not atomic means that you could end up with scenarios where messages either get missed due to the non-deterministic nature of process scheduling, or you get garbled messages because a process doesn't complete a read and leaves part of a message in the pipe.
It seems to me that both signal and socket can be used for this job,
how do you decide which one to use actually?
Using signals for IPC is sort of inconvenient and primitive. You should really be choosing between Unix sockets (not TCP ones!) and pipes.
Pipes are generally easier to program with, since they guarantee that a single write under the size of PIPE_BUF is atomic. They do have their limitations however. For example, when the writer is faster than the reader, the writer starts to block when the pipe buffer gets full. The size of this buffer by default is around 64k, and it cannot be changed without recompiling the kernel, at least in Linux. Pipes are also unidirectional, which means that you'll have to keep a pair of pipes in each process, one for reading and one for writing.
Unix sockets have a configurable send buffer size and a more advanced programming interface.