C fifo keeps blocked - c

I'm currently studying multithreading with C, but there is something I don't quite understand with our named pipe excersize.
We are expected to do an implementation of file search system that finds files and adds to a buffer with one process and the second process should take filenames from threads of first one, finds the search query inside that file and returns the position to first process via pipe. I did nearly all of it but i'm confused how to do the communication between two processes.
Here is my code that does the communication:
main.c
void *controller_thread(void *arg) {
pthread_mutex_lock(&index_mutex);
int index = t_index++; /*Get an index to thread*/
pthread_mutex_unlock(&index_mutex);
char sendPipe[10];
char recvPipe[10];
int fdsend, fdrecv;
sprintf(sendPipe, "contrl%d", (index+1));
sprintf(recvPipe, "minion%d", (index+1));
mkfifo(sendPipe, 0666);
execlp("minion", "minion", sendPipe, recvPipe, (char*) NULL);
if((fdsend = open(sendPipe, O_WRONLY|O_CREAT)) < 0)
perror("Error opening pipe");
if((fdrecv = open(recvPipe, O_RDONLY)) < 0)
perror("Error opening pipe");
while(1) {
char *fileName = pop(); /*Counting semaphore from buffer*/
if(notFile(fileName))
break;
write(fdsend, fileName, strlen(fileName));
write(fdsend, search, strlen(search));
char place[10];
while(1) {
read(fdrecv, place, 10);
if(notPlace(place)) /*Only checks if all numeric*/
break;
printf("Minion %d searching %s in %s, found at %s\n", index,
search, fileName, place);
}
}
}
From the online resources I found, I think this is the way to handle the fifo inside the main. I tried to write a test minion just to make sure it works, so here it is
minion.c
int main(int argc, char **argv) {
char *recvPipe = argv[1];
char *sendPipe = argv[2];
char fileName[100];
int fdsend, fdrecv;
return 0;
fdrecv = open(recvPipe, O_RDONLY);
mkfifo(sendPipe, 0666);
fdsend = open(sendPipe, O_WRONLY|O_CREAT);
while(1) {
read(fdrecv, fileName, 100);
write(fdsend, "12345", 6);
write(fds, "xxx", 4);
}
return 0;
}
When I run this way, the threads get blocked and prints no response if I change to O_NONBLOCK to mode of open. Then it prints "Error opening pipe no such device or address" error, so I know that somehow I couldn't open the recvPipe inside minion but I don't know what is the mistake

Among the problems with your code is an apparent misunderstanding about the usage of execlp(). On success, this function does not return, so the code following it will never be executed. One ordinarily fork()s first, then performs the execlp() in the child process, being certain to make the child terminate if the execlp() fails. The parent process may need to eventually wait for the forked child, as well.
Additionally, it is strange, and probably undesirable, that each process passes the O_CREAT flag when it attempts to open the write end of a FIFO. It should be unnecessary, because each one has just created the FIFO with mkfifo(). Even in the event that mkfifo() fails or that some other process removes it before it can be opened, you do not want to open with O_CREAT because that will get you a regular file, not a FIFO.
Once you fix the execlp() issue, you will also have a race condition. The parent process relies on the child to create one of the FIFOs, but does not wait for that process to do so. You will not get the desired behavior if the parent reaches its open attempt before the child completes its mkfifo().
I suggest having the parent create both FIFOs, before creating the child process. The child and parent must cooperate by opening the both ends of one FIFO before proceeding to open both ends of the other. One's open for reading will block until the other opens the same FIFO for writing.
Or you could use ordinary (anonymous) pipes (see pipe()) instead of FIFOs. These are created open on both ends, and they are more natural for communication between processes related by inheritance.
In any event, be sure to check the return values of your function calls. Almost all of these functions can fail, and it is much better to detect and handle that up front than to sort out the tangle that may form when you assume incorrectly that every call succeeded.

Fifos need some synchronization at open time. By default open(s) are blocking, so that an open for read is blocked until some other open the same fifo for writing, and the converse (this to let peers to be synchronized for a communication). You can use O_NONBLOCK to open for reading while there is no actual open peer, but the converse is false, because opening for writing while there is no reading peer leads to an error (letting a process trying to write while there is no reader is considered as non sense).
You may read Linux Fifo manual entry for example.

Related

How to use pipe between parent and child process after call to popen?

I want to communicate with a child process like the following:
int main(int argc, char *argv[])
{
int bak, temp;
int fd[2];
if (pipe(fd) < 0)
{
// pipe error
exit(1);
}
close(fd[0]);
dup2(STDOUT_FILENO, fd[1]);
fflush(stdout);
bak = dup(1);
temp = open("/dev/null", O_WRONLY);
dup2(temp, 1);
close(temp );
Mat frame;
std::vector<uchar> buf;
namedWindow( "Camera", WINDOW_AUTOSIZE );
VideoCapture cam(0 + CAP_V4L);
sleep(1);
if (!cam.isOpened())
{
cout << "\nCould not open reference " << 0 << endl;
return -1;
}
for (int i=0; i<30; i++)
{
cam>>frame;
}
//cout<<"\nCamera initialized\n";
/*Set the normal STDOUT back*/
fflush(stdout);
dup2(bak, 1);
close(bak);
imencode(".png",frame, buf);
cout<<buf.size()<<endl;
ssize_t written= 0;
size_t s = 128;
while (written<buf.size())
{
written += write(fd[1], buf.size()+written, s);
}
cout<<'\0';
return 0;
}
The process corresponding to the compilation of the source code above is called from the parent with popen.
Note that I am writing to the std out that has been duplicated with a pipe.
The parent will read the data and resend them to UDP socket.
If I do something like this:
#define BUFLEN 128
FILE *fp;
char buf[BUFLEN];
if ((fp = popen("path/to/exec", "r")) != NULL)
{
while((fgets(buf, BUFLEN, fp)!=NULL))
{
sendto(sockfd, buf, strlen(buf),0, addr, alen);
}
}
the program is working i.e. the receiver of sendto will receive the data.
I tried to use a pipe as done in the child process:
int fd[2];
if (pipe(fd) < 0)
{
// pipe error
exit(1);
}
close(fd[1]);
dup2(STDIN_FILENO, fd[0]);
if ((fp = popen("path/to/exec", "r")) != NULL)
{
while((read(fd[0], buf, BUFLEN) > 0)
{
sendto(sockfd, buf, strlen(buf),0, addr, alen);
}
}
but with this are not sent.
So how to use pipe in this case to achieve the same behaviour of the first case? Should I do dup2(STDIN_FILENO, fd[0]); or dup2(STDOUT_FILENO, fd[0]);?
I am using the sandard(s) since the file descriptors are inherited by the child process so should not require any other effort. That is why I thought I can use pipe but is that so?
In the parent:
if (pipe(fd) < 0)
{
// pipe error
exit(1);
}
close(fd[0]);
you get a pipe, and then immediately close one end of it. This pipe is now useless, because no-one will ever be able to recover the closed end, and so no data can flow through it. You have converted a pipe into a hollow cylinder sealed at one end.
Then in the child:
if (pipe(fd) < 0)
{
// pipe error
exit(1);
}
close(fd[1]);
you create another unrelated pipe, and seal this at the other end. The two pipes are not connected, and now you have two separate hollow cyclinders, each sealed at one end. Nothing can flow through either of them.
If putting something in the first cylinder made it appear in the other, that'd be a pretty good magic trick. Without sleight of hand or cleverly arranged mirrors, the solution is to create one pipe, keep both ends open and push data through it.
The usual way to manually set up a pipe from which a parent process can read a child process's standard output has these general steps:
parent creates a pipe by calling pipe()
parent fork()s
parent closes (clarification: its copy of) the write end of the pipe
child dupes the write end of the pipe onto its standard output via dup2()
child closes the original file descriptor for the write end of the pipe
(optional) child closes (clarification: its copy of) the read end of the pipe
child execs the desired command, or else performs the wanted work directly
The parent can then read the child's output from the read end of the pipe.
The popen() function does all of that for you, plus wraps the parent's pipe end in a FILE. Of course, it can and will set up a pipe going in the opposite direction instead if that's what the caller requests.
You need to understand and appreciate that in the procedural scheme presented above, it is important which actions are performed by which process, and in what order relative to other actions in the same process. In particular, the parent must not close the write end of the pipe before the child is launched, because that renders the pipe useless. The child inherits the one-end-closed pipe, through which no data can be conveyed.
With respect to your latter example, note also that redirecting the standard input to the read end of the pipe is not part of the process for either parent or child. The fact that your pipe is half-closed, so that nothing can ever be read from it anyway, is just icing on the cake. Moreover, the parent clobbers its own standard input this way. That's not necessarily wrong, but the parent does not even rely on it.
Overall, however, there is a bigger picture that you seem not to appreciate. Even if you performed the redirection you seem to want in the parent, so that it could be inherited by the child, popen() performs its own redirection to a pipe of its own creation. The FILE * it returns is the means by which you can read the child's output. No previous output redirection you may have performed is relevant (clarification: of the child's standard output).
In principle, an approach similar to yours could be used to create a second redirection going the other way, but at that point the convenience factor of popen() is totally lost. It would be better go take the direct pipe / fork / dup2 / exec route all the way through if you want to redirect the child's input and output.
Applying all that to your first example, you have to appreciate that although a process can redirect its own standard streams, it cannot establish a pipe to its parent process that way. The parent needs to provide the pipe, else it has no knowledge of it. And when a process dupes one file descriptor onto another, that replaces the original with the new, closing the original if it is open. It does not redefine the original. And of course, in this case, too, a pipe is useless once either end is no longer open anywhere.

understanding pipe() function

I'm trying to understand how pipe() function works and I have the following program example
int main(void)
{
int fd[2], nbytes;
pid_t childpid;
char string[] = "Hello, world!\n";
char readbuffer[80];
pipe(fd);
if((childpid = fork()) == -1)
{
perror("fork");
exit(1);
}
if(childpid == 0)
{
/* Child process closes up input side of pipe */
close(fd[0]);
/* Send "string" through the output side of pipe */
write(fd[1], string, (strlen(string)+1));
exit(0);
}
else
{
/* Parent process closes up output side of pipe */
close(fd[1]);
/* Read in a string from the pipe */
nbytes = read(fd[0], readbuffer, sizeof(readbuffer));
printf("Received string: %s", readbuffer);
}
return(0);
}
My first question is what benefits do we get from closing the file descriptor using close(fd[0]) and close(fd[1]) in child and parent processes. Second, we use write in child and read in parent, but what if parent process reaches read before child reaches write and tries to read from pipe which has nothing in it ? Thanks!
Daniel Jour gave you 99% of the answer already, in a very succinct and easy to understand manner:
Closing: Because it's good practice to close what you don't need. For the second question: These are potentially blocking functions. So reading from an empty pipe will just block the reader process until something gets written into the pipe.
I'll try to elaborate.
Closing:
When a process is forked, its open files are duplicated.
Each process has a limit on how many files descriptors it's allowed to have open. As stated in the documentation: each side of the pipe is a single fd, meaning a pipe requires two file descriptors and in your example, each process is only using one.
By closing the file descriptor you don't use, you're releasing resources that are in limited supply and which you might need further on down the road.
e.g., if you were writing a server, that extra fd means you can handle one more client.
Also, although releasing resources on exit is "optional", it's good practice. Resources that weren't properly released should be handled by the OS...
...but the OS was also written by us programmers, and we do make mistakes. So it only makes sense that the one who claimed a resource and knows about it will be kind enough to release the resource.
Race conditions (read before write):
POSIX defines a few behaviors that make read, write and pipes a good choice for thread and process concurrency synchronization. You can read more about it on the Rational section for write, but here's a quick rundown:
By default, pipes (and sockets) are created in what is known as "blocking mode".
This means that the application will hang until the IO operation is performed.
Also, IO operations are atomic, meaning that:
You will never be reading and writing at the same time. A read operation will wait until a write operation completes before reading from the pipe (and vice-versa)
if two threads call read in the same time, each will get a serial (not parallel) response, reading sequentially from the pipe (or socket) - this make pipes great tools for concurrency handling.
In other words, when your application calls:
read(fd[0], readbuffer, sizeof(readbuffer));
Your application will wait forever for some data to be available and for the read operation to complete (which it will once 80 (sizeof(readbuffer)) bytes were read, or if the EOF status changed during a read).

Parent child interprocess comunication - is keeeping pipes open OK?

I have to implement a program in which a process sends data it has received from parent process to its child process, waits until the child sends him processed data back, and then return processed data to child process (so e.g. in case of 4 processes the data flow would look like this P1->P2->P3->P4->P3->P2->P1). For means of interprocess communication I need to use pipes. Here's an approach I planned to take:
./child
// Assert argv contains 2 pipe descriptors - for reading
// from parent and for writing to parent, both of type char[]
// I'm not handling system errors currently
int main(int argc, char *argv[]) {
int read_dsc, write_dsc;
read_dsc = atoi(argv[1]);
write_dsc = atoi(argv[2]);
char data[DATA_SIZE];
read (read_dsc, data, DATA_SIZE - 1);
close (read_dsc);
// Process data...
(...)
// Pass processed data further
int pipeRead[2]; // Child process will read from this pipe
int pipeWrite[2]; // Child process will write into this pipe
pipe(pipeRead);
pipe(pipeWrite);
switch(fork()) {
case 0:
close (pipeRead[1]);
close (pipeWrite[0]);
char pipeReadDsc[DSC_SIZE];
char pipeWriteDsc[DSC_SIZE];
printf (pipeReadDsc, "%d", pipeRead[0]);
printf (pipeWriteDsc, "%d", pipeWrite[1]);
execl ("./child", "child", pipeReadDsc, pipeWriteDsc, (char *) 0);
default:
close(pipeRead[0]);
close(pipeWrite[1]);
wait(0);
read (pipeWrite[0], data, DATA_SIZE - 1);
close (pipeWrite[0]);
// Pass data to parent process
write (write_dsc, data, DATA_SIZE - 1);
close (write_dsc);
}
}
High level description of my solution is as follows: make 2 pipes, one for writing to child process, one for reading from child process. Wait until child process finishes and then read from read pipe and pass data to parent.
The problem is I don't know whether this approach is correct. I've read somewhere that not closing unused pipes is an error as it clutters OS file descriptors and there shouldn't be many opened pipes at once. Here however we're keping unclosed pipe for reading from a child and potentially if there are n processes, there are n opened pipes when process number n processes it's data (all parent processes are waiting for data to come back). However I can't see any other way to solve this problem...
So - is my solution correct? If it isn't, how should I approach this problem?
Yes your solution is correct. But there is problems in your code:
case 0 is the child, you will benefit in redirecting pipe ends onto standard input and output (use dup or dup2); passing descriptor ids to the child is weird.
default is the parent, so you need to write before reading.
"not closing unused pipes is an error" : it is not an error but may cause problems (detecting the end of a communication would be difficult or impossible), but it seems that you correctly close all non useful pipe ends in your code, so ok. In general the number of open pipes is not really an issue, as open files...

How to send continuous stream of data from one process to another via EXECL

I am writing C program which constantly generates two string values named stateName and timer (with the rate of five times per second). I need to concatenate and pass them to another process called ProcessNo3_TEST which is responsible for tokenizing and also displaying them.
The problem is I don't know how to pass them continuously via execl. I had a couple of attempts but none of them were successful. Here is my code which works fine for a single pair of values (e.g. UP2 and 98):
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#define READ 0
#define WRITE 1
int FIFO[2];
char fileDescriptor[10];
char* stringMaker( char *s1,char *s2 );
int main()
{
char lengthInChar[15],msg[200];
int msgLength,i;
char *stateName, *timer;
if (pipe(FIFO) == -1)
{
printf("cannot create pipe\n");
exit(1);
}
sprintf(fileDescriptor, "%d", FIFO[READ]);
stateName = "UP2"; // for instance
timer = "98"; // for instance
msgLength = strlen(stateName) + strlen(timer) +3;
strcpy(msg, stringMaker(stateName, timer) );
write(FIFO[WRITE], msg, msgLength);
switch (fork())
{
case 0:
sprintf(lengthInChar, "%d", msgLength);
execl("ProcessNo3_TEST", "ProcessNo3_TEST", lengthInChar, fileDescriptor, NULL);
exit(1);
case -1:
perror("fork() failed-->");
exit(2);
default:
break;
}
sleep(10);
exit(0);
}
char* stringMaker( char *s1,char *s2 )
{
char *s3;
strcpy(s3,s1);
strcat(s3,"-");
strcat(s3,s2);
strcat(s3,"-");
strcat(s3,"\0");
return s3;
}
Can anyone help on this please?
(I am running CygWin on Windows by the way)
----------UPDATE-------------
As advised in comments below, I found a good example of fdopen() which solved my problem. (Link)
Although one can arrange to pass one pipe end's file descriptor number (in string form) as a program argument, normally one would instead redirect the child process's standard streams to read from (in this case) or write to the pipe. This is usually achieved via dup2(), which you would apply in the child process, after forking but before execl().
Parent and child processes then communicate via the pipe. In this case, the parent writes to the writing end and the child reads from the reading end. Either or both can wrap their file descriptor in a stream by passing it to fdopen(). Then you can use stdio functions with it. Note, too, that after the fork, each process should close the FD for the pipe end it does not intend to use.
If are set on using execl() (often people prefer not to, but sometimes it has it's benefits) than you should use named pipes instead of unanonymous. Anononymous pipe end is lost after execl(). But if you have a named pipe, you can pass it's name as an argument to the execl(), open it in child process and use there.

What happens if a child process won't close the pipe from writing, while reading?

Given the following code:
int main(int argc, char *argv[])
{
int pipefd[2];
pid_t cpid;
char buf;
if (argc != 2) {
fprintf(stderr, "Usage: %s \n", argv[0]);
exit(EXIT_FAILURE);
}
if (pipe(pipefd) == -1) {
perror("pipe");
exit(EXIT_FAILURE);
}
cpid = fork();
if (cpid == -1) {
perror("fork");
exit(EXIT_FAILURE);
}
if (cpid == 0) { /* Child reads from pipe */
close(pipefd[1]); /* Close unused write end */
while (read(pipefd[0], &buf, 1) > 0)
write(STDOUT_FILENO, &buf, 1);
write(STDOUT_FILENO, "\n", 1);
close(pipefd[0]);
_exit(EXIT_SUCCESS);
} else { /* Parent writes argv[1] to pipe */
close(pipefd[0]); /* Close unused read end */
write(pipefd[1], argv[1], strlen(argv[1]));
close(pipefd[1]); /* Reader will see EOF */
wait(NULL); /* Wait for child */
exit(EXIT_SUCCESS);
}
return 0;
}
Whenever the child process wants to read from the pipe, it must first close the pipe's side from writing. When I remove that line close(pipefd[1]); from the child process's if,
I'm basically saying that "okay, the child can read from the pipe, but I'm allowing the parent to write to the pipe at the same time"?
If so, what would happen when the pipe is open for both reading & writing? No mutual exclusion?
Whenever the child process wants to read from the pipe, it must first close the pipe's side from writing.
If the process — parent or child — is not going to use the write end of a pipe, it should close that file descriptor. Similarly for the read end of a pipe. The system will assume that a write could occur while any process has the write end open, even if the only such process is the one that is currently trying to read from the pipe, and the system will not report EOF, therefore. Further, if you overfill a pipe and there is still a process with the read end open (even if that process is the one trying to write), then the write will hang, waiting for the reader to make space for the write to complete.
When I remove that line close(pipefd[1]); from the child's process IF, I'm basically saying that "okay, the child can read from the pipe, but I'm allowing the parent to write to the pipe at the same time"?
No; you're saying that the child can write to the pipe as well as the parent. Any process with the write file descriptor for the pipe can write to the pipe.
If so, what would happen when the pipe is open for both reading and writing — no mutual exclusion?
There isn't any mutual exclusion ever. Any process with the pipe write descriptor open can write to the pipe at any time; the kernel ensures that two concurrent write operations are in fact serialized. Any process with the pipe read descriptor open can read from the pipe at any time; the kernel ensures that two concurrent read operations get different data bytes.
You make sure a pipe is used unidirectionally by ensuring that only one process has it open for writing and only one process has it open for reading. However, that is a programming decision. You could have N processes with the write end open and M processes with the read end open (and, perish the thought, there could be processes in common between the set of N and set of M processes), and they'd all be able to work surprisingly sanely. But you'd not readily be able to predict where a packet of data would be read after it was written.
fork() duplicates the file handles, so you will have two handles for each end of the pipe.
Now, consider this. If the parent doesn't close the unused end of the pipe, there will still be two handles for it. If the child dies, the handle on the child side goes away, but there's still the open handle held by the parent -- thus, there will never be a "broken pipe" or "EOF" arriving because the pipe is still perfectly valid. There's just nobody putting data into it anymore.
Same for the other direction, of course.
Yes, the parent/child could still use the handle to write into their own pipe; I don't remember a use-case for this, though, and it still gives you synchronization problems.
When the pipe is created it is having two ends the read end and write end. These are entries in the User File descriptor table.
Similarly there will be two entries in the File table with 1 as reference count for both the read end and the write end.
Now when you fork, a child is created that is the file descriptors are duplicated and thus the reference count of both the ends in the file table becomes 2.
Now "When I remove that line close(pipefd[1])" -> In this case even if the parent has completed writing, your while loop below this line will block for ever for the read to return 0(ie EOF). This happens since even if the parent has completed writing and closed the write end of the pipe, the reference count of the write end in the File table is still 1 (Initially it was 2) and so the read function still is waiting for some data to arrive which will never happen.
Now if you have not written "close(pipefd[0]);" in the parent, this current code may not show any problem, since you are writing once in the parent.
But if you write more than once then ideally you would have wanted to get an error (if the child is no longer reading),but since the read end in the parent is not closed, you will not be getting the error (Even if the child is no more there to read).
So the problem of not closing the unused ends become evident when we are continuously reading/writing data. This may not be evident if we are just reading/writing data once.
Like if instead of the read loop in the child, you are using only once the line below, where you are getting all the data in one go, and not caring to check for EOF, your program will work even if you are not writing "close(pipefd[1]);" in the child.
read(pipefd[0], buf, sizeof(buf));//buf is a character array sufficiently large
man page for pipe() for SunOS :-
Read calls on an empty pipe (no buffered data) with only one
end (all write file descriptors closed) return an EOF (end
of file).
A SIGPIPE signal is generated if a write on a pipe with only
one end is attempted.

Resources