I created and written to a named pipe in C under Linux. For how long the text that is written in there is saved in the named pipe?
From what I have done, and the bytes of the pipe file after my program is run I suppose that the text is not preserved in the pipe after the program ends. In the mkfifo manual there is no info about this. I know that ordinary pipes are destroyed after the process that have created them is closed. But what about named pipes, that are still in your file system after the program has finished?
This is the code I use to create a named pipe and to write/read from it.
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <string.h>
#include <sys/stat.h>
#include <fcntl.h>
int main(int argc, char *argv[]) {
int FIFOFileDescriptorID;
FIFOFileDescriptorID = mkfifo(argv[1], 0660);
int ProccesID = fork();
if (ProccesID == 0) {
int TempFileDescriptor = 0;
char buffer[512] = "Some random text goes here...";
TempFileDescriptor = open(argv[1], O_WRONLY);
write(TempFileDescriptor, &buffer, sizeof(buffer));
close(TempFileDescriptor);
} else {
int TempFileDescriptor = 0;
char buffer[512];
TempFileDescriptor = open(argv[1], O_RDONLY);
read(TempFileDescriptor, &buffer, sizeof(buffer));
close(TempFileDescriptor);
printf("Received string: %s\n", buffer);
}
return 0;
}
After I have run this program and created and use the pipe for write/read, I run another one – just to read the text from the given pipe. Indeed, there was no text there.
I will exam this thing better, because there is a good change, after I start the program do delete/create the pipe again.
It'll not save anything. When you read/write something to the named pipe, it the process will be blocked unless some other process writes/reads from the same named pipe.
The file stays in the file-system. But the content goes away when reading/writing finishes.
From linux manual,
Once you have created a FIFO special file in this way, any process
can open it for reading or writing, in the same way as an ordinary file.
However, it has to be open at both ends simultaneously before you can
proceed to do any input or output operations on it. Opening a FIFO for
reading normally blocks until some other process opens the same FIFO for
writing, and vice versa.
Here is some code I wrote up to test named pipes. I made sure to handle all errors:
cleanup in SIGPIPE
Look at Wikipedia: http://en.wikipedia.org/wiki/Named_pipe - named pipes persist beyond the lifetime of the process that created or used them, until they are explicitly deleted.
Related
I'm trying to understand what is behind this behaviour in my parent process.
Basically, I create a child process and connect its stdout to my pipe. The parent process continuously reads from the pipe and does some stuff.
I noticed that when inserting the while loop in the parent the stdout seems to be lost, nothing appears on the terminal etc I thought that the output of stdout would somehow go to the pipe (maybe an issue with dup2) but that doesn't seem to be the issue. If I don't continuously fflush(stdout) in the parent process, whatever I'm trying to get to the terminal just won't show. Without a while loop in the parent it works fine, but I'm really not sure why it's happening or if the rest of my implementation is problematic somehow.
Nothing past the read system call seems to be going to the stdout in the parent process. Assuming the output of inotifywait in the pipe is small enough ( 30 > bytes ), what exactly is wrong with this program?
What I expect to happen is the stdout of inotifywait to go to the pipe, then for the parent to read the message, run strtok and print the file name (which only appears in stdout when I fflush)
Running the program with inotify installed and creating any file in the current directory of the program should be enough. Removing the while loop does print the created file's name (as expected).
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <signal.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <fcntl.h>
#include <errno.h>
int main(void) {
char b[100];
int pipefd;
if (mkfifo("fifo", 0666) == -1) {
if (errno != EEXIST) {
perror("mkfifo");
exit(EXIT_FAILURE);
}
}
pid_t pid = fork();
if (pid < 0) {
perror("fork");
exit(1);
}
if ((pipefd = open("fifo", O_RDWR)) < 0) {
perror("open pipe");
exit(EXIT_FAILURE);
}
if (pid == 0) {
dup2(pipefd, 1);
const char* dir = ".";
const char* args[] = {"inotifywait", dir, "-m", "-e",
"create", "-e", "moved_to", NULL};
execvp("inotifywait", (char**)args);
perror("inotifywait");
} else {
while (1) {
fflush(stdout); // the output only appears in stdout with this here
if (read(pipefd, b, 30) < 0) {
perror("problem # read");
exit(1);
}
char filename[30];
printf("anything");
sscanf(b, "./ CREATE %s", filename);
printf("%s", filename);
}
}
}
The streams used by the C standard library are designed in such a way that they are normally buffered (except for the standard error stream stderr).
The standard output stream is normally line buffered, unless the output device is not an interactive device, in which case it is normally fully buffered. Therefore, in your case, it is probably line buffered.
This means that the buffer will only be flushed
when it is full,
when an \n character is encountered,
when the stream is closed (e.g. during normal program termination),
when reading input from an unbuffered or line-buffered stream (in certain situations), or
when you explicitly call fflush.
This explains why you are not seeing the output, because none of the above are happening in your infinite loop (when you don't call fflush). Although you are reading input, you are not doing this from a C standard library FILE * stream. Instead, you are bypassing the C runtime library (e.g. glibc) by using the read system call directly (i.e. you are using a file descriptor instead of a stream).
The simplest solution to your problem would probably be to replace the line
printf("%s", filename);
with:
printf("%s\n", filename);
If stdout is line-buffered (which should be the case if it is connected to a terminal), then the input should automatically be flushed after every line and an explicit call to fflush should no longer be necessary.
I'm trying to read a text file's string from a process, then deliver the string to another process via named pipes on LINUX. The problem is when i type './reader text.txt = receiver' to the console the recieving process' read() function returns an error if i put the line
fcntl(fd, F_SETFL, O_NONBLOCK);
or gets stuck on read() function if i remove it.
heres the process that reads the string (reader)
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <errno.h>
#include <fcntl.h>
#include <sys/wait.h>
int main(int argc,char *argv1[]){
if(argc==4 && strcmp(argv1[2],"=") == 0){
char mesaj[99999]; //message to be delivered
char line[150];
FILE *fp =fopen(argv1[1],"r"); //reading from a text file
if(fp==NULL){
printf("text file error!");
exit(1);
}
while(fgets(line,sizeof(line),fp)){
strcat(mesaj,line); //append every line to message
}
fclose(fp);
mesaj[strlen(mesaj)-1]='\0';
int n =strlen(mesaj)+1;
//printf("got the text %s\n",mesaj);
if(mkfifo("myFifo",0777)== -1 && errno!= EEXIST){
printf("Pipe error");
exit(1);
}
printf("opening\n");
int fd= open("myFifo",O_RDWR);
if(fd==-1){
printf("open error");
exit(1);
}
printf("opened");
if( write(fd, mesaj,sizeof(char)*n)==-1){
printf("write error");
exit(1);
}
printf("written");
close(fd);
printf("closed");
fflush(stdout);
char mesajSizeChar[n];
sprintf(mesajSizeChar, "%d", n);
char *args[]={mesajSizeChar,NULL}; //send the message size as parameter for the other process
char program[]="./";
strcat(program,argv1[3]); // recieved process name as parameter
execv(program,args); // call the other process
perror("execv");
return 0;
}
}
and heres the recieving process (reciever)
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <errno.h>
#include <fcntl.h>
int main(int argc,char *argv1[]){
int mesajSize=atoi(argv1[0]); //convert message size to integer
char mesaj[99999];
printf("\ncame here\n");
int fd= open("myFifo",O_RDWR);
fcntl(fd, F_SETFL, O_NONBLOCK);
printf("\nopen \n");
if(fd==-1)
printf("pipe error\n");
if(read(fd,mesaj,sizeof(char)*mesajSize)==-1)
printf("read error\n");
printf("read \n");
printf("\nworked: %s \n",mesaj);
close(fd);
return 0;
}
The problem is that you closed the pipe in the first process. A pipe doesn't have any permanent storage, it can only hold data while it's open by at least one process. When you close the pipe, the data that you've written to it is discarded.
As a result, when the second process tries to read from the pipe, there's nothing available.
You need to keep the pipe FD open when you execute the second process. Get rid of close(fd); in the reader program.
To use a FIFO or pipe, sender and receiver must run concurrently, but you are trying to run them sequentially. A FIFO or pipe has no persistent storage, so the system does not allow you to write to one unless unless at least one process has the read end open, so as to be able to read it.
Ordinarily, attempts to open a FIFO for writing will block while there are no readers, and vice versa. Your reader is working around this by opening the FIFO for both reading and writing, even though it intends only to write. You will find that if it tries to send too much data to the FIFO then it blocks, because nothing is reading the data, and pipes / FIFOs have limited buffer capacity. When it closes the FIFO's fd, leaving no process with it open, all data previously written to it are lost.
Your receiver also erroneously opens the FIFO for both reading and writing, whereas it should open it only for reading. There being no data to read from it, I would expect attempts to read from it to block indefinitely, unless you put it into non-blocking mode. This seems to be exactly what you describe.
To fix it, I would suggest
taking the code that starts the receiver out of the reader. Instead, start the reader and receiver separately. Alternatively, the reader may start out by fork()ing, with the resulting child process execv()ing the receiver.
The reader should open the FIFO with flag O_WRONLY, and the receiver should open it with mode O_RDONLY.
You should find a different way to convey the message length from reader to receiver, or, better, to avoid needing to tell it the message length in advance at all. You could, for instance, send an initial fixed-length message that conveys the length of the main message data, but more typical would be for the receiver to just keep reading data until it sees EOF.
The reader will cause the receiver to see EOF on the FIFO by closing it, either explicitly or by terminating. This depends on the receiver having it open in read-only mode, however, and there being no other writers.
The reader probably should not attempt to buffer the whole message in memory at once. It should not, in any case, assume that a write() call will transfer the full number of bytes requested -- the return value will tell you how many actually were transferred. You need to be prepared to use multiple write() calls in a loop to transfer all the data.
Similarly, the receiver cannot rely on a single read() call to transfer the full number of bytes requested in one call, even if it has some way to know how many are coming. As with write(), you need to be prepared to use multiple read()s to transfer all the data.
I'm so confused with this, I need to create named pipes using mkfifo (i know how to do this) the program before would use fork to create child processes that would do something, but now I have to replace fork with poll() to watch multiple streams (that's the part I don't get). In more detailed, when i run my program in terminal, its suppose to make the mkfifo files and then wait till a stream comes in, hence just stay there, not closing. Then I open up a new terminal, and need to input exactly this into terminal "cat file1 > (name of the mkfifo files)" and what that should do is make the program read the data that was in file1, on any of the input pipes made from mkfifo. I've looked everywhere but can never put things together to make it work.
this is what i have so far
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <poll.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <fcntl.h>
#include "battleFieldReader.h" //this is where all the data computation is done
main(int argc, char *argv[])
{
if (sscanf (argv[1], "%i", &N) !=1 )
{
printf ("ERROR: Please input an integer.\n");
_exit(EXIT_SUCCESS);
}
struct pollfd pfd[2];
pid = getpid();
printf("pid=%d\n",pid);
N = atoi(argv[1]);
signal(SIGTERM, removePipes);
int i;
char fileName[32];
char fileName2[32];
snprintf (fileName, sizeof(fileName), "%d_%di", pid, 0);
mkfifo(fileName, 0666);
pfd[0].fd = open(fileName, O_RDONLY | O_NDELAY);
pfd[0].events = POLLIN;
snprintf (fileName2, sizeof(fileName2), "%d_%do", pid, 0);
mkfifo(fileName2, 0666);
pfd[1].fd = open(fileName2, O_WRONLY | O_NDELAY);
pfd[1].events = POLLIN;
while(1)
{
int n;
n = poll(pfd, 2, 3000);
if(n < 1)
{
printf("waiting...\n");
continue;
}
if(pfd[0].revents & POLLIN)
{
printf("test\n");
/*ideally this is where i would put a method to compute data but its just an infinite loop, its suppose to stop, so it can do the same thing whenever another file comes on, but I dont know how to stop it either*/
}
}
}
whats happening is I'm creating a pipe 2N times, one for input and output for whatever process id is running the program. then wait till something comes in on one of the pipes and then spit out what needs to be done with the file data. Can anyone clear things with me, if I'm going in the right direction or something.
poll tells you if you can read from or write to a file descriptor without blocking (or without getting an EAGAIN/EWOULDBLOCK result if the fd in non-blocking).
Your code has a number of obvious problems:
You say you want to create 2N fifos, but then you only create 2. Your pfd array has a fixed size of 2, too. You'll need a bigger array and loop to create more.
You open an O_WRONLY file descriptor to write to a pipe, but then you set the events field to POLLIN, which will test for input available. You want POLLOUT to test for output possible.
In your processing loop, you poll two fds, but you only check pfd[0] for availability, and then you never do anything with it. You should read pfd[0].fd after the pfd[0].revents & POLLIN check succeeds. You should also check pfd[1].revents & POLLOUT and write data to pfd[1].fd if that succeeds.
I am trying to pass data from my perl script to my c program using a pipe (uni-directional).
I need to find a way to to do this without messing with the child programs STDIN or STDOUT, so I try creating a new handle and passing the fd.
I create 2 IO::Handles and create a pipe. I write to one end of the pipe and attempt to pass the File descriptor of the other end of the pipe to my child program that is being execed. I pass the file descriptor by setting an ENV variable. Why does this not work? (It does not print out 'hello world'). As far as I know, file descriptors and pipes are inherited by the child when exec'd.
Perl script:
#!/opt/local/bin/perl
use IO::Pipe;
use IO::Handle;
my $reader = IO::Handle->new();
my $writer = IO::Handle->new();
$reader->autoflush(1);
$writer->autoflush(1);
my $pipe = IO::Pipe->new($reader, $writer);
print $writer "hello world";
my $fh = $reader->fileno;
$ENV{'MY_FD'} = $fh;
exec('./child') or print "error opening app\n";
# No more code after this since exec replaces the current process
C Program, app.c (Compiled with gcc app.c -o child):
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
int main(int argc, char ** argv) {
int fd = atoi(getenv("MY_FD"));
char buf[12];
read(fd, buf, 11);
buf[11] = '\0';
printf("fd: %d\n", fd);
printf("message: %s\n", buf);
}
Output:
fd: 3
message:
The message is never passed through the pipe to the C program. Any suggestions?
Your pipe file descriptors are set FD_CLOEXEC, and so are closed upon exec().
Perl's $^F controls this behavior. Try something like this, before you call IO::Pipe->new:
$^F = 10; # Assumes we don't already have a zillion FDs open
Alternatively, you can with Fcntl clear the FD_CLOEXEC flag yourself after creating the pipe.
I found the solution. Some people said that it was not possible with exec, that it would not see pipes or file descriptors, but that was not correct.
Turns out that perl closes/invalidates all fd > 2 automatically unless you say otherwise.
Adding the following flags to the FD fixes this problem (where READ is the handle here, NOT STDIN):
my $flags = fcntl(READ, F_GETFD, 0);
fcntl(READ, F_SETFD, $flags & ~FD_CLOEXEC);
Your program is failing because exec calls another program and never returns. It isn't designed for communication with another process at all.
You probably wrote the above code based on the IO::Pipe documentation, which says "ARGS are passed to exec". That isn't what it means, though. IO::Pipe is for communication between two processes within your Perl script, which are created by fork. They mean the execution of the new process, rather than a call to exec in your own code.
Edit: for one-directional communication, all you need is open with a pipe:
open my $prog, '|-', './child' or die "can't run program: $!";
print {$prog} "Hello, world!";
Rodrigo, I can tell you that your file descriptor is no longer valid when you exec into the c app.
Please be aware that I just say it is INVALID, but it still exists in the environment variables. The FD=3 will continue existing until the whole process ends.
You can check the fd by fcntl. The code is listing below
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
int main(int argc, char ** argv) {
int fd = atoi(getenv("MY_FD"));
char buf[12];
read(fd, buf, 11);
buf[11] = '\0';
printf("fd: %d, if fd still valid: %d\n", fd, fcntl(fd, F_GETFD));
printf("strlen %d\n", (int)strlen(buf));
printf("message: %s\n", buf);
}
You can see that MY_FD=3 will always in ENV as the process doesn't destroy itself, so you can get fd as 3. But, this file descriptor has been invalid. so the result of fcntl(fd, F_GETFD) will be -1, and the length you read from fd will be 0.
That's why you will never see the "hello world" sentence.
One more thing, #dan1111 is right, but you don't need to open a new pipe, as you have already done so.
All you need to is just set MY_FD=0, like
$ENV{'MY_FD'} = 0;
The STDIN/OUT is another independent process that always exists, so the pipe will not broken down when your perl app exec into c app. That's why you can read from what you input in app.
If your requirement is writing from another file hanle, please try to make that file handle an independent process and always exist, just like STDIN.
I have written a C utility for Linux that checks the contents of /proc/net/dev once every second. I open the file using fopen("/proc/net/dev", "r") and then fclose() when I'm done.
Since I'm using a 'pseudo' file rather than a real one, does it matter if I open/close the file each time I read from it, or should I just open it when my app starts and keep it open the whole time? The utility is launched as a daemon process and so may run for a long time.
It shouldn't matter, no. However, there might be issues with caching/buffering, which would mean it's actually best (safest) to do as you do it, and re-open the file every time. Since you do it so seldom, there's no performance to be gained by not doing it, so I would recommend keeping your current solution.
What you want is unbuffered reading. Assuming you can't just switch to read() calls, open the device, and then set the stream to unbuffered mode. This has the additional advantage that there is no need to close the stream when you're done. Just rewind it, and start reading again.
FILE *f = fopen("/proc/net/dev", "r");
setvbuf(f, NULL, _IONBF, 0);
while (running)
{
rewind(f);
...do your reading...
}
The pseudo files in "/proc" are dangerous for daemons because if the kernel decides to drop them they just vanish leaving you with an invalid FILE * struct. That means that your strategy is the only correct one to treat a file in "/proc" (but no one is going to expect that "/proc/net/dev" is removed by the kernel during runtime).
In general (especially for files in "/proc/[PID]") one should open files in "/proc" before an operation and close them as soon as possible after the operation is done.
See this example code. It forks and reads the "/proc/[PID]/status" file of the child, once before the child has exited and once during the cleanup of the child.
#include <unistd.h>
#include <time.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/wait.h>
int main(int argc, char** argv){
pid_t child=fork();
if(child==0){
sleep(1);
} else {
char path[256],buffer[256]; int status,read_length;
sprintf(path,"/proc/%i/status",child);
//do a read while the child is alive
FILE *fd=fopen(path,"r");
if(fd!=0){
read_length=fread(buffer,1,255,fd);
printf("Read: %i\n",read_length);
fclose(fd);
}
//repeat it while the child is cleaned up
fd=fopen(path,"r");
wait(&status);
if(fd!=0){
read_length=fread(buffer,128,1,fd);
printf("Read: %i\n",read_length);
fclose(fd);
}
}
}
The result is as follows
f5:~/tmp # ./a.out
Read: 255
Read: 0
So you see, you could easily get an unexpected result from files in "/proc" if they get deleted by the kernel during you program runtime.