I am trying to pass data from my perl script to my c program using a pipe (uni-directional).
I need to find a way to to do this without messing with the child programs STDIN or STDOUT, so I try creating a new handle and passing the fd.
I create 2 IO::Handles and create a pipe. I write to one end of the pipe and attempt to pass the File descriptor of the other end of the pipe to my child program that is being execed. I pass the file descriptor by setting an ENV variable. Why does this not work? (It does not print out 'hello world'). As far as I know, file descriptors and pipes are inherited by the child when exec'd.
Perl script:
#!/opt/local/bin/perl
use IO::Pipe;
use IO::Handle;
my $reader = IO::Handle->new();
my $writer = IO::Handle->new();
$reader->autoflush(1);
$writer->autoflush(1);
my $pipe = IO::Pipe->new($reader, $writer);
print $writer "hello world";
my $fh = $reader->fileno;
$ENV{'MY_FD'} = $fh;
exec('./child') or print "error opening app\n";
# No more code after this since exec replaces the current process
C Program, app.c (Compiled with gcc app.c -o child):
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
int main(int argc, char ** argv) {
int fd = atoi(getenv("MY_FD"));
char buf[12];
read(fd, buf, 11);
buf[11] = '\0';
printf("fd: %d\n", fd);
printf("message: %s\n", buf);
}
Output:
fd: 3
message:
The message is never passed through the pipe to the C program. Any suggestions?
Your pipe file descriptors are set FD_CLOEXEC, and so are closed upon exec().
Perl's $^F controls this behavior. Try something like this, before you call IO::Pipe->new:
$^F = 10; # Assumes we don't already have a zillion FDs open
Alternatively, you can with Fcntl clear the FD_CLOEXEC flag yourself after creating the pipe.
I found the solution. Some people said that it was not possible with exec, that it would not see pipes or file descriptors, but that was not correct.
Turns out that perl closes/invalidates all fd > 2 automatically unless you say otherwise.
Adding the following flags to the FD fixes this problem (where READ is the handle here, NOT STDIN):
my $flags = fcntl(READ, F_GETFD, 0);
fcntl(READ, F_SETFD, $flags & ~FD_CLOEXEC);
Your program is failing because exec calls another program and never returns. It isn't designed for communication with another process at all.
You probably wrote the above code based on the IO::Pipe documentation, which says "ARGS are passed to exec". That isn't what it means, though. IO::Pipe is for communication between two processes within your Perl script, which are created by fork. They mean the execution of the new process, rather than a call to exec in your own code.
Edit: for one-directional communication, all you need is open with a pipe:
open my $prog, '|-', './child' or die "can't run program: $!";
print {$prog} "Hello, world!";
Rodrigo, I can tell you that your file descriptor is no longer valid when you exec into the c app.
Please be aware that I just say it is INVALID, but it still exists in the environment variables. The FD=3 will continue existing until the whole process ends.
You can check the fd by fcntl. The code is listing below
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
int main(int argc, char ** argv) {
int fd = atoi(getenv("MY_FD"));
char buf[12];
read(fd, buf, 11);
buf[11] = '\0';
printf("fd: %d, if fd still valid: %d\n", fd, fcntl(fd, F_GETFD));
printf("strlen %d\n", (int)strlen(buf));
printf("message: %s\n", buf);
}
You can see that MY_FD=3 will always in ENV as the process doesn't destroy itself, so you can get fd as 3. But, this file descriptor has been invalid. so the result of fcntl(fd, F_GETFD) will be -1, and the length you read from fd will be 0.
That's why you will never see the "hello world" sentence.
One more thing, #dan1111 is right, but you don't need to open a new pipe, as you have already done so.
All you need to is just set MY_FD=0, like
$ENV{'MY_FD'} = 0;
The STDIN/OUT is another independent process that always exists, so the pipe will not broken down when your perl app exec into c app. That's why you can read from what you input in app.
If your requirement is writing from another file hanle, please try to make that file handle an independent process and always exist, just like STDIN.
Related
I am trying to use a pipe to rederect stdout into a pipe and read it later. I will use this later with fork(), where the child process starts a different program that I need to comunicate with. This is my Code:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <string.h>
int main(){
printf("Starting Test\n");
int myPipe[2], nbytes;
char readbuffer[80];
pipe(myPipe);
int backup = dup(1); //store stdout
if (dup2(1,myPipe[1])< 1){printf("error");} //copy stdout in the input end of my pipe
printf("in pipe\n"); //print something in stdout -> in my pipe
nbytes = read(myPipe[0],readbuffer,sizeof(readbuffer)); //read output of my pipe
dup2(myPipe[1],backup); // restore stdout
printf("recived: %s",readbuffer); //prit out what I recived
return 0;
}
I expected it to print out:
Starting Test
recived: in pipe
But the output that I get is:
Starting Test
in pipe
recived: #����U
So I assume that stdout was not copied properly, as I get the "in pipe" before the "recived: ..." But the dup2() call throws no errors.
I read some tutorials, mostly this one https://tldp.org/LDP/lpg/node11.html but I can't find my error... Thank you for your help!
The code has a couple of problems:
In dup2(1,myPipe[1]) the parameters are back to front. That makes mypipe[1] be the same as 1. But instead you need it to be the other way around: dup2(myPipe[1],1)
dup2(myPipe[1],backup) is also wrong. That makes backup be the same as mypipe[1]. What you want instead is to make 1 the same as backup: dup2(backup, 1).
Smaller problem but printf does not output a NUL character. So the read will not result in a valid NUL terminated string. Solve that by initialising: char readbuffer[80] = "";
I'm so confused with this, I need to create named pipes using mkfifo (i know how to do this) the program before would use fork to create child processes that would do something, but now I have to replace fork with poll() to watch multiple streams (that's the part I don't get). In more detailed, when i run my program in terminal, its suppose to make the mkfifo files and then wait till a stream comes in, hence just stay there, not closing. Then I open up a new terminal, and need to input exactly this into terminal "cat file1 > (name of the mkfifo files)" and what that should do is make the program read the data that was in file1, on any of the input pipes made from mkfifo. I've looked everywhere but can never put things together to make it work.
this is what i have so far
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <poll.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <fcntl.h>
#include "battleFieldReader.h" //this is where all the data computation is done
main(int argc, char *argv[])
{
if (sscanf (argv[1], "%i", &N) !=1 )
{
printf ("ERROR: Please input an integer.\n");
_exit(EXIT_SUCCESS);
}
struct pollfd pfd[2];
pid = getpid();
printf("pid=%d\n",pid);
N = atoi(argv[1]);
signal(SIGTERM, removePipes);
int i;
char fileName[32];
char fileName2[32];
snprintf (fileName, sizeof(fileName), "%d_%di", pid, 0);
mkfifo(fileName, 0666);
pfd[0].fd = open(fileName, O_RDONLY | O_NDELAY);
pfd[0].events = POLLIN;
snprintf (fileName2, sizeof(fileName2), "%d_%do", pid, 0);
mkfifo(fileName2, 0666);
pfd[1].fd = open(fileName2, O_WRONLY | O_NDELAY);
pfd[1].events = POLLIN;
while(1)
{
int n;
n = poll(pfd, 2, 3000);
if(n < 1)
{
printf("waiting...\n");
continue;
}
if(pfd[0].revents & POLLIN)
{
printf("test\n");
/*ideally this is where i would put a method to compute data but its just an infinite loop, its suppose to stop, so it can do the same thing whenever another file comes on, but I dont know how to stop it either*/
}
}
}
whats happening is I'm creating a pipe 2N times, one for input and output for whatever process id is running the program. then wait till something comes in on one of the pipes and then spit out what needs to be done with the file data. Can anyone clear things with me, if I'm going in the right direction or something.
poll tells you if you can read from or write to a file descriptor without blocking (or without getting an EAGAIN/EWOULDBLOCK result if the fd in non-blocking).
Your code has a number of obvious problems:
You say you want to create 2N fifos, but then you only create 2. Your pfd array has a fixed size of 2, too. You'll need a bigger array and loop to create more.
You open an O_WRONLY file descriptor to write to a pipe, but then you set the events field to POLLIN, which will test for input available. You want POLLOUT to test for output possible.
In your processing loop, you poll two fds, but you only check pfd[0] for availability, and then you never do anything with it. You should read pfd[0].fd after the pfd[0].revents & POLLIN check succeeds. You should also check pfd[1].revents & POLLOUT and write data to pfd[1].fd if that succeeds.
The following simplified piece of code is executed by a thread in the background. The thread runs until he is told to exit (by user input).
In the code below I have removed some error checking for better readability. Even with error checking the code works well and both the master and the slave are created and/or opened.
...
int master, slave;
char *slavename;
char *cc;
master = posix_openpt(O_RDWR);
grantpt(master);
unlockpt(master);
slavename = ptsname(master);
slave = open(slavename, O_RDWR);
printf("master: %d\n",master);
printf("slavename: %s\n",slavename);
On my machine the output is the following:
master: 3
slavename: /dev/pts/4
So I thought that opening an xterm with the command xterm -S4/3 (4 = pt-slave, 3 = pt-master) while my program is running should open a new xterm window for the created pseudoterminal. But xterm just starts running without giving an error or any further informations but does not open a window at all. Any suggestions on that?
EDIT:
Now with Wumpus Q. Wumbley's help xterm starts normally, but I can't redirect any output to it. I tried:
dup2(slave, 1);
dup2(slave, 2);
printf("Some test message\n");
and opening the slave with fopen and then using fprinf. Both didn't work.
The xterm process needs to get access to the file descriptor somehow. The intended usage of this feature is probably to launch xterm as a child process of the one that created the pty. There are other ways, though. You could use SCM_RIGHTS file descriptor passing (pretty complicated) or, if you have a Linux-style /proc filesystem try this:
xterm -S4/3 3<>/proc/$PID_OF_YOUR_OTHER_PROGRAM/fd/3
'
You've probably seen shell redirection operators before: < for stdin, > for stdout, 2> for stderr (file descriptor 2). Maybe you've also seen other file descriptors being opend for input or output with things like 3<inputfile 4>outputfile. Well the 3<> operator here is another one. It opens file descriptor 3 in read/write mode. And /proc/PID/fd/NUM is a convenient way to access files opened by another process.
I don't know about the rest of the question. I haven't tried to use this mode of xterm before.
OK, the trick with /proc was a bad idea. It's equivalent to a fresh open of /dev/ptmx, creating a new unrelated pty.
You're going to have to make the xterm a child of your pty-creating program.
Here's the test program I used to explore the feature. It's sloppy but it revealed some interesting things. One interesting thing is that xterm writes its window ID to the pty master after successful initialization. This is something you'll need to deal with. It appears as a line of input on the tty before the actual user input begins.
Another interesting thing is that xterm (the version in Debian at least) crashes if you use -S/dev/pts/2/3 in spite of that being specifically mentioned in the man page as an allowed format.
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
int main(void)
{
int master;
char *slavename, window[64], buf[64];
FILE *slave;
master = posix_openpt(O_RDWR);
grantpt(master);
unlockpt(master);
slavename = ptsname(master);
printf("master: %d\n", master);
printf("slavename: %s\n", slavename);
snprintf(buf, sizeof buf, "-S%s/%d", strrchr(slavename,'/')+1, master);
if(!fork()) {
execlp("xterm", "xterm", buf, (char *)0);
_exit(1);
}
slave = fopen(slavename, "r+");
fgets(window, sizeof window, slave);
printf("window: %s\n", window);
fputs("say something: ", slave);
fgets(buf, sizeof buf, slave);
fprintf(slave, "you said %s\nexiting in 3 seconds...\n", buf);
sleep(3);
return 0;
}
I created and written to a named pipe in C under Linux. For how long the text that is written in there is saved in the named pipe?
From what I have done, and the bytes of the pipe file after my program is run I suppose that the text is not preserved in the pipe after the program ends. In the mkfifo manual there is no info about this. I know that ordinary pipes are destroyed after the process that have created them is closed. But what about named pipes, that are still in your file system after the program has finished?
This is the code I use to create a named pipe and to write/read from it.
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <string.h>
#include <sys/stat.h>
#include <fcntl.h>
int main(int argc, char *argv[]) {
int FIFOFileDescriptorID;
FIFOFileDescriptorID = mkfifo(argv[1], 0660);
int ProccesID = fork();
if (ProccesID == 0) {
int TempFileDescriptor = 0;
char buffer[512] = "Some random text goes here...";
TempFileDescriptor = open(argv[1], O_WRONLY);
write(TempFileDescriptor, &buffer, sizeof(buffer));
close(TempFileDescriptor);
} else {
int TempFileDescriptor = 0;
char buffer[512];
TempFileDescriptor = open(argv[1], O_RDONLY);
read(TempFileDescriptor, &buffer, sizeof(buffer));
close(TempFileDescriptor);
printf("Received string: %s\n", buffer);
}
return 0;
}
After I have run this program and created and use the pipe for write/read, I run another one – just to read the text from the given pipe. Indeed, there was no text there.
I will exam this thing better, because there is a good change, after I start the program do delete/create the pipe again.
It'll not save anything. When you read/write something to the named pipe, it the process will be blocked unless some other process writes/reads from the same named pipe.
The file stays in the file-system. But the content goes away when reading/writing finishes.
From linux manual,
Once you have created a FIFO special file in this way, any process
can open it for reading or writing, in the same way as an ordinary file.
However, it has to be open at both ends simultaneously before you can
proceed to do any input or output operations on it. Opening a FIFO for
reading normally blocks until some other process opens the same FIFO for
writing, and vice versa.
Here is some code I wrote up to test named pipes. I made sure to handle all errors:
cleanup in SIGPIPE
Look at Wikipedia: http://en.wikipedia.org/wiki/Named_pipe - named pipes persist beyond the lifetime of the process that created or used them, until they are explicitly deleted.
Basically I want to do in C (and without buffering) the same as this bash-script:
#!/bin/sh
cat ./fifo_in | myprogram > ./fifo_out
In other words I want to exec "myprogram" and redirect its stdin and stdout to two pipes which have been created previously.
Another program is feeding data into fifo_in and reading out of fifo_out.
Of course it would be easy to just read from ./fifo_in, buffer it in the parent and write to myprogram's stdin (and reverse for stdout and ./fifo_out) but I think there is probably a way to let "myprogram" read/write directly from/to the fifos without buffering in the parent process.
Edit:
Eugen's answer seems to be the correct one, but I cannot get it to work.
I use this function on the C-side, which seems correct to me:
pid_t execpipes(const char *wd, const char *command, const char *pipename)
{
char pipename_in[FALK_NAMESIZE];
char pipename_out[FALK_NAMESIZE];
strcpy(pipename_in, FALKPATH);
strcat(pipename_in, "/");
strcat(pipename_in, FALK_FIFO_PATH);
strcat(pipename_in, "/");
strncat(pipename_in, pipename, FALK_NAMESIZE-2);
strcpy(pipename_out, pipename_in);
strcat(pipename_out, "R");
pid_t pid;
pid = fork();
if (pid < 0)
{ //Error occured
perror("fork");
exit(1);
}
if (pid == 0)
{
chdir(wd);
d("execpipes: pipename_in=\"%s\"\n", pipename_in);
d(" pipename_out=\"%s\"\n", pipename_out);
freopen(pipename_in,"r",stdin);
freopen(pipename_out,"w",stdout);
d("execpipes: command=\"%s\"\n", command);
execl("/bin/sh", "sh", "-c", command, (char *)NULL); // using execv is probably faster
// Should never get here
perror("execl");
exit(1);
}
return pid;
}
I read and write the pipes from a PHP-script (only relevant part posted):
$pipe_in = fopen($fp.$pipename, "w");
$DEBUG .= "Write to pipe_in\n";
$ret = fwrite($pipe_in, $in);
$pipe_out = fopen($fp.$pipename.'R', "r");
$DEBUG .= "Read from pipe_out\n";
$atext = fread($pipe_out, 200000); // Program hangs here
The program is started correctly, receives the input via $pipe_in correctly, processes the data correctly and (because it ran fine for many months) I assume it puts out the data correctly to stdout, but when I try to read from $pipe_out, it hangs. I know that the pipes themselves are set up correctly because if I don't open $pipe_out, the program does not get any input - which makes sense because there is no reader for $pipe_out and therefore the pipeline is not complete. So I can open $pipe_out, but I cannot read anything from it, which is quite strange.
Edit2:
Program works now, thanks guys - For some reason the first pipe has to be closed before you can read from the second pipe:
$pipe_in = fopen($fp.$pipename, "w");
$pipe_out = fopen($fp.$pipename.'R', "r");
$DEBUG .= "Write to pipe_in\n";
$ret = fwrite($pipe_in, $in);
fclose($pipe_in);
$DEBUG .= "Read from pipe_out\n";
$atext = fread($pipe_out, 200000);
fclose($pipe_out);
unlink($fp.$pipename);
unlink($fp.$pipename.'R');
I'd write a small wrapper for myprogram, that does
freopen("./fifo_in","r",stdin)
freopen("./fifo_out","w",stdout)
(Ofcourse not with constant paths!), then execve myprogram
Korn shell supports coprocesses, which I think effectively does what you ask: read from a pipe and write to a pipe (which can be stdout and stdin of a C process)
http://www.dartmouth.edu/~rc/classes/ksh/coprocesses.html
How about
myprogram < ./fifo_in > ./fifo_out
?
As for getting rid of the buffering: Since your program directly reads/writes the pipes, the buffering shouldn't hurt you.
An important point is that the process which writes fifo_in should flush properly so you don't have to wait. The same goes for your output: As soon as a "work unit" is complete, flush your stdout which will make the data available to whoever reads the output pipe.
But you can't do anything in myprogram to make the writer of fifo_in flush its buffers.
[EDIT] To do this from C (without the help of a shell), use code like this:
- Put the names of the two pipes into local variables on the stack
- Call `fork()`. If that returns '0', then open the two fifos with `freopen()` [like Eugen suggested][1]
- Call `execve` to launch the real exec.
That's (in a nutshell) what the shell is doing when it runs commands. Make sure the parent process (the one where fork() returns a PID != 0) handles the signal SIGCHLD
Perhaps you are looking of a named pipe? For example:
mkfifo fifo_in
As a test stub for my_program.c, to read fifo_in via the buffered stdin:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int main(void) {
char buf[80];
if (!freopen("./fifo_in", "r", stdin)) {
perror("freopen");
exit(EXIT_FAILURE);
}
while (!ferror(stdin)) {
while (fgets(buf, sizeof buf, stdin))
fputs(buf, stdout);
sleep(1);
}
return 0;
}
Then as a test for the writer, using the bash shell:
for x in {1..10}; do
echo $x
echo $x >> fifo_in
sleep 1
done
Notes:
I'd prefer to use unbuffered I/O.
The writer, at least on my machine, blocks until there is a reader.
The reader, in this sample, cannot tell when the writer is finished.