static int pipefd[2];
static pid_t cpid = 0;
void sigtimeout(int num) {
kill(cpid, 9);
close(pipefd[1]);
close(pipefd[0]);
pipe(pipefd);
write(pipefd[1], "T/O\n", 5);
}
void settimer(float time) {
struct itimerval timer;
timer.it_value.tv_sec = (int)time;
timer.it_value.tv_usec = (timeout - (int)time) * 1000000;
timer.it_interval.tv_sec = 0;
timer.it_interval.tv_usec = 0;
setitimer(ITIMER_REAL, &timer, NULL);
}
pid_t popen2(char *cmd) {
if (pipe(pipefd) == -1)
return -1;
int pid;
if ((pid = fork()) == -1)
return -1;
if (pid == 0) {
close(STDIN_FILENO);
close(STDERR_FILENO);
dup2(pipefd[1], STDOUT_FILENO);
close(pipefd[0]);
close(pipefd[1]);
execlp("sh", "sh", "-c", cmd, NULL);
_exit(EXIT_SUCCESS);
} else
settimer(timeout);
return pid;
}
void getcmd(const Block *block, char *output)
{
if (block->signal)
{
output[0] = block->signal;
output++;
}
strcpy(output, block->icon);
char *cmd;
if (button)
{
cmd = strcat(exportstring, block->command);
cmd[14] = '0' + button;
cpid = popen2(cmd);
if (cpid == -1) {
close(pipefd[0]);
close(pipefd[1]);
return;
}
cmd[16] = '\0';
}
else
{
cmd = block->command;
cpid = popen2(cmd);
if (cpid == -1) {
close(pipefd[0]);
close(pipefd[1]);
return;
}
}
button = 0;
waitpid(cpid, 0, 0);
settimer(0);
kill(cpid, 9);
close(pipefd[1]);
int i = strlen(block->icon);
read(pipefd[0], output+i, CMDLENGTH-i-delimLen);
close(pipefd[0]);
for (char *c = output; *c; c++)
if (*c == '\n') {
c[1] = '\0';
break;
}
i = strlen(output);
if (delim[0] != '\0') {
//only chop off newline if one is present at the end
i = output[i-1] == '\n' ? i-1 : i;
strncpy(output+i, delim, delimLen);
}
else
output[i++] = '\0';
}
So I'm trying to modify dwmblocks to add a timeout functionality. That way, if a command hangs, the whole status bar doesn't freeze.
Everything seems to work fine, but there is one little catch.
My code makes my whole linux system crash, cause it leaves some file descriptor open every time it runs a command. As you probably know, linux has protections against that, so every other application on my system that tries to open a file descriptor crashes too.
Thing is, I'm literally closing every pipe I open in my code, even the ones that open automatically when you fork. I just can't figure out what the problem is.
I'd really appreciate any help.
BTW: I'm only putting the relevant code here, because the problem is with the file descriptors. This is the only place where I work with file descriptors in my code.
Feel free to ask for more parts of the code if you feel it's relevant in some way :)
Your popen2() function leaves both pipe ends in the parent process open. Parent should always close the write end (that the child process writes to). Perhaps that is the one that leaks.
If you were to print (int)getpid() to show the process ID, you could list the pseudofiles in /proc/PID/fd/ as those describe the file descriptor that process has currently open. (If you use ls -laF /proc/PID/fd/ with the process ID number instead of PID, you can even see what/where the descriptors are open.)
In this answer here, I recently showed how one can implement a safe_pipe(), run(), and wait_all_children() functions for exploring complicated piping schemes between child processes. safe_pipe() sets the close-on-exec flag for each pipe descriptor by default, and run() clears that flag only in the child process if it uses such a descriptor, so the forked child processes do not have the other pipes' descriptors open at all. The included example.c also shows how the parent process must close all pipe ends used by the child processes when the child processes have been started, so that the child processes correctly detect end of inputs. (If the parent process has write ends open, then it could write to the pipe, so any child reading from the read end of that pipe will never see end-of-input.)
If necessary or helpful, I'm willing to explain how safe_pipe() and run() and wait_all_children() do what they do, and also why. In particular, run() uses an extra pipe between the child process prior to exec, and the parent process, to detect problems exec'ing the specified binary (and other errors). While there may be typos in there, the two are very robust implementations, not leaking resources etc.
Well this is embarassing hahaha
I had a typo in my settimer function : timeout was supposed to be time.
So the real problem was due to a racing condition, even though my piping system is far from perfect.
That goes to show how hard bugs can be to debug sometimes.
Related
I have a legacy piece of code where I am delegating certain logic to a child process. The requirement is that parent writes to the pipe and the child reads on it. After reading, child writes something to a new pipe and parent reads from it.
In the code below, function send() is called periodically by a dedicated thread in the parent process.
LaunchWorker() ensures that the child process is forked only on the first call to it.
1) I am not being able to figure out how to close the read and write ends of the two descriptors so that the old data written to the pipe is flushed on every write.
2) Also is calling pipe() twice needed for the two descriptors?
Any input to make this code work will be greatly appreciated.
Thanks!
typedef struct
{
WebClient* wc; // user defined class
string url;
string data;
size_t dataLen;
DownloadObserver* downloadCallback; // user defined class
string* token;
}Payload;
PayLoad pl;
static pid_t worker_pid = 0;
int fd1[2];
int fd2[2];
bool done = false;
void LaunchWorker()
{
if (worker_pid != 0)
{
return;
}
pipe(fd1);
pipe(fd2);
worker_pid = fork();
}
void send()
{
//populate pl;
LaunchWorker();
if (worker_pid == 0)
{
while(true)
{
close(fd1[1]);
close(fd2[0]);
int cr = read(fd1[0], &pl, sizeof(Payload));
if (cr > 0)
{
// Upload some data to a remote http endpoint - uses libcurl
//if upload success, done = true
int cw = write(fd2[1], &done, sizeof(bool));
if (cw > 0)
{
// success
}
else
{
// failure
}
}
else
{
// failure
}
}
}
else if (workper_pid > 0)
{
close(fd1[0]);
close(fd2[1]);
int pw = write(fd1[1], &pl, sizeof(Payload));
if (pw > 0)
{
int pr = read(fd2[0], &done, sizeof(bool));
if (pr > 0)
{
// do something with value read
}
else
{
// failure
}
}
else
{
failure
}
}
}
Firstly, you need to close the unused ends of the pipe immediately after the fork, so the parent should close the read end of fd1 and the write end of fd2 and the child should close the write end of fd1 and the read end of fd2. If you don't do this, the pipes will not be closed when you finish.
Then, when you want to stop everything, I would have the writer of each pipe close the write end of the pipe. i.e.
parent closes write end of fd1
child reads eof on fd1 and closes it
child closes the write end of fd2
parent reads eof on fd2 and closes it.
Or the other way around if the child initiates shut down.
Finally, the parent process should issue the wait system call to collect the exit result of the child.
And yes, you do need two pipes. Pipes are one way.
I have been making simple shell which performs pipe.
Here is some code for operating pipe syntax.
int fd[2];
int stdin_copy;
int stdout_copy;
int status;
char * msg;
if (pipe(fd) == -1) {
perror("pipe");
exit(1);
}
// fd[0] : process read from fd[0]
// fd[1] : process write to fd[1]
if (execok(pr_words) == 0) { /* is it executable? */
status = fork(); /* yes; create a new process */
if (status == -1) { /* verify fork succeeded */
perror("fork");
exit(1);
} else if (status == 0) { /* in the child process... */
stdout_copy = dup(1);
close(1); // close standard output
dup(fd[1]);
close(fd[0]);
close(fd[1]); // close and fd[1] will be stdout
pr_words[l_nwds] = NULL; /* mark end of argument array */
status = execve(path, pr_words, environ); /* try to execute it */
perror("execve"); /* we only get here if */
exit(0); /* execve failed... */
}
/*------------------------------------------------*/
/* The parent process (the shell) continues here. */
/*------------------------------------------------*/
else if (status > 0) { // in the parent process....
wait( & status); /* wait for process to end */
if (execok(af_words) == 0) {
if (pipe(fd2) == -1) {
perror("pipe");
exit(1);
}
status = fork();
if (status == -1) {
perror("fork");
exit(1);
} else if (status == 0) { // in the child process...
stdin_copy = dup(0);
close(0);
dup(fd[0]);
close(fd[1]);
close(fd[0]);
read(fd[0], readbuffer, sizeof(readbuffer));
af_words[r_nwds] = NULL; /* mark end of argument array */
status = execve(path, af_words, environ); /* try to execute it */
} else if (status > 0) {
wait( & status);
msg = "over";
write(2, msg, strlen(msg));
close(fd[0]);
close(fd[1]);
dup2(stdin_copy, 0);
dup2(stdout_copy, 1);
close(stdin_copy);
close(stdout_copy);
printf("%s", "hi");
}
} else {
/*----------------------------------------------------------*/
/* Command cannot be executed. Display appropriate message. */
/*----------------------------------------------------------*/
msg = "*** ERROR: '";
write(2, msg, strlen(msg));
write(2, af_words[0], strlen(af_words[0]));
msg = "' cannot be executed.\n";
write(2, msg, strlen(msg));
}
}
} else {
/*----------------------------------------------------------*/
/* Command cannot be executed. Display appropriate message. */
/*----------------------------------------------------------*/
msg = "*** ERROR: '";
write(2, msg, strlen(msg));
write(2, pr_words[0], strlen(pr_words[0]));
msg = "' cannot be executed.\n";
write(2, msg, strlen(msg));
}
pr_words and af_words is two-dimensional pointer containing command, right-side and left-side of pipe. (ex. ls | cat -> pr_words = "ls\0" , af_words = "cat\0")
And, first I make child process using fork() and register fd[1] for standard output. (and also save stdin file descriptor before closing stdin) And after execute left side of command, make other child process for handling right side of command.
Similarly, I saved stdout file descriptor before close stdout and made fd[0] standard input. By using input from first outcome of execve function, I thought every outcome would be saved in fd[1]. (Because this was currently registered as std output).
And, finally, restore pipe input and output to standard output. (I don't want to use dup2 but I have no choice because of my lack of knowledge )
However, in execution of this code, after I enter the 'ls | cat', there is no output. Furthermore, I set every entry of terminal will print '#'. (which means that '# ls' or '# cat' ...) But, after enter above pipe command, that program even does not print '#'.
I guess input and output stream of this program are completely twisted after dealing with pipe command.
How can I fix it? I mean, I want save outcome of first execve into fd[1] and after using this fd[1] for performing second execve, make final outcome will be printed through stdout file description.
I see a few issues with your code at least:
First off, you shouldn't wait() on the first process before starting the second one. A pipe only has a few KB of buffer in it, after which your shell will hang if the first child process tries to continue to write there. You need to start both children before wait()ing for each of them. Just move the first wait(&status) call down next to the other one. You'll probably want to use waitpid or something later so you know which one finished first and which status goes to which, but you can address that once you get the basics working.
Secondly, all variables and file descriptor mappings in your program are copied when you fork(). Therefore, you don't need to save stdin or stdout in either child process, because none of the changes you make in the child processes will affect the parent. Furthermore, because you only initialize stdin_copy and stdout_copy in the child processes, the versions of those variables you use in the parent process after the second fork() are uninitialized. This is what's causing the parent shell's I/O to be messed up after executing this code. You don't actually need to do anything in the parent after forking the second time to maintain the original stdin and stdout there -- you never change them in that process before that point. You probably want to remove all of this from the post-fork parent code:
close(fd[0]);
close(fd[1]);
dup2(stdin_copy, 0);
dup2(stdout_copy, 1);
close(stdin_copy);
close(stdout_copy);
Thirdly, why are you reading from the pipe before calling execve() in the second child? That's just going to strip data out of the pipe that your exec'd child will never see. That's probably what's causing the pipe itself to appear not to work. You probably want to remove this:
read(fd[0], readbuffer, sizeof(readbuffer));
Lastly, this line probably needs to go before the execok() call (and similarly for the other similar one):
pr_words[l_nwds] = NULL; /* mark end of argument array */
The skeleton of the code should look about like this, leaving off error handling and execok checks, and demonstrating the use of waitpid() if you want to know which status code is for which child:
int child_pid[2];
child_pid[0] = fork();
if (child_pid[0] == 0) {
// first child, close stdout and replace with pipe, then exec
} else {
child_pid[1] = fork();
if (child_pid[1] == 0) {
// second child, close stdin and replace with pipe, then exec
} else {
// parent, and now we have the pids of the children
waitpid(child_pid[0], &status, 0); // wait for first child
waitpid(child_pid[1], &status, 0); // wait for second child
// *do not* mess with stdin/stdout, they are okay here
}
}
I am trying to run ls|wc using execvp. So I create a pipe and then fork to create a child. I close the appropriate(read./write) end in parent/child and then map the other end to stdout/stdin. Then I run the ls in parent using execvp and wc in child. When I run the program it says
wc:standard input:bad file descriptor.
0 0 0
wc: -:Bad file descriptor
Here is my code:
int main()
{
//int nbBytes = 0; //stream length
int pfd_1[2]; //file descriptor
//char buffer[MAX_FILE_LENGTH];
char* arg[MAX_FILE_LENGTH];
pid_t processPid;
//Create a pipe
if(pipe(pfd_1) == -1)
{
printf("Error in creating pipe");
return 0;
}
//Create a child
processPid = fork();
if(processPid == -1)
{
printf("Erro in fork");
exit(1);
}
else if(processPid == 0) //Child
{
//redirect read end file descriptor to standard input
dup2(pfd_1[0],0);
//Close the write end
if(close(pfd_1[1] == -1))
{
printf("Error in closing the write end file descriptor");
exit(1);
}
arg[0] = "wc";
//arg[1] = "-l";
arg[1] = '\0';
if(execvp(arg[0],arg) == -1)
{
printf("Error in executing ls");
}
}
else //Parent
{
//redirect standard output to the file descriptor
dup2(pfd_1[1],1);
//Close the read end
if(close(pfd_1[0] == -1))
{
printf("Error in closing the read end from parent");
exit(1);
}
//Command
arg[0] = "ls";
arg[1] = "/proc/1/status";
arg[2] = '\0';
if(execvp(arg[0],arg) == -1)
{
printf("Error in executing ls");
}
}
}
Any idea what might be wrong? Why would it consider standard input as bad file descriptor? My understanding was since the stdin and read end file descriptor are aliases so the wc -l would read whatever the output is from the parent process. Do I need to do scanf to read from the stdin?
The problem is in this line:
if(close(pfd_1[1] == -1))
You are closing the result of pfd_1[1] == -1, which is by necessity equal to 0 (as they will never be equal). The correct line would probably be:
if (close(pfd_1[1]) == -1)
Note that you do this again later in attempting to close the read end in the parent process.
If you're going to fork children, you have to call wait() in the parent process in order to avoid "zombie" child processes. So you don't want to overlay the parent process that did the original process forking with another executable via exec.
One quick way to setup a series of pipes in the way you want would be to fork a child for each executable you want to run, and read that data back into a buffer in the parent. Then feed that data from the first child into a new child process that the parent forks off. So each child is fed data from the parent, processes the data, and writes the data back to the parent process, which stores the transformed data in a buffer. That buffer is then fed to the next child, etc., etc. The final results of the data in the buffer are the final output of the pipe.
Here's a little pseudo-code:
//allocate buffer
unsigned char buffer[SIZE];
for (each executable to run in pipeline)
{
pipes[2];
pipe(pipes);
pid_t pid = fork();
if (pid == 0)
{
//setup the pipe in the child process
//call exec
}
else
{
//setup the pipe in the parent process
if (child executable is not the first in the pipeline)
{
//write contents of buffer to child process
}
//read from the pipe until the child exits
//store the results in buffer
//call wait, and maybe also check the return value to make sure the
//child returned successfully
wait(NULL);
//clean up the pipe
}
}
I'm trying to write a program that will spawn an arbitrary number of child processes and pipe between them, similar to a command line pipeline. In my case I'm trying to do "ls -l | more" and output that to stdout, then have the parent continue executing more commands.
I have the following code as a minimal example:
int main (int argc, const char * argv[]) {
int fd[2];
pipe(fd);
chdir("/directory/with/lots/of/files");
// Create one child process for more
int pid = fork();
if (pid == 0) {
close(fd[1]);
int ret = dup2(fd[0],0);
if (ret < 0) perror("dup2");
char *argv[10];
argv[0] = "more"; argv[1] = NULL;
execvp("more", argv);
}
// Create another child process for ls
int pid2 = fork();
if (pid2 == 0) {
int ret = dup2(fd[1],1);
if (ret < 0) perror("dup2");
char *argv[10];
argv[0] = "ls"; argv[1] = "-l";
argv[2] = NULL;
execvp("ls", argv);
}
// wait for the more process to finish
int status;
waitpid(pid, &status, 0);
printf("Done!\n");
return 0;
}
Now, when I execute the program (enclosed in a main() function of course) what I end up with is more, which is expected. I'll hit "d" to page down more's output and "u" to go up, and it seems to work fine. But when I reach the bottom, instead of exiting like more does, it just leaves a blank line. Ctrl-C works to exit it but it exits the entire program, meaning the "Done!" line never gets printed. A movie is available here that illustrates what happens (note that at the very end I press Ctrl-C to get back to bash).
Any thoughts on this? I'm just trying to figure out how to change it to where instead of going to a blank line after more reaches the bottom, more quits and returns to the parent process so it can continue executing.
You need to close() at least the writing end of your pipe, otherwise more will never see EOF. For example:
...
// close parent's pipes
close(fd[0]);
close(fd[1]);
// wait for the more process to finish
int status;
waitpid(pid, &status, 0);
printf("Done!\n");
return 0;
}
I think it is because of the wait() function. Following the logic, your second child process outputs to the first child process, meaning it should end first than the second one.
In your wait function you are waiting for the first process to end, but you are not waiting for the second process. That means that if the second process does not send an EOF to the output ever, your first process won't end, i guess.
You can try to wait for the second process instead of the first and find out if that's the problem.
I'm trying to develop a shell in Linux as an Operating Systems project. One of the requirements is to support pipelining (where calling something like ls -l|less passes the output of the first command to the second). I'm trying to use the C pipe() and dup2() commands but the redirection doesn't seem to be happening (less complains that it didn't receive a filename). Can you identify where I'm going wrong/how I might go about fixing that?
EDIT: I'm thinking that I need to use either freopen or fdopen somewhere since I'm not using read() or write()... is that correct?
(I've heard from others who've done this project that using freopen() is another way to solve this problem; if you think that would be better, tips for going that direction would also be appreciated.)
Here's my execute_external() function, which executes all commands not built-in to the shell. The various commands in the pipe (e.g. [ls -l] and [less]) are stored in the commands[] array.
void execute_external()
{
int numCommands = 1;
char **commands;
commands = malloc(sizeof(char *));
if(strstr(raw_command, "|") != NULL)
{
numCommands = separate_pipeline_commands(commands);
}
else
{
commands[0] = malloc(strlen(raw_command) * sizeof(char));
commands[0] = raw_command;
}
int i;
int pipefd[2];
for (i = 0; i < numCommands; i++)
{
char **parameters_array = malloc(strlen(commands[i]) * sizeof(char *));
int num_params;
num_params = str_to_str_array(commands[i], parameters_array);
if (numCommands > 1 && i > 0 && i != numCommands - 1)
{
if (pipe(pipefd) == -1)
{
printf("Could not open a pipe.");
}
}
pid_t pid = fork();
pmesg(2, "Process forked. ID = %i. \n", pid);
int status;
if (fork < 0)
{
fprintf(to_write_to, "Could not fork a process to complete the external command.\n");
exit(EXIT_FAILURE);
}
if (pid == 0) // This is the child process
{
if (numCommands > 1) { close(pipefd[1]); } // close the unused write end of the pipe
if (i == 0) // we may be pipelining and this is the first process
{
dup2(1, pipefd[1]); // set the source descriptor (for the next iteration of the loop) to this proc's stdout
}
if (i !=0 && (i != numCommands-1)) // we are pipelining and this is not the first or last process
{
dup2(pipefd[0], 0); // set the stdin of this process to the source of the previous process
}
if (execvp(parameters_array[0], parameters_array) < 0)
{
fprintf(to_write_to, "Could not execute the external command. errno: %i.\n", errno);
exit(EXIT_FAILURE);
}
else { pmesg(2, "Executed the child process.\n");}
}
else
{
if (numCommands > 1) { close(pipefd[0]); } // close the unused read end of the pipe
if (backgrounding == 0) { while(wait(&status) != pid); }// Wait for the child to finish executing
}
free(parameters_array);
}
free(commands);
}
It looks like there are a couple of bugs going on in your code.
First, all your dup2's are only in the child. In order to connect a pipe you will need to dup2 the stdout of the parent to the write end pipefd[1] of the pipe. Then you would hook up the read end to stdin.
Also it looks like on of your dup2's is backwards with dup2 fildes is duplicated to fildes2. So when you reassign stdin you want dup2(in, 0) and for stdout you want dup2(out, 1).
So a stripped down piece of piping code is going to look like:
int pipefd[2];
pipe(pipefd);
pid_t pid = fork();
if (pid == 0) //The child
{
dup2(pipefd[0], 0);
}
else
{
dup2(pipefd[1], 1);
}