i'm trying to call an external program from my code with some arguments. As i'm trying to see how different parameters change it's output i have to run it multiple times (about 1000 times). Every time the external program runs i'm just interested in one line of its output although it is printing a lot of (for my purpose) useless stuff. The line i'm interested in is right above the special identifier("some_signal") in the output. So i thought i'll wait till this line appears and read the line above.
I tried the following:
pid_t pid = 0;
int pipefd[2];
FILE* output;
char line[256]; // pipe read buffer
char prev_line[256]; // prev. line pipe buffer
char signal[] = "some_signal\n";
int status = 0;
double obj_Val;
pipe (pipefd); //create pipe
pid = fork (); //span child process
if (pid == 0)
{
// redirect child's output to pipe
close (pipefd[0]);
dup2 (pipefd[1], STDOUT_FILENO);
dup2 (pipefd[1], STDERR_FILENO);
execl ("/some/path",
"some/path",
"some_argument", (char*) NULL);
}
else if (pid < (pid_t) 0)
{
printf("fork failed \n");
return EXIT_FAILURE;
}
else
{
// get child output to pipe
close (pipefd[1]);
output = fdopen (pipefd[0], "r");
while (fgets (line, sizeof(line), output), signal != NULL)
{
if(strcmp(line, signal) == 0)
{
obj_Val = atof (prev_line);
kill (pid, SIGTERM);
waitpid (pid, &status, WNOHANG);
kill (pid, SIGKILL);
waitpid (pid, &status, 0);
break;
}//end if
strcpy (prev_line, line);
}//end while
}
This works fine for like 100 runs or so and then one out of two errors occurs. The first one is a segmentation fault. The second one is the calling program printing out all the output of the called program (without the line containing the wanted signal) and goes into an infinite loop (my guess is, since the signal is missing the while loop won't terminate).
Maybe someone can provide a hint or a link where to look and what to look for, or preferably tell me what i'm doing wrong.
Your while condition is broken: signal != NULL is always true, as signal is an array; and because of the comma , operator the entire condition is always true.
And because of the following, you're not checking the return value of fgets, which means that nothing was read and the buffer is uninitialized.
Also prev_line is not initialized before you do atof on it for the first time.
In any case, compile with -Wall -Werror and fix the remaining errors.
Related
So my program needs to pipe multiple processes and read the number of bytes each process output has.
The way I implemented it, in a for loop, we have two children:
Child 1: dups output and executes the process
Child 2: reads the output and writes it for the next input
Currently, child 1 executes the process and the child 2 reads its output, but it doesn't seem to write it in the right place because in the second loop iteration it prints the output to the screen and blocks.
for (int i = 0; i < processes; i++) {
int result = socketpair(PF_LOCAL, SOCK_STREAM, 0, apipe[i]);
if (result == -1) {
error_and_exit();
}
int pid;
int pid2;
pid = fork_or_die();
// child one points to STDOUT
if (pid == FORK_CHILD) {
if (dup2(apipe[i][1], STDOUT_FILENO) == -1)
error_and_exit();
if (close(apipe[i][1]) == -1)
error_and_exit();
if (close(apipe[i][0]) == -1)
error_and_exit();
if (execlp("/bin/sh", "sh", "-c", tabCommande[i], (char *)NULL) == -1)
error_and_exit();
}
pid2 = fork_or_die();
//CHILD 2 reads the output and writes if for the next command to use
if(pid2 == FORK_CHILD){
FILE *fp;
fp = fopen("count", "a");
close(apipe[i][1]);
int count=0;
char str[4096];
count = read(apipe[i][0], str, sizeof(str)+1);
close(apipe[i][0]);
write(STDIN_FILENO, str, count);
fprintf(fp, "%d : %d \n ", i, count);
fclose(fp);
}
}
Your second child does “write(STDIN_FILENO, …); that’s not a conventional way of using standard input.
If standard input is a terminal, then the device is usually opened for reading and writing and the three standard I/O channels are created using dup() or dup2(). Thus you can read from the outputs and write to the input — but only if the streams are connected to a login terminal (window). If the input is a pipe, you can't successfully write to it, nor can you read from the output if it is a pipe. (Similarly if the input is redirected from a file or the output is redirected to a file.) This terminal setup is done by the process that creates the terminal window. It is background information explaining why writing to standard input appears on the terminal.
Anyway, that's what you're doing. You are writing to the terminal via standard input. Your minimum necessary change is to replace STDIN_FILENO with STDOUT_FILENO.
You are also going to need a loop around the reading and writing code. In general, processes generate lots of output in small chunks. The close on the input pipe will be outside the loop, of course, not between the read() and write() operations. You should check that the write() operations write all the data to the output.
You should also have the second child exit after it closes the output file. In this code, I'd probably open the file after the counting loop (or what will become the counting loop), but that's mostly a stylistic change, keeping the scope of variables to a minimum.
You will probably eventually need to handle signals like SIGPIPE (or ignore it so that the output functions return errors when the pipe is closed early). However, that's a refinement for when you have the basic code working.
Bug: you have:
count = read(apipe[i][0], str, sizeof(str)+1);
This is a request to the o/s to give you a buffer overflow — you ask it to write more data into str than str can hold. Remove the +1!
Minor note: you don’t need to check the return value from execlp() or any of that family of functions. If the call succeeds, it doesn’t return; if it returns, it failed. Your code is correct to exit after the call to execlp(), though; that's good.
You said:
I replaced STDIN_FILENO to STDOUT_FILENO in the second child but it doesn't seem to solve the issue. The output is still shown in the terminal and there's a pipe blockage after.
That observation may well be correct, but it isn't something that can be resolved by studying this code alone. The change to write to an output stream is necessary — and in the absence of any alternative information, writing to STDOUT_FILENO is better than writing to STDIN_FILENO.
That is a necessary change, but it is probably not a sufficient change. There are other changes needed too.
Did you set up the inputs and outputs for the pair of children this code creates correctly? It is very hard to know from the code shown — but given that it is not working as you intended, it's a reasonable inference that you did not get all the plumbing correct. You need to draw a diagram of how the processes are intended to operate in the larger context. At a minimum, you need to know where the standard input for each process comes from, and where its standard input goes. Sometimes, you need to worry about standard error too — most likely though, in this case, you can quietly ignore it.
This is what I think your code could look like — though the comments in it describe numerous possible variants.
#include <sys/socket.h>
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
/* The code needs these declarations and definition to compile */
extern _Noreturn void error_and_exit(void);
extern pid_t fork_or_die(void);
extern void unknown_function(void);
static ssize_t copy_bytes(int fd1, int fd2);
#define FORK_CHILD 0
int processes;
int apipe[20][2];
char *tabCommande[21];
void unknown_function(void)
{
for (int i = 0; i < processes; i++)
{
int result = socketpair(PF_LOCAL, SOCK_STREAM, 0, apipe[i]);
if (result == -1)
error_and_exit();
int pid1 = fork_or_die();
// child one points to STDOUT
if (pid1 == FORK_CHILD)
{
if (dup2(apipe[i][1], STDOUT_FILENO) == -1)
error_and_exit();
if (close(apipe[i][1]) == -1)
error_and_exit();
if (close(apipe[i][0]) == -1)
error_and_exit();
execlp("/bin/sh", "sh", "-c", tabCommande[i], (char *)NULL);
error_and_exit();
}
//CHILD 2 reads the output and writes if for the next command to use
int pid2 = fork_or_die();
if (pid2 == FORK_CHILD)
{
close(apipe[i][1]);
ssize_t count = copy_bytes(apipe[i][0], STDOUT_FILENO);
FILE *fp = fopen("count", "a");
if (fp == NULL)
error_and_exit();
/*
** Using %zd for ssize_t is a reasonable guess at a format to
** print ssize_t - but it is a guess. Alternatively, change the
** type of count to long long and use %lld. There isn't a
** documented, official (fully standardized by POSIX) conversion
** specifier for ssize_t AFAIK.
*/
fprintf(fp, "%d : %zd\n ", i, count);
fclose(fp);
exit(EXIT_SUCCESS);
}
/*
** This is crucial - the parent has all the pipes open, and the
** child processes won't get EOF until the parent closes the
** write ends of the pipes, and they won't get EOF on the inputs
** until the parent closes the read ends of the pipe.
**
** It could be avoided if the first child creates the pipe or
** socketpair and then creates the second child as a grandchild
** of the main process. That also alters the process structure
** and reduces the number of processes that the original parent
** process has to wait for. If the first child creates the
** pipe, then the apipe array of arrays becomes unnecessary;
** you can have a simple int apipe[2]; array that's local to the
** two processes. However, you may need the array of arrays so
** that you can chain the outputs of one process (pair of
** processes) to the input of the next.
*/
close(apipe[i][0]);
close(apipe[i][1]);
}
}
static ssize_t copy_bytes(int fd1, int fd2)
{
ssize_t tbytes = 0;
ssize_t rbytes;
char buffer[4096];
while ((rbytes = read(fd1, buffer, sizeof(buffer))) > 0)
{
ssize_t wbytes = write(fd2, buffer, rbytes);
if (wbytes != rbytes)
{
/*
** There are many possible ways to deal with this. If
** wbytes is negative, then the write failed, presumably
** irrecoverably. The code could break the loop, reporting
** how many bytes were written successfully to the output.
** If wbytes is zero (pretty improbable), it isn't clear
** what happened. If wbytes is positive, then you could add
** the current value to tbytes and try to write the rest in
** a loop until everything has been written or an error
** occurs. You pays your money and takes your pick.
*/
error_and_exit();
}
tbytes += wbytes;
}
if (tbytes == 0 && rbytes < 0)
tbytes = rbytes;
return tbytes;
}
You could add #include <signal.h> and signal(SIGPIPE, SIG_IGN); to the code in the second child.
I am creating a simple Linux command shell in C. I am having trouble understanding where my code is having problems. "commands" is a list of strings of Linux commands that I want to be executed concurrently as the children processes of one parent. When all are done executing, I want the parent to exit the function. However, when I call exit(0), the for loop continues and parses the current command again, causing the args to be executed again in execvp. Am I using fork() and wait() correctly here? I have tried using waitpid() as well with no luck.
void executeShell(char** commands){
char **arr = commands;
char *c;
pid_t pid, wpid;
int status = 0;
for (c = *arr; c; c=*++arr){
// printf("%d-\n", strcmp(command, "exit"));
if (strcmp(c, "exit") == 0){
EXIT = 1;
return;
}
printf("Running command \'%s\'...\n", c);
char** args = parseStringToTokenArray(c, " ");
free(args);
/* fork and execute the command */
pid = fork();
if(pid < 0){
perror("fork() error\n");
return;
}
/* child process executes command */
else if (pid == 0){
/* 'cd' is not a part of /bin library so handle it manually */
if (strcmp(args[0], "cd") == 0){
changeDirectory(args[1]);
}
else if (strcmp(args[0], "sdir") == 0){
searchDirectory(args[1]);
}else{
/* parse commands with arguments */
execvp(args[0], args);//execute the command
}
exit(0);// <-----command is finished, so why no exit?
}
}
/* wait for children to complete */
while((wpid = wait(&status)) > 0);
}
If execvp succeeds, the entire child process address space is replaced by the program invoked by execvp(). This means that the exit(0) will only ever be invoked following your two special cases i.e. cd and sdir. As far as your code is concerned execvp() should never return, unless there is an error.
A further problem is that you free args immediately after allocating it and then go on to use it in your child processes. This is undefined behaviour.
The only problem I see with your wait code is that, if any of the children block waiting for user input, the parent will block waiting for the child to exit.
The cd code, has no effect on any process except the child in which it is executed. The parent's current directory is not affected. As you state in the comments, this can bet fixed by handling cd in the parent without forking.
I am using a subroutine named read_from_pipe in my child process as below to read whatever is in the pipe and display it:
void read_from_pipe(int fileDescriptr)
{
FILE *stream;
int c;
if (fileDesc_Is_Valid(fileDescriptr) == TRUE)
{
stream = fdopen(fileDescriptr, "r");
while ((c = fgetc(stream)) != EOF)
putchar(c);
fclose(stream);
}
else
perror("Reading from pipe failed -->");
}
fileDesc_Is_Valid is another subroutine which checks the existance of file descriptor.
The problem is that because I have used waitpid(pid, &status, 0); statement in my parent to wait for child to finish off its tasks, the compiler gets stuck in the first cold run at while loop when pipe is actually empty. How can I AND another condition in my while to let compiler simply ignore empty pipes?
It's actually very simple, you just need to ignore the SIGPIPE signal, and it's done with a single function-call:
signal(SIGPIPE, SIG_IGN);
When the pipe is empty, instead of raising the SIGPIPE signal, reading from the pipe will return an end-of-file value (the underlying read system-call will return 0).
I have program I cant modify, as is, and I need to execute it, write some data to its stdin and get the answer from its stdout in programmatic manner, automated.
What is the simpliest way to do this?
I suppose something like this pseudo-C-code
char input_data_buffer[] = "calculate 2 + 2\nsay 'hello world!'";
char output_data_buffer[MAX_BUF];
IPCStream ipcs = executeIPC("./myprogram", "rw");
ipcs.write(input_data_buffer);
ipcs.read(output_data_buffer);
...
PS: I thought of popen, but AFAIK there is only one-way pipes in linux
EDIT:
It is supposed it will be one-message-from-each-side communication. Firstly parent side send input to child process' stdin, then child provides output to its stdout and exits, meanwhile parent reads its stdout. Now about communication termination: I think when child process exits it will send EOF terminator to stdout, so parent will know exactly whether child done, on the other hand it is guaranteed that parent knows what kind of input child expects for.
Generally this program (parent) - a student's solution tester. It takes paths to two other executables from CLI, the first is student's program to test, the second is etalon correctly working program, which solves very same problem.
Input/output of students programs is strictly specified, so tester run both programs and compares its outputs for lots of random inputs, all mismatches will be reported.
Input/output max size is estimated at few hundreds kilobytes
Example: ..implement insertion sort algorithm ... first line there is sequence length ... second line there is sequence of numbers a_i where |a_i| < 2^31 - 1...
output first line must be sum of all elements, the second line must be sorted sequence.
Input:
5
1 3 4 6 2
Expected output:
16
1 2 3 4 6
Read Advanced Linux Programming -which has at least an entire chapter to answer your question- and learn more about execve(2), fork(2), waitpid(2), pipe(2), dup2(2), poll(2) ...
Notice that you'll need (at least in a single-threaded program) to multiplex (with poll) on the input and the output of the program. Otherwise you might have a deadlock: the child process could be blocked writing to your program (because the output pipe is full), and your program could be blocked reading from it (because the input pipe is empty).
BTW, if your program has some event loop it might help (and actually poll is providing the basis for a simple event loop). And Glib (from GTK) provides function to spawn processes, Qt has QProcess, libevent knows them, etc.
Given that the processing is simply a question of one message from parent to child (which must be complete before the child responds), and one message from child to parent, then it is easy enough to handle:
Create two pipes, one for communication to child, one for communication to parent.
Fork.
Child process duplicates the relevant ends of the pipes (read end of 'to-child' pipe, write end of 'to-parent' pipe) to standard input, output.
Child closes all pipe file descriptors.
Child execs test program (or prints a message to standard error reporting failure and exits).
Parent closes the irrelevant ends of the pipes.
Parent writes the message to the child and closes the pipe.
Parent reads the response from the child and closes the pipe.
Parent continues on its merry way.
This leaves the child process lying around as a zombie. If the parent is going to do this more than once, or just needs to know the exit status of the child, then after closing the read pipe, it will wait for the child to die, collecting its status.
All this is straight-forward, routine coding. I'm sure you could find examples on SO.
Since apparently there are no suitable examples on Stack Overflow, here is a simple implementation of the code outlined above. There are two source files, basic_pipe.c for the basic piping work, and myprogram.c which is supposed to respond to the prompts shown in the question. The first is almost general purpose; it should probably loop on the read operation (but that hasn't mattered on the machine I tested it on, which is running an Ubuntu 14.04 derivative). The second is very specialized.
System calls
pipe()
fork()
dup2()
execv()
waitpid()
close()
read()
write()
basic_pipe.c
#include <assert.h>
#include <errno.h>
#include <stdarg.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <sys/wait.h>
static char msg_for_child[] = "calculate 2 + 2\nsay 'hello world!'\n";
static char cmd_for_child[] = "./myprogram";
static void err_syserr(const char *fmt, ...);
static void be_childish(int to_child[2], int fr_child[2]);
static void be_parental(int to_child[2], int fr_child[2], int pid);
int main(void)
{
int to_child[2];
int fr_child[2];
if (pipe(to_child) != 0 || pipe(fr_child) != 0)
err_syserr("Failed to open pipes\n");
assert(to_child[0] > STDERR_FILENO && to_child[1] > STDERR_FILENO &&
fr_child[0] > STDERR_FILENO && fr_child[1] > STDERR_FILENO);
int pid;
if ((pid = fork()) < 0)
err_syserr("Failed to fork\n");
if (pid == 0)
be_childish(to_child, fr_child);
else
be_parental(to_child, fr_child, pid);
printf("Process %d continues and exits\n", (int)getpid());
return 0;
}
static void be_childish(int to_child[2], int fr_child[2])
{
printf("Child PID: %d\n", (int)getpid());
fflush(0);
if (dup2(to_child[0], STDIN_FILENO) < 0 ||
dup2(fr_child[1], STDOUT_FILENO) < 0)
err_syserr("Failed to set standard I/O in child\n");
close(to_child[0]);
close(to_child[1]);
close(fr_child[0]);
close(fr_child[1]);
char *args[] = { cmd_for_child, 0 };
execv(args[0], args);
err_syserr("Failed to execute %s", args[0]);
/* NOTREACHED */
}
static void be_parental(int to_child[2], int fr_child[2], int pid)
{
printf("Parent PID: %d\n", (int)getpid());
close(to_child[0]);
close(fr_child[1]);
int o_len = sizeof(msg_for_child) - 1; // Don't send null byte
if (write(to_child[1], msg_for_child, o_len) != o_len)
err_syserr("Failed to write complete message to child\n");
close(to_child[1]);
char buffer[4096];
int nbytes;
if ((nbytes = read(fr_child[0], buffer, sizeof(buffer))) <= 0)
err_syserr("Failed to read message from child\n");
close(fr_child[0]);
printf("Read: [[%.*s]]\n", nbytes, buffer);
int corpse;
int status;
while ((corpse = waitpid(pid, &status, 0)) != pid && corpse != -1)
err_syserr("Got pid %d (status 0x%.4X) instead of pid %d\n",
corpse, status, pid);
printf("PID %d exited with status 0x%.4X\n", pid, status);
}
static void err_syserr(const char *fmt, ...)
{
int errnum = errno;
va_list args;
va_start(args, fmt);
vfprintf(stderr, fmt, args);
va_end(args);
if (errnum != 0)
fprintf(stderr, "(%d: %s)\n", errnum, strerror(errnum));
exit(EXIT_FAILURE);
}
myprogram.c
#include <stdio.h>
int main(void)
{
char buffer[4096];
char *response[] =
{
"4",
"hello world!",
};
enum { N_RESPONSES = sizeof(response)/sizeof(response[0]) };
for (int line = 0; fgets(buffer, sizeof(buffer), stdin) != 0; line++)
{
fprintf(stderr, "Read line %d: %s", line + 1, buffer);
if (line < N_RESPONSES)
{
printf("%s\n", response[line]);
fprintf(stderr, "Sent line %d: %s\n", line + 1, response[line]);
}
}
fprintf(stderr, "All done\n");
return 0;
}
Example output
Note that there is no guarantee that the child will complete before the parent starts executing the be_parental() function.
Child PID: 19538
Read line 1: calculate 2 + 2
Sent line 1: 4
Read line 2: say 'hello world!'
Sent line 2: hello world!
All done
Parent PID: 19536
Read: [[4
hello world!
]]
PID 19538 exited with status 0x0000
Process 19536 continues and exits
You can use expect to achieve this:
http://en.wikipedia.org/wiki/Expect
This is what a usual expect program would do:
# Start the program
spawn <your_program>
# Send data to the program
send "calculate 2 + 2"
# Capture the output
set results $expect_out(buffer)
Expect can be used inside C programs using expect development library, so you can translate previous commands directly into C function calls. Here you have an example:
http://kahimyang.info/kauswagan/code-blogs/1358/using-expect-script-cc-library-to-manage-linux-hosts
You can also use it from perl and python which usually are usually easier to program for these type of purposes than C.
my question is can I write an integer to pipe ? and how ?
I need to make 3 processes first one generate 2 numbers, second make sum of the numbers, third print the result (USING PIPE)
Thanks all
The complicated part of what you're trying to do is creating the pipeline. You could just have the shell do it for you...
$ ./makenumbers | ./addnumbers | ./printresult
but that's boring, eh? And you have to have three executables. So let's have a look at what those vertical bars are doing at the C level.
You create a pipe with the pipe system call. You reassign standard input/output with dup2. You create new processes with fork, and you wait for them to terminate with waitpid. A program to set the whole thing up would look something like this:
int
main(void)
{
pid_t children[2];
int pipe1[2], pipe2[2];
int status;
pipe(pipe1);
pipe(pipe2);
children[0] = fork();
if (children[0] == 0)
{
/* in child 0 */
dup2(pipe1[1], 1);
generate_two_numbers_and_write_them_to_fd_1();
_exit(0);
}
children[1] = fork();
if (children[1] == 0)
{
/* in child 1 */
dup2(pipe1[0], 0);
dup2(pipe2[1], 1);
read_two_numbers_from_fd_0_add_them_and_write_result_to_fd_1();
_exit(0);
}
/* parent process still */
dup2(pipe2[0], 0);
read_a_number_from_fd_0_and_print_it();
waitpid(children[0], &status, 0);
waitpid(children[1], &status, 0);
return 0;
}
Please note:
I left out all error handling, because that would make the program about three times longer. Your instructor wants you to include error handling.
Similarly, I left out checking the exit status of the children; your instructor also wants you to check that.
You do not need the dup2 calls; you could just pass the pipe fd numbers to the subroutine calls. But if you were exec-ing a new binary in the child, which is more typical, you would need them. You would then also have to worry about making sure all file descriptors numbered 3 and higher were closed.
There is a reason I am using _exit instead of exit. Try to figure out what it is.
You need to use read and write instead of stdio.h calls in the subroutines called from child processes. The reason is related to the reason I am using _exit.
Since a pipe is just a file, you can use the fprintf() function to convert a random number to text and write that to the pipe. For instance:
FILE *pipe = popen("path/to/your/program", "w");
if (pipe != NULL) {
fprintf(pipe, "%d\n", rand());
pclose(pipe);
}