OK I have a program that creates two pipes -> forks -> the child's stdin and stdout are redirected to one end of each pipe -> the parent is connected to the other ends of the pipes and tries to read the stream associated with the child's output and print it to the screen (and I will also make it write to the input of the child eventually).
The problem is, when the parent tries to fgets the child's output stream, it just stalls and waits until the child dies to fgets and then print the output. If the child doesn't exit, it just waits forever. What is going on? I thought that maybe fgets would block until SOMETHING was in the stream, but not block all the way until the child gives up its file descriptors.
Here is the code:
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <sys/types.h>
#include <unistd.h>
int main(int argc, char *argv[]) {
FILE* fpin;
FILE* fpout;
int input_fd[2];
int output_fd[2];
pid_t pid;
int status;
char input[100];
char output[100];
char *args[] = {"/somepath/someprogram", NULL};
fgets(input, 100, stdin); // the user inputs the program name to exec
pipe(input_fd);
pipe(output_fd);
pid = fork();
if (pid == 0) {
close(input_fd[1]);
close(output_fd[0]);
dup2(input_fd[0], 0);
dup2(output_fd[1], 1);
input[strlen(input)-1] = '\0';
execvp(input, args);
}
else {
close(input_fd[0]);
close(output_fd[1]);
fpin = fdopen(input_fd[1], "w");
fpout = fdopen(output_fd[0], "r");
while(!feof(fpout)) {
fgets(output, 100, fpout);
printf("output: %s\n", output);
}
}
return 0;
}
The child should probably fflush() its output, and/or terminate lines properly. Otherwise the I/O buffering can hang on to the data for quite a while.
You can try to set the O_NONBLOCK flag (using fcntl()) on the child's output file descriptor before handing over control, but that will require you to change your parent code's accordingly. As pointed out in comments though, this won't help you overcome the buffering done at the C standard library level if the child uses FILE-based I/O.
Related
I am developing a simple shell program, a command line interpreter and I wanted to read input from the file line by line, so I used getline() function. At the first time, the program works correctly, however, when it reaches the end of the file, instead of terminating, it starts to read a file from the start and it runs infinitely.
Here are some codes in main function that are related to getline():
int main(int argc,char *argv[]){
int const IN_SIZE = 255;
char *input = NULL;
size_t len = IN_SIZE;
// get file address
fileAdr = argv[2];
// open file
srcFile = fopen(fileAdr, "r");
if (srcFile == NULL) {
printf("No such file!\n");
exit(-1);
}
while (getline( &input, &len, srcFile) != -1) {
strtok(input, "\n");
printf("%s\n", input);
// some code that parses input, firstArgs == input
execSimpleCmd(firstArgs);
}
fclose(srcFile);
}
I am using fork() in my program and most probably it causes this problem.
void execSimpleCmd(char **cmdAndArgs) {
pid_t pid = fork();
if (pid < 0) {
// error
fprintf(stderr, "Fork Failed");
exit(-1);
} else if (pid == 0) {
// child process
if (execvp(cmdAndArgs[0], cmdAndArgs) < 0) {
printf("There is no such command!\n");
}
exit(0);
} else {
// parent process
wait(NULL);
return;
}
}
In addition, sometimes the program reads and prints a combinations of multiple lines. For example, if an input file as below:
ping
ww
ls
ls -l
pwd
it prints something like pwdg, pwdww, etc. How to fix it?
It appears that closing a FILE in some cases seeks the underlying file descriptor back to the position where the application actually read to, effectively undoing the effect of the read buffering. This matters, since the OS level file descriptors of the parent and the child point to the same file description, and the same file offset in particular.
The POSIX description of fclose() has this phrase:
[CX] [Option Start] If the file is not already at EOF, and the file is one capable of seeking, the file offset of the underlying open file description shall be set to the file position of the stream if the stream is the active handle to the underlying file description.
(Where CX means an extension to the ISO C standard, and exit() of course runs fclose() on all streams.)
I can reproduce the odd behavior with this program (on Debian 9.8):
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
int main(int argc, char *argv[]){
FILE *f;
if ((f = fopen("testfile", "r")) == NULL) {
perror("fopen");
exit(1);
}
int right = 0;
if (argc > 1)
right = 1;
char *line = NULL;
size_t len = 0;
// first line
getline(&line, &len, f);
printf("%s", line);
pid_t p = fork();
if (p == -1) {
perror("fork");
} else if (p == 0) {
if (right)
_exit(0); // exit the child
else
exit(0); // wrong way to exit
} else {
wait(NULL); // parent
}
// rest of the lines
while (getline(&line, &len, f) > 0) {
printf("%s", line);
}
fclose(f);
}
Then:
$ printf 'a\nb\nc\n' > testfile
$ gcc -Wall -o getline getline.c
$ ./get
getline getline2
$ ./getline
a
b
c
b
c
Running it with strace -f ./getline clearly shows the child seeking the file descriptor back:
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f63794e0710) = 25117
strace: Process 25117 attached
[pid 25116] wait4(-1, <unfinished ...>
[pid 25117] lseek(3, -4, SEEK_CUR) = 2
[pid 25117] exit_group(1) = ?
(I didn't see the seek back with a code that didn't involve forking, but I don't know why.)
So, what happens is that the C library on the main program reads a block of data from the file, and the application prints the first line. After the fork, the child exits, and seeks the fd back to where the application level file pointer is. Then the parent continues, processes the rest of the read buffer, and when it's finished, it continues reading from the file. Because the file descriptor was seeked back, the lines starting from the second are again available.
In your case, the repeated fork() on every iteration seems to result in an infinite loop.
Using _exit() instead of exit() in the child fixes the problem in this case, since _exit() only exits the process, it doesn't do any housekeeping with the stdio buffers.
With _exit(), any output buffers are also not flushed, so you'll need to call fflush() manually on stdout and any other files you're writing to.
However, if you did this the other way around, with the child reading and buffering more than it processes, then it would be useful for the child to seek back the fd so that the parent could continue from where the child actually left.
Another solution would be not to mix stdio with fork().
I am trying to write a shell script in
C language. In short details, if user enters a command I send it to the system() call, if user enters more than one command like "ls;whoami" I parse it and create child processes to execute all of them. Now it works but my methods such as gets() and getting input by the user does not seem well and when I put multi commands, prompt text becomes unseen. Do you have any suggestion or if you see any mistakes or wrong usage because I am not the C guy then I would be grateful.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#define RESET "\033[0m"
#define BOLDGREEN "\033[1m\033[32m" /* Bold Green */
#define BOLDMAGENTA "\033[1m\033[35m" /* Bold Magenta */
char input[50];
char command[50];
char *inputget;
char *dotcomma;
char *p;
pid_t pid;
void welcome(){
char* welcomestr = "\n\TEST\n\n";
printf("%s%s%s",BOLDMAGENTA,welcomestr,RESET);
}
void prompt(){
char *username = getenv("USER");
char hostname[1024];
gethostname(hostname, 1024);
char currentDirectory[256];
getcwd(currentDirectory, 256);
printf("%s%s#%s%s%s ~%s $ %s",BOLDGREEN,username,hostname,RESET,BOLDBLUE,currentDirectory,RESET);
}
int main(int argc, char* argv[]){
inputget=input;
welcome();
prompt();
gets(inputget);
if(argc == 1) {
while(strcmp(inputget, "quit")!=0){
p = strsep(&inputget, ";\n");
while(p != NULL){
pid_t parent = getpid();
pid = fork();
if (pid==-1){
perror("failed to fork");
}
else if (pid==0){
system(p);
exit(0);
}else{
p = strsep(&inputget, ";\n");
}
}
wait(NULL);
prompt();
scanf("%s",input);
inputget=input;
}
exit(0);
}else{
//get argc 2 and and read-run commands from text.file
}
}
Let's begin with the worst part: using gets. Don't use this function anymore,
it's dangerous and deprecated. You should use fgets instead.
fgets(input, sizeof input, stdin);
There is no reason why any of these variables
char input[50];
char command[50];
char *inputget;
char *dotcomma;
char *p;
pid_t pid;
have to be global variables, declare them in main.
Stream buffers like stdout are buffered, content is physically written on
the device once the buffer is full or you call fflush to flush the buffer.
One exception is stdout when is connected to a terminal, in that case printf
will flush immediately when a newline is printed. That's why you almost always
see that the format of printf statements end with \n, like
printf("Your age is %d\n", age);
When you don't want to print a newline, because you are printing something like
a prompt, then you should flush stdout yourself.
void prompt(){
char *username = getenv("USER");
char hostname[1024];
gethostname(hostname, 1024);
char currentDirectory[256];
getcwd(currentDirectory, 256);
printf("%s%s#%s%s%s ~%s $ %s",BOLDGREEN,username,hostname,RESET,BOLDBLUE,currentDirectory,RESET);
fflush(stdout); // <-- you need this here
}
The last thing is where you are executing wait. The problem is the
synchronization between the children and the parent process. Once a child is
created, it begins to run immediately. If the child is also printing to stdout
and you don't synchronize with the parent, then there's no guarantee which
output will be printed first.
In your case, if the user enters cmd1;cmd2;cmd3, you are forking 3 times
but you are only doing one wait after you've forked all children. That means
that the three children will run concurrently and the order of their output is
undefined. After all children are forked, you finally do wait(NULL), but this
only waits for one child, then you execute prompt() but remember, the other
children might be still running and hence the output of the prompt might come
before the output of the other children that are running. That is perhaps what
you've been observing.
If you want to emulate the shell, then cmd2 can only start after cmd1 is
finished and cmd3 only after cmd2 is finished. Only when the three commands
are finished you can execute prompt(). That means that you have to wait for
every child to end before the next child can be forked. That's why you have to
move the wait in the parent block before the next fork is called.
// "quit\n" because fgets does not remove the newline
while(strcmp(inputget, "quit\n") != 0) {
p = strsep(&inputget, ";\n");
while(p != NULL) {
if(p[0] == 0)
{
// handles "empty fields", when
// two delimiters come one after
// the other
p = strsep(&inputget, ";\n");
continue;
}
pid = fork();
if (pid==-1) {
perror("failed to fork");
}
else if (pid==0) {
system(p);
exit(0);
} else {
wait(NULL); // <-- here the parent waits
// until the child is finished
p = strsep(&inputget, ";\n");
}
}
prompt();
fgets(input, sizeof input, stdin);
inputget = input;
}
Also note that I'm not using scanf here. scanf("%s"... reads until the first
non-white character, so a command like cat /etc/fstab will only read cat and
your shell will only execute cat and it would block until you close stdin
(by pressing Ctrl+D). The next time, scanf won't wait
for user input and will read /etc/fstab instead and try to execute
/etc/fstab, which will fail, as /etc/fstab is not a script or binary.
That's why it's better to use fgets.
I'd also use a longer buffer for the user input, depending on the command
length, 49 bytes is too short. I'd use 1024 or more. Or you can use getline to
fetch a whole line without the worry about buffer sizes.
So,my programm should do this: I write in the parent process a message to the file "vaterkind" and then read it with the child process, write it to a string and put it on the screen. I tried with this code:
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
#include <string.h>
#include <sys/stat.h>
int main()
{
FILE *vaterkind, *kindvater;
char path1[]={"/home/user/Documents/vaterkind"}, path2[]={"/home/user/Documents/kindvater"}; //paths to files
int ret, readyv=0, readyk=0;
mkfifo(path1,0666); //makes fifos
mkfifo(path2,0666);
ret = fork();
if(ret > 0) //parent
{
char data_in[50];
char data_out[50];
puts("String");
gets(data_in);
vaterkind = open(path1,O_WRONLY); //dad is able to write
kindvater = open(path2,O_RDONLY); //child is able to read
write(vaterkind,data_in,strlen(data_in)); //write input in vaterkind
puts("String sent");
readyv = 1; // set ready
}
else if(ret == 0) // child
{
char data[50],hex[50],ascii[50];
int i = 0, j = 0;
vaterkind = open(path1,O_RDONLY); //dad is able to read
kindvater = open(path2,O_WRONLY); //child is able to write
read(vaterkind,data,strlen(data)); //read input and write to data
puts(data);
puts("Child finished");
return 0;
}
else
{
puts("Fork failed");
}
sleep(1);
return 0;
}
But when I start the program I first get the message "String" then a symbol (don't know why a symbol is here) then "Child finished" then I can do the gets from the parent and then "String sent" it looks like this:
String
Child finished
input
String sent
Can somebody help out please?
In the child branch
char data[50];
...
read(vaterkind,data,strlen(data))
At that point data contains garbage and applying strlen to it makes no sense at all.
(That's not even mentioning that the parent sends a string without a zero-terminator and the child never bothers to zero-terminate what it received.)
You are supposed to develop a communication protocol of some sort to make sure that at any point the child knows how many bytes it should read from FIFO. For example, the parent can first send the length of the string and only then the contents of the string. The child begins by reading the length and proceeds from there.
Your read in child process is not blocked.
so it wont wait till parent process is finished.
you have to use a pipe from parent to child.
Write into pipe from parent end.
read from the pipe in child.
This blocks the child process, till reading is complete.
Below is simple design for your understanding and how to do it.
int main(int argc, char *argv[])
{
//Two file descriptors for pipe read and write.
int fd[2];
// create a pipe with defined file descriptors
pipe(fd);
// fork() returns 0 for child process.
//so below is a parent process.
if (fork() != 0)
{
// close read-descriptor, you are only writing in parent.
close(fd[0]);
//write into the pipe
write(fd[1], "your data to write", sizeof("your data"));
// close the write descriptor
close(fd[1]);
}
else
{
// close the write-descriptor, you are only reading here
close(fd[1]);
// read here. this will block the process till your reading complete.
read(fd[0], "your variable to read", sizeof("your variable"));
// close the read-descriptor
close(fd[0]);
}
return 0;
}
I'm trying to run a program with a specific standard input. I succeed by using a file descriptor of a file where there is what I want to put in the stdin, but I fail to write directly on the stdin :
$cat input.test
echo Hello
$
Code C :
int main(int argc, char **argv)
{
int fd = 0;
fd = open("input.test", O_CREAT);
close(STDIN_FILENO);
dup2(fd, STDIN_FILENO);
char *const args[] = { "bash", NULL };
execvp("bash", args);
}
That works :
$./a.out
Hello
$
But if I try to write directly on the STDIN using pipe the program displays nothing and keeps running :
int main(int argc, char **argv)
{
int fds[2];
pipe(fds);
close(STDIN_FILENO);
dup2(fds[1], STDIN_FILENO);
write(fds[1], "echo Hello;", 11); // Résults are identics with fds[0]
char *const args[] = { "bash", NULL };
execvp("bash", args);
}
Thanks for your help
Cordially,
Bastien.
EDIT Problem solved:
Thanks for your answers, here the code which works :
int main(void)
{
int fd[2];
pid_t pid;
if (pipe(fd) < 0)
return EXIT_FAILURE;
if ((pid = fork()) < 0)
return EXIT_FAILURE;
else if (pid != 0) { /* father */
close(fd[1]);
dup2(fd[0], STDIN_FILENO);
execlp("bash", "bash", (char *)0);
} else { /* son */
close(fd[0]);
write(fd[1], "echo hello\n", 11);
}
return EXIT_SUCCESS;
}
You need to dup the read side of the pipe to stdin, not the write side. (And write to the write side, obviously.)
#include <unistd.h>
#include <string.h>
#include <stdio.h>
int main(int argc, char **argv)
{
int fds[2];
char cmd[] = "echo hello\nexit\n";
pipe(fds);
close(STDIN_FILENO);
dup2(fds[0], STDIN_FILENO);
write(fds[1], cmd, strlen(cmd));
char *const args[] = { "bash", NULL };
execvp("bash", args);
return 0;
}
Make sure you check the return values of all those functions though, you'll never manage to debug your code if you don't.
execv and friends replace the current running program with the specified one; they do not return - execution continues at the start of new program instead.
So what you normally do is fork and, in one of the forks, call execv. You then read and write through the pipe from your program continuing in the other fork. There are usually popen functions to do this in most languages; sadly in POSIX the popen() is strictly read or write and not bidirectional.
Luckily, I've made, tested and published a popen3 function. This gives you back three file descriptors - one for stdin to the process, and two for stdout and stderr. You can then use write() on the stdin.
When you call pipe, fd[ 0 ] is open for reading, and fd[ 1 ] is open for writing. You should be dup'ing stdin on the read side ( fd[ 0 ]) and writing to the write side( fd[ 1 ]). Check the return value of write: it is probably -1.
But there is a larger issue. You never close either side of the pipe. bash may block on a read and never do anything until the write side of the pipe is closed. You should close both sides of the pipe after you dup and write. (Or set FD_CLOEXEC).
Also note that doing it the way you do, you're dependent on pipe buffer size. If you write too much, write will be blocked as there's no reader. Do do it reliably, you should fork(), do exec in the child and write to the pipe in the parent. This way the pipe will have a reader and you will be able to write as much data as you want into it.
I'm confused about what I'm doing wrong when I'm attempting to output to a file after I've execed a second program.
Say I have input file that has the following names:
Marty B. Beach 7 8
zachary b. Whitaker 12 23
Ivan sanchez 02 15
Jim Toolonganame 9 03
After my programs finish, it will convert the student's names to their usernames and output it to a file such as this:
mbb0708
zbw1223
is0215
jt0903
Currently as my program stands, it outputs nothing to the file and the terminal seems to be in an infinite loop despite self testing my converter program before and making sure it outputs names correctly to stdout.
I'm not sure what I'm doing wrong here? First time programming with pipes. I know have to make use of the read and write commands extract the data, but with the dup2 command is that necessary for the read command alone?
manager.c
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
int main(int argc, char** argv)
{
pid_t pid;
int nbytes;
/*Buffer to hold data from pipe*/
char buffer[BUFSIZ + 1];
/*Pipe Information*/
int commpipe[2];
if(pipe(commpipe))
{
fprintf(stderr, "Pipe failed.\n");
return EXIT_FAILURE;
}
if((pid = fork()) == -1)
{
fprintf(stderr,"Fork error. Exiting.\n");
exit(1);
}
else if(pid == 0)
{
/*This is the child process. Close our copy of the write end of the file descriptor.*/
close(commpipe[1]);
/* Connect the read end of the pipe to standard input*/
dup2(commpipe[0], STDIN_FILENO);
/*Program will convert the Student's name to their respective names*/
execl("converter","converter",NULL);
/*Exit if failure appears*/
exit(EXIT_FAILURE);
}
else
{
FILE *file;
file = fopen("usernames.txt","a+"); //append a file(add text to a file or create a file it does not exist)
/*Close or copy of the read end of the file descriptor */
//close(commpipe[1]);
nbytes = write(commpipe[1], buffer, BUFSIZ);
//Read from pipe here first?
//Output to usernames.txt the usernames of the user from the pipe.
fprintf(file, "%s", buffer);
/*Wait for the child process to finish*/
waitpid(pid, NULL, 0);
}
return 0;
}
One problem is that after manager has sent all the data to converter, the manager is not closing commpipe[1]. Because of that, converter will never get EOF on stdin so will not exit.
Most likely manager isn't getting any data back from converter due to buffering. Some implementations of stdio use full-buffer buffering (as opposed to line-buffering) when not writing to a terminal. Once you fix the previous error and get the process to close, that will flush stdout. You can also consider adding fflush(stdout) after your puts line.
Have a look at the OpenGroup site, there's an example that looks similar to yours. I suggest you get the sample working first with some hard coded. Once that is working, add the code to read and write the results.
I made some minor changes to get the example working:
#include <stdlib.h>
#include <unistd.h>
#include <stdio.h>
#include <assert.h>
int main(int argc, char** argv){
int fildes[2];
const int BSIZE = 100;
char buf[BSIZE];
ssize_t nbytes;
int status;
status = pipe(fildes);
if (status == -1 ) {
/* an error occurred */
printf("Error!\n");
exit(-1);
}
printf("Forking!\n");
switch (fork()) {
case -1: /* Handle error */
printf("Broken Handle :(\n");
break;
case 0: /* Child - reads from pipe */
printf("Child!\n");
close(fildes[1]); /* Write end is unused */
nbytes = read(fildes[0], buf, BSIZE); /* Get data from pipe */
/* At this point, a further read would see end of file ... */
assert(nbytes < BSIZE); /* Prevent buffer overflow */
buf[nbytes] = '\0'; /* buf won't be NUL terminated */
printf("Child received %s", buf);
close(fildes[0]); /* Finished with pipe */
fflush(stdout);
exit(EXIT_SUCCESS);
default: /* Parent - writes to pipe */
printf("Parent!\n");
close(fildes[0]); /* Read end is unused */
write(fildes[1], "Hello world\n", 12); /* Write data on pipe */
close(fildes[1]); /* Child will see EOF */
/* Note that the Parent should wait for a response from the
child here, because the child process will be terminated once
the parent exits */
exit(EXIT_SUCCESS);
}
return 0;
}
As I understand, your converter program reads lines from stdin and writes them to stdout. As a pipe is a uni-directional entity, you will need TWO of them to communicate with the manager - one to send data to the converter and one to receive output from it.
Maybe you should consider enhancing the converter to take (as optional arguments) the name of an input and output file.