I am writing a sender/reader IPC C program for class, and I am having trouble setting the O_NONBLOCK flag to 0 so that my reader will block when the buffer it is attempting to read from is empty. Here are the functions I am using:
int set_nonblock_flag(int desc, int value)
{
int oldflags = fcntl(desc, F_GETFL, 0);
if (oldflags == -1)
return -1;
if (value != 0)
oldflags |= O_NONBLOCK;
else
oldflags &= ~O_NONBLOCK;
return fcntl(desc, F_SETFL, oldflags);
}
main()
main ()
{
int fd[2], nbytes;
char readbuff[26];
int r_pid = 0;
int s_pid = 0;
/* THIS IS ALL UPDATED!*/
fd[0] = open("fd.txt",O_RDONLY);
fd[1] = open("fd.txt",O_WRONLY);
set_nonblock_flag(fd[0], 0);
set_nonblock_flag(fd[1], 0);
/* END UPDATES */
pipe(fd);
r_pid = fork();
if (r_pid < 0) /* error */
{
fprintf( stderr, "Failed to fork receiver\n" );
exit( -1 );
}
else if (r_pid == 0) /* this is the receiver */
{
fprintf( stdout, "I, %d am the receiver!\n", getpid() );
close( fd[1] ); /* close write end */
nbytes = read( fd[0], readbuff, 1 );
printf ("nonblocking flag = %d\n", fcntl(fd, F_GETFL, 0));
printf ("Nbytes read: %d\n", nbytes );
}
... /* rest of function removed */
The line printf ("nonblocking flag = %d\n", fcntl(fd, F_GETFL, 0));
is just returning -1 as the flag status. Shouldn't it be 0 if it is cleared?
You are calling set_nonblock_flag with first argument as an array of ints.
Here's a snippet from fcntl manpage. The first argument should be a file descriptor.
SYNOPSIS
#include <fcntl.h>
int fcntl(int fildes, int cmd, ...);
DESCRIPTION
The fcntl() function shall perform the operations described below on open files. The fildes argument is a file
descriptor.
I think you want to first call pipe and then call set_nonblock_flag. So, I think what you really want is the following:
int fd[2];
...
pipe(fd);
set_nonblock_flag(fd[0], 0);
Related
Can one do non-blocking I/O on a pipe? fcntl fails to set O_NONBLOCK. Page 918 of The Linux Programming Interface includes a table 'Semantics of reading n bytes from pipe or FIFO (p)'. This table lists the behaviour of pipes and FIFO's with one column titled O_NONBLOCK enabled? This would imply that you can set the O_NONBLOCK flag on a pipe. Is this correct? The following code fails to set the flag, fcntl(2) does not report an error though.
#include <unistd.h>
#include <stdio.h>
#include <string.h>
#include <fcntl.h>
int main()
{
int fds[2];
pid_t pid;
char wr_buf[100];
char rd_buf[100];
pipe(fds);
pid = fork();
if ( pid )
{
while (1 )
{
memcpy( wr_buf, "abcdefghi\0",10);
write( fds[1], wr_buf, 10);
sleep(2);
}
}
else
{
int retval = fcntl( fds[0], F_SETFL, fcntl(fds[0], F_GETFL) | O_NONBLOCK);
printf("Ret from fcntl: %d\n", retval);
while (1)
{
ssize_t r=read( fds[0], rd_buf, 10 );
printf("read: %d\n", r);
if ( r > 0 )
{
printf("Buffer: %s\n", rd_buf);
}
else
{
printf("Read nothing\n");
perror("Error was");
sleep(1);
}
}
}
}
There is nothing special to pipe and O_NONBLOCK. The following example work as expected. I did not check every retval from every call to make the example a bit more readable. A real world application must do the checks.
#include <unistd.h>
#include <stdio.h>
#include <string.h>
#include <fcntl.h>
int main()
{
int fds[2];
pid_t pid;
char buf[100];
pipe(fds);
pid = fork();
if ( pid )
{
while (1 )
{
memcpy( buf, "abcdefghi\0",10);
write( fds[1], buf, 10);
sleep(2);
}
}
else
{
int retval = fcntl( fds[0], F_SETFL, fcntl(fds[0], F_GETFL) | O_NONBLOCK);
printf("Ret from fcntl: %d\n", retval);
while (1)
{
ssize_t r=read( fds[0], buf, 10 );
printf("read: %d\n", r);
if ( r > 0 )
{
printf("Buffer: %s\n", buf);
}
else
{
printf("Read nothing\n");
perror("Error was");
sleep(1);
}
}
}
}
After writing my example I inspect your code and found:
flags = fcntl(pfd[0], F_GETFD);
flags |= O_NONBLOCK;
if (fcntl(pfd[0], F_SETFD, flags))
Please change F_SETFD to F_SETFL and also for the get operation. You would not change the file descriptor flags but the file status flags :-)
From man 3 fcntl:
File descriptor flags
The following commands manipulate the flags associated with a file
descriptor. Currently, only one such flag is defined: FD_CLOEXEC, the
close-on-exec flag. If the FD_CLOEXEC bit is 0, the file descriptor
will remain open across an execve(2), otherwise it will be closed.
File status flags
Each open file description has certain associated status flags, ini‐
tialized by open(2) and possibly modified by fcntl(). Duplicated file
descriptors (made with dup(2), fcntl(F_DUPFD), fork(2), etc.) refer to
the same open file description, and thus share the same file status
flags.
F_SETFL (int)
Set the file status flags to the value specified by arg. File
access mode (O_RDONLY, O_WRONLY, O_RDWR) and file creation flags
(i.e., O_CREAT, O_EXCL, O_NOCTTY, O_TRUNC) in arg are ignored.
On Linux this command can change only the O_APPEND, O_ASYNC,
O_DIRECT, O_NOATIME, and O_NONBLOCK flags. It is not possible
to change the O_DSYNC and O_SYNC flags; see BUGS, below.
I want to my program to execute md5sum command to generate a hash given a file, and then storing the hash into an array (char *) variable.
I have read about popen(), but it's involving FILE * variables and I'd like to use only file descriptors.
Is there any way to do this?
As I noted in my comment, it is perfectly possible to implement the functionality of popen() but have the function return a file descriptor instead of a file stream like popen() does. There isn't a standard library function for the task. You'll need to create a pipe and fork. The child will do plumbing so that the standard output of the command goes to the write end of the pipe (and the read end is closed) and then executes the command. The parent will close the write end of the pipe, read the response from the read end of the pipe, and close that. It's not really all that hard — it is mildly fiddly, that's all.
The code corresponding to pclose() is a bit trickier. Should the code wait for the child to die, or at least attempt to collect the zombie? If so, how will it know which PID is appropriate wait for? It is tempting to just say "call close() with the returned file descriptor", but that could leave zombies around. Should it wait for the child to die, or should it just collect the corpse if the child has died, leaving it to other code to deal with the zombies? The solution implemented in the code below:
Limits file descriptors to 128 (including standard I/O channels).
Records the PID associated with a file descriptor in the fixed-size array pids.
Waits on the child with either 0 (unconditional wait) or WNOHANG using waitpid() and the saved PID associated with the file descriptor.
Reports the child status if the child has exited.
Otherwise reports success — 0.
It would be feasible to alter the design so that the array of PID values for each file descriptor is dynamically allocated. You control whether the dpclose() function waits for the child to exit or does not wait if it has not already exited.
The code does no signal handling. That is another layer of complication.
/* SO 6557-1879 */
/* #include "dpopen.h" */
#ifndef DPOPEN_H_INCLUDED
#define DPOPEN_H_INCLUDED
#include <fcntl.h> /* O_RDONLY or O_WRONLY for mode in dpopen() */
#include <sys/wait.h> /* WNOHANG for options in dpclose() */
/* dpopen() - similar to popen(), but returning a file descriptor */
/* The value in mode must be O_RDONLY or O_WRONLY */
extern int dpopen(char *cmd, int mode);
/* dpclose() - similar to pclose(), but working with a file descriptor returned by dpopen() */
/* The value in options must be 0 or WNOHANG */
/* The return value is the exit status of the child if available, 0 if not, or -1 if there is a problem */
extern int dpclose(int fd, int options);
#endif /* DPOPEN_H_INCLUDED */
#include <assert.h>
#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>
enum { MAX_PDOPEN_FD = 128 };
static pid_t pids[MAX_PDOPEN_FD];
int dpopen(char *cmd, int mode)
{
if (cmd == 0 || (mode != O_RDONLY && mode != O_WRONLY))
{
errno = EINVAL;
return -1;
}
int fd[2];
if (pipe(fd) != 0)
return -1;
/*
** Avoid embarrassment in debug builds if someone closed file
** descriptors too enthusiastically, and double check at run-time
** for non-debug builds. In some ways, it isn't very necessary as a
** runtime check - the circumstances are implausible. It is
** possible to code around fd[0] == STDIN_FILENO and fd[1] ==
** STDERR_FILENO, etc, but it is very messy to do so (having to
** avoid closing file descriptors, etc). It is simpler to close the
** two new file descriptors and return -1 with errno set to EINVAL
** if they overlap with the standard I/O descriptors. If this
** problem is detected, the program is already screwed up because at
** least one of standard input, standard output or standard error
** was closed.
*/
assert(fd[0] > STDERR_FILENO && fd[1] > STDERR_FILENO);
if (fd[0] <= STDERR_FILENO || fd[1] <= STDERR_FILENO)
{
close(fd[0]);
close(fd[1]);
errno = EINVAL;
return -1;
}
if (fd[0] >= MAX_PDOPEN_FD || fd[1] >= MAX_PDOPEN_FD)
{
close(fd[0]);
close(fd[1]);
errno = EMFILE;
return -1;
}
/*
** Prepare for forking - minimal step. See SO 5011-0992
** (https://stackoverflow.com/q/50110992 and
** https://stackoverflow.com/a/50112169/): "Why does forking my
** process cause the file to be read infinitely?"
** See also SO 0297-9209 (https://stackoverflow.com/q/2979209 and
** https://stackoverflow.com/a/34247021) "Using fflush(stdin)",
** noting that Standard C and POSIX diverge somewhat; POSIX mandates
** behaviour that the C standard does not. It would be possible to
** ensure standard input is 'clean' using code such as:
**
** if (lseek(fileno(stdin), 0L, SEEK_CURR) >= 0)
** fflush(stdin);
**
** Standard error is normally not a problem; by default, it is not
** fully buffered.
*/
fflush(stdout);
pid_t pid = fork();
if (pid < 0)
{
close(fd[0]);
close(fd[1]);
return -1;
}
if (pid == 0)
{
/* Child */
if (mode == O_RDONLY)
dup2(fd[1], STDOUT_FILENO);
else
dup2(fd[0], STDIN_FILENO);
close(fd[0]);
close(fd[1]);
char *argv[] = { "/bin/sh", "-c", cmd, 0 };
execv(argv[0], argv);
exit(EXIT_FAILURE);
}
/* Parent */
if (mode == O_RDONLY)
{
close(fd[1]);
pids[fd[0]] = pid;
return fd[0];
}
else
{
close(fd[0]);
pids[fd[1]] = pid;
return fd[1];
}
}
int dpclose(int fd, int options)
{
if (fd <= STDERR_FILENO || fd >= MAX_PDOPEN_FD || pids[fd] == 0 ||
(options != 0 && options != WNOHANG))
{
errno = EINVAL;
return -1;
}
if (close(fd) != 0)
return -1;
pid_t corpse;
int status;
pid_t child = pids[fd];
pids[fd] = 0;
if ((corpse = waitpid(child, &status, options)) == child)
return status;
return 0;
}
int main(void)
{
int fd1 = dpopen("ls -ltr", O_RDONLY);
int fd2 = dpopen("cat > ls.out; sleep 10", O_WRONLY);
if (fd1 < 0 || fd2 < 0)
{
fprintf(stderr, "failed to create child processes\n");
exit(EXIT_FAILURE);
}
char buffer[256];
ssize_t rbytes;
while ((rbytes = read(fd1, buffer, sizeof(buffer))) > 0)
{
ssize_t wbytes = write(fd2, buffer, rbytes);
if (wbytes != rbytes)
{
fprintf(stderr, "Failed to write data\n");
close(fd1);
close(fd2);
exit(EXIT_FAILURE);
}
}
if (dpclose(fd1, WNOHANG) < 0 || dpclose(fd2, 0) < 0)
{
fprintf(stderr, "failed to close pipes correctly\n");
exit(EXIT_FAILURE);
}
return 0;
}
See also Why does forking my process cause the file to be read infinitely? and
my answer.
See also Using fflush(stdin) and
my answer, noting also the information in SO 5011-0992.
I am doing a code review right now and I was blown away by the amount of code that the guy wrote just to execute one script (with hard-coded path and no input arguments) and read the output out of it. (BTW he had a lot of bugs in it.)
I encountered a similar issue before and it was suggested to me that doing pipe/fork/exec manually is "more secure". I am aware of two potential problems:
As system() and popen() execute shell commands, it is possible to slip potentially harmful environment variable values to the program executed this way
Another is that when the command is constructed from user input. I can imagine subshells doing all kinds of harmful things and so on.
I was wondering if suggesting to use popen() instead would be OK in this case. It would greatly simplify the code. The second point is not an issue as there is no user input. By using env -i to clean the environment before executing the script should make the first issue go away:
FILE *fp = popen("/usr/bin/env -i /path/to/some/fancy/script.sh", "r");
/* ... */
Are there any other potential issues I am missing, or is doing the script execution "manually" still worth the effort?
This is technically not your answer to the question on how to call popen() safely, but the answer to the question you should have asked: "How to make a better popen()"
The function child_spawn(argv, env, flags) will set up pipes for communicating with a child process, and spawn the child. It will return a
struct child that holds the child pid and file descriptors for communication.
argv is a NULL terminated string array of command and arguments, while env is a NULL terminated string array of environment variables. If env is NULL, the child will inherit the environment from the parent.
So argv should have the form
const char* argv[] = {"/bin/ls", "-l", NULL};
And env should have the form
const char **env = NULL;
or
const char *env[] =
{
"PATH=/bin:/usr/bin",
"HOME=/tmp",
"SHELL=/bin/sh",
NULL
};
When you are finished with the child process, child_wait() will close the file descriptors associated with the child and wait for it to exit.
To use child_spawn() as a substitute of popen() you call it like this:
struct child c = child_spawn(argv, NULL, CHILD_PIPE_STDOUT);
You may now read from c->fd_out to get the content of the childs stdout.
The constants CHILD_PIPE_STDIN, CHILD_PIPE_STDOUT and CHILD_PIPE_STDERR may be "or"-ed together to have a valid file descriptors in c->fd_in, c->fd_out, c->fd_err
Be aware that if you spawn a child with CHILD_PIPE_STDIN|CHILD_PIPE_STDOUT, there is a risk of deadlock when reading and writing, unless you do non-blocking io.
The function my_system() is an example on how to implement a safer system() using child_spawn()
/*
We have to #define _GNU_SOURCE to get access to `char **environ`
*/
#define _GNU_SOURCE
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <sys/sendfile.h>
#include <stdio.h>
#include <stdlib.h>
#include <error.h>
#include <errno.h>
#include <string.h>
struct child
{
pid_t pid;
int fd_in;
int fd_out;
int fd_err;
};
static void
close_if_valid(int fd)
{
if (fd != -1) close(fd);
}
/*
Closes all file-descriptors for child communication
and waits for child to exit
returns status value from waitpid().
see `man waitpid` on how to interpret that value
*/
int child_wait(struct child *c)
{
close_if_valid(c->fd_in);
close_if_valid(c->fd_out);
close_if_valid(c->fd_err);
int status;
pid_t p = waitpid(c->pid, &status, 0);
if (p == 0)
error(1, errno, "waitpid() failed");
return status;
}
int
dup_if_valid(int fd1, int fd2)
{
if (fd1 != -1 && fd1 != fd2)
return dup2(fd1, fd2);
return fd2;
}
pid_t
child_spawn_fd(const char *const argv[], const char *const env[],
int in, int out, int err)
{
fflush(stdout);
pid_t p = fork();
if (p)
return p;
/***********************
We are now in child
***********************/
/*
Set file descriptors to expected values,
-1 means inherit from parent
*/
if (dup_if_valid(in, 0) == -1)
goto CHILD_ERR;
if (dup_if_valid(out, 1) == -1)
goto CHILD_ERR;
if (dup_if_valid(err, 2) == -1)
goto CHILD_ERR;
/*
close all unneeded file descriptors
This will free resources and keep files and sockets belonging to
the parent from beeing open longer than needed
On *BSD we may call `closefrom(3);`, but this may not exits
on Linux. So we loop over all possible file descriptor numbers.
A better solution, is to look in `/proc/self/fs`
*/
int max_fd = sysconf(_SC_OPEN_MAX);
for (int fd = 3; fd <= max_fd; fd++)
close(fd);
if (env)
environ = (char **)env;
/* Change to execvp if command should be looked up in $PATH */
execv(argv[0], (char * const *)argv);
CHILD_ERR:
_exit(1);
}
#define CHILD_PIPE_STDIN (1 << 0)
#define CHILD_PIPE_STDOUT (1 << 1)
#define CHILD_PIPE_STDERR (1 << 2)
#define READ_END 0
#define WRITE_END 1
struct child
child_spawn(const char * const argv[], const char * const env[], int flags)
{
int in_pipe[2] = {-1, -1};
int out_pipe[2] = {-1, -1};
int err_pipe[2] = {-1, -1};
if (flags & CHILD_PIPE_STDIN)
if (pipe(in_pipe))
error(EXIT_FAILURE, errno, "pipe(in_pipe) failed");
if (flags & CHILD_PIPE_STDOUT)
if (pipe(out_pipe))
error(EXIT_FAILURE, errno, "pipe(out_pipe) failed");
if (flags & CHILD_PIPE_STDERR)
if (pipe(err_pipe))
error(EXIT_FAILURE, errno, "pipe(err_pipe) failed");
pid_t p = child_spawn_fd(argv, env,
in_pipe[READ_END],
out_pipe[WRITE_END],
err_pipe[WRITE_END]);
if (p == -1)
error(EXIT_FAILURE, errno, "fork() failed");
close_if_valid(in_pipe[READ_END]);
close_if_valid(out_pipe[WRITE_END]);
close_if_valid(err_pipe[WRITE_END]);
struct child c =
{
.pid = p,
.fd_in = in_pipe[WRITE_END],
.fd_out = out_pipe[READ_END],
.fd_err = err_pipe[READ_END],
};
return c;
}
/*
Safer implementation of `system()`. It does not invoke shell, and takes
command as NULL terminated list of execuatable and parameters
*/
int
my_system(const char * const argv[])
{
struct child c = child_spawn(argv, NULL, 0);
int status = child_wait(&c);
if (WIFEXITED(status))
return WEXITSTATUS(status);
else
return -1;
}
int
main (int argc, char **argv)
{
printf("Running 'ls -l' using my_system()\n");
printf("---------------------------------\n");
fflush(stdout);
const char * ls_argv[] =
{
"/bin/ls",
"-l",
NULL
};
int e = my_system(ls_argv);
printf("---------\n");
printf("\exit code ---> %d\n", e);
printf("\nRunning 'ls -l' using child_spawn() and reading from stdout\n");
printf("-----------------------------------------------------------\n");
fflush(stdout);
struct child c = child_spawn(ls_argv, NULL, CHILD_PIPE_STDOUT);
/*
Read from the childs stdout and write to current stdout
*/
size_t copied = 0;
while (1)
{
char buff[4096];
ssize_t rlen = read(c.fd_out, buff, 4096);
if (rlen == -1)
error(EXIT_FAILURE, errno, "read() failed");
if (rlen == 0)
break;
size_t written = 0;
while (written < rlen)
{
ssize_t wlen = write(1, buff + written, rlen - written);
if (wlen == -1)
error(EXIT_FAILURE, errno, "write() failed");
written += wlen;
}
copied += written;
}
/* Wait for child to end */
int status = child_wait(&c);
printf("---------\n");
if (WIFEXITED(status))
{
printf(" ---> child exited normally with exit code %d and with %ld bytes copied\n",
WEXITSTATUS(status),
copied);
}
else
printf(" ---> child exited by som other reason than _exit()");
printf("\nWriting to Elmer Fudd filter\n");
const char *quote = "Be very very quiet, I'm hunting rabbits!\n";
printf("Original text: %s", quote);
printf("-----------------------------------------------------------\n");
fflush(stdout);
const char *fudd_filter[] =
{"/bin/sed", "-e" "s/r/w/g", NULL};
struct child c2 = child_spawn(fudd_filter, NULL, CHILD_PIPE_STDIN);
size_t qlen = strlen(quote);
const char *q = quote;
while (qlen)
{
ssize_t wlen = write(c2.fd_in, q, qlen);
if (wlen == -1)
error(EXIT_FAILURE, errno, "write() failed");
q += wlen;
qlen -= wlen;
}
child_wait(&c2);
}
My program have to send some bytes of information by using unnamed pipes.
I have a txt file named "input" which is supposed to be read by the program and it's information have to be send and write in another file named "output". Also i must use read(), write(), open() functions.
My code look like this:
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <time.h>
#include <string.h>
#define BUFSIZE 25
int main( int argc, char *argv[] ) {
srand(time(NULL));
pid_t pid;
int mypipefd[2];
int ret;
char buf[BUFSIZE];
int output;
int stream;
int nbytes;
ret = pipe(mypipefd);
if( ret == -1 ) {
perror( "pipe error");
exit(1);
}
pid = fork();
if( pid == -1 ) {
perror( "FORK ERROR...");
exit(2);
}
if( pid == 0 ) {
/* CHILD */
printf(" Child process...\n");
stream = open("input.txt", O_RDONLY);
if (close(mypipefd[0]) == -1 ) {
perror("ERROR CLOSING PIPE");
exit(3);
}
while ( (nbytes = read(stream, buf, BUFSIZE)) > 0 ) {
sleep(rand() %2);
write(mypipefd[1], buf, nbytes );
}
if ( close(stream) == -1 ) {
perror("ERROR CLOSING STREAM");
exit(4);
}
}
else {
/* PARENT */
printf(" Parent process...\n");
output = open("output.txt", O_CREAT | O_WRONLY, 00777);
while ( (nbytes = read(mypipefd[0], buf, BUFSIZE)) > 0 ) {
write(output, buf, nbytes);
}
printf("buf: %s\n", buf);
if (close(output) == -1) {
perror("ERROR CLOSING OUTPUT");
exit(5);
}
if (close(mypipefd[1]) == -1 ) {
perror("ERROR CLOSING PIPE");
exit(6);
}
}
return 0;
}
Unfortunately the code is not working terminal screen
Before I tried while loop and was sending all the information at once, it worked, but output file looked like this output file
while the input file look like this input file
The primary bug was that the parent must do close(mypipefd[1]) before the parent read loop (and not after). This prevented the parent from seeing EOF on the pipe after the child was done writing.
Also, you were missing a waitpid in the parent.
The printf for buf in the parent was in the wrong place [after the read loop]. At that point, buf can't be guaranteed to have the correct data or that it's correctly zero-terminated. That's why stdout had some garbage chars at the end.
So, In addition to outputting to the output file, the loop should output to stdout, but should use fwrite as buf can't be guaranteed to be zero terminated.
I had missed that in my initial post, so I've corrected it.
As per my top comments, the child should loop on a [possible] partial write to the pipe. I coded that.
Here's the version with the bugs annotated and fixed:
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <time.h>
#include <string.h>
#include <sys/types.h>
#include <sys/wait.h>
#define BUFSIZE 25
int main( int argc, char *argv[] ) {
srand(time(NULL));
pid_t pid;
int mypipefd[2];
int ret;
char buf[BUFSIZE];
int output;
int stream;
int nbytes;
ret = pipe(mypipefd);
if( ret == -1 ) {
perror( "pipe error");
exit(1);
}
pid = fork();
if( pid == -1 ) {
perror( "FORK ERROR...");
exit(2);
}
if( pid == 0 ) {
/* CHILD */
printf(" Child process...\n");
stream = open("input.txt", O_RDONLY);
if (close(mypipefd[0]) == -1 ) {
perror("ERROR CLOSING PIPE");
exit(3);
}
while ( (nbytes = read(stream, buf, BUFSIZE)) > 0 ) {
sleep(rand() %2);
#if 0
write(mypipefd[1], buf, nbytes );
#else
// NOTE: this _should_ work but adds extra at the end
int off;
int wlen;
for (off = 0; nbytes > 0; off += wlen, nbytes -= wlen) {
wlen = write(mypipefd[1], buf + off, nbytes );
if (wlen <= 0)
break;
}
#endif
}
if ( close(stream) == -1 ) {
perror("ERROR CLOSING STREAM");
exit(4);
}
// NOTE/FIX: child must close it's side of the pipe
#if 1
close(mypipefd[1]);
#endif
}
else {
/* PARENT */
printf(" Parent process...\n");
// NOTE/FIX: this must be closed _before_ the read loop -- holding it
// open prevents parent from seeing EOF on pipe
#if 1
if (close(mypipefd[1]) == -1 ) {
perror("ERROR CLOSING PIPE");
exit(6);
}
#endif
#if 1
printf("buf: ");
#endif
output = open("output.txt", O_CREAT | O_WRONLY, 00777);
while ( (nbytes = read(mypipefd[0], buf, BUFSIZE)) > 0 ) {
write(output, buf, nbytes);
#if 1
fwrite(buf,1,nbytes,stdout);
#endif
}
// NOTE/BUG: the buffer at this point will only have the data from
// the _last_ read and may not be null terminated
#if 0
printf("buf: %s\n", buf);
#else
printf("\n");
#endif
if (close(output) == -1) {
perror("ERROR CLOSING OUTPUT");
exit(5);
}
// NOTE/BUG: this must be closed _before_ the parent's read loop
#if 0
if (close(mypipefd[1]) == -1 ) {
perror("ERROR CLOSING PIPE");
exit(6);
}
#endif
// NOTE/FIX: this is missing (prevents orphan/zombie child process)
#if 1
waitpid(pid,NULL,0);
#endif
}
return 0;
}
UPDATE:
but i don't understand what does "for" loop do here
A write to a pipe can generate a "short write" (e.g. you want to write 20 but the return value (i.e. number of bytes actually written) comes back with 15. You have to index into the buffer and write the remaining bytes in subsequent writes.
There is a kernel limit on how many bytes can be written in a single atomic write (e.g.) if you did write(mypipefd[1],buf,10000000), the kernel doesn't have space allocated for such a large write, so it will return the value of what it could append to the pipe buffer [in the kernel].
Also, let's say the kernel pipe buffer can hold 64 bytes. And you write buffers of size 64 to it. Maybe the reader is reading only 32 bytes. So, the first write is fine. Then reader reads out 32 bytes. So, the next write to the pipe of 64, there is only space for 32 bytes, so the write will return 32
Program have to display: "buf: This is ra" then "buf: ndom text"
Okay, I've fixed that
At last, I need to implement error handling everywhere.
I've annotated places where I'd add error and handling, along with some things to look for.
Anyway, here's an updated version. I've left in the // NOTE/* comments but removed the #if/#endif pairs to make an easier read.
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <time.h>
#include <string.h>
#include <sys/types.h>
#include <sys/wait.h>
#define BUFSIZE 25
int
main(int argc, char *argv[])
{
srand(time(NULL));
pid_t pid;
int mypipefd[2];
int ret;
char buf[BUFSIZE];
int output;
int stream;
int nbytes;
ret = pipe(mypipefd);
if (ret == -1) {
perror("pipe error");
exit(1);
}
pid = fork();
if (pid == -1) {
perror("FORK ERROR...");
exit(2);
}
if (pid == 0) {
/* CHILD */
printf(" Child process...\n");
stream = open("input.txt", O_RDONLY);
if (close(mypipefd[0]) == -1) {
perror("ERROR CLOSING PIPE");
exit(3);
}
while ((nbytes = read(stream, buf, BUFSIZE)) > 0) {
sleep(rand() % 2);
// NOTE/FIX: writing to pipes _can_ generate a _short_ write. that
// is, (e.g.) if the length given to write is 20, the return value
// may be only 15. this means that the remaining 5 bytes must be
// sent in a second/subsequent write
int off;
int wlen;
for (off = 0; nbytes > 0; off += wlen, nbytes -= wlen) {
wlen = write(mypipefd[1], buf + off, nbytes);
if (wlen < 0) {
perror("ERROR WRITING TO FILE");
exit(3);
}
if (wlen == 0)
break;
}
}
if (close(stream) == -1) {
perror("ERROR CLOSING STREAM");
exit(4);
}
// NOTE/FIX: child must close it's side of the pipe
// NOTE/ERRCODE: check error code here
close(mypipefd[1]);
}
else {
/* PARENT */
printf(" Parent process...\n");
// NOTE/FIX: this must be closed _before_ the read loop -- holding it
// open prevents parent from seeing EOF on pipe
if (close(mypipefd[1]) == -1) {
perror("ERROR CLOSING PIPE");
exit(6);
}
// NOTE/ERRCODE: this should be checked for -1 (i.e. output file
// could not be opened for file permission, etc. or other reasons
// similar to those for the file write below)
output = open("output.txt", O_CREAT | O_WRONLY, 00777);
// NOTE/FIX: we read one less than buffer size to allow for adding an
// artificial zero byte at the end
while ((nbytes = read(mypipefd[0], buf, BUFSIZE - 1)) > 0) {
// NOTE/ERRCODE: error handling _could_ be added here but it would
// be rare (e.g. filesystem has an I/O error because it's full or
// marked R/O because of an I/O error on the underlying disk)
write(output, buf, nbytes);
// write partial buffer to stdout
buf[nbytes] = 0;
printf("buf: %s\n",buf);
}
if (close(output) == -1) {
perror("ERROR CLOSING OUTPUT");
exit(5);
}
// NOTE/FIX: this is missing (prevents orphan/zombie child process)
// NOTE/ERRCODE: yes, this _can_ have an error return but here it's
// unlikely because we _know_ that pid is valid
// what can be done is to do:
// int status;
// waitpid(pid,&status,0)
// then process the return code from the child using the W* macros
// provided (e.g. WIFEXITED, WSTATUS) on status
waitpid(pid, NULL, 0);
}
return 0;
}
Can one do non-blocking I/O on a pipe? fcntl fails to set O_NONBLOCK. Page 918 of The Linux Programming Interface includes a table 'Semantics of reading n bytes from pipe or FIFO (p)'. This table lists the behaviour of pipes and FIFO's with one column titled O_NONBLOCK enabled? This would imply that you can set the O_NONBLOCK flag on a pipe. Is this correct? The following code fails to set the flag, fcntl(2) does not report an error though.
#include <unistd.h>
#include <stdio.h>
#include <string.h>
#include <fcntl.h>
int main()
{
int fds[2];
pid_t pid;
char wr_buf[100];
char rd_buf[100];
pipe(fds);
pid = fork();
if ( pid )
{
while (1 )
{
memcpy( wr_buf, "abcdefghi\0",10);
write( fds[1], wr_buf, 10);
sleep(2);
}
}
else
{
int retval = fcntl( fds[0], F_SETFL, fcntl(fds[0], F_GETFL) | O_NONBLOCK);
printf("Ret from fcntl: %d\n", retval);
while (1)
{
ssize_t r=read( fds[0], rd_buf, 10 );
printf("read: %d\n", r);
if ( r > 0 )
{
printf("Buffer: %s\n", rd_buf);
}
else
{
printf("Read nothing\n");
perror("Error was");
sleep(1);
}
}
}
}
There is nothing special to pipe and O_NONBLOCK. The following example work as expected. I did not check every retval from every call to make the example a bit more readable. A real world application must do the checks.
#include <unistd.h>
#include <stdio.h>
#include <string.h>
#include <fcntl.h>
int main()
{
int fds[2];
pid_t pid;
char buf[100];
pipe(fds);
pid = fork();
if ( pid )
{
while (1 )
{
memcpy( buf, "abcdefghi\0",10);
write( fds[1], buf, 10);
sleep(2);
}
}
else
{
int retval = fcntl( fds[0], F_SETFL, fcntl(fds[0], F_GETFL) | O_NONBLOCK);
printf("Ret from fcntl: %d\n", retval);
while (1)
{
ssize_t r=read( fds[0], buf, 10 );
printf("read: %d\n", r);
if ( r > 0 )
{
printf("Buffer: %s\n", buf);
}
else
{
printf("Read nothing\n");
perror("Error was");
sleep(1);
}
}
}
}
After writing my example I inspect your code and found:
flags = fcntl(pfd[0], F_GETFD);
flags |= O_NONBLOCK;
if (fcntl(pfd[0], F_SETFD, flags))
Please change F_SETFD to F_SETFL and also for the get operation. You would not change the file descriptor flags but the file status flags :-)
From man 3 fcntl:
File descriptor flags
The following commands manipulate the flags associated with a file
descriptor. Currently, only one such flag is defined: FD_CLOEXEC, the
close-on-exec flag. If the FD_CLOEXEC bit is 0, the file descriptor
will remain open across an execve(2), otherwise it will be closed.
File status flags
Each open file description has certain associated status flags, ini‐
tialized by open(2) and possibly modified by fcntl(). Duplicated file
descriptors (made with dup(2), fcntl(F_DUPFD), fork(2), etc.) refer to
the same open file description, and thus share the same file status
flags.
F_SETFL (int)
Set the file status flags to the value specified by arg. File
access mode (O_RDONLY, O_WRONLY, O_RDWR) and file creation flags
(i.e., O_CREAT, O_EXCL, O_NOCTTY, O_TRUNC) in arg are ignored.
On Linux this command can change only the O_APPEND, O_ASYNC,
O_DIRECT, O_NOATIME, and O_NONBLOCK flags. It is not possible
to change the O_DSYNC and O_SYNC flags; see BUGS, below.