I'm trying to write a program that reads some text from a file and prints it to the screen. The parent will read the content of the file write it to n number of pipes and the children will read it and then print it.
So far this is what I've got:
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <string.h>
int main (void)
{
pid_t pid;
char c;
FILE *fd;
char buf[100];
int N_CHILDREN = 2;
int p[N_CHILDREN][2];
int i,j;
for(i=0; i<N_CHILDREN; i++)
{
pipe(p[i]);
}
fd=fopen("123.txt","r");
for(j=0; j < N_CHILDREN;j++)
{
pid = fork ();
if (pid == 0)
{
close (p[j][1]);
while(read(p[j][0], &fd,sizeof(buf)) > 0)
printf("\n%c",&fd);
}
if (pid < 0)
{
//Fork Failed
fprintf (stderr, "Fork failure.\n");
return EXIT_FAILURE;
}
if ( pid > 0) //Parent
{
close (p[j][0]);
write(p[j][1], fd ,sizeof(buf));
}
}
}
Problem is it's not really reading the content from the file. I've tried sending it a string of characters instead of reading from a file and it worked as intended, both children printed the message one time and the program ended.
Any thoughts about it? After reading the manuals I still can't see where the problem is.
You are confusing C Standard I/O streams (created with fopen(); written to with fprintf() et al., read with fscanf() et al.) with Unix file descriptor I/O (created with open() or pipe() et al., written to with write() et al., read with read() et al.)
Standard I/O functions take an opaque FILE * as a handle; Unix I/O functions take a file descriptor (a small int) as a handle.
Once you understand the conceptual difference, I'm sure you will realize that
FILE *fd = ...
read(..., &fd, ...);
is reading into a pointer-to-FILE -- not terribly useful :-)
Several problems here:
you make bad usage of read function by passing &fd, which is a FILE*. This function needs a pointer to the "buffer" to print, here I guess buf.
you don't check errors. For example if fopen fails.
you never read data from your file, so you have "nothing" to send to children.
you have to get returned value of read (in children) because it is the effective amount of data that you get. So it is the amount of data that you have to print after that (to stdout).
So here is an example code, see comments inside:
// put here all the needed includes (see manpages of functions)
// it is better to create a function for the child: the code
// is easier to read
// the child just get the file descriptor to read (the pipe)
void child(int fd) {
char buf[100]; // buffer to store data read
int ret; // the number of bytes that are read
// we read from 'fd', into 'buf'. It returns the number of bytes
// really read (could be smaller than size). Return <=0 when over
while((ret = read(fd, buf, sizeof(buf))) > 0) {
// write the 'ret' bytes to STDOUT (which as file descriptor 1)
write(1, buf, ret);
}
}
int main (void) {
pid_t pid;
char buf[100];
int N_CHILDREN = 2;
int p[N_CHILDREN][2];
int i,j, ret;
int fdi;
// create the pipes
for(i=0; i<N_CHILDREN; i++) {
if (pipe(p[i]) == -1) {
perror("pipe"); // ALWAYS check for errors
exit(1);
}
}
// open the file (with 'open' not 'fopen', more suitable for
// reading raw data
fdi = open("123.txt",O_RDONLY);
if (fdi < 0) {
perror("open"); // ALWAYS check for errors
exit(1);
}
// just spawn the children
for(j=0; j < N_CHILDREN;j++) {
pid = fork();
if (pid < 0) {
perror("fork"); // ALWAYS check for errors
exit(1);
}
if (pid == 0) { // child
close(p[j][1]); // close the writing part
child(p[j][0]); // call child function with corresp. FD
exit(0); // leave : the child should do nothing else
}
}
// don't need that part
for(j=0; j<N_CHILDREN; j++) {
close(p[j][0]); // close the read-part of pipes
}
// need to read file content, see comment in child() function
while ((ret = read(fdi, buf, sizeof(buf))) > 0) {
// write the data to all children
for(j=0; j<N_CHILDREN; j++) {
write(p[j][1], buf , ret); // we write the size we get
}
}
// close everithing
for(j=0; j<N_CHILDREN; j++) {
close(p[j][1]); // needed, see text after
}
close(fdi); // close read file
return(0); // main returns a int, 0 is "ok"
}
You have to close every parts of pipes when not needed or when it is over. Until a file descriptor is open a read will block the process. Only when last write counterpart is closed the read returns <=0.
Note: 1. the correct usage of read/write function 2. checking for errors 3. reading from the file and writing to the pipe(s) 4. dealing with effective amount of data read (ret variable) so that you can write (to "screen" or to an other file descriptor the right amount of data.
You're not reading anything in to buf as far as I can tell.
Related
I am trying to write to a file and display the output of the thing i wrote with another process. The code i come up with:
void readLine (int fd, char *str) {
int n;
do {
n = read (fd, str, 1);
} while (*str++ != '\0');
}
int main(int argc,char ** argv){
int fd=open("sharedFile",O_CREAT|O_RDWR|O_TRUNC,0600);
if(fork()==0){
char buf[1000];
while(1) {
readLine(fd,buf);
printf("%s\n",buf);
}
}else{
while(1){
sleep(1);
write(fd,"abcd",strlen("abcd")+1);
}
}
}
the output i want (each result spaced from the other with a period of one second):
abcd
abcd
abcd
....
Unfortunately this code doesn't work, it seems that the child process (the reader of the file "sharedFile") reads junk from the file because somehow it reads values even when the file is empty.
When trying to debug the code, readLine function never reads the written file correctly,it always reads 0 bytes.
Can someone help?
First of all, when a file descriptor becomes shared after forking, both the parent and child are pointing to the same open file description, which means in particular that they share the same file position. This is explained in the fork() man page.
So whenever the parent writes, the position is updated to the end of the file, and thus the child is always attempting to read at the end of the file, where there's no data. That's why read() returns 0, just as normal when you hit the end of a file.
(When this happens, you should not attempt to do anything with the data in the buffer. It's not that you're "reading junk", it's that you're not reading anything but are then pretending that whatever junk was in the buffer is what you just read. In particular your code utterly disregards the return value from read(), which is how you're supposed to tell what you actually read.)
If you want the child to have an independent file position, then the child needs to open() the file separately for itself and get a new fd pointing to a new file description.
But still, when the child has read all the data that's currently in the file, read() will again return 0; it won't wait around for the parent to write some more. The fact that some other process has a file open for writing don't affect the semantics of read() on a regular file.
So what you'll need to do instead is that when read() returns 0, you manually sleep for a while and then try again. When there's more data in the file, read() will return a positive number, and you can then process the data you read. Or, there are more elegant but more complicated approaches using system-specific APIs like Linux's inotify, which can sleep until a file's contents change. You may be familiar with tail -f, which uses some combination of these approaches on different systems.
Another dangerous bug is that if someone else writes text to the file that doesn't contain a null byte where expected, your child will read more data than the buffer can fit, thus overrunning it. This can be an exploitable security vulnerability.
Here is a version of the code that fixes these bugs and works for me:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
void readLine (int fd, char *str, size_t max) {
size_t pos = 0;
while (pos < max) {
ssize_t n = read(fd, str + pos, 1);
if (n == 0) {
sleep(1);
} else if (n == 1) {
if (str[pos] == '\0') {
return;
}
pos++;
} else {
perror("read() failure");
exit(2);
}
}
fprintf(stderr, "Didn't receive null terminator in time\n");
exit(2);
}
int main(int argc, char ** argv){
int fd=open("sharedFile", O_CREAT|O_RDWR|O_TRUNC, 0600);
if (fd < 0) {
perror("parent opening sharedFile");
exit(2);
}
pid_t pid = fork();
if (pid == 0){
int newfd = open("sharedFile", O_RDONLY);
if (newfd < 0) {
perror("child opening sharedFile");
exit(2);
}
char buf[1000];
while (1) {
readLine(newfd, buf, 1000);
printf("%s\n",buf);
}
} else if (pid > 0) {
while (1){
sleep(1);
write(fd,"abcd",strlen("abcd")+1);
}
} else {
perror("fork");
exit(2);
}
return 0;
}
I create a function exec_in_child which takes the command arguments, pipe file descriptors (fds), read_flag and write_flag as input. When write_flag is set to 1, the child process should duplicate stdout to fds[1], and then execute the command. When read_flag is set to 1, the child should duplicate the stdin to fds[0] and the execute the command.
Do I have to close one end of the pipe when I'm reading/writing to
the other end?
The code below doesn't work. I'm trying to execute /bin/ls inside a child process, write the stdout to the pipe, and then read
it off in the parent process and print it. I'm not able to read in
the parent process.
Can I read and write to the pipe inside the same process without closing other? This situation arises when I want to child to read
from pipe, execute, and then write to the pipe.
#include <stdio.h> /* printf */
#include <stdlib.h>
#include <string.h> /* strlen, strcpy */
int exec_in_child(char *arguments[], const int temp[], int , int);
int main()
{
ssize_t bytes_read;
char *curr_dir = (char *)malloc(500);
int pipefd[2];
if (pipe(pipefd) == -1) {
perror("pipe");
exit(EXIT_FAILURE);
}
char *arguments[] = {"/bin/pwd",0};
exec_in_child(arguments, pipefd, 0, 1);
bytes_read = read(pipefd[0], curr_dir, strlen(curr_dir));
printf("%s = %d\n", "bytes read from pipe" ,(int)bytes_read);
printf("%s: %s\n","character read from the pipe",curr_dir);
return 0;
}
int exec_in_child(char * arguments[], const int fds[], int read_flag, int write_flag) {
pid_t pid;
pid = fork();
if (pid < 0) {
perror("Error: Fork Failed");
}
else if (pid == 0){ /*inside the child process */
if (read_flag == 1) {
dup2(fds[0], 0);
perror("Dup2 stdin");
}
if (write_flag == 1) {
dup2(fds[1], 1);
perror("Dup2 stdout");
}
execv(arguments[0], arguments);
perror("Error in child");
exit(1);
} /* if (pid == 0) */
else {
while(pid != wait(0));
} /* if(pid < 0) */
return 0;
}
I get this result:
hmwk1-skk2142(test) > ./a.out
Dup2 stdout: Success
bytes read from pipe = 0
character read from the pipe:
To answer your questions:
1) You do not need to close either end of the pipe in order to use the other end. However, you generally want to close any end(s) of the pipe you're not using. The biggest reason to do this is that the pipe will only close when all open write file descriptors are closed.
2) Your code isn't working because you're using strlen() improperly. This function calculates the length of a string by searching for the null (0) character. When you malloc() the storage for curr_dir you have no guarantee of what resides there (though it will usually be zeroed, as in this case).
Thus, your call strlen(curr_dir) returns zero, and the read() system call thinks you want to read up to zero bytes of data. Change your read call to the following:
bytes_read = read(pipefd[0], curr_dir, 500);
and your code will work perfectly.
3) You can read and write to any pipe you've got a valid file descriptor to. A single process can absolutely read and write the same pipe.
Parent has opened a file to read, I fork two children to read from file and write on different files.
child 1 reads the first line, and child 2,reads nothing. When I do an ftell, it reaches the end.
Can anyone please explain this behaviour?
f[0] = fopen("input", "r");
for ( i = 1; i <= 2; i++ ){
if ((pid = fork()) != 0){
waitpid(pid);
}
else
{
snprintf ( buffer, 10, "output%d", i );
printf("opening file %s \n",buffer);
f[i] = fopen( buffer, "w");
fgets(buff2, 10, f[0]);
fprintf(f[i], "%s", buff2);
fclose(f[i]);
_exit(0);
}
}
fclose(f[0]);
Your problem is buffering. stdio reads files on fully buffered mode by default, which means a call to fgets(3) will actually read a huge block of characters from the file, buffer everything, and then return the first line, while leaving the rest in the buffer, in the perspective of being called again in the future (remember that stdio strives for minimizing the number of read(2) and write(2) syscalls). Note that stdio buffering is a user-space thing; all the kernel sees is a single process reading a huge block on that file, and so the cursor is updated accordingly.
Common block sizes are 4096 and 8192; your input file is probably smaller than that and so the first process that calls fgets(3) ends up reading the whole file, leaving the cursor in the end. Buffering is tricky.
What can you do? One solution I can think of is to disable buffering (since this is an input stream we're talking about, we can't use line buffered mode, because line buffering is meaningless for input streams). So if you disable buffering on the input stream before forking, everything will work. This is done with setvbuf(3).
Here's a working example:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <errno.h>
static FILE *f[3];
static char buffer[128];
static char buff2[128];
int main(void) {
pid_t pid;
int i;
if ((f[0] = fopen("input", "r")) == NULL) {
perror("Error opening input file");
exit(EXIT_FAILURE);
}
if (setvbuf(f[0], NULL, _IONBF, 0) < 0) {
perror("setvbuf(3) failed");
exit(EXIT_FAILURE);
}
for (i = 1; i <= 2; i++) {
if ((pid = fork()) < 0) {
perror("fork(2) failed");
exit(EXIT_FAILURE);
}
if (pid != 0) {
if (waitpid(pid, NULL, 0) < 0) {
perror("waitpid(2) failed");
exit(EXIT_FAILURE);
}
} else {
snprintf(buffer, sizeof(buffer), "output%d", i);
printf("opening file %s\n", buffer);
if ((f[i] = fopen(buffer, "w")) == NULL) {
perror("fopen(2) failed");
exit(EXIT_FAILURE);
}
errno = 0;
if (fgets(buff2, sizeof(buff2), f[0]) == NULL) {
if (errno != 0) {
perror("fgets(3) error");
exit(EXIT_FAILURE);
}
}
fprintf(f[i], "%s", buff2);
fclose(f[i]);
exit(EXIT_SUCCESS);
}
}
fclose(f[0]);
return 0;
}
Note that this may incur a significant performance hit. Your code will be making a lot more syscalls, and it might be too expensive for huge files, but it doesn't seem to be a problem since apparently you're dealing with relatively small input files.
Here's an extract of my fork() man page:
The child process has its own copy of the parent's descriptors. These descriptors reference the same underlying objects, so that, for instance, file pointers in file objects are shared between the child and the parent, so that an lseek(2) on a descriptor in the child process can affect a subsequent read or write by the parent. This descriptor copying is also used by the shell to establish standard input and output for newly created processes as well as to set up pipes.
So your behaviour is completely normal. If you want your child to have its own file descriptor, it should open its own file.
For example, you could do the following:
for ( i = 1; i <= 2; i++ )
{
if ((pid = fork()) != 0)
{
waitpid(pid);
}
else
{
f[0] = fopen("input", "r"); // New
snprintf ( buffer, 10, "output%d", i );
printf("opening file %s \n",buffer);
f[i] = fopen( buffer, "w");
fgets(buff2, 10, f[0]);
fprintf(f[i], "%s", buff2);
fclose(f[i]);
fclose(f[0]); //New
_exit(0);
}
}
Also, you should check for errors (almost all the functions in your else could fail with error).
I would like to read in a file descriptor line by line until there is nothing else to read. I expected read(2) to return zero when the end of the file descriptor is reach, but on the contrary, it seems to wait for the file descriptor to be closed. I would like to avoid closing my file descriptor as I have more to write in it.
Here is a comprehensive example I have done :
#include <stdio.h>
#include <stdlib.h>
size_t read_line(int fd, char* buffer);
int main()
{
int fd[2];
if (pipe(fd) != 0) {
fprintf(stderr,"Pipe error.\n");
exit(1);
}
if (fork() == 0) {
close(fd[0]);
int i;
for (i = 0; i < 10; ++i)
dprintf(fd[1], "FOO\n");
pause();
exit(0);
}
close(fd[1]);
char buffer[20];
while (read_line(fd[0], buffer) > 0)
printf("%s\n", buffer);
printf("BAR !!\n");
exit(0);
}
size_t read_line(int fd, char* buffer)
{
char char_buf;
size_t buffer_size = 0;
while (read(fd, &char_buf, 1) > 0 && char_buf != '\n')
buffer[buffer_size++] = char_buf;
buffer[buffer_size] = '\0';
return buffer_size;
}
My read_line function should be returning zero at the end. It works properly if I replace "pause();" with "close(fd[1]);", but obviously, it is not my goal as I would avoid to loose the pipe.
Thanks !
There is no way to do this by checking the return value of read. The reason being that while the file descriptor is empty, read will just wait. Your problem is similar to that of waiting until you have no input from a scanf, if that makes it easier for you to understand.
A solution is to have a specific value which will stop the loop. For example if you were reading non-negative integers, this value could be -1.
Another solution comes from the fact that if a pipe is empty and no process has the pipe open for write, a read returns 0. So when you are done writing, close the file destriptor p[1].(p[1] must also be closed from the sender process).
I'm not sure if I am even barking up the right tree here... but here goes.
I'm trying to pass data from my parent process to all children. It's a simple server program that basically will keep a list of connected clients and then send the routing table of connected clients to every client. This is eventually going to include a struct of information about each client... but for right now I just want to get every forked process to get the same information from the parent.
In the parent process, first I set up my pipes and set them to nonblocking (for when there isn't any new data available in the pipe). After a connection is made with a client the number of entries variable is increased to reflect this new connection. I then fork a child process to a new function and update my array of pipes with the new number of table entries (I have 10 pipes at the moment to see if I needed to keep a separate pipe for each child).
pid_t pid;
int numchildren;
int i, n;
/* Create the pipes. */
for(i = 0; i < 10; i++)
{
if (pipe (mypipe[i]))
{
fprintf (stderr, "Pipe failed.\n");
return EXIT_FAILURE;
}
}
for(i = 0; i < 10; i++)
{
for(n=0; n<2; n++)
{
// Get previous flags
int f = fcntl(mypipe[i][n], F_GETFL, 0);
// Set bit for non-blocking flag
f |= O_NONBLOCK;
// Change flags on fd
fcntl(mypipe[i][n], F_SETFL, f);
}
//close(mypipe[i][0]);
}
pid = fork();
if (pid == (pid_t) 0)
{
close (mypipe[numentries-1][1]);
recievecmds(new_fd, mypipe[numentries-1][0]);
close(new_fd);
return EXIT_SUCCESS;
}
else if (pid < (pid_t) 0)
{
fprintf (stderr, "Fork failed.\n");
return EXIT_FAILURE;
}
else
{
sprintf (buf,"%d",numentries);
for(i = 0; i < 10; i++)
write(mypipe[i][1], buf, strlen(buf));
memset(&buf, 0, sizeof buf);
}
And then I try to read whats in the pipe in the recievecmds() function:
nbytes = read(mypipe[childindex][0], buf, sizeof(buf));
The first client connected tells me numentries = 1, the second client tells me numentries = 2 and so on. I mean I really don't even see the point for a pipe since it seems that whatever I put in the pipe I could just pass it in the function I called on the fork. Am I going about this the wrong way? It's been very frustrating trying to figure this out. How can I keep all of my child processes updated concurrently from my parent process?
Thank you so much in advance.
edit - My main problem was that I was redeclaring the pipe everytime in an infinite while loop. Very dumb mistake, immediately realized that was probably the root of my problem. However, while now the first child/pipe combo contains the correct data... the second does not. I'll see if I can figure this out on my own, thanks for the advice!
Of course now I'm running into problems because I manually select an option to get the data off the pipe. I'm going to have to think up a way to maybe either get the data for all pipes every time it's updated or make sure to get just the newest data (probably just one char at a time).
Thanks for putting up with me guys! And I apologize about not posting the whole program... but there's quite a bit. I definitely should have mentioned that I have it in an infinite loop.
Various observations
Don't make the pipes non-blocking; you want the children to block when there's no data. At least, in the early phases of the design; later, you may want to let them get on with work when there's no data waiting.
You need to be careful with your plumbing. The parent needs 10 pipes, one for each child. But it only needs the write end of the pipe, not the read end.
The children each need one pipe, for reading. Any superfluous pipes (for example, the write ends of the pipes that the parent had already opened before forking the Nth child) need to be closed.
You could consider using threads - in which case you could perhaps pass the data to the children. But in the long term, it appears that you will be periodically passing data to the children, and then you need a mechanism to get the data to them (other than the function call).
Pipes are 'easy' as long as you pay meticulous attention to which file descriptors are in use. Close all the descriptors you do not need.
The parent will have to loop around all ten pipes writing the same data to each.
It will also need to consider what to do if a child exits. It should close the pipe (no use any more), and decide whether to start a new child (but how will it ensure that the new child has all the accumulated information it needs?).
Watch out for SIGPIPE - maybe install a handler, or maybe use SIG_IGN and detect write errors instead of signals.
Working code
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
enum { NUM_CHILDREN = 10 };
enum { NUM_MESSAGES = 10 };
static int write_pipes[NUM_CHILDREN];
static int n_pipes;
static void be_childish(int *pipe)
{
int i;
char buffer[32];
int nbytes;
int pid = getpid();
close(pipe[1]);
for (i = 0; i < n_pipes; i++)
close(write_pipes[i]);
printf("Child %d\n", pid);
while ((nbytes = read(pipe[0], buffer, sizeof(buffer))) > 0)
{
printf("Child %d: %d %.*s\n", pid, nbytes, nbytes, buffer);
fflush(0);
}
printf("Child %d: finished\n", pid);
exit(0);
}
int main(void)
{
pid_t pid;
int i, j;
/* Create the pipes and the children. */
for (i = 0; i < NUM_CHILDREN; i++)
{
int new_pipe[2];
if (pipe(new_pipe))
{
int errnum = errno;
fprintf(stderr, "Pipe failed (%d: %s)\n", errnum, strerror(errnum));
return EXIT_FAILURE;
}
if ((pid = fork()) < 0)
{
int errnum = errno;
fprintf(stderr, "Fork failed (%d: %s)\n", errnum, strerror(errnum));
return EXIT_FAILURE;
}
else if (pid == 0)
{
be_childish(new_pipe);
}
else
{
close(new_pipe[0]);
write_pipes[n_pipes++] = new_pipe[1];
}
}
for (i = 0; i < NUM_MESSAGES; i++)
{
char message[30];
int len;
snprintf(message, sizeof(message), "Message %d", i);
len = strlen(message);
for (j = 0; j < n_pipes; j++)
{
if (write(write_pipes[j], message, len) != len)
{
/* Inferior error handling; first failure causes termination */
fprintf(stderr, "Write failed (child %d)\n", j);
exit(1);
}
}
sleep(1);
}
printf("Parent complete\n");
return 0;
}
I'd suggest using a shared memory segment. Your parent and child processes can mmap the same file, and read/write state to it. This can be an actual file, or an anonymous memory segment. This is exactly what Apache does with their ScoreBoardFile.