So I've been working on code for my OS class, and in my project, I have to create a new child process for each file on the command line, and pipe information from the child to the parent. We're supposed to be re-creating Unix's wc utility (I've handled this part already).
What I've tried thus far is:
for(i=0; i<argcount; i++){
int pid;
pid = fork();
if(pid == 0){
/* Child Process */
/* Close read pipes */
close(l_pipe[0]);
close(w_pipe[0]);
close(c_pipe[0]);
wc(lflag, wflag, cflag, filenames[i]);
} else {
/* Parent Process for piping */
/* Close write pipes */
close(l_pipe[1]);
close(w_pipe[1]);
close(c_pipe[1]);
/* Read from pipes */
read(l_pipe[0], &buffer, sizeof(count_t));
lines+=buffer;
read(w_pipe[0], &buffer, sizeof(count_t));
words+=buffer;
read(c_pipe[0], &buffer, sizeof(count_t));
bytes+=buffer;
}
}
However, this creates as many parents as there are children, which is obviously wrong. I'm not exactly sure where I should be forking. I have to use a pipeline between the child and parent processes, and I'm certain that the parent needs to read() as many times as the child write()s.
Thank you for any suggestions you can provide.
As well as the issues raised by Greg Hewgill in his answer, I observe the following issues:
You could have the loop fork and run the child processes, while the parent part simply goes back to the next iteration of the loop.
You would then have a loop to read from the pipes.
Before that loop, the parent process would close the write ends of all three pipes (otherwise it will never see EOF on the pipes).
The loop to read from the pipes should read from each pipe in turn, rather than draining each pipe in turn.
I'm assuming that you have count_t buffer; — it helps to show variable declarations. If you have some sort of char buffer[sizeof(count_t)] instead, then you have all sorts of problems, large and small.
When the second child is created in your current scheme, the write ends of the pipes are all closed, so the second and subsequent children will be unable to send anything to the parent. You must move the three calls to close the write ends of the pipes so it is outside the loop.
Your pipes must be in global variables so that the wc function can use them. This isn't the end of the world, but it is often neater to avoid global variables. This is a second-order problem; you have other more major problems to fix first.
If you need to associate sizes with individual files, you have more bookkeeping to do. At the moment, you only aggregate grand totals. In that case, your current synchronous design is likely appropriate. Failing that, you'd have the children write a PID or another ID number plus the count on the pipe in one operation (to ensure the operation is atomic). Individual write operations are not interleaved unless they're too big for the pipe's internal buffer, which won't be a problem for a couple of integers.
There's a couple of things I notice straight away.
In your child process branch, you're not calling _exit() after wc(). This means that your child process will loop around and start forking itself for further children.
In your parent process branch, you are waiting for the response from the child you just spawned before proceeding to create the next child. So essentially you've serialised the process and won't take advantage of multiple processes.
Related
I have a C program that forks a child process at some point in a loop. The child process waits for the parent process to finish its job (some numerical calculations). If things go wrong, the parent process aborts and the child process should continue from the state when it was forked and retry the calculation with some modifications. Otherwise, the parents keeps running, and the child process should be killed.
The communication between the parent and child process is through a memory mapped file, which only has 1 byte as a character that indicates the status of the parent process.
The memory map is done like this
char child_flag[]="W";
fp1 = fopen( "child_interface.dat","wb");
// the interface file has two bytes, but only one is meaningful to the program
fwrite(child_flag, 1, sizeof(child_flag), fp1);
fclose(fp1);
printf("child_interface.dat created\n");
if(mmap_child_flag() ==0) {
printf("memory map of parent-child interface successful.\n");
fflush(stdout);
}
The wait loop in the child process is like this
child_pid = fork();
if (child_pid ==0) { /* child process, wait for parent process to finish*/
mmap_child_flag();
while(child_file[0]=='W' ){ //Child waits
usleep(100000);
}
if(child_file[0]=='R'){ // run child process (as a new parent process)
child_file[0]='W';
goto label2;
}
if(child_file[0]=='K'){ //Kill child process
exit(0);
}
}
The problem is that the child process seems to get stuck in the sleep while loop, even when the parent process has set the status to 'K' (checked in the file that is memory mapped). This code has been run on several linux based super computers, and the behavior seems very inconsistent. On some platforms, it can run smoothly, but on some others, it constantly get stuck in the while loop. Sometimes, if I add some statements inside the while loop after the usleep call, it can then run just fine.
However, I'm not sure if the sleep while loop is the root cause of this problem. My guess is that because the process has almost nothing to do except to check a byte in the memory, the system let it sleep all the time and somehow "forget" to let it check the memory. Can such thing happen in the Linux system?
This the function that does the actual mapping
/* Memory map for parent-child processes interface */
int mmap_child_flag()
{
int fd_child;
struct stat st_child;
// open files
if ((fd_child = open("child_interface.dat", O_RDWR)) == -1){
perror("open child_interface.dat");
exit(1);
}
// stat
if (stat("child_interface.dat", &st_child) == -1){
perror("stat of child_interface.dat");
exit(1);
}
// map, child_file is global char array
child_file = mmap(0, st_child.st_size, PROT_WRITE, MAP_SHARED, fd_child, 0);
if (child_file == (char *)(-1)) {
perror("mmap child_interface.dat");
exit(1);
}
return 0;
}
The problem is that the child process seems to get stuck in the sleep while loop, even when the parent process has set the status to 'K' (checked in the file that is memory mapped).
There are several odd things about your program, with one of them being that you are using shared memory for this task at all. See below for a better approach.
Issues with the current approach
As to the question as it stands, however, you have a synchronization problem. The contents of the mapped memory are being changed outside the scope of the child process, but you've given it no reason to suspect that that might be the case. The compiler can therefore assume that if the wait loop condition is satisfied when it is first evaluated, then it will be satisfied on every subsequent evaluation, too.
For a more complicated interaction, you might need to set up a process-shared mutex or similar to guard access to the shared memory, but for this, it would probably be sufficient to declare child_file as a pointer to volatile char.
A better approach
You want the child to wait for a one- or maybe two-byte instruction from the parent. You presently do this by polling the contents of a shared memory segment, but that's complex to set up and use, as you discovered. It would be a lot easier to use a pipe to convey the needed information from parent to child:
setup: Declare an array. Call pipe().
child use: The child performs a blocking read() on the pipe.
parent use: write() the message to the pipe when ready, then close it. Or just close it.
Note that the pipe itself then provides adequate synchronization, and that there is no need for a wait loop. Note also that the child can detect the case that the parent dies without sending any message, which your shared memory approach does not support.
A shared memory region is good for sharing a lot of data, but it is a bad way to communicate between processes. The reason is that you can't get a notification that something has been changed, nor do you get a notification if the other user of the shared memory died.
To communicate between two processes, use pipe() if you need to create a one-way communication channel, or if you need bidirectional communication, use socketpair(). You can use poll() to wait for the other side to send some data. You will also get notified if the process on the other side terminated.
You were using a loop like this:
while(child_file[0]=='W' ){ //Child waits
usleep(100000);
}
This is bad, since you are wasting on average 50 ms of time that you could have spent doing something useful. Apart from that, there is also the problem that both the compiler and the CPU can sometimes change the order in which things are written to memory. If you have more data in child_file than just the flag at the start, then this might be an issue, unless you use atomics or explicit barriers.
I am attempting to write a program which forks and waits for his child to finish, then the child does some work on an input and then forks the same way it's parent does and so on.
Now, I know that forking copies to the child the array of file descriptors and that I should close the ones associated with the parent, but I can't figure out which are the parents. Do I need to give to my child it's parents pid?
I've been trying to wrap my head around it for the better part of an hour and I think I have some kind of a mind block because I can't come to a conclusion.
TL;DR: As a child process how do I know which file descriptors belong to my parent?
Just after the fork (and before any exec function) your child process has the same state as its parent process (except for the result of the fork, which is 0 only in the child). So you know what are the file descriptors, since you have coded the program running in parent&child. On Linux you might also read the /proc/self/fd/ directory, see proc(5).
You might close most file descriptors after the fork and before the exec; you could code something like
for (int fd=3; fd<64; fd++) (void) close(fd);
we are starting from 3 which is after STDERR_FILENO which is 2, and we are stopping arbitrarily at 64, and the cast to (void) on the close call means to the reader that we don't care about failing close.... Of course, if you have e.g. some pipe(7)-s to communicate between parent and child you'll be careful to avoid closing their relevant file descriptor(s).
(However, doing a closing loop like above is poor taste and old fashion)
In general, you'll be careful in your program to set the close-on-exec flag on most file descriptors (e.g. fcntl(2) on F_SETFD operation and FD_CLOEXEC flag, or directly open(2) with O_CLOEXEC), then the execve(2) (done in most child processes after the fork) would close them.
Can someone help me with this small problem please? I am trying to create a fork bomb and record how many processes a computer creates before something weird happens, but I can't figure out how to write the processes into a text file. I looked around the internet and couldn't find a solution for my problem and I tried doing different things but that didn't work with me also. If you guys can help me out on this matter that would be awesome! Here's my code so far (far from perfect, I know.. D: ).
while(1)
{
fork(); // Fork bomb
write(fd, '\n', sizeof(buf));
}
Thanks in advance
Open a file for write+append.
Each forked process will inherit the file descriptor.
In each forked child process, write a single null byte to the file descriptor.
When everything crashes, the size of the file, in bytes, will tell you how many processes were started.
Another idea that would write the number of processes, instead of writing that many bytes
This is a bit complex. I'm writing this answer just for the sake of completeness(and fun!).
Use a process as the "master process". (The easiest way is using the starting process.) Each time a new process is created, a signal(you can use SIGUSR1) would be sent to the master process, so the master process can increment its process counter. (note that incrementing the counter in every process wouldn't work because their memory are not shared.) Once fork() has failed, another signal is sent to the master. Once all children have failed(a child would signal the failure only once and the master would have a failure counter in addition to the process counter), the master will write the process counter to the file, and kill all processes in its process group(kill() can kill not only a single process but also a process group).
Cautions:
You may need to use nice() to avoid the children from preventing the master to execute.
In order to prevent the children from forking again before all children are killed, you may need to suspend the children before terminating them.
My C homework program multiplies two matrices. Each entry of the product matrix is calculated by a child process created with fork(). The child calculates this and sends the data to the parent using a pipe. So if the product matrix has a size 10x10, I will create 100 child processes. Works fine.
Then I noticed that for very large matrices, this was not working. After doing some inspection, I realized that it's because it won't create any more child processes after a certain number (the pid returns negative). Like, some sort of limit, which makes sense.
Indeed, I can't expect my computer to allow a program to spawn several thousands of child processes, so obviously my program can't multiply super large matrices. Alright.
Then it occurred to me: well, I don't need all the child processes immediately. I could make 100, let them do their thing, and then create the next 100, and so on until all the necessary calculations are done for my matrix product.
My program is essentially a loop that iterates rows * columns times. Each iteration, it makes a child process. So I decided that, for every 100 iterations, I would put a sleep() thing. My hope was that when the sleep() thing was done, the 100 previous child processes would die out, "freeing" all the space necessary for the next 100 batch. Alas, it did not make a difference: program behaves exactly the same (except that it is slower, of course).
So, given that the sleep() thing did not work, my suspicion is that I am not properly "killing" the child processes. This is how a child process dies:
// Close the pipe!
close(fd[0]);
close(fd[1]);
// Exit
exit(0);
And the parent, after reading the data, also closes the pipe:
read(fd[0], buffer, sizeof(buffer));
close(fd[0]);
close(fd[1]);
So my question is: since I am unable to make new child processes because I have already created too many, it seems like I am not properly disposing of the old processes. How can I dispose them correctly?
What sleep(3) does is putting the process to temporarily sleep, but still it keeps it alive.
As you stated, you need to make sure the children-process are dead before creating more. To do that, children need to exit(2) properly when they finish their job, and parent process should use wait(2) on them in order to reap them, and make space in the process table.
A possible solution is to fork(2) some children processes to do some part of the job, then wait(2) them to finish before forking more of them to do some other part of the job.
I've searched and searched to no avail, so I've finally come to ask for some help.
My assignment involves (in C running on RedHat Linux) creating two child processes, each of which write characters for an arbitrary amount of iterations to a pipe. This pipe is shared with the parent process that created them. The parent reads from the pipe while the children still have characters to write to it. Once the children have exited and the pipe is empty, the parent(main) can then terminate.
Here is a quick & dirty example of the logic of the code I have written.
main()
{
//create pipe
fork(); //childA
//writes to pipe
fork(); //childB
//writes to pipe
//parent reading
while(condition) {
//read from pipe + print chars to terminal
}
}
Now, my question is regarding the condition of the while loop.
I need to read when the children are blocked from writing due to a full pipe, but I cannot figure out what type of condition would allow me to do this. Any help would be absolutely amazing.
Is it a requirement that the pipe needs to be full when you read? Or is it simply a requirement that you keep reading from both children, even if one of the pipes is full so the child is blocked on writing?
I don't know of any standard way to tall if a pipe is full. You could try using the FIONREAD ioctl to determine how much data there is to read on the pipe, and compare that against PIPE_BUF, but that may not work properly if there is less than PIPE_BUF data in the pipe, but the child is doing a write that would put it over PIPE_BUF; that call may block without filling up the pipe. I would not rely on this technique working.
The usual way to keep reading from two file descriptors, regardless of which one is ready, is to use the select system call. This allows you to wait until one or the other file descriptor has data available. This means that the parent process won't block trying to read from one child which doesn't have any data available, while the other child is blocking because its buffer is full.
edit: After re-reading your question, it sounds like there is actually only one pipe, that both children are writing to. Is that correct? Again, the question comes up about whether you are required to wait until the children are blocking. If the parent simply reads from the pipe, it will block until one of the children has written to the pipe; you don't need to do anything special.
If your assignment requires you to wait until the pipe is actually full, I'd be interested in seeing the exact wording, because I'm not sure why you would want to do that.
edit 2: In response to your questions in the comment:
Does the code for my parent process need to follow the code for my two child processes within the program?
No, there is no requirement about the order of the code for the parent or child processes. In order to distinguish the code the parent runs from the code the children run, you check the return value of fork(). If the return value is 0, you are in the child process. If it is not, you are in the parent; if the return value is -1, then there was an error, if it's positive, it is the PID of the child process.
So, you can write code like this:
int pid = fork();
if (pid) {
// in the parent, check if pid is -1 for errors
} else {
// in the child
}
Or:
int pid = fork();
if (pid == 0) {
// in the child, do whatever you need to do and...
exit(0);
}
// in the parent; since the child calls exit() above, control will never
// reach here in the child. Or you could do an execl() in the child, which
// replaces the current program with another one, so again, control will
// never reach here within the child process.
How can I keep reading from the pipe until the two children terminate AND the pipe is empty?
Just keep reading until read returns 0. read on the read end of the pipe will not return 0 until all processes have closed the write end of the pipe, and all data has been read out of the pipe. If something still has it open, but there is no data in the pipe, read will block until data is written to the pipe.
One gotcha is to remember to close the write end of the pipe in the parent process before trying to read; otherwise, when the parent tries to do a read after the children have finished, it will block waiting for it to close its own pipe.