I'm using ImageIO's get_reader() function on a video in a context where the process doing this is going to be forked. This reader is then continually randomly indexed (using the get_data method). Eventually, I run into an error imageio.core.format.CannotReadFrameError: Could not read frame -1. If the process is not forked, this error doesn't occur. Is ImageIO's get_reader() not fork safe? Or does my problem lie elsewhere?
Related
I have an issue with write to the same console with two processes(1 parent create child process) that interacts between them with a pipe.
every process has multiple threads and has to write to the console.
sometimes I find that one process start to write and the other process stops it in the middle and writes in its place and so it turns out that the text is blurred.
of course, every process has his own semaphores for his own threads but the problem is between the two processes.
There is a way to fix this issue?
I am currently working on a multi-client server that uses select() to handle multiple clients. However, when a client sends a message that needs heavy calculations, I have to create a a new thread using pthread_create() so that my server can remain responsive to other messages from clients. Once the calculation is done for that client, I need to be able to return a message to the client. But I am not sure how I can know if that thread is finished and how to get it's final result. Obviously I cant use pthread_join() as that blocks my server program while running that new thread. So does C offer a function that I can use to get the end result of that child thread? I would like to avoid using Global Variables as well.
You can just check if the thread has finished before joining it from the main thread (which will be non blocking)
You should get how to do it from here : How do you query a pthread to see if it is still running?
Otherwise you can probably just send back the answer from the child thread, you can pass connection information as parameter of the thread function.
If you want the child thread to wake up the thread that is waiting in select() when it has finished processing, you can use pipe() to create a pipe. The thread calling select() adds the read side of the pipe to its file descriptor set, and the child thread writes to the write side of the pipe when it has finished its work.
You can even have it send the result over the pipe, if the result isn't too large.
Question
Is it possible to disable the raising of a signal (SIGPIPE) when writing to a pipe() FD, without installing my own signal handler or disabling/masking the signal globally?
Background
I'm working on a small library that occasionally creates a pipe, and fork()s a temporary child/dummy process that waits for a message from the parent. When the child process receives the message from the parent, it dies (intentionally).
Problem
The child process, for circumstances beyond my control, runs code from another (third party) library that is prone to crashing, so I can't always be certain that the child process is alive before I write() to the pipe.
This results in me sometimes attempting to write() to the pipe with the child process' end already dead/closed, and it raises a SIGPIPE in the parent process. I'm in a library other customers will be using, so my library must be as self-contained and transparent to the calling application as possible. Installing a custom signal handler could break the customer's code.
Work so far
I've got around this issue with sockets by using setsockopt(..., MSG_NOSIGNAL), but I can't find anything functionally equivalent for pipes. I've looked at temporarily installing a signal handler to catch the SIGPIPE, but I don't see any way to limit its scope to the calling function in my library rather than the entire process (and it's not atomic).
I've also found a similar question here on SO that is asking the same thing, but unfortunately, using poll()/select() won't be atomic, and there's the remote (but possible) chance that the child process dies between my select() and write() calls.
Question (redux)
Is there any way to accomplish what I'm attempting here, or to atomically check-and-write to a pipe without triggering the behavior that will generate the SIGPIPE? Additionally, is it possible to achieve this and know if the child process crashed? Knowing if it crashed lets me build a case for the vendor that supplied the "crashy" library, and lets them know how often it's failing.
Is it possible to disable the raising of a signal (SIGPIPE) when writing to a pipe() FD [...]?
The parent process can keep its copy of the read end of the pipe open. Then there will always be a reader, even if it doesn't actually read, so the condition for a SIGPIPE will never be satisfied.
The problem with that is it's a deadlock risk. If the child dies and the parent afterward performs a blocking write that cannot be accommodated in the pipe's buffer, then you're toast. Nothing will ever read from the pipe to free up any space, and therefore the write can never complete. Avoiding this problem is one of the purposes of SIGPIPE in the first place.
You can also test whether the child is still alive before you try to write, via a waitpid() with option WNOHANG. But that introduces a race condition, because the child could die between waitpid() and the write.
However, if your writes are consistently small, and if you get sufficient feedback from the child to be confident that the pipe buffer isn't backing up, then you could combine those two to form a reasonably workable system.
After going through all the possible ways to tackle this issue, I discovered there were only two venues to tackle this problem:
Use socketpair(PF_LOCAL, SOCK_STREAM, 0, fd), in place of pipes.
Create a "sacrificial" sub-process via fork() which is allowed to crash if SIGPIPE is raised.
I went the socketpair route. I didn't want to, since it involved re-writing a fair bit of pipe logic, but it's wasn't too painful.
Thanks!
Not sure I follow: you are the parent process, i.e. you write to the pipe. You do so to send a message after a certain period. The child process interprets the message in some way, does what it has to do and exits. You also have to have it waiting, you can't get the message ready first and then spawn a child to handle it. Also just sending a signal would not do the trick as the child has to really act on the content of the message, and not just the "do it" call.
First hack which comes to mind would be that you wont close the read side of the pipe in the parent. That allows you to freely write to the pipe, while not hurting child's ability to read from it.
If this is not fine, please elaborate on the issue.
I've got a program where main process forks into 4 children that cooperate with each other:
process0 is opening FIFO file (O_WRONLY), reading 1 byte at a time from STDIN using read function, writing into FIFO using write and closing FIFO file
process1 is waiting for the shared memory to be empty (I'm using first byte of shared memory table if(tab[0] == 0) to check if its empty) opening FIFO file (O_RDONLY), reading from it, translating this one byte into hex and saving it into shared memory. It then closes fifo and sets tab[0] which is shared memory table to 1.
process2 is reading from shared memory if tab[0] == 1. After reading it writes data into pipe
process3 is reading from pipe and writing into STDIN
this all works perfect. The problem started when I wanted to add signals. I'm using semaphores to synchronize p0 and p1, signals to synchronize p1 and p2 and message queue to synchronize p2 and p3. It also works fine except for the times when for example process1 is in the sleeping mode. It goes into this mode when it wants to read from fifo and has to wait for data to be transferred.. I guess. I've been reading about it.
processes hierarchy
Here's what I've found that I think may be the cause:
"When the process makes a system call while in user mode (1), it moves into state 2 where it begins to run in kernel mode. Assume at this point that the system call made was to read a file on the hard disk. Because the read is not carried out immediately, the process goes to sleep, waiting on the event that the system has read the disk and the data is ready. It is now in state 4. When the data is ready, the process is awakened. This does not mean it runs immediately, but rather it is once again ready to run in main memory (3). "
I think I understand it but how can I avoid this? I want my program to react to signals always. Is there some way to change processes state from sleeping to running when I send the signal through "kill" ? Can I somehow tell the process to stop "waiting on the event that the system has read the disk and the data is ready" ?
here's my code if someone wants to look at it:
Program's code
If I understand your question correctly, process 1 is getting hung up at line 442 (source) and not responding to signals because it's in the read.
The obvious answer is just don't do a blocking read(). Check the descriptor to see if anything is there to read and move on if not. Read up on fcntl/ioctl and how to do non-blocking reads.
Your observations is likely that (most) system call defaults to restarting if a signal occurs. Shortly said, that means code stuck in a system call wakes up to user-space when a signal is delivered, but resumes the system call after the signal handler has been run.
If you use sigaction() to establish a signal handler instead of the signal() function., system calls will not restart when a signal is caught , but rather fail and set errno to EINTR.
That means you you have to handle the fact that system calls can "fail" anywhere due to a signal being delivered.
(signal() by default will cause system calls on linux restart, it can be controlled by some feature macros before including the signal.h header. Whether sigaction() causes the system call to restart is controlled by the flag SA_RESTART when establishing a signal handler)
I am creating a GTK application in C. I am creating a child process using fork and then replacing it with execve("crawler",arg,env); which crawls my home directory and stores all the filenames in a file.
Now, this child process takes some time (about 2-5 minutes).
In the mean time, when this child process is running, the main GTK parent program is waiting.
But when the child process is running, after some time, the GTK application hangs.
I have tried gdk_thread_enter()/leave() in my main function.
But still I am the application is hanging.
Please, point out a mistake if any or else suggest any modification.
execve does not create a child process, it replaces the current process with the child. Are you sure you used fork() first, and then execve() from within the child?
EDIT since you're already using fork/execve, perhaps the child process is still interacting with Gtk somehow. Best to use Glib/Gtk+-specific functions for invoking the crawler -- try, for instance, g_spawn_command_line_async
If what you mean by "when this child process is running, the main GTK parent program is waiting" is that your code executes a wait(), waitid(), waitpid() in its main thread, then the app will indeed suspend execution until a child terminates (unless you've selected NOHANG option).
If your Gtk app doesn't need to coordinate further with your crawler program, just use the previously-mentioned
g_spawn_command_line_async routine, and do not set G_SPAWN_DO_NOT_REAP_CHILD. If you do need to coordinate, you could set that flag, and create a GChildWatch source, or perhaps could use one of the g_spawn pipe routines.
As I interpret gdk_thread_enter()/leave(), they are locking or unlocking threading, rather than running or stopping new threads. Gtk callbacks run in the main thread, so as indicated above, a blocking waitpid() in a callback will hang the Gtk app. A non-blocking waitpid() in a timer callback (eg) is not a problem, however.