I ran across some problems with GtkSubprocess, and I figured out that it is related to using threads, and is there a way to make it immune to concurrency problems?
I have this program that does some operations on a file, which are individually represented by GtkListBoxRows. When the GSubprocess finishes, and I attempt to remove the list box row, the program segfaults. BTW, each file has its own process, so if a user loads 10 files, there will be 10 threads (this is managed by GThreadPool). Interestingly, if I comment out the code that launches the process, and the code that blocks the thread function till the process finishes, the program does not segfault. So I deduced that GSubprocess is having problems with concurrency. The error produced varies a lot, so this must be due to time-related problems.
I wanted to use GSubprocess because it is relatively easy to get the output of the command, which I need. Will I need to move my invocations of GSubprocess outside of the thread function?
I found out that it is not safe, due to its internal implementation in the GTK+ source code. And you should not even use threads in an application as well, as stated here. Here is my workaround: create the process in the main loop, and wait for the process to terminate using the async version of the call. Thus you avoid threads.
Related
My excuses for the newbie question:
I used gdbus-codegen to generate a C-code D-Bus client proxy, running on a Yocto Linux embedded system. The generated code calls g_initable_new(), returning a GInitable *.
At this point, two new threads appear: from the stack trace, I see that one calls g_main_loop_run(), and the other g_main_context_iteration(). So I gather that under the hood, g_initable_new() started a new main loop. OK so far.
When I'm done, I call g_object_unref() on my GInitable *. That works too, but the two threads are still running. How do I quit that loop?
Thanks in advance!
As the other #Philip confirmed, GDBus creates worker threads that can apparently not be stopped. Boo hiss.
My initial concern was that I didn't want my child process to inherit some useless - and potentially harmful - threads. But actually, fork() takes care of that, as noted in the manpage: "The child process is created with a single thread — the one that called fork()." So this concern is unfounded.
But I discovered another good reason to spawn the D-Bus code: I have a setuid program, and as soon as I call setreuid() to reduce the privileges of my process, the gdbus worker threads fail (which makes sense).
So that appears to be my answer: you can't quit the GDBus worker threads, and if that's not OK, isolate the GDBus stuff in a process of its own.
I am new to C programming. I used to think using exit() was the cleanest way of process termination (as it is capable of removing temporary files, closing open files, normal process termination...), but when I tried man exit command on the terminal (Ubuntu 16.04.5, gcc 5.4.0) I saw the following line:
The exit() function uses a global variable that is not protected, so
it is not thread-safe.
After that I tried to make some research about better replacement for exit() (to change my programming behavior from the beginning). While doing that I faced with this question in which side effects of exit() is mentioned and it is suggested to use atexit() properly to solve the problem (at least partially).
There were some cases in which using abort() was preferred over exit(). On top of that, this question suggests that atexit() might also be harmful.
So here are my questions:
Is there any general and better way of process terminating (which is guaranteed to clean like exit() and is not harmful for the system at any case)?
If the answer to the first question is NO!, what is the best possible way of process terminating (including the cases in which they are most useful)?
what is the best possible way of process terminating
If going single threaded just use exit(), as your code is not going multi-threaded.
Else make sure all but one thread have ended before the last thread and then safely call exit() because of 1. above.
Given that power/hardware fails can happen at any time, the imposs.. extreme difficulty of reliably terminating threads with user code and the chaotic nature of the use of memory pools etc. in many non-trivial multithreaded apps, it is better to design apps and systems that can clean temp files etc. on start-up, rather than trying to micro-manage shutdown.
'Clean up all the resources you allocate before you exit' sounds like good advice in a classroom or lecture, but quickly becomes a whole chain of albatross round your neck when faced with a dozen threads, queues and pools in a continually changing dynamic system.
If you can, if you are running under a non trivial OS, let it do its job and clean up for you. It's much better at it than your user code will ever be.
I have a multi-threaded application in a POSIX/Linux environment - I have no control over the code that creates the pthreads. At some point the process - owner of the pthreads - receives a signal.
The handler of that signal should abort,cancel or stop all the pthreads and log how many pthreads where running.
My problem is that I could not find how to list all the pthreads running in process.
There doesn't seem to be any portable way to enumerate the threads in a process.
Linux has pthread_kill_other_threads_np, which looks like a leftover from the original purely-userland pthreads implementation that may or may not work as documented today. It doesn't tell you how many threads there were.
You can get a lot of information about your process by looking in /proc/self (or, for other processes, /proc/123). Although many unices have a file or directory with that name, the layout is completely different, so any code using /proc will be Linux-specific. The documentation of /proc is in Documentation/filesystems/proc.txt in the kernel source. In particular, /proc/self/task has a subdirectory for each thread. The name of the subdirectory is the LWP id; unfortunately, [1][2][3] there doesn't seem to be a way to associate LWP ids with pthread ids (but you can get your own thread id with gettid(2) if you work for it). Of course, reading /proc/self/task is not atomic; the number of threads is available atomically through /proc/self/status (but of course it might change before you act on it).
If you can't achieve what you want with the limited support you get from Linux pthreads, another tactic is to play dynamic linking tricks to provide your own version of pthread_create that logs to a data structure you can inspect afterwards.
You could wrap ps -eLF (or another command that more closely reads just the process you're interested in) and read the NLWP column to find out how many threads are running.
Given that the threads are in your process, they should be under your control. You can record all of them in a data structure and keep track.
However, doing this won't be race-condition free unless it's appropriately managed (or you only ever create and join threads from one thread).
Any threads created by libraries you use are their business and you should not be messing with them directory, or the library may break.
If you are planning to exit the process of course, you can just leave the threads running anyway, as calling exit() terminates them all.
Remember that a robust application should be crash-safe anyway, so you should not depend upon shutdown behaviour to avoid data loss etc.
I'm not sure if the title accurately describes what I want to do but here's the rub:
We have a large and hairy codebase (not-invented-here courtesy of Elbonian Code Slaves) which currently compiles as one big binary which internally creates several pthreads for various specific tasks, communicating through IPC messages.
It's not ideal for a number of reasons, and several of the threads would be better as independent autonomous processes as they are all individual specific "workers" rather than multiple instances of the same piece of code.
I feel a bit like I'm missing some trick, is our only option to split off the various thread code and compile each as a standalone executable invoked using system() or exec() from the main blob of code? It feels clunky somehow.
If you want to take a part of your program that currently runs as a thread, and instead run it as a separate process launched by your main program, then you have two main options:
Instead of calling pthread_create(), fork() and in the child process call the thread-start function directly (do not use any of the exec-family functions).
Compile the code that the the thread executes as a separate executable. Launch that executable at need by the standard fork / exec sequence. (Or you could use system() instead of fork/exec, but don't. Doing so needlessly brings the shell into it, and also gives you much less control.)
The former has the disadvantage that each process image contains a lot of code that it will never use, since each is a complete copy of everything. Inasmuch as under Linux fork() uses copy-on-write, however, that's mostly an address-space issue, not a resource-wastage issue.
The latter has the disadvantage that the main program needs to be able to find the child programs on the file system. That's not necessarily a hard problem, mind you, but it is substantially different from already having the needed code at hand. If there is any way that any of the child programs would be independently useful, however, then breaking them out as separate programs makes a fair amount of sense.
Do note, by the way, that I do not in general accept your premise that it is inappropriate to implement specific for-purpose workers as threads. If you want to break out such tasks, however, then the above are your available alternatives.
Edited to add:
As #EOF pointed out, if you intend that after the revamp your main process will still be multi-threaded (that is, if you intend to convert only some threads to child processes) then you need to be aware of a significant restriction placed by POSIX:
If a multi-threaded process calls fork(), [...] to avoid errors, the child process may only execute async-signal-safe operations until such time as one of the exec functions is called.
On the other hand, I'm pretty sure the relevant definition of "multi-threaded" is that the process has multiple live threads at the time fork() is called. It should not present a problem if the child processes are all forked off before any additional threads are created, or after all but one thread is joined.
This question already has answers here:
Closed 13 years ago.
Possible Duplicate:
Linux API to list running processes?
How can I detect hung processes in Linux using C?
Under linux the way to do this is by examining the contents of /proc/[PID]/* a good one-stop location would be /proc/*/status. Its first two lines are:
Name: [program name]
State: R (running)
Of course, detecting hung processes is an entirely separate issue.
/proc//stat is a more machine-readable format of the same info as /proc//status, and is, in fact, what the ps(1) command reads to produce its output.
Monitoring and/or killing a process is just a matter of system calls. I'd think the toughest part of your question would really be reliably determining that a process is "hung", rather than meerly very busy (or waiting for a temporary condition).
In the general case, I'd think this would be rather difficult. Even Windows asks for a decision from the user when it thinks a program might be "hung" (on my system it is often wrong about that, too).
However, if you have a specific program that likes to hang in a specific way, I'd think you ought to be able to reliably detect that.
Seeing as the question has changed:
http://procps.sourceforge.net/
Is the source of ps and other process tools. They do indeed use proc (indicating it is probably the conventional and best way to read process information). Their source is quite readable. The file
/procps-3.2.8/proc/readproc.c
You can also link your program to libproc, which sould be available in your repo (or already installed I would say) but you will need the "-dev" variation for the headers and what-not. Using this API you can read process information and status.
You can use the psState() function through libproc to check for things like
#define PS_RUN 1 /* process is running */
#define PS_STOP 2 /* process is stopped */
#define PS_LOST 3 /* process is lost to control (EAGAIN) */
#define PS_UNDEAD 4 /* process is terminated (zombie) */
#define PS_DEAD 5 /* process is terminated (core file) */
#define PS_IDLE 6 /* process has not been run */
In response to comment
IIRC, unless your program is on the CPU and you can prod it from within the kernel with signals ... you can't really tell how responsive it is. Even then, after the trap a signal handler is called which may run fine in the state.
Best bet is to schedule another process on another core that can poke the process in some way while it is running (or in a loop, or non-responsive). But I could be wrong here, and it would be tricky.
Good Luck
You may be able to use whatever mechanism strace() uses to determine what system calls the process is making. Then, you could determine what system calls you end up in for things like pthread_mutex deadlocks, or whatever... You could then use a heuristic approach and just decide that if a process is hung on a lock system call for more than 30 seconds, it's deadlocked.
You can run 'strace -p ' on a process pid to determine what (if any) system calls it is making. If a process is not making any system calls but is using CPU time then it is either hung, or is running in a tight calculation loop inside userspace. You'd really need to know the expected behaviour of the individual program to know for sure. If it is not making system calls but is not using CPU, it could also just be idle or deadlocked.
The only bulletproof way to do this, is to modify the program being monitored to either send a 'ping' every so often to a 'watchdog' process, or to respond to a ping request when requested, eg, a socket connection where you can ask it "Are you Alive?" and get back "Yes". The program can be coded in such a way that it is unlikely to do the ping if it has gone off into the weeds somewhere and is not executing properly. I'm pretty sure this is how Windows knows a process is hung, because every Windows program has some sort of event queue where it processes a known set of APIs from the operating system.
Not necessarily a programmatic way, but one way to tell if a program is 'hung' is to break into it with gdb and pull a backtrace and see if it is stuck somewhere.