I have a library which registers an atfork handler (via pthread_atfork()) which does not support multiple threads when fork() is called. In my case, I don't need the forked environment to be usable because all I want is to call exec() right after the fork(). So, I want the fork() but without any atfork handlers. Is that possible? Do I miss any important edge cases?
For background info, the library is OpenBlas, the issue is described here and here.
You could use vfork() (NPTL implementation doesn't call fork handlers). Although POSIX has removed vfork from the standard, it's likely available on your implementation.
Fork handlers established using pthread_atfork(3) are not called when
a multithreaded program employing the NPTL threading library calls
vfork(). Fork handlers are called in this case in a program using
the LinuxThreads threading library. (See pthreads(7) for a
description of Linux threading libraries.)
Or, posix_spawn(). This is similar to vfork. Man page says:
According to POSIX, it unspecified whether fork handlers established with pthread_atfork(3) are called when posix_spawn() is invoked. On glibc, fork handlers are called only if the child is created using fork(2).
Or, syscall and directly use SYS_clone. SYS_clone is the system call number used to create threads and processes on Linux. So syscall(SYS_clone, SIGCHLD, 0); should work, provided you would exec immediately.
syscall(SYS_fork); (as answered by Shachar) would likely work too. But note that SYS_fork not available on some platforms (e.g., aarch64, ia64). SYS_fork is considered as obsolete in Linux and it's only there for backward compatibility and Linux kernel uses SYS_clone for creating all "types" of processes.
(Note: These options are mostly limited to glibc/Linux).
Yes. The following should work on Linux (and, I think, all glibc based platforms):
#define _GNU_SOURCE
#include <unistd.h>
#include <sys/syscall.h>
...
syscall(SYS_fork);
This bypasses the library and directly calls the system call for fork. You might run into trouble if your platform does not implement fork as a single system call. For Linux, that simply means that you should use clone instead.
With that in mind, I'm not sure I'd recomment doing that. Since you're a library, you have no idea why someone registered an atfork. Assuming it's irrelevant is bad programming practice.
So you lose portability in order to do something that may or may not break stuff, all in the name of, what? Saving a few function calls? Personally, I'd just use fork.
Related
Background, from POSIX:
A process shall be created with a single thread. If a multi-threaded process calls fork(), the new process shall contain a replica of the calling thread and its entire address space, possibly including the states of mutexes and other resources. Consequently, to avoid errors, the child process may only execute async-signal-safe operations until such time as one of the exec functions is called.
The difficulty is that we generally don't know if we're a multi-threaded process, since threads may have been created by library code. And "async-signal-safe" is a quite-severe restriction.
It is nonsensical to ask "how many threads are there", since if other threads are still running, they may be exiting or creating new threads while asking. We can, however, get answers (or partial answers) to simpler questions:
Is it even possible for other threads to exist?
Am I the only thread that ever existed?
Am I the only thread that exists right now?
...
For simplicity's sake let's assume:
we're not in a signal handler
nobody is mad enough to invoke UB by calling pthread_create or C11's thrd_create from a signal handler
nobody is doing threads outside of pthreads, C11, and C++11
C++11 threads appear to always be implemented in terms of pthreads (on platforms that support fork, at least)
C11 threads are very similar to pthreads, although we sometimes have to handle the functions separately.
Answers that involve arcane implementation details are encouraged, as long as they are (fairly) stable.
Some partial answers (more still needed):
Question 1 is addressed by libstdc++'s __gthread_active_p() for several libc implementations. The header is compatible with C, but it a static function in a C++-only part of the include path, and also relies on the existence of macro __GXX_WEAK__ which is only predefined for C++. (libc++ unconditionally pulls in pthreads)
Unfortunately, this is dangerously unreliable for the dlopen case (race conditions in correct user code), see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=78017
Question 2 can be addressed by installing interceptors for pthread_create and thrd_creat. But this can potentially be finicky (see comments in gthr.h about interceptors).
If calling clock_gettime with CLOCK_PROCESS_CPUTIME_ID and CLOCK_THREAD_CPUTIME_ID differs, this may be proof that another thread has existed, but beware of races, resolution, and clock settability (setting these clocks is not possible on Linux, but POSIX potentially allows it)
Question 3 is the interesting one, anyway:
GDB is likely to know the answer, but spawning a whole other process seems unnecessary (e.g. answers involving ps should be rewritten to use /proc/ directly)), and it may not have permission anyway
libthread_db.so exists but appears undocumented except in the original Solaris version. It looks like it might be possible to implement the proc_service.h callbacks for the current process however, if we ignore the "stop" part ...
On Linux, if gettid() != getpid(), you're not the main thread, thus there probably are at least two threads. (it's possible for the main thread to call pthread_exit, but this is weird)
A (somewhat) more portable version of the preceding: use __attribute__((constructor)) (or politely ask your caller) to stash the value of pthread_self() for the main thread. Unfortunately, there is a disturbing comment in libstdc++'s <thread> header (grep for __GLIBC__) about returning 0 (but I cannot reproduce this).
On Linux if /proc is mounted and accessible, you can enumerate /proc/self/task/. The code to do this is portable, it will just fail on OSes that don't provide this. (are there others that do provide this much?). Is /proc/self/status or /proc/self/stat any more portable? They have less information (and stat is hard to parse securely), but we probably don't need any more. (need to test these for the "main thread exited" case)
On GLIBC, we could possibly read the debug symbols to find the multiple_threads flag (sometimes global, sometimes part of struct pthread - ugh). But this is probably similar to libthread_db.so
Similarly MUSL has a count (minus one) and a linked list ... though it prefers to take an internal lock first. If we're only reading, is it safe to skip that?
If we block a signal and then kill the current process (not thread) with it, and our thread isn't the one that receives it, we know that other threads must exist to handle it. But there's no way to know how long to wait, and signals are dangerous global state anyway.
On Linux, unshare(2) ignores CLONE_THREAD for single-threaded processes and errors for multithreaded processes! (There's also some harder cases with user namespaces but I don't think they're needed)
On Linux, SELinux's setcon(3) is guaranteed to fail for multithreaded processes under certain conditions. This requires investigation; it takes some steps to correlate the kernel implementation to userland headers (there is a userland library involved).
From grepping kernel sources those are the only 2 that use specific functions, but there's nothing stopping other functions from being implemented on the same data structures.
In Windows, you can execute a process using CreateProcess().
Now I want to know how to execute a process in Linux, so far I only found that you can do that by calling fork() and then calling exec().
Is this the only way to execute a process in Linux?
Linux provides both high-level and low-level interfaces for starting new processes.
Low-level interfaces
The primary low-level interface is the one you have already discovered: fork(). This is often followed quickly by one of the several exec() functions, but an exec() is unneeded if it is satisfactory for the new process to execute the same code as the original. That's more common than it may sound to someone more used to the Windows API.
POSIX also defines posix_spawn() family, which serve as special-purpose alternatives to fork() + exec() for certain circumstances to which the latter are not well suited. posix_spawn() is implemented on Linux via the clone() library function (see next).
Although they are not in POSIX, Linux also provides clone() and vfork() as alternatives to fork(). On modern Linux these use the same system system call that fork() does, but not the fork() library function itself. There's not much reason to use vfork() any longer -- it is perhaps indicative that POSIX used to have it, but removed it nearly a decade ago. Linux-specific clone(), on the other hand, has some interesting behaviors that are occasionally useful. It provides a more general interface to the fork system call than any of the other functions discussed here do, but it is not portable. Generally speaking, prefer fork() unless you need something it cannot provide.
High(er)-level interfaces
Linux provides some higher-level interfaces as well. The primary two are system(), which executes a shell command and waits for it to complete, and popen(), which launches a shell command with either its standard input or standard output connected to a pipe, by which the concurrently-running parent process can communicate with it. Both of these are specified by POSIX, and on POSIX systems they are specified to operate via fork() + exec(). Of course Windows has system(), too, and _popen(), but not fork() or any direct analog.
Summary
Overall, then, a userspace Linux process can start a new process only by forking, but that is to be distinguished from calling the fork() library function, even indirectly. There are at least six different C library functions in GNU / Linux that can serve as an interface for starting a new process. Some of those interfaces furthermore permit the new process to execute the same code (a copy of the process image of the original) indefinitely, which is sometimes a useful thing to do. It is the fork() part of fork() + exec() that starts a new process; the exec() part merely changes what code the new process runs.
I've been looking at creating Unix dæmons, and there seem to be two methods. The long-winded one, which seems to come up when searching is to call fork(), setsid(), fork() again, chdir() to somewhere safe, set umask() and, finally, close() stdin, stdout and stderr.
Running man daemon, however, brings up information on a daemon() function, which seems to do all the same stuff as above. Are there any differences between the two approaches or is daemon() just a convenience function that does the same thing as the long-winded method? Is either one better, especially for a novice C programmer?
The daemon function is not defined in POSIX, so its implementation (if any) could behave differently on different platforms.
On Linux with glibc, daemon only does one fork, optionally chdirs (but only to /, you can't specify a path), does not touch umask, and does not close the std* descriptors (it optionally reopens them to /dev/null though). (source)
So it depends on the platform, and at least one implementation does less than what you do. If you need all of what you're doing, stick with that (or stick to a platform where the daemon function does exactly that).
Note that daemon is not conforming to any standard. Better use standard conforming functions (like POSIX-defined fork and setsid).
The daemon call summarizes the long-winded fork procedure, and I don't recall any implementation that does anything more.
Since daemon() is a high-level concept, it's definitely to be preferred for novice and experienced programmers.
I'm trying to run an application in C, but the only way I could find that is reasonably easy to use works like this:
system("command here");
It works, of course, but it's really slow (especially when repeating this a lot). I'm just wondering if there is a way of running a program without having to interact with a shell, something like python's subprocess module.
I have heard of execl, and I would use that (forking it first, of course), but I'm wondering if there is a simpler way that wouldn't require forking first.
EDIT: I also want to be able to know the return code of the program
As I'm sure you already know, system already employs the fork/exec strategy. I understand you want to circumvent the shell and are looking for a simple approach, I'm just saying you could just as easily write a function to wrap the fork/exec pattern as is done in system. Indeed it would probably be most straightforward to just do that. An alternative as Gabe mentioned in the comments is posix_spawn.
A faster (but apparently discouraged) alternative is vfork() / exec, but this is generally discouraged and is obsolete in the latest POSIX standards.
4.3BSD; POSIX.1-2001 (but marked OBSOLETE). POSIX.1-2008 removes the
specification of vfork().
It's meant to be immediately followed by an exec or _exit. Otherwise all kinds of weird bugs can arise since the virtual memory pages and page tables aren't duplicated (child uses same data/heap/stack segments). The parent/calling process blocks until the child execs or _exits. Regular fork's modern implementations have copy-on-write semantics which approach the speed of vfork, without the potential bugs incurred by vfork's memory sharing semantics.
If you want even further control over memory-sharing semantics and process inheritance, and the consequent potential speed-up (and are on Linux), look into clone() (wrapper for system-call sys_clone()) which is what some process-creating system calls delegate their work to. Be sure to carefully comb over all of the various flags.
You can use waitpid to get the exit status of the process.
If neither system() nor popen() provides the mechanism you need, then the easy way to do it is with fork() and execv() (or, perhaps, execl(), but the argument list must be fixed at compile time, not variable, to use it). Really! It is not hard to do fork() and exec(), and any alternative will encapsulate that processing.
The Python subprocess module is simply hiding fork() and exec() for you behind a convenient interface. That's probably appropriate for a high-level language like Python. C is a lower-level language and doesn't really need the complexity.
The hard way to do it is with posix_spawn(). You have to create arguments to describe all the actions you want done in the child between the fork() and the exec(), which is far harder to set up than it is to simply do the fork(), make the changes, and then use exec() after all. This (posix_spawn()) is what you get when you design the code to spawn a child process without visibly using fork() and exec() and ensure that it can handle almost any reasonable circumstance.
You'll need to consider whether you need to use wait() or waitpid() or a variant to determine when the child is complete. You may need to consider whether to handle the SIGCHLD signal (which will notify you when a child dies).
I've been looking at creating Unix dæmons, and there seem to be two methods. The long-winded one, which seems to come up when searching is to call fork(), setsid(), fork() again, chdir() to somewhere safe, set umask() and, finally, close() stdin, stdout and stderr.
Running man daemon, however, brings up information on a daemon() function, which seems to do all the same stuff as above. Are there any differences between the two approaches or is daemon() just a convenience function that does the same thing as the long-winded method? Is either one better, especially for a novice C programmer?
The daemon function is not defined in POSIX, so its implementation (if any) could behave differently on different platforms.
On Linux with glibc, daemon only does one fork, optionally chdirs (but only to /, you can't specify a path), does not touch umask, and does not close the std* descriptors (it optionally reopens them to /dev/null though). (source)
So it depends on the platform, and at least one implementation does less than what you do. If you need all of what you're doing, stick with that (or stick to a platform where the daemon function does exactly that).
Note that daemon is not conforming to any standard. Better use standard conforming functions (like POSIX-defined fork and setsid).
The daemon call summarizes the long-winded fork procedure, and I don't recall any implementation that does anything more.
Since daemon() is a high-level concept, it's definitely to be preferred for novice and experienced programmers.