Is there an easy way to determine if a certain process is running?
I need to know if an instance of my program is running in the background, and if not fork and create the background process.
Normally the race-free way of doing this is:
Open a lock file / pid file for writing (but do not truncate it)
Attempt to take an exclusive lock on it (using fcntl or flock) without blocking
If that fails with EAGAIN, then the other process is already running.
The file descriptor should now be inherited by the daemon and left open for its lifetime
The advantage of doing this over simply storing a PID, is that if somebody reuses the PID, you won't get a false positive.
The biggest problem with storing the pid in the file is that a low-numbered pid used by a system start up daemon can get reused on a subsequent reboot by a different daemon. I have seen this happen.
This is usually done using pidfiles: a file in /var/run/[name].pid containing only the process ID returned by fork().
if pidfile exists:
exit()
else:
create pidfile
pid = start_background()
pidfile.write(pid)
On shutdown: remove pidfile
Linux software, by far and large does not care about the exclusivity of programs, only the resources they use. "Caring" is most often provided by the implementation (E.G. the infrastructure of the distro).
For instance, if you want to run a program, but that program locks up or turns zombie and you have no way to kill it, or it's running as a different user performing some other function. Why should the program care whether another copy of itself is running? Having it do so only seems like an unnecessary restriction.
If it's a process that opens a socket (like a TCP port), have the program fail if it can't open the socket. If it needs exclusive access to a file, have it fail if it can't get it. Support a PID file, but don't make it mandatory.
You'll see this methodology all over GNU software, which is part of what makes it so versatile.
Related
I need to solve a concurrency assignment for my operating systems class. I don't want the solution here, but I am lacking one part.
We should write a process that writes to file, reads from it and then deltetes it. This process we should run two times in two different shells. No fork here for simplicity. Process A should write, Process B then read and then Process should delete the file. Afterwards they switch roles.
I understand that you can achieve atomicity easily by locking. With while loops around the read-, and write sections etc. you can also get further control. But when I run process A and then process B, process B will spin before the write seciton until it achieves the lock and not got into reading when process A releases the lock. So my best guess is to have a read and a write lock. This information must be shared somehow between the processes. The only way I can think of is some global variable, but since both processes hold copies of the variables, I think this is not possible. Another way would be to have a read lock file and a write lock file, but that seems overly complicated to me.
Is there a better way?
You can use semaphores to ensure the writer and deleter wait for the previous process to finish its job. (Use man sem_init for details)
When running multiple processes with semaphores, it should be created using shared mem (man shm_open for more details).
You will need as many semaphores as the number of pipelines in this process.
You can use file as a lock. Two processes try to create a file with a previously agreed upon name using the O_EXCL flag. Only one will succeed. The one that succeeds gets the access to the resource. So in this case process A should try to create a file with name say, foo, with O_EXCL flag and, if successful, it should go ahead and write to file the information. After its work is complete, Process A should unlink foo. Process B should try to create file foo with O_EXCL flag, and if successful, try to read the file created by Process A. After its attempt is over, Process B should unlink the file foo. That way only one process will be accessing the file at any time.
Your problem (with files and alternating roles in the creation/deletion of files) seems to be a candidate to use the O_EXCL flag on opening/creating the file. This flag makes the open(2) system call to succeed in creating a file only if the file doesn't exist, so it makes the file to appear as a semaphore itself. Each process can liberate the lock (A or B) but the one that does, just liberates the lock and makes the role of owning again accessible.
You will see that both processes try to use one of the roles, but if they both try to use the owner role, one of them will succeed, and the other will fail.
Just enable a SIGINT signal handler on the owning process, to allow it to delete the file in case it gets signalled, or you will leave the file and after that no process will be able to assume the owning role (at least you will need to delete it manually).
This was the first form of locking feature in unix, long before semaphores, shared memory or other ways to block processes existed. It is based on the atomicity of system calls (you cannot execute two system calls on the same file simultaneously)
I am using Shared Memory in a Client-Server Model.
When my Server gets killed off by the user by using sigkill instead of sigterm / sigint I can't do anything about it (as intended), but my Shared Memory Object and Semaphores still exist in /dev/shm/.
The next time I start my Server I want to create a new object with exactly the same name, and - as intented - this fails and I exit my program.
The user would need to remove the objects on his own - which is certainly not the best thing.
How can I handle this?
I could just call shm_open() without the O_EXCL flag, ultimately destroying the purpose of this flag. Because maybe there is already an instance of my server running and uses this object.
Pulseaudio seems to use a combination of digits to keep it's objects distinct and doesn't get affected by killing it with -9, so there seems to be a way.
One option is to use IPC_RMID (see this).
This options marks the shm segment for cleanup after the last process attached to it disappears.
For semaphores, you can look at robust mutexes. It does require you to code for an extra error case (when a process holding the mutex dies).
You also have the option of using file locks, which are released if the process terminates (see lockf()).
An outline of a possible server initialization:
Open pid file w/O_EXCL | O_WRONLY
if success write pid
close
Open shm w/O_CREAT
done
open pid file w/O_RDONLY
read pid
Use kill(0) to see if server alive
If yes, exit
If no, remove pid file
Close pid, start at top again
Pid files are usually located in the /var/run dir and are named foobar.pid where foobar is the program name
As the parent process is using huge mount of memory, fork may fail with errno of ENOMEM under some configuration of kernel overcommit policy. Even though the child process may only exec low memory-consuming program like ls.
To clarify the problem, when /proc/sys/vm/overcommit_memory is configured to be 2, allocation of (virtual) memory is limited to SWAP + MEMORY * ration(default to 50%).
When a process forks, virtual memory is not copied thanks to COW. But the kernel still need to allocate virtual memory space. As an analogy, fork is like malloc(virtual memory space size) which will not allocate physical memory and writing to shared memory will cause copy of virtual memory and physical memory is allocated. When overcommit_memory is configured to be 2, fork may fail due to virtual memory space allocation.
Is it possible to fork a process without inherit virtual memory space of parent process in the following conditions?
if the child process calls exec after fork
if the child process doesn't call exec and will not using any global or static variable from parent process. For example, the child process just do some logging then quit.
As Basile Starynkevitch answered, it's not possible.
There is, however, a very simple and common solution used for this, that does not rely on Linux-specific behaviour or memory overcommit control: Use an early-forked slave process do the fork and exec.
Have the large parent process create an unix domain socket and fork a slave process as early as possible, closing all other descriptors in the slave (reopening STDIN_FILENO, STDOUT_FILENO, and STDERR_FILENO to /dev/null). I prefer a datagram socket for its simplicity and guarantees, although a stream socket will also work.
In some rare cases it is useful to have the slave process execute a separate dedicated small helper program. In most instances this is not necessary, and makes security design much easier. (In Linux, you can include SCM_CREDENTIALS ancillary messages when passing data using an Unix domain socket, and use the process ID therein to verify the identity/executable the peer is using the /proc/PID/exe pseudo-file.)
In any case, the slave process will block in reading from the socket. When the other end closes the socket, the read/receive will return 0, and the slave process will exit.
Each datagram the slave process receives, describes a command to execute. (Using a datagram allows using C strings, delimited with NUL characters, without any escaping etc.; using an Unix stream socket typically requires you to delimit the "command" somehow, which in turn means escaping the delimiters in the command component strings.)
The slave process creates one or more pipes, and forks a child process. This child process closes the original Unix socket, replaces the standard streams with the respective pipe ends (closing the other ends), and executes the desired command. I personally prefer to use an extra close-on-exec socket in Linux to detect successful execution; in an error case, the errno code is written to the socket, so that the slave-parent can reliably detect the failure and the exact reason, too. If success, the slave-parent closes the unnecessary pipe ends, replies to the original process about the success, with the other pipe ends as SCM_RIGHTS ancillary data. After sending the message, it closes the rest of the pipe ends, and waits for a new message.
On the original process side, the above process is sequential; only one thread may execute start executing an external process at a time. (You simply serialize the access with a mutex.) Several can run at the same time; it is only the request to and response from the slave helper that is serialized.
If that is an issue -- it should not be in typical cases -- you can for example multiplex the connections, by prefixing each message with an ID number (assigned by the parent process, monotonically increasing). In that case, you'll probably use a dedicated thread on the parent end to manage the communications with the slave, as you certainly cannot have multiple threads reading from the same socket at the same time, and expect deterministic results.
Further improvements to the scheme include things like using a dedicated process group for the executed processes, setting limits to them (by setting limits to the slave process), and executing the commands as dedicated users and groups by using a privileged slave.
The privileged slave case is where it is most useful to have the parent execute a separate helper process for it. In Linux, both sides can use SCM_CREDENTIALS ancillary messages via Unix domain sockets to verify the identity (PID, and with ID, the executable) of the peer, making it rather straightforward to implement robust security. (But note that /proc/PID/exe has to be checked more than once, to catch the attacks where a message is sent by a nefarious program, quickly executing the appropriate program but with command-line arguments that cause it to exit soon, making it occasionally look like the correct executable made the request, while a copy of the descriptor -- and thus the entire communications channel -- was in control of a nefariuous user.)
In summary, the original problem can be solved, although the answer to the posed question is No. If the executions are security-sensitive, for example change privileges (user accounts) or capabilities (in Linux), then the design has to be carefully considered, but in normal cases the implementation is quite straight-forward.
I'd be happy to elaborate if necessary.
No, it is not possible. You might be interested by vfork(2) which I don't recommend. Look also into mmap(2) and its MAP_NORESERVE flag. But copy-on-write techniques are used by the kernel, so you practically won't double the RAM consumption.
My suggestion is to have enough swap space to not being concerned by such an issue. So setup your computer to have more available swap space than the largest running process. You can always create some temporary swap file (e.g. with dd if=/dev/zero of=/var/tmp/swapfile bs=1M count=32768 then mkswap /var/tmp/swapfile) then add it as a temporary swap zone (swapon /var/tmp/swapfile) and remove it (swapoff /var/tmp/swapfile and rm /var/tmp/swapfile) when you don't need it anymore.
You probably don't want to swap on a tmpfs file system like /tmp/ often is, since tmpfs file systems are backed up by swap space!.
I dislike memory overcommitment and I disable it (thru proc(5)). YMMV.
I'm not aware of any way to do (2), but for (1) you could try to use vfork which will fork a new process without copying the page tables of the parent process. But this generally isn't recommended for a number of reasons, including because it causes the parent to block until the child performs an execve or terminates.
This is possible on Linux. Use the clone syscall without the flag CLONE_THREAD and with the flag CLONE_VM. The parent and child processes will use the same mappings, much like a thread would; there is no COW or page table copying.
madvise(addr, size, MADV_DONTFORK)
Alternatively, you can call munmap() after fork() to remove the virtual addresses inherited from the parent process.
This question already has answers here:
How to create a single instance application in C or C++
(15 answers)
Closed 9 years ago.
I have a binary and it's a daemon and it's developed in C. I want to add a check at the beginning of my program to guarantee that the binary is launched only one time. My binary runs on Linux.
Any suggestions?
A common method is to put a PID file in /var/run. After your daemon starts successfully, you flock write its PID to this file. At startup, you check the value of the PID in this file, if it exists. If there is no PID currently running, it's safe for the application to startup. If the PID exists, perform a check to see if that PID is an instance of your executable. If it's not, it is also safe to startup. You should delete the file on exit, but it's not strictly necessary.
The best way to do this, in my opinion, is not to do it. Let your initialization scheme serialize instances of the daemon: systemd, runit, supervise, upstart, launchd, and so on can make sure there are no double invocations.
If you need to invoke your daemon "by hand," try the linux utility flock(1) or a 3rd-party utility like setlock. Both of these will run the daemon under the protection of a (perhaps inherited) lockfile which remains locked for the life of the program.
If you insist upon adding this functionality to the daemon itself (which, in my opinion, is complication that most daemons don't need), choose a lockfile and keep it exclusively flock(2)d. Unlike most pidfile/process table approaches, this approach is not race-prone. Unlike POSIX system semaphores, this mechanism will correctly handle the case of a crashed daemon (the lock vanishes when the process does).
There may be other easy serializations, too. If your daemon binds to a socket, you know that EADDRINUSE probably means that another instance is running...
Fork and execute this:
pidof nameOfProgram
If it returns a value, you know your program is running!
The other classic method is to have a lock file - the program creates a file, but only if that file does not already exist. If the file does exist, it presumes there's another copy of the program running. Because the program could crash after creating the file, smarter versions of this have ways to detect that situation.
I want only one copy of my program in the system. How can I look for another copies in the system from C-code? I want something like that
# program &
[1] 12586
# program &
Program is already running
The best idea I have is making .lock-files. But I didn't find any guildlines about them.
Thank you.
One daemon I wrote opened a UNIX domain socket for regular client-daemon communication. Other instances then checked whether they could connect to that socket. If they could, another instance was currently running.
Edit: As noted by #psmears, there's a race condition. The other instances should just try to create that same listening socket. That will fail if it is already in use.
Lock files work more often than that special case. You may create an (empty) file in a well known location and then use file locks, say with fcntl(2) and F_SETLK and F_GETLK to set a lock on that file or determine whether a lock is held. May not work over NFS. Locks are cleared when your process dies, so this should work, and is portable (at least to HP-UX). Some daemons like to dump their pid into that file if they determine that no other instance is currently running.
You can use named sempahores, which is a very standard approach to this problem.
Your program calls semctl() to find if there are any active sempahores, then checks to see if you can run. If you find none, then you create the sempahore.
The OS handles the problem of processes being killed off with kill -9 and leaving sempahores.
You need to read the man page for semctl and sem_open for your machines to see what that mechanism
is.