File locking in C using flock is commonly used to implement cross-platform cooperative inter-process locking/mutex.
I have an implementation (mac/linux/win) that works well, but it is not robust to file deletion (at least under Linux).
One or more process have started creating and using lockfile (/tmp/lockfile) and cooperatively interlock on a shared resource with it.
Some time later, I manually delete the lockfile (rm /tmp/lockfile). The running process keep on cooperating with each other, but any new process that wants to start using the same resource lock and lockfile breaks the overall mutex logic. It will create a new version of the /tmp/lockfile that is somehow different to the one already in used in already running process.
What can be done to prevent the lockfile from being unlinked while any process has it open?
What other solutions can be used?
I can't use a semaphore because I need the lock to self-release if the owning process crashes.
You can indeed use a semaphore. Linux provides the flag SEM_UNDO, which will undo the semaphore-operation on process termination (see semop(2)).
The rm command does not actually delete files. Rather, it unlinks them from the file system, as if via the unlink(2) syscall. The files will not be removed from disk as long as any process holds them open (or any other hard links to them exist), and processes that do hold them open continue to refer to the same file, even though it no longer appears in directory listings. Nothing prevents another file from being created and linked to the file system at the same place as the previous one, but that is an altogether different file. This behavior is desirable for consistent program behavior, and some programs intentionally use it to their advantage for managing temporary files.
There is nothing you can do to prevent a process with sufficient privileges from unlinking the lock file. Any process that has sufficient privilege to create the lock file has sufficient privilege to unlink it, with the consequences you describe. One usually mitigates this problem by creating a temporary file with an unpredictable name for use with flock(), so that the file name or an open file handle must be exchanged between processes that want to synchronize actions by locking that file. For the particular case of child processes, you can rely on the child inheriting open file descriptors from its parent to enable them to get at the lock file even if it has been unlinked.
On the other hand, if you are relying on a lock file with a well-known name then the solution may be to create the file in advance, make root its owner and the owner of every directory in the path of hard links leading to it, and deny all other users write access the file and the directories. You could consider further wrapping it up with mandatory access controls (SELinux policy) if you wanted to be even more careful.
Related
I'd like to create shared library in C for linux, some abstract implementation of database management. Shared library whould be responsible for read file containing database and write differences into it. But I have no idea how to handle multiprocessing problems of file handling for this case eg.: App1 try to write differences into database file and App2 has currently opened file with database to read it. In the case of this example I'd like to inform app1 that file is currently open and delay write sequence until App2 will finish database file read.
I was thinking of using some mutual exclusion mechanisms or by using global enum variable to manage current file status, but after I read some of posts I understood that every application that uses shared library create it's own copy in memory and they don't share any memory section during work.
Shared library whould be responsible for read file containing database and write differences into it.
That is possible but quit complicated solution.
While you would need to make sure that multiple processes do not interfere with each other. It's possible to do so with file logs (see man flock), and record locking in man fcntl but you must insure that multiple processes update disjoint file chunks "in place" (without resizing the file).
This is also prone to deadlocks -- if one of the processes takes a lock on a region and then goes into infinite loop, other processes may get stuck as well.
A much simpler solution involves client-server, where the server implements all writes and clients send it read and modify requests.
P.S. There are existing libraries implementing either approach. You will likely save yourself several months of development time using existing solutions.
I want to take backup of a file once it reaches a particular size. The file is associated with stdout stream for multiple process.
I doubt if freopen can be used in this case as it is associated with multiple process.
Though it may succeed with the process which executing that, other process references for stdout going to NULL on UNIX platform. I don't know how it behave on windows.
Renaming that file also not allowing on windows as multiple process are opened that file.
Is there any way to forcefully rename of the file.? On Unix, if I move the file all the process are referring the moved file. This also fine, later I have a mechanism to inform other process to reopen the file.
How to achieve this on windows with C programming.
We have a shared folder which contains some files that needs to be processed.
We also have 3 UNIX servers which runs a shell script which take and process one file each time. At the end of the script the file it's moved away.
The 3 UNIX server doesn't communicate each other, and they are not aware of each other.
In your opinion what is the best way to guarantee that each file will be processed one time, without raising concurrent access issues\error ?
So or so you need some type of a file locking mechanism. Some of the possibilities:
You can create a temporary lock file for every files on work. For example, for file name.ext you will need to create a name.ext.lock, just before you start its processing. If this file already exists - also, the creation fails with a "file exists", it means somebody is already working on it, thus you shouldn't do anything with it.
Second, you could use advisory locks. Advisory locking doesn't already work on every type of file sharing, and they have only libc-level interface, so you can't use them from shell scripting. I suggest to dig into the manual of the flock libc api call.
Third, it were the hardest and it is deeply unix-specific. It were the mandatory lock. Mandatory locking means that the locks are effective even against the processes, who don't know anything from them. You can read more about them here: Mandatory file lock on linux
In your place I did the first, if I can modify the workings of the processing (for example, if I can hook them with a script or even I am developing the processing script). If not, you need probably the third, although it doesn't always work.
I want to delete an executable after I start the process.
I tried by putting unlink and it work fine, but I want my executable to continue running.
Is using the unlink approach correct? Are there any issues with using this approach?
On Unix, there shouldn't be any problems unlinking the executable of a running process.
When you unlink a file, the directory entry is removed, but the inode and the underlying data are not freed until all existing references to the file (i.e. hard links and open handles) are released.
I'm implementing an FTP-like protocol in Linux kernel 2.4 (homework), and I was under the impression that if a file is open for writing any subsequent attempt to open it by another thread should fail, until I actually tried it and discovered it goes through.
How do I prevent this from happening?
PS: I'm using open() to open the file.
PS2: I need to be able to access existing files. I just want to prevent them being written to simultaneously.
You could keep a list of open files, and then before opening a file check to see if it has already been opened by another thread. Some issues with this approach are:
You will need to use a synchronization primitive such as a Mutex to ensure the list is thread-safe.
Files will need to be removed from the list once your program is finished with them.
System-level file locking is process-based, so you cannot use it. You will need to use process-level locking. For example, by defining a mutex (lock) using pthreads.
Use the O_CREATE and O_EXCL flags to open(). That way the call will fail if the file already exists.