I'm writing a (very small) webserver in C language on Windows.
I need exclusive lock file both on reading and writing files, i've read msdn documentation about locking etc. and I've found the function LockFileEx with the OVERLAPPED structure and an Event hEvent, I read also about how they work but the question is:
- In a web server we have lots of files, when a thread locks for example the file "test.txt"(exclusive lock) because for there was a request of this file, how can I synchronize another thread that wants to get the lock on the same file?
thank you.
Take a look at the use of mutex objects. They should solve that problem for you.
Threads that need access to the lock file can request a lock for it and be queued. When the current thread is done, it releases its lock and the next requesting thread is granted the lock.
Related
I have read the sqlite.org contents to know about the sqlite locking mechanism.
As per my understanding sqlite uses its own locking mechanism to prevent from multiple processes / threads writing at once on same file.
https://www.sqlite.org/lockingv3.html
https://www.sqlite.org/wal.html
Do we need explicit lock (like semaphore mutex) for write queries across one or more process?
When WAL mode enabled, If process-1 executing read query while process-2 writing to the DB , will process-1 gets db_locked/ db_busy error?
I need to create named lock that work correctly with multi-thread application for Linux. Each instance of application could use more than one named-lock with different names.
I know about fcntl/flock, but it doesn't work if try to lock twice from different thread of one application or from one thread.
I know about open(..., O_CREATE | O_EXCL), but this file-lock will not be removed if application was killed by signal KILL or was crashed with segmentation fault and there is needed manual removing of lock-files after restart application.
Any another ways?
If you just need to run under modern Linux, you could use file-private locks. If that's not an option, you'll have to build your own thread-safe locking abstraction on top of fcntl locks. SQLite is public domain and has implemented that, so you could look at that for inspiration. If GPLed code is okay: OpenJDK has another, incompatible implementation of the same thing.
O_EXCL does not perform locking (beyond the file creation step), so that's usually not helpful.
Other options are System V and POSIX semaphores, but these usually do not work as well as fcntl locks when processes day. A robust, process-shared mutex in a file mapping could be an option as well, but you need to be careful to stay within the POSIX semantics as far as serialization to disk is concerned (basically, you need to reinitialize the mutex every time the application starts from scratch, after a reboot or libc update).
Lets say 4 simultaneous processes are running on a processor, and data needs to be copied from an HDFS (used with Spark) file system to a local directory. Now I want only one process to copy that data, while the other processes just wait for that data to be copied by the first process.
So, basically, I want some kind of a semaphore mechanism, where every process tries to obtain semaphore to try copying the data, but only one process gets the semaphore. All processes who failed to acquire the semaphore would then just wait for the semaphore to be cleared (the process who was able to acquire the semaphore would clear it after its done with copying), and when its cleared they know the data has already been copied. How can I do that in Linux?
There's a lot of different ways to implement semaphores. The classical, System V semaphore way is described in man semop and more broadly in man sem_overview.
You might still want to do something more easily scalable and modern. Many IPC frameworks (Apache has one or two of those, too!) have atomic IPC operations. These can be used to implement semaphores, but I'd be very very careful.
Generally, I regularly encourage people who write multi-process or multi-threaded applications to use C++ instead of C. It's often simpler to see where a shared state must be protected if your state is nicely encapsulated in an object which might do its own locking. Hence, I urge you to have a look at Boost's IPC synchronization mechanisms.
In addition of Marcus Müller's answer, you could use some file locking mechanism to synchronize.
File locking might not work very well on networked or remote file systems. You should use it on a locally mounted file system (e.g. Ext4, BTRFS, ...) not on a remote one (e.g. NFS)
For example, you might adopt the convention that your directory contains (or else you'll create it) some .lock file and use an advisory lock flock(2) (or a POSIX lockf(3)) on that .lock file before accessing the directory.
If using flock, you could even lock the directory directly....
The advantage of using such a file lock approach is that you could code shell scripts using flock(1)
And on Linux, you might also use inotify(7) (e.g. to be notified when some file is created in that directory)
Notice that most solutions are (advisory, so) presupposing that every process accessing that directory is following some convention (in other words, without more precautions like using flock(1), a careless user could access that directory - e.g. with a plain cp command -, or files under it, while your locking process is accessing the directory). If you don't accept that, you might look for mandatory file locking (which is a feature of some Linux kernels & filesystems, AFAIK it is sort-of deprecated).
BTW, you might read more about ACID properties and consider using some database, etc...
Hello every one I am making a program using filing I know how to read an write in a file .But please can any one help me about the file read write locks in C programming.Like how to insert lock and how to release it especially in forking .Please any give a small example or a tutorial as i didn't file any thing about file locks in c
Thanks
File locking is not part of C, but is dependent on the operating system. Since you talk abour forking I assume you are using UNIX or a UNIX-like system (e.g. Linux or BSD.)
In that case you can use the flock or lockf functions. These locks are preserved on forking, which means that multiple processes can have an exclusive lock to the same file if the lock was acquired in the parent process before the fork.
On Windows it can be specified in the CreateFilecall, or later with the LockFile or LockFileEx functions.
Is there any way to perform POSIX shared synchronization objects cleanup especially on process crash? Locked POSIX semaphores unblock is most desired thing but automatically 'collected' queues / shared memory region would be nice too. Another thing to keep eye on is we can't in general use signal handlers because of SIGKILL which cannot be caught.
I see only one alternative: some external daemon which accepts subscriptions and 'keep-alive' requests working as watchdog so not having notifications about some object it could close / unlock object in accordance to registered policy.
Has anyone better alternative / proposition? I never worked seriously with POSIX shared objects before (sockets were enough for all my needs and are much more useful by my opinion) and I did not found any applicable article. I'd gladly use sockets here but can't because of historical reasons.
Rather than using semaphores you could use file locking to co-oridinate your processes. The big advanatge of file locks being that they are released if the process terminates. You can map each semaphore onto a lock for a byte in a shared file and know that locks will get released on exit; in mosts version of unix the bytes you lock don't even have to exist. There is code for this in Marc Rochkind's book Advanced Unix Programming 1st edition, don't know if it's in the latest 2nd edition though.
I know this question is old, but another great solution is POSIX robust mutexes. They automatically unlock and enter an "inconsistent flag" state when the owner dies, and the next thread to attempt locking the mutex gets an EOWNERDEAD error but succeeds in becoming the new owner of the mutex. It's then able to clean up whatever state the mutex was protecting (which could be in a very bad inconsistent state due to asynchronous termination of the previous owner!) and mark the mutex as consistent again before unlocking it.
See the documentation on robust mutexes here:
http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_mutex_lock.html
The usual way is to work with signal handlers. Just catch the signals and call the cleanup functions.
But your watchdog daemon has some merits, too. It would surely make the system more simple to understand and manage. To make it more simple to administrate, your application should start the daemon when it's not running and the daemon should be able to clean up any residue from the last crash.