I'm implementing an FTP-like protocol in Linux kernel 2.4 (homework), and I was under the impression that if a file is open for writing any subsequent attempt to open it by another thread should fail, until I actually tried it and discovered it goes through.
How do I prevent this from happening?
PS: I'm using open() to open the file.
PS2: I need to be able to access existing files. I just want to prevent them being written to simultaneously.
You could keep a list of open files, and then before opening a file check to see if it has already been opened by another thread. Some issues with this approach are:
You will need to use a synchronization primitive such as a Mutex to ensure the list is thread-safe.
Files will need to be removed from the list once your program is finished with them.
System-level file locking is process-based, so you cannot use it. You will need to use process-level locking. For example, by defining a mutex (lock) using pthreads.
Use the O_CREATE and O_EXCL flags to open(). That way the call will fail if the file already exists.
Related
I'd like to create shared library in C for linux, some abstract implementation of database management. Shared library whould be responsible for read file containing database and write differences into it. But I have no idea how to handle multiprocessing problems of file handling for this case eg.: App1 try to write differences into database file and App2 has currently opened file with database to read it. In the case of this example I'd like to inform app1 that file is currently open and delay write sequence until App2 will finish database file read.
I was thinking of using some mutual exclusion mechanisms or by using global enum variable to manage current file status, but after I read some of posts I understood that every application that uses shared library create it's own copy in memory and they don't share any memory section during work.
Shared library whould be responsible for read file containing database and write differences into it.
That is possible but quit complicated solution.
While you would need to make sure that multiple processes do not interfere with each other. It's possible to do so with file logs (see man flock), and record locking in man fcntl but you must insure that multiple processes update disjoint file chunks "in place" (without resizing the file).
This is also prone to deadlocks -- if one of the processes takes a lock on a region and then goes into infinite loop, other processes may get stuck as well.
A much simpler solution involves client-server, where the server implements all writes and clients send it read and modify requests.
P.S. There are existing libraries implementing either approach. You will likely save yourself several months of development time using existing solutions.
File locking in C using flock is commonly used to implement cross-platform cooperative inter-process locking/mutex.
I have an implementation (mac/linux/win) that works well, but it is not robust to file deletion (at least under Linux).
One or more process have started creating and using lockfile (/tmp/lockfile) and cooperatively interlock on a shared resource with it.
Some time later, I manually delete the lockfile (rm /tmp/lockfile). The running process keep on cooperating with each other, but any new process that wants to start using the same resource lock and lockfile breaks the overall mutex logic. It will create a new version of the /tmp/lockfile that is somehow different to the one already in used in already running process.
What can be done to prevent the lockfile from being unlinked while any process has it open?
What other solutions can be used?
I can't use a semaphore because I need the lock to self-release if the owning process crashes.
You can indeed use a semaphore. Linux provides the flag SEM_UNDO, which will undo the semaphore-operation on process termination (see semop(2)).
The rm command does not actually delete files. Rather, it unlinks them from the file system, as if via the unlink(2) syscall. The files will not be removed from disk as long as any process holds them open (or any other hard links to them exist), and processes that do hold them open continue to refer to the same file, even though it no longer appears in directory listings. Nothing prevents another file from being created and linked to the file system at the same place as the previous one, but that is an altogether different file. This behavior is desirable for consistent program behavior, and some programs intentionally use it to their advantage for managing temporary files.
There is nothing you can do to prevent a process with sufficient privileges from unlinking the lock file. Any process that has sufficient privilege to create the lock file has sufficient privilege to unlink it, with the consequences you describe. One usually mitigates this problem by creating a temporary file with an unpredictable name for use with flock(), so that the file name or an open file handle must be exchanged between processes that want to synchronize actions by locking that file. For the particular case of child processes, you can rely on the child inheriting open file descriptors from its parent to enable them to get at the lock file even if it has been unlinked.
On the other hand, if you are relying on a lock file with a well-known name then the solution may be to create the file in advance, make root its owner and the owner of every directory in the path of hard links leading to it, and deny all other users write access the file and the directories. You could consider further wrapping it up with mandatory access controls (SELinux policy) if you wanted to be even more careful.
I want to create a project to lock file and folders in ubuntu by face detection through opencv using C language. Can you please let me know it is possible and how can i do it.
Can't help you with the opencv part, but "lock file and folders" could mean a few things:
You want to change permissions of files so that a given user/group can/cannot
access them. If this is the case, you want the chmod function.
See man 2 chmod. Seems like this is probably what you're after?
Usually, "file locking" on Linux refers to a means to prevent other processes from accessing a file without changing permissions via either:
Mandatory file locking via lockf (or fcntl).
Advisory file locking via flock.
If file locking is what you're after, here are the "see also" documents referred to by the man pages on lockf and/or flock:
https://www.kernel.org/doc/Documentation/filesystems/mandatory-locking.txt
https://www.kernel.org/doc/Documentation/filesystems/locks.txt
Note: Others have indicated you might want to use the C++ API for opencv. All of these functions should work just fine from C++ too.
We have a shared folder which contains some files that needs to be processed.
We also have 3 UNIX servers which runs a shell script which take and process one file each time. At the end of the script the file it's moved away.
The 3 UNIX server doesn't communicate each other, and they are not aware of each other.
In your opinion what is the best way to guarantee that each file will be processed one time, without raising concurrent access issues\error ?
So or so you need some type of a file locking mechanism. Some of the possibilities:
You can create a temporary lock file for every files on work. For example, for file name.ext you will need to create a name.ext.lock, just before you start its processing. If this file already exists - also, the creation fails with a "file exists", it means somebody is already working on it, thus you shouldn't do anything with it.
Second, you could use advisory locks. Advisory locking doesn't already work on every type of file sharing, and they have only libc-level interface, so you can't use them from shell scripting. I suggest to dig into the manual of the flock libc api call.
Third, it were the hardest and it is deeply unix-specific. It were the mandatory lock. Mandatory locking means that the locks are effective even against the processes, who don't know anything from them. You can read more about them here: Mandatory file lock on linux
In your place I did the first, if I can modify the workings of the processing (for example, if I can hook them with a script or even I am developing the processing script). If not, you need probably the third, although it doesn't always work.
I'm writing a TFTP server program for university, which needs exclusive access to the files it opens for reading. Thus it can be configured that if a file is locked by another process that it waits for the file to become unlocked.
Is there any way on Win32 to wait for a file become unlocked without creating a handle for it first?
The reason I ask, is that if another process calls CreateFile() with a dwShareMode that is incompatible to the one my process uses, I won't even be able to get a file handle to use for waiting on the lock using LockFileEx().
Thanks for your help in advance!
If you take a look at the Stack Overflow questions What Win32 API can be used to find the process that has a given file open? and SYSTEM_HANDLE_INFORMATION structure, you will find links to code that can be used to enumerate processes and all open handles of each running process. This information can be used to obtain a HANDLE to the process that has the file open as well as its HANDLE for the file. You would then use DuplicateHandle() to create a copy of the file HANDLE, but in the TFTP process' handle table. The duplicated HANDLE could then be used by the TFTP process with LockFileEx().
This solution relies on an internal function, NtQuerySystemInformation(), and an undocumented system information class value that can be used to enumerate open handles. Note that this feature of NtQuerySystemInformation() "may be altered or unavailable in future versions of Windows". You might want to use an SEH handler to guard against access violations were that to happen.
As tools from MS like OH and Process Explorer do it, it is definitely possible to get all the handles opened by a process. From there to wait on what you'd like the road is still long, but it is a beginning :)
If you have no success with the Win32 API, one place to look at is for sure the NT Native API http://en.wikipedia.org/wiki/Native_API
You can start from here http://msdn.microsoft.com/en-us/library/windows/desktop/ms724509%28v=vs.85%29.aspx and see if it works with the SystemProcessInformation flag.
Look also here for a start http://nsylvain.blogspot.com/2007/09/how-list-all-open-handles.html
The native API is poorly documented, but you can find resources online (like here http://www.osronline.com/article.cfm?id=91)
As a disclaimer, I should add that the Native API is somehow "internal", and therefore subject to change on future versions. Some functions, however, are exposed also publicly in the DDK, at kernel level, so the likelihood of these functions to change is low.
Good luck!