I have two processes that will be running, one that will be reading from shared memory (mmap) and one that will be writing to that shared memory (mmap). These processes are started separately in two different terminals, but they need to be synchronized so that while one process is writing, it writes the full amount before the other process reads from the memory. All of the posts I have seen relating to shared memory mutex locks have been spawning threads/processes from a single main program. Is there any way to create a shared mutex lock that can be used by two separate programs?
Sorry, but you are out of luck. Pthreads library does not have a concept of 'named' mutex, so two independent processes can't reliably share one.
Yes, you can create a mutex in shared memory and than use this mutex from the other process, but there is no way you can ensure the mutex is fully initialized by the first process when you are checking it in the second. For trully independent programms I strongly recommend using semaphores.
You can create a shared mutex into an mmapped file. If you're using Linux and have a sufficiently new kernel, you can even create an unlinked temp file, mmap it; initialize the mutex and only then link it to the final location. Or you can use file locking to deny access to it until the initialization has been completed.
The semaphore example from pthread_mutexattr_init POSIX manuals at linux.die.net did work on my Linux 4.2.0-27 Ubuntu.
Related
I'm working with semaphores in C , especifically to control the access to a shared memory zone in linux. but there is one thing that I can't understand.
I am using a mutex to control the access to a specific zone because i have 2 processes that must read/write from that zone. the thing is, when we use the fork() to create a new child process, the whole program is "copied" to another program as if they were two seperate programs right ? so, when i do V(mutex) in one process, how does the other one know he can't access ?
I know its a noob question but nobody could explain this to me until now.
After the fork neither process is going to know about the memory actions of the other because they are separate copies. You have to put your shared variables in shared memory, including mutexes and semaphores. Then all the processes are operating on the same resource.
For unrelated (i.e. non-forked) process there are usually system facilities (e.g. named semaphores) that each process can open based on a path name or similar method that each can use to find and use the resource.
You synchronisation objects must be placed in process shared memory, for example created with mmap (... MAP_ANONYMOUS ...). In addition, they must have the PTHREAD_PROCESS_SHARED attribute set, for example, by using pthread_mutexattr_setpshared.
See here:
Semaphores and Mutex for Thread and Process Synchronization
So mutex in practice is often used in threads, which makes sharing trivial. For processes however, mutex could be stored as a part of the shared mem.
For semaphores however, linux has built in library, which identifies global semaphores by keys. See below.
http://beej.us/guide/bgipc/output/html/multipage/semaphores.html
Or you can use other IPC to sync. Signals, for example.
Hope this helps.
I am trying to create a shared memory which will be used by multiple processes. these processes communicate with each other using MPI calls (MPI_Send, MPI_Recv).
I need a mechanism to control the access of this shared memory I added a question yesterday to see if MPI provides any facility to do that. Shared memory access control mechanism for processes created by MPI , but it seems that there is no such provision by MPI.
So I have to choose between named semaphore or flock.
For named semaphore if any of the process dies abruptly without calling sem_cloe(), than that semaphore always remains and can be seen by ll /dev/shm/. This results in deadlock sometimes(if I run the same code again!), for this reason I am currently thinking of using flock.
Just wanted to confirm if flock is best suited for this type of operation ?
Are there any disadvantages of using flock?
Is there anything else apart from named semaphore and flock that can be used here ?
I am working on C under linux.
You can also use a POSIX mutex in shared memory; you just have to set the "pshared" attribute on it first. See pthread_mutexattr_setpshared. This is arguably the most direct way to do what you want.
That said, you can also call sem_unlink on your named semaphore while you are still using it. This will remove it from the file system, but the underlying semaphore object will continue to exist until the last process calls sem_close on it (which happens automatically if the process exits or crashes).
I can think of two minor disadvantages to using flock. First, it is not POSIX, so it makes your code somewhat less portable, although I believe most Unixes implement it in practice. Second, it is implemented as a system call, so it will be slower. Both pthread_mutex_lock and sem_wait use the "futex" mechanism on Linux, which only does a system call when you actually have to wait. This is only a concern if you are grabbing and releasing the lock a lot.
I'm writing a small piece of software in C using the pthread library.
I have a bunch of threads that need write access to a shared structure containing a dynamically allowed char array but I can guarantee that two different threads will never try to access the same element of this array. My question is: should I use something like a semaphore or a mutex or isn't it necessary?
If your threads only read information, then no lock is needed.
If your threads modify information other threads don't see, no lock is needed.
If there is a single place that can be modified by one thread and used by others, you need to use a mutex.
In your case the data are not shared between the threads and since the data is not shared between the threads, no synchronization mechanismn is required.
Well I think you answered the question yourself!
The purpose of mutexes is to protect against concurrent access of different threads on some resources. If you can guarantee that by design your threads will never concurrently access (read or write) the same memory area then you don't need mutex protection.
I am new to threads and processes.
I have code that works fine right now with forking the code into multiple processes. However each process needs to add to a global variable, but from what I read, each time the process forks, it takes a copy of the global, and adds them independently. Is there a way to join them, like you can with threads?
Different processes can communicate and exchange data via shared memory.
On linux, you can look:
man shm_overview
for attaching a memory segment on several processes
and
man sem_overview
for the semaphore library for controlling parallel access.
You should define a struct with two fields, one for your global and one for a semaphore. Then, before any forking occurs, create some shared memory in the parent process big enough to hold this struct and initialize one there. In the children, map in the shared memory so they can access the global. All processes, parent and children, should obey the rules of the semaphore when accessing the global.
To avoid unnecessary blocking which can hurt performance, try not to hold the semaphore too long. When reading the global, make a quick copy of it in a process and use that, rather than holding the semaphore for the entire time you are using its value. Likewise, when changing the global, prepare your changes ahead of time (before you grab the semaphore) and, once you have the semaphore, copy them in all at once. Sometimes your work depends on reading and writing the global without it changing in between being read and written. In this case, some blocking may be inevitable.
It is not clear what platform you are on, but all major PC and server platforms (Windows, Linux/Unix/Mac OS) have support for shared memory and semaphores. The APIs may be different, but the functionality you need is there.
CreateFileMapping is used to create shared memory in win32. Do need to synchronized shared memory read/write or it is done automatically ?
Your question was not clear about whether you use this for shared memory between threads or processes, so just to be sure: for threads you dont need a file-mapping, all memory in a single process is visible to all threads (and needs sync).
To use it for shared memory between processes: yes you have to sync accesses to it explicitly because the system can not know whether consecutive writes to it are meant to be grouped as a single transaction or not. Tip to do this: To sync them you can not use criticalsections (they only work for threads within a process), you could use:
http://msdn.microsoft.com/en-us/library/aa904937%28v=VS.85%29.aspx