Do semaphores and mutexes synchronize both threads and processes, or only threads, or only processes?
Edit: my mistake it's C, not shell. Since I programmed through the shell I mistook it. And one more thing: are communication and synchronization the same?
SysV IPC semaphores synchronize between processes.
With POSIX semaphores, you can choose via the pshared argument to sem_init(3) With pshared=1, you can synchronize among all tasks (possibly both threads and processes) that have access to the region of shared memory that you've placed the semaphore at.
Pthread mutexes work on threads.
(Pthreads also has condition variables which are functionally equivalent to semaphores).
The thread versions can have better performance as it's the process that can manage the shared counter, whereas with semaphores, the kernel has to do it.
Synchronization is about determining whether a task can go on or has to wait in coordination with other tasks, and it involves communication, e.g., via incrementing and decrementing semaphores, which are shared among tasks.
Related
I have 2 threads, the producer thread receives some data over an internet socket, constructs an object and adds it to a queue and the consumer thread pops items from that same queue and does some processing.
Since both threads modify the queue (by enqueuing and dequeuing respectively) I think using a mutex in both threads should be sufficient. Is this a good approach or should I use a semaphore? If so, why?
Either can be made to work. It depends on details of the actual design. How many producer/consumer requests will you allow simultaneously? Is it truly limited to two threads, or will the code spawn others as more requests occur?
I found this short blog interesting. It discusses and compares both the Mutex and Semaphore and may give you some ideas.
"A mutex object allows multiple process threads to access a single shared resource but only one at a time. On the other hand, semaphore
allows multiple process threads to access the finite instance of the
resource until available. In mutex, the lock can be acquired and
released by the same process at a time."
Examples in C
One queue accessed by multiple threads using mutex
Blocking Queue in C++ using semaphores (Not C specific, but concept will be same.)
I want to use anonymous semaphores in shared memory to synchronise multiple processes.
While multiple POSIX operating systems provide anonymous semaphores (through sem_init and associated functions), macOS doesn't support them.
I discovered that the Mach semaphores are available on macOS, but in semaphore_create, I see no equivalent to to the pshared parameter of sem_init and I have a hard time finding documentation indicating that Mach/XNU semaphores can actually be used for synchronising processes and not just threads. Is it possible?
Here is how I am using the semaphores currently (note that sema is a structure that contains different kinds of semaphores depending on the target OS, for instance it contains a sem_t semaphore on Linux):
For initialising the semaphore:
task_t task = current_task();
semaphore_create(task, &sema->semaphore, SYNC_POLICY_FIFO, value)
For waiting on it:
semaphore_wait(sema->semaphore);
For waiting on it with a timeout (equivalent functionality to sem_timedwait on Linux):
mach_timespec_t time;
time.tv_sec = seconds;
time.tv_nsec = useconds;
semaphore_timedwait(sema->semaphore, time);
For signalling it:
semaphore_signal(sema->semaphore);
All of the above code examples are actually contained in corresponding functions that allow to provide an OS agnostic API, the test calls I'm doing on these functions on Linux work as expected, but on macOS the process waiting on the semaphore is never signaled from the other process.
What am I doing wrong?
Edit
I found this in the docs:
Semaphores can be used any place where mutexes can occur. This
precludes their use in interrupt handlers or within the context of the
scheduler, and makes it strongly discouraged in the VM system.
What does the "context of the scheduler" refer to exactly? Would it be if you was writing your own scheduler or does this refer to any interaction with the scheduler (eg: processes).
In all cases, am I right that VM means that it shouldn't be used with processes that make use of the virtual memory, in other terms... all user processes?
Mach semaphores can be used across processes, but it requires moving their port handles with mach_msg to another process, and it's a bit of a pain.
But - why go there, when you have not POSIX semaphores, but Sys V ones? sem_init may not be available, but Sys V semaphores are fully supported.
Specifically you're looking at the following sys calls:
semsys
semctl
semget
semop : on top of which you have sem_post(2) and a few other sem_* operations
sem_unlink
sem_close
sem_open takes a semaphore path on the filesystem, creating a named semaphore which will have visibility in your other process as well. sem_wait, etc..
I have two processes that will be running, one that will be reading from shared memory (mmap) and one that will be writing to that shared memory (mmap). These processes are started separately in two different terminals, but they need to be synchronized so that while one process is writing, it writes the full amount before the other process reads from the memory. All of the posts I have seen relating to shared memory mutex locks have been spawning threads/processes from a single main program. Is there any way to create a shared mutex lock that can be used by two separate programs?
Sorry, but you are out of luck. Pthreads library does not have a concept of 'named' mutex, so two independent processes can't reliably share one.
Yes, you can create a mutex in shared memory and than use this mutex from the other process, but there is no way you can ensure the mutex is fully initialized by the first process when you are checking it in the second. For trully independent programms I strongly recommend using semaphores.
You can create a shared mutex into an mmapped file. If you're using Linux and have a sufficiently new kernel, you can even create an unlinked temp file, mmap it; initialize the mutex and only then link it to the final location. Or you can use file locking to deny access to it until the initialization has been completed.
The semaphore example from pthread_mutexattr_init POSIX manuals at linux.die.net did work on my Linux 4.2.0-27 Ubuntu.
I don't really get the difference between semaphores and condition variables...
What can be used with threads and what can't, when should I use what?
In windows semaphores are used for inter-process thread synchronization. Whereas, on the other hand condition variable can only be used for thread synchronization in a process. Visit this link for information on some thread synchronization techniques used in windows:
http://www.codeproject.com/Articles/7953/Thread-Synchronization-for-Beginners
Also, condition variables are atomic in nature. If you are a windows programmer, you can use critical sections for thread synchronization in a process.
I have 3 questions about thread and process communication.
Can the Linux function msgget(), msgsnd(), and msgrcv() be invoked by multiple threads in one process? These functions in different threads are to attempt to access(r/w) one process' message queue. Are all race conditions supposed to be taken care by the system? If not, is there any good method to support threads and send a message to its main thread(process)?
Can semop() function be used to synchronize threads in one process?
There is a shared memory which have the following entities to access.
process
several threads in one process.
Do I have to use semaphore of inter-process level and a semaphore of threads level at the same time? Any simple way to handle this?
A lot of question. :) thanks.
Can the Linux function msgget(), msgsnd(), and msgrcv() be invoked by multiple threads in one process?
You do not need to worry about race conditions, the system will take care of that, there is no race condition with these calls.
can semop() function be used to synchronize threads in one process?
Yes, read more in the documentation
Do I have to use semaphore of inter-process level and a semaphore of threads level?
Any resource which is shared globally among threads or processes is subject to race conditions due to one or more threads or processes trying to access it at the very same time, So you need to synchronize the access to such a shared global resource.