I'm currently working on a school projet (C language) which aims to make processes who can write into shared memory, one by one.
I can't use multi-threading neither fork.
I can have only one executable, which must be run as many times as we want.
For example if i do : ./main john then ./main toto, John would be able to write in shared memory while Toto has to wait that John unlock the semaphore.
But I just can't find any documentation who explain clearly how it works without doing threads, forks etc.
May someone can help me ? Thanks
Assuming that Linux is used:
Because multi-threading is not allowed, you may ignore pthread-semaphores.
Maybe system-V IPC semaphores are what you are looking for:
https://man7.org/linux/man-pages/man7/sysvipc.7.html
Those can be used between processes.
The system-V IPC also provides functions for shared memory handling.
Posix-semaphores and shm API are also another alternative:
https://linux.die.net/man/7/sem_overview
https://man7.org/linux/man-pages/man7/shm_overview.7.html
https://opensource.com/article/19/4/interprocess-communication-linux-storage has examples for using POSIX semaphores to synchronize shared memory access (in the second part of that article, past the file lock example).
It is not unlike synchronizing threads, except that it uses operating system facilities for inter process communication.
POSIX semaphores are available on POSIX systems, notably many *nixes.
Related
I'm taking over some C code running in Linux (Centos) with extensive use of semaphores.
The way the code is written :
./Program1
This program launches a bunch of processes which makes use of mutexes and semaphores.
./Program2
This program also launches a bunch of processes which makes use of mutexes and semaphores.
I've realised that Program1 and Program2, they make use of semaphores with the same names.
In Linux C programming, can different programs use the same semaphores?
My guess is no, but the same naming is confusing the hell out of me. They are using the same source code to launch and handle the semaphores.
The semaphores are invoked using the following commands:
semget
semctl
semop
I've read that these are called processes semaphores.. if Program1 creates SEMAPHORE1, can Program2 access SEMAPHORE1?
Appreciate any help here, thanks!
Assuming you mean named semaphores (or even unnamed semaphores stored in shared memory, both which can be created with sem_open), they generally are shared amongst processes.
Semaphores using semget and related calls use an ID key rather than a name but their usage patterns are similar.
Semaphores are one of the IPC (inter-process communication) methods.
You can create a one-process-only semaphore by using an unnamed variant in non-shared memory and this will only be accessible to threads of the given process but, in my experience, that's not a common use case. The semget family of calls can also give you process-private semaphores.
Mutexes, on the other hand, tend to be more used within a single process for inter-thread communication but there is even a variant of them that can work inter-process.
You create a pthread_mutexattr (attribute) which allows sharing of mutexes and then use that attribute when initialising the mutex you want to share. Obviously, the mutex needs to be in shared memory so that multiple processes can get at it.
I'm working with semaphores in C , especifically to control the access to a shared memory zone in linux. but there is one thing that I can't understand.
I am using a mutex to control the access to a specific zone because i have 2 processes that must read/write from that zone. the thing is, when we use the fork() to create a new child process, the whole program is "copied" to another program as if they were two seperate programs right ? so, when i do V(mutex) in one process, how does the other one know he can't access ?
I know its a noob question but nobody could explain this to me until now.
After the fork neither process is going to know about the memory actions of the other because they are separate copies. You have to put your shared variables in shared memory, including mutexes and semaphores. Then all the processes are operating on the same resource.
For unrelated (i.e. non-forked) process there are usually system facilities (e.g. named semaphores) that each process can open based on a path name or similar method that each can use to find and use the resource.
You synchronisation objects must be placed in process shared memory, for example created with mmap (... MAP_ANONYMOUS ...). In addition, they must have the PTHREAD_PROCESS_SHARED attribute set, for example, by using pthread_mutexattr_setpshared.
See here:
Semaphores and Mutex for Thread and Process Synchronization
So mutex in practice is often used in threads, which makes sharing trivial. For processes however, mutex could be stored as a part of the shared mem.
For semaphores however, linux has built in library, which identifies global semaphores by keys. See below.
http://beej.us/guide/bgipc/output/html/multipage/semaphores.html
Or you can use other IPC to sync. Signals, for example.
Hope this helps.
I am trying to create a shared memory which will be used by multiple processes. these processes communicate with each other using MPI calls (MPI_Send, MPI_Recv).
I need a mechanism to control the access of this shared memory I added a question yesterday to see if MPI provides any facility to do that. Shared memory access control mechanism for processes created by MPI , but it seems that there is no such provision by MPI.
So I have to choose between named semaphore or flock.
For named semaphore if any of the process dies abruptly without calling sem_cloe(), than that semaphore always remains and can be seen by ll /dev/shm/. This results in deadlock sometimes(if I run the same code again!), for this reason I am currently thinking of using flock.
Just wanted to confirm if flock is best suited for this type of operation ?
Are there any disadvantages of using flock?
Is there anything else apart from named semaphore and flock that can be used here ?
I am working on C under linux.
You can also use a POSIX mutex in shared memory; you just have to set the "pshared" attribute on it first. See pthread_mutexattr_setpshared. This is arguably the most direct way to do what you want.
That said, you can also call sem_unlink on your named semaphore while you are still using it. This will remove it from the file system, but the underlying semaphore object will continue to exist until the last process calls sem_close on it (which happens automatically if the process exits or crashes).
I can think of two minor disadvantages to using flock. First, it is not POSIX, so it makes your code somewhat less portable, although I believe most Unixes implement it in practice. Second, it is implemented as a system call, so it will be slower. Both pthread_mutex_lock and sem_wait use the "futex" mechanism on Linux, which only does a system call when you actually have to wait. This is only a concern if you are grabbing and releasing the lock a lot.
i am porting program from GNU/Linux to VxWorks, i am having a problem regarding to fork() and i can't find alternatives ; VxWork's API provide two useful calls taskSpawn( ) and rtpSpawn( ) to spwan RTP/Task but these API do NOT duplicate the calling process (fork does). does anyone have idea about porting/workaround fork() to Vxworks?
VxWorks API Reference
If I remember my vxWork correctly - you can't. fork() requires virtual memory management, something I believe VxWorks 5.5 does not provide, at least not the full semantics needed to implement fork. (it was added in vxwork 6 though if I am not mistaken).
I don't know anything about VxWorks memory model but it may be impossible to port fork. The reason for this is that when a process is forked, the memory of the original process is copied into the new process. Importantly, the two processes must use the same internal virtual addresses otherwise things like pointers are going to break.
Obviously, the two processes must have different physical addresses which means in order to fork, one requires a platform that has a memory management unit (MMU) and the kernel must support a memory model that allows programs to share the same virtual addresses. This is why there is no fork equivalent for creating a new thread.
In addition to this, copying a large process can be very expensive. So Linux uses what is called copy-on-write. This means all fork does is mark all memory pages read-only. When a write is attempted, an interrupt is generated and only then is the memory page copied.
It is unlikely that a Real Time Operating System RTOS will support copy-on-write because it means that memory write times are not bounded and violates the realtime guarantees of the OS.
It is therefore much easier not to support fork at all just implement APIs for spawning brand new processes without the duplication.
As part of my homework project I had to implement a library that implements a pipe using shared memory. Both the anonymous and named pipe .
I chose the Posix implementation , meaning , I used the following calls :
mmap()
shm_open()
ftruncate()
shm_unlink()
For semaphores and synchronization
sem_init()
sem_getvalue()
sem_wait()
sem_post()
(I might forgot one or two calls)
My TA told me that he prefers that I'd implement that library with System V version ,
however since I'm in the middle of my exams , I have no extra time to do that (would take a least a week , I guess) .
My questions are :
What's is the difference between a pipe that's implemented in Posix vs a pipe that's implemented in System-V ?
What calls would I need for implementing the above library using the Sys V version ?
Thanks
With regards to question (1) - there should be no differences between a Posix implementation of a pipe and a System V implementation of a pipe - if you're implementing a library with a set of routines, then the user should not see any difference between the two implementations.
For the developer, the shared memory calls that are used for System V are: shmget to create shmat to access and shmctl to destroy (shmdt is used to unmap the shared memory segment from the current process).
You use a call to ftok which converts a filename into the key that you will use.
Note that the key is the magic uniquification item that distinguishes between the different shared memory/semaphore items.
For getting semaphores you use semget, to lock and unlock you use semop and to destroy it you use semctl.
System V semaphores and shared memory segments can survive beyond the execution of a program - i.e. if the program terminates without destroying them, then they will remain in the system until they are destroyed or recreated either programmatically, or using ipcrm