Can you use a pipe to share threads between two processes in C? I am writing my own shell and want to know if this implementation would be more optimal, if it is even possible?
Each thread is specific to a process and share memory of the calling thread.
If you want efficient inter process communication, you can use shared memory.
See http://man7.org/linux/man-pages/man7/shm_overview.7.html
Related
I'm taking over some C code running in Linux (Centos) with extensive use of semaphores.
The way the code is written :
./Program1
This program launches a bunch of processes which makes use of mutexes and semaphores.
./Program2
This program also launches a bunch of processes which makes use of mutexes and semaphores.
I've realised that Program1 and Program2, they make use of semaphores with the same names.
In Linux C programming, can different programs use the same semaphores?
My guess is no, but the same naming is confusing the hell out of me. They are using the same source code to launch and handle the semaphores.
The semaphores are invoked using the following commands:
semget
semctl
semop
I've read that these are called processes semaphores.. if Program1 creates SEMAPHORE1, can Program2 access SEMAPHORE1?
Appreciate any help here, thanks!
Assuming you mean named semaphores (or even unnamed semaphores stored in shared memory, both which can be created with sem_open), they generally are shared amongst processes.
Semaphores using semget and related calls use an ID key rather than a name but their usage patterns are similar.
Semaphores are one of the IPC (inter-process communication) methods.
You can create a one-process-only semaphore by using an unnamed variant in non-shared memory and this will only be accessible to threads of the given process but, in my experience, that's not a common use case. The semget family of calls can also give you process-private semaphores.
Mutexes, on the other hand, tend to be more used within a single process for inter-thread communication but there is even a variant of them that can work inter-process.
You create a pthread_mutexattr (attribute) which allows sharing of mutexes and then use that attribute when initialising the mutex you want to share. Obviously, the mutex needs to be in shared memory so that multiple processes can get at it.
I'm working with semaphores in C , especifically to control the access to a shared memory zone in linux. but there is one thing that I can't understand.
I am using a mutex to control the access to a specific zone because i have 2 processes that must read/write from that zone. the thing is, when we use the fork() to create a new child process, the whole program is "copied" to another program as if they were two seperate programs right ? so, when i do V(mutex) in one process, how does the other one know he can't access ?
I know its a noob question but nobody could explain this to me until now.
After the fork neither process is going to know about the memory actions of the other because they are separate copies. You have to put your shared variables in shared memory, including mutexes and semaphores. Then all the processes are operating on the same resource.
For unrelated (i.e. non-forked) process there are usually system facilities (e.g. named semaphores) that each process can open based on a path name or similar method that each can use to find and use the resource.
You synchronisation objects must be placed in process shared memory, for example created with mmap (... MAP_ANONYMOUS ...). In addition, they must have the PTHREAD_PROCESS_SHARED attribute set, for example, by using pthread_mutexattr_setpshared.
See here:
Semaphores and Mutex for Thread and Process Synchronization
So mutex in practice is often used in threads, which makes sharing trivial. For processes however, mutex could be stored as a part of the shared mem.
For semaphores however, linux has built in library, which identifies global semaphores by keys. See below.
http://beej.us/guide/bgipc/output/html/multipage/semaphores.html
Or you can use other IPC to sync. Signals, for example.
Hope this helps.
I am trying to work out a way to synchronize two processes which share data.
Basically I have two processes linked using shared memory. I need process A to set some data in the shared memory area, then process B to read that data and act on it.
The sequence of events I am looking to have is:
B blocks waiting for data available signal
A writes data
A signals data available
B reads data
B blocks waiting for data not available signal
A signals data not available
All goes back to the beginning.
In other terms, B would block until it got a "1" signal, get the data, then block again until that signal went to "0".
I have managed to emulate it OK using purely shared memory, but either I block using a while loop which consumes 100% of CPU time, or I use a while loop with a nanosleep in it which sometimes misses some of the signals.
I have tried using semaphores, but I can only find a way to wait for a zero, not for a one, and trying to use two semaphores just didn't work. I don't think semaphores are the way to go.
There will be numerous processes all accessing the same shared memory area, and all processes need to be notified when that shared memory has been modified.
It's basically trying to emulate a hardware data and control bus, where events are edge rather than level triggered. It's the transitions between states I am interested in, rather than the states themselves.
So, any ideas or thoughts?
Linux has its own eventfd(2) facility that you can incorporate into your normal poll/select loop. You can pass eventfd file descriptor from process to process through a UNIX socket the usual way, or just inherit it with fork(2).
Edit 0:
After re-reading the question I think one of your options is signals and process groups: start your "listening" processes under the same process group (setpgid(2)), then signal them all with negative pid argument to kill(2) or sigqueue(2). Again, Linux provides signalfd(2) for polling and avoiding slow signal trampolines.
If 2 processes are involved you can use a file , shared memory or even networking to pass the flag or signal. But if the processes are more, there may be some suitable solutions in modifying the kernel. There is one shared memory in your question, right ?! How the signals are passed now ?!
In linux, all POSIX control structures (mutex, conditions, read-write-locks, semaphores) have an option such that they also can be used between processes if they reside in shared memory. For the process that you describe a classic mutex/condition pair seem to fit the job well. Look into the man pages of the ..._init functions for these structures.
Linux has other proper utilities such as "futex" to handle this even more efficiently. But these are probably not the right tools to start with.
1 Single Reader & Single Writer
1 Single Reader & Single Writer
This can be implemented using semaphores.
In posix semaphore api, you have sem_wait() which will wait until value of the semaphore count is zero once it is incremented using sem_post from other process the wait will finish.
In this case you have to use 2 semaphores for synchronization.
process 1 (reader)
sem_wait(sem1);
.......
sem_post(sem2);
process 2(writer)
sem_wait(sem2);
.......
sem_post(sem1);
In this way you can achieve synchronization in shared memory.
Is there a way to pass data (ex: int value) from one process to another process in c?
In my experience, we just can send signal from one process to another. But looks like there is no way to "attach" some information along with that signal to another process.
With the sigqueue function, you can pass a single integer or pointer along with a signal (but keep in mind, pointers will be useless if the target of the signal is another process, since different processes don't share address space).
Some other methods are pipes, shared memory (POSIX or SysV style), files, ...
You can use one of the various Inter Process Communication Mechanisms available.
Use Google. As a reference you can also look here
A clean, portable, powerful way is use Socket.
You can use pipes to do that. The main purpose of pipes is to communicate data between different processes.
Pipes are the simplest mechanism offered by the operating system for inter-process communication. A pipe is a communication buffer between two processes: it has two descriptors, one for writing another for reading. Write and read operations are done in a FIFO order (first-in-first-out).
There are two kinds of pipes: unnamed pipes and named pipes (also known as FIFOs).
Unnamed pipes only allow communication between hierarchically related processes (parent and child processes);
Named pipes allow the communication between any process. A special file is created in the file-system through
If you want some example code just go here: http://pastebin.com/1W216nyN
I think we can use global variable between process, not sure but. If any one tried then please let me know. If we use a header which contain extern valriable , we can use this in another main() which is nothing but a independednt program (process). but we have to link the two main() together which executing.
my question is somewhat conceptual, how is parent process' data shared with child process created by a fork() call or with a thread created by pthread_create()
for example, are global variables directly passed into child process and if so, does modification on that variable made by child process effect value of it in parent process?
i appreciate partial and complete answers in advance, if i'm missing any existing resource, i'm sorry, i've done some search on google but couldn't find good results
thanks again for your time and answers
The semantics of fork() and pthread_create() are a little different.
fork() will create a new process, where the global variables will be separate between the parent and children. Most OS implementations will use copy-on-write semantics, meaning that both the parent and child process will use the same physical memory pages for all global variables until one of the processes attempts to edit the physical memory, at which point a copy of that page is made, so that now each process gets its own copy and does not see the other process's, so that the processes are isolated.
pthread_create() on the other hand, creates a new thread within the same process. The new thread will have a separate stack space from the other running threads of the same process, however the global variables and heap space are shared between all threads of the same process. This is why you often need a mutex to coordinate access to a shared piece of memory between multiple threads of the same process.
TL;DR version: with fork(), you don't see the other guy's changes; with pthread_create() you do.
A fork creates an almost exact copy of the calling process, including memory and file descriptors. Global variables are copied along with everything else, but they are not in any way linked to the parent process. Since file descriptors are also copied, parent and child can interact via these (as long as they're setup properly, usually via pipe or socketpair).
There's a big difference between processes created by fork and between threads created with pthread_create. Processes don't share global variables and should communicate through pipes, sockets, or other tools provided by the OS. A good solution is MPI - which is a message-passing library for inter-process communication.
Threads are quite different. A thread created with pthread_create shares all the global variables with its caller. Moreover, the caller can pass an arbitrary structure into the thread, and this structure will also be shared. This means that one should be extremely careful when programming with threads - such amounts of sharing are dangerous and error prone. The pthread API provides mutexes and conditions for robust synchronization between threads (although it still requires practice and expertise to implement correctly).