I'm writing a C program in which I need to pass messages between child processes and the main process. But the thing is, I need to do it without using the functions like msgget() and msgsnd()
How can I implement? What kind of techniques can I use?
There are multiple ways to communicate with children processes, it depends on your application.
Very much depends on the level of abstraction of your application.
-- If the level of abstraction is low:
If you need very fast communication, you could use shared memory (e.g. shm_open()). But that would be complicated to synchronize correctly.
The most used method, and the method I'd use if I were in your shoes is: pipes.
It's simple, fast, and since pipes file descriptors are supported by epoll() and those kind of asynchronous I/O APIs, you can take advantage from this fact.
Another plus is that, if your application grows, and you need to communicate with remote processes (processes that are not in your local machine), adapting pipes to sockets is very easy, basically it's still the same reading/writing from/to a file descriptor.
Also, Unix-domain sockets (which in other platforms are called "named pipes") let you to have a server process that creates a listening socket with a very well known name (e.g. an entry in the filesystem, such as /tmp/my_socket) and all clients in the local machine can connect to that.
Pipes, networking sockets, or unix-domain sockets are very interchangeable solutions, because - as said before - all involve reading/writing data from/to a file descriptor, so you can reuse the code.
The disadvantage with a file descriptor is that you're writing data to a stream of bytes, so you need to implement the "message streaming protocol" of your messages by yourself, to "unstream" your messages (marshalling/unmarshalling), but that's not so complicated in the most of the cases, and that also depends on the kind of messages you're sending.
I'd pass on other solutions such as memory mapped files and so on.
-- If the level of abstraction is higher:
You could use a 3rd party message passing system, such as RabbitMQ, ZMQ, and so on.
I am implementing a very basic C server that allows clients to chat. Right now I am using fork(), but I am having trouble having clients see each others' messages.
It also seems that all clients get the same file descriptor from accept(). Basically, I have a while loop where I test if someone wants to connect with select(), accept() their connection, and fork(). After that I read input and try to pass them to all users (whom I am keeping in a list). I can copy/paste my code if necessary.
So, is it possible to have the clients communicate with processes, or do I have to use pthreads?
Inter-process communication -IPC- (in general) don't care about client vs server (except at connect phase). A given process can have both a client and a server role (on different sockets), and would use poll(2) or the older select on several sockets in some event loop.
Notice that processes have each their own virtual address space, while threads share the same virtual address space (the one of their containing process). Read some pthread tutorial, and some book on POSIX programming (perhaps the old ALP). Be aware that a lot of information regarding processes can be queried on Linux thru /proc/ (see proc(5) for more). In particular, the virtual address space of process of pid 1234 can be obtained thru /proc/1234/maps and its opened file descriptors thru /proc/1234/fd/ and /proc/1234/fdinfo/ etc....
However, it is simpler to code a common server keeping the shared state, and dispatching messages to clients.
You could design a protocol where the clients have some way to initiate that IPC. For example, if all the processes are on the same machine, you could have a protocol which transmits a file path used as unix(7) socket address, or as fifo(7), and each "client" process later initiate (with some connect) a direct communication with another "client". It might be unwise to do so.
Look also into libraries like 0mq. They often are free software, so you can study their source code.
For many reasons, I would like to use unix domain sockets for IPC between two processes.
Each process reacts to asynchronous events of some specific kind from the outside world by writing to the socket and communicating this event to the second process and - at the same time - each process also needs to read data coming from the other socket to do some stuff. In substance, in this model there would be one socket and two threads per process: one for possibly blocking reads, and one for the writings.
I would like to know if it's possible to use unix domain sockets for concurrently reading and writing from/to each process independently, without making use of any explicitly locking in that safety would implicitly guaranteed by these kind of sockets. If yes, I'd also like to know where this guarantee is officially claimed.
The only relevant difference between an AF_LOCAL socket and an AF_INET socket is that AF_LOCAL sockets are local to the current computer. Creating an AF_LOCAL socket and binding it is no difference than creating an AF_INET socket and binding it to localhost.
The path used for binding AF_LOCAL sockets is only used for connecting to the socket, nothing else.
So if you create a connection-oriented AF_LOCAL socket (using SOCK_STREAM or SOCK_SEQPACKET) then each connection is unique and you can have multiple processes connecting through the same listening (passive) AF_LOCAL socket.
I want to create one server and one client(two separate programs) where in server creates
two named pipes(i guess thats the minimum required for bidirectional flow of traffic) and then
the client is started,and Client and server should be able to send and recieve data both ways
all the time(full duplex types).I think that would require me to have non-blocking named pipes.Would like some help as i have been able to create half duplex type of communication
but struggling to make a contineous seamless transfer of data between client and server happen.
Thanks
Possible options:
Local domain sockets: AF_LOCAL family with SOCK_STREAM, SOCK_DGRAM, SOCK_SEQPACKET type. The socket can be "in-memory", meaning you connect to it using a unique string, or it can be a socket file in the filesystem. It works just like any network socket, full-duplex.
Two pipes: one for reading, one for writing (vice versa for the other process). Might be a bit more complicated keeping track of two pipes, as opposed to the local domain socket.
Helpful link Check out the part on Pipes and the one on Unix Sockets.
Have you considered using select() to handle the reading named pipe?
What are they and how do they work?
Context happens to be SQL Server
Both on Windows and POSIX systems, named-pipes provide a way for inter-process communication to occur among processes running on the same machine. What named pipes give you is a way to send your data without having the performance penalty of involving the network stack.
Just like you have a server listening to a IP address/port for incoming requests, a server can also set up a named pipe which can listen for requests. In either cases, the client process (or the DB access library) must know the specific address (or pipe name) to send the request. Often, a commonly used standard default exists (much like port 80 for HTTP, SQL server uses port 1433 in TCP/IP; \\.\pipe\sql\query for a named pipe).
By setting up additional named pipes, you can have multiple DB servers running, each with its own request listeners.
The advantage of named pipes is that it is usually much faster, and frees up network stack resources.
--
BTW, in the Windows world, you can also have named pipes to remote machines -- but in that case, the named pipe is transported over TCP/IP, so you will lose performance. Use named pipes for local machine communication.
Unix and Windows both have things called "Named pipes", but they behave differently. On Unix, a named pipe is a one-way street which typically has just one reader and one writer - the writer writes, and the reader reads, you get it?
On Windows, the thing called a "Named pipe" is an IPC object more like a TCP socket - things can flow both ways and there is some metadata (You can obtain the credentials of the thing on the other end etc).
Unix named pipes appear as a special file in the filesystem and can be accessed with normal file IO commands including the shell. Windows ones don't, and need to be opened with a special system call (after which they behave mostly like a normal win32 handle).
Even more confusing, Unix has something called a "Unix socket" or AF_UNIX socket, which works more like (but not completely like) a win32 "named pipe", being bidirectional.
Linux Pipes
First In First Out (FIFO) interproccess communication mechanism.
Unnamed Pipes
On the command line, represented by a "|" between two commands.
Named Pipes
A FIFO special file. Once created, you can use the pipe just like a normal file(open, close, write, read, etc).
To create a named pipe, called "myPipe", from the command line (man page):
mkfifo myPipe
To create a named pipe from c, where "pathname" is the name you would like the pipe to have and "mode" contains the permissions you want the pipe to have (man page):
#include <sys/types.h>
#include <sys/stat.h>
int mkfifo(const char *pathname, mode_t mode);
According to Wikipedia:
[...] A traditional pipe is "unnamed" because it exists anonymously and persists only for as long as the process is running. A named pipe is system-persistent and exists beyond the life of the process and must be "unlinked" or deleted once it is no longer being used. Processes generally attach to the named pipe (usually appearing as a file) to perform IPC (inter-process communication).
Compare
echo "test" | wc
to
mkdnod apipe p
wc apipe
wc will block until
echo "test" > apipe
executes
This is an exeprt from Technet (so not sure why the marked answer says named pipes are faster??):
Named Pipes vs. TCP/IP Sockets
In a fast local area network (LAN) environment, Transmission Control Protocol/Internet Protocol (TCP/IP) Sockets and Named Pipes clients are comparable with regard to performance. However, the performance difference between the TCP/IP Sockets and Named Pipes clients becomes apparent with slower networks, such as across wide area networks (WANs) or dial-up networks. This is because of the different ways the interprocess communication (IPC) mechanisms communicate between peers.
For named pipes, network communications are typically more interactive. A peer does not send data until another peer asks for it using a read command. A network read typically involves a series of peek named pipes messages before it starts to read the data. These can be very costly in a slow network and cause excessive network traffic, which in turn affects other network clients.
It is also important to clarify if you are talking about local pipes or network pipes. If the server application is running locally on the computer that is running an instance of SQL Server, the local Named Pipes protocol is an option. Local named pipes runs in kernel mode and is very fast.
For TCP/IP Sockets, data transmissions are more streamlined and have less overhead. Data transmissions can also take advantage of TCP/IP Sockets performance enhancement mechanisms such as windowing, delayed acknowledgements, and so on. This can be very helpful in a slow network. Depending on the type of applications, such performance differences can be significant.
TCP/IP Sockets also support a backlog queue. This can provide a limited smoothing effect compared to named pipes that could lead to pipe-busy errors when you are trying to connect to SQL Server.
Generally, TCP/IP is preferred in a slow LAN, WAN, or dial-up network, whereas named pipes can be a better choice when network speed is not the issue, as it offers more functionality, ease of use, and configuration options.
Pipes are a way of streaming data between applications. Under Linux I use this all the time to stream the output of one process into another. This is anonymous because the destination app has no idea where that input-stream comes from. It doesn't need to.
A named pipe is just a way of actively hooking onto an existing pipe and hoovering-up its data. It's for situations where the provider doesn't know what clients will be eating the data.
Inter-process communication (mostly) for Windows Applications. Similar to using sockets to communicate between applications in Unix.
MSDN
Named pipes in a unix/linux context can be used to make two different shells to communicate since a shell just can't share anything with another.
Furthermore, one script instantiated twice in the same shell can't share anything through the two instances. I found a use for named pipes when coding a daemon that contains the start() and stop() function, and I wanted to use the same script to perform the two actions.
Without named pipes (or any kind of semaphore) starting the script in the background is not a problem. The thing is when it finishes you just can't access the instance in background.
So when you want to send him the stop command you just can't: running the same script without named pipes and calling the stop() function won't do anything since you are actually running another instance.
The solution was to implement two pipes, one READ and the other WRITE when you start the daemon. Then make him, among its other tasks, listen to the READ pipe. Then the Stop() function contains a command that will write a message in the pipe, that will be handled by the background running script that will perform an exit 0. This way our second instance of the same script has only on task to do: tell the first instance to stop.
This way one and only one script can start and stop itself.
Of course you have different ways to do it by triggering the stop via a touch for example. But this one is nice and interesting to code.
Named pipes is a windows system for inter-process communication. In the case of SQL server, if the server is on the same machine as the client, then it is possible to use named pipes to tranfer the data, as opposed to TCP/IP.