I have two file descriptors created with socket() and both are connected to separate hosts. I want anything received on the first socket to be immediately sent on the second and vice versa.
I know can achieve this manually with a combination of select(), send() and recv(), but is there a more direct way to tell the kernel to simply pipe the output from one into the other?
No. You can use a tool like netcat which does this for you (so you don't have to write the code) but even netcat contains a loop that copies the data.
Related
I am currently measuring performance of named pipe to compare with another library.
I need to simulate a clients (n) / server (1) situation where server read messages and do a simple action for every written messages. So clients are writers.
My code now work, but if I add a 2nd writer, the reader (server) will never see the data and will not receive forever. The file is still filled with the non-read data at the end and read method will return 0.
Is it ok for a single named pipe to be written by multiple-process? Do I need to initialize it with a special flag for multiple-process?
I am not sure I can/should use multiple writers on a single pipe. But, I am not sure also it would be a good design to create 1 pipe for each clients.
Would it be a more standard design to use 1 named pipe per client connection?
I know about Unix Domain Name Socket and it will be use later. I need o make the named pipes work.
I have to make a simple IRC client/server programs for my IT school. The subject asks us to use select(2) for socket polling but forbids us to use O_NONBLOCK sockets.
Your server will accept multiple simultaneous connections.
Attention, the use of fork is prohibited. So you should imperatively use select
Your server must not be blocking.
This has nothing to do with non-blocking sockets, which are prohibited (so do not use fcntl(s, O_NONBLOCK))
I’m wondering if it is even possible to design a non-blocking server (which does not fork) with blocking sockets even using select(2).
Here is a simple example: let say we have a simple text protocol with one command per line. Each client has a buffer. When select(2) tells us a client is ready for read(2), we read until we found a \n in the client buffer, therefor we process the command. With non-blocking sockets, we would read until EAGAIN.
Let imagine now that we are using blocking sockets and a malicious client sends text with no line-break. select(2) tells us data is available, we then read(2) on the client. But we will never read the expected \n. Instead of returning EAGAIN, the syscall will block indefinitely. This is a denial of service attack.
Is it really possible to design a non-blocking server with blocking-sockets and select(2) (no fork(2))?
Yes, you read once from the socket that select tells you is ready. If the read contains the \n, then process that line. Otherwise, store any data that was received, and immediately go back to the select.
This means of course, that for every open socket, you must maintain state information, and a buffer of data read so far. This allows the code to process each read independently, without the need to finish a full line before going back to the select.
It's impossible.
select() blocks, and therefore so does any program that calls it.
The behaviour defined by Posix for send() in blocking mode is that it blocks until all the data supplied has been transferred to the socket send buffer. Unless you're going to delve into low-water marks and so on, it is impossible to know in advance whether there is enough room in he socket send buffer for any given send() to complete without blocking, and therefore impossible for any program that calls send() not to block.
Note that select() doesn't help you with this. It can tell when you when there is some room, but not when there is enough.
I am doing some network testing, and I am connecting between linux boxes with 2 small C programs that are just using function:
connect()
After connection some small calculations are made and recorded to local file, and I instruct one of the programs to close the connection and then run a netcat listener on the same port. The first program then retries the connection and connects to netcat.
I wondered if someone could advise if it is possible to maintain the initial connection whilst freeing the port and pass the connection to netcat on that port (so that the initial connection is not closed).
Each TCP connection is defined by the four-tuple (target IP address, target port, source IP address, source port), so there is no need to "free up" the port on either machine.
It is very common for a server process to fork() immediately after accept()ing a new connection. The parent process closes its copy of the connection descriptor (returned by accept()), and waits for a new connection. The child process closes the original socket descriptor, and executes the desired program or script that should handle the actual connection. In many cases the child moves the connection descriptor to standard input and standard output (using dup2()), so that the executed script or program does not even need to know it is connected to a remote client: everything it writes to standard output is sent to the remote client, and everything the remote client sends is readable from standard input.
If there is an existing process that should handle the connection, and there is an Unix domain socket connection (stream, datagram or seqpacket socket; makes no difference) between the two processes, it is possible to transfer the connection descriptor as an SCM_RIGHTS ancillary message. See man 2 sendmsg, man 2 recvmsg, man 3 cmsg, and man 7 unix for details. This only works on the same machine over an Unix domain socket, because the kernel actually duplicates the descriptor from one process to the other; really, the kernel does some funky magic to make this happen.
If your server-side logic is something like
For each incoming connection:
Do some calculations
Store calculations into a file
Store incoming data from the connection into a file (or standard output)
then I recommend using pthreads. Just create the desired number of threads, have all of them wait for an incoming connection by calling accept() on the listening socket, and have each thread handle the connection by themselves. You can even use stdio.h I/O for the file I/O. For more complex output -- multiple statements per chunk --, you'll need a pthread_mutex_t per output stream, and remember to fflush() it before releasing the mutex. I suspect a single multithreaded program that does all that, and exits nicely if interrupted (SIGINT aka CTRL+C), should not exceed three hundred lines of C.
If you only need to output data from a stream socket, or if you only need to write input to it, you can treat it as a file handle.
So in a Posix program you can use dup2() to duplicate the socket handle to the value 1, which is standard output. Then close the original handle. Then use exec() to overwrite your program with "cat", which will write the output from standard input aka filehandle 1 aka your socket.
I'm still new to C socket programming but I was able to create some simple client and server programs.
I'm programming a server that listens for TCP connections, its duty is answering to clients' requests and then close the communication when the client sends a special sequence of bytes (or when it disconnects, of course).
I started coding the server using the accept() function inside an endless loop: the server waits for a client, accept()'s it, does all the stuff, close()'s the socket descriptor at the end and goes back again waiting to accept a new client.
Since I want to serve one client at a time I called the listen function in this way: listen(mysocket, 1);
Everything worked pretty well, but then a new problem came out. The server part explained above runs in a separated thread (let's call it thread #2) and the main thread (thread #1) must be able to tell it to terminate. I created a global variable then, if this variable is set to 1 (by thread #1) thread #2 must terminate. The problem is that thread #2 gets stuck when the accept() function is called, thus it can't periodically check the global variable.
I clearly needed a timeout value for that function: "if there isn't a connection to accept, check the value of the global variable, continue waiting for new connection if set to 0 or terminate if set to 1".
I googled for a solution then and found that the select() function does the thing that I need. It's a little bit different though, I discovered for the first time the fd_set and all the FD_* macros. I modified the server part to make it work with the select() function and everything works really nice, but here comes the last problem, the one that I'm not able to solve.
If call the listen function this way: listen(socket, 1); but the server still accepts and serves multiple connections at the same time. Does this depend because select() works with fd_set's? I'm using some examples I found on the web and, when a connection is accepted, it creates a new socket descriptor that goes in the set with all the others.
I'd like to accept the connection of just one client, I wrote a simple code that recognizes if the connecting client should be served or not, but is there a way to disconnect it server side? I know that I have to use the close() function to close a socket descriptor, but when using select() I'm working with fd_set's and I don't really know what to do to close them.
Or, is there a way to limit the number of socket descriptors in a set? I found the FD_SETSIZE macro, but I wasn't able to make it work and I'm not even sure if it fixes the problem.
Thank you for your time!
The listen() function has a backlog argument that determines how many incoming request may be pending before they may be turned away. This is worded carefully so that the OS implementation can support more than what you specify in the listen() call. You may not be able to control the exact number of backlogged connections.
If you must support only one client at a time, then accept a second connection but tell the new client that the connection is not available at this time, and then close the new connection. This also has the benefit that you have the opportunity to tell the client why the connection is not available.
You can put the listen() socket in the fd_set, too. (I don't understand from your question if you already do this) If the select indicates that listen socket is readable, you can call accept on the listen socket. accept returns exactly one fd. You can put the returned fd into the fd_set, too. Threads are more or less orthogonal to this. (systemcalls are atomic), but you can of course screw things up.
The backlog parameter for listen is more or less unrelated; it just specifies how many embryonic sockets the system should maintain simultaneously (the connection buildup uses resources, even before the socket becomes usable, as indicated by listen returning the fresh fd) Once the three-way handshake is completed accept will return the fd, and the embryonic connections are born as the new socket fd.
The getdtablesize() macro is a remainder from old times. In modern implementations it is based on getrlimit().
The underlying problem with FD_SETSIZE is that fd_set must be a lvalue, such that it can be assigned. So it must be fixed size. (it is probably an array inside a struct) To avoid fixed sized structures you could use poll() instead of select() (or use multiple smaller fd_sets ;-)
if you're using the windows API , you could try Critical Sections.
I want to write a asynchronous socket server in C, but before I do I'm doing some research. Looking at the select() socket example shown here: http://www.gnu.org/s/hello/manual/libc/Server-Example.html#Server-Example I can see that the example program will only accept one client per select loop (if I'm reading it right). So if there are 20 clients and two more try to connect, will it only accept the 21st client then process the other 20 (worst case, assuming all 20 others require reading) and THEN accept the 22nd? Would it be better if I break the loop after accepting a client so it can select() again and take care of all pending clients before processing connected ones? Or does that defeat the purpose of using select()? Thanks.
The server pattern shown in the example you linked to is fine; there isn't any significant problem introduced by the loop only accepting one socket per iteration.
The key point to keep in mind is that in a well-designed select() loop, the only place the process should ever block is inside the select() call. In particular, if coded correctly, the server will never block inside send(), recv(), or accept(). It's best to set all sockets to non-blocking mode (via fcntl(fd, F_SETFL, O_NONBLOCK)) in order to guarantee this behavior.
Given that, the precise ordering of "which clients get serviced first within any particular event loop iteration" doesn't matter, because all clients' sockets get handled very soon after they have data ready-for-read (or buffer space ready-for-write), and all new connections get accepted quickly.
select() leaves it up to the programmer to decide how to handle notifications. One call to select() can indicate that any or all sockets have bytes to be read and some connection requests to process. The application can process all the notifications before calling select() or process one notification before calling select() again.
You can use poll() on the listening socket after accept() to see if there are more clients waiting to connect.
Note the the number of concurrent connect-attempts is handled by the backlog parameter to listen(server_sock, backlog) - backlog is 1 in the sample you reference.