Related
I have written a proxy which also duplicates traffic. I am trying to duplicate network traffic to a replica server which should receive all the inputs and also process all the requests. However only the responses on the main server are visible to the client. The high level workflow is as follows
Thread 1. Take input from client forward it to a pipe in non-blocking way, and to the server
Thread 2. Read from server and send to client
Thread 3. Read from pipe and forward to replica server
Thread 4. Read from replica server and drop
The code is available in this gist: https://gist.github.com/nipunarora/679d49e81086b5a75195ec35ced646de
The test seems to work for smaller data and transactions, but I seem to be getting the following error when working with iperf and larger data sets:
Buffer overflow? : Resource temporarily unavailable
The specific part in the code where the problem is stemming from:
void forward_data_asynch(int source_sock, int destination_sock) {
char buffer[BUF_SIZE];
int n;
//put in error condition for -1, currently the socket is shutdown
while ((n = recv(source_sock, buffer, BUF_SIZE, 0)) > 0)// read data from input socket
{
send(destination_sock, buffer, n, 0); // send data to output socket
if( write(pfds[1],buffer,n) < 0 )//send data to pipe
{
//fprintf(stats_file,"buffer_overflow \n");
//printf("format string" ,a0,a1);
//int_timeofday();
perror("Buffer overflow? ");
}
//DEBUG_PRINT("Data sent to pipe %s \n", buffer);
}
shutdown(destination_sock, SHUT_RDWR); // stop other processes from using socket
close(destination_sock);
shutdown(source_sock, SHUT_RDWR); // stop other processes from using socket
close(source_sock);
}
The reading process is as follows:
void forward_data_pipe(int destination_sock) {
char buffer[BUF_SIZE];
int n;
sleep(10);
//put in error condition for -1, currently the socket is shutdown
while ((n = read(pfds[0], buffer, BUF_SIZE)) > 0)// read data from pipe socket
{
//sleep(1);
//DEBUG_PRINT("Data received in pipe %s \n", buffer);
send(destination_sock, buffer, n, 0); // send data to output socket
}
shutdown(destination_sock, SHUT_RDWR); // stop other processes from using socket
close(destination_sock);
}
Please note, the pipe has been defined as follows:
/** Make file descriptor non blocking */
int setNonblocking(int fd)
{
int flags;
/* If they have O_NONBLOCK, use the Posix way to do it */
#if defined(O_NONBLOCK)
/* Fixme: O_NONBLOCK is defined but broken on SunOS 4.1.x and AIX 3.2.5. */
if (-1 == (flags = fcntl(fd, F_GETFL, 0)))
flags = 0;
return fcntl(fd, F_SETFL, flags | O_NONBLOCK);
#else
/* Otherwise, use the old way of doing it */
flags = 1;
return ioctl(fd, FIOBIO, &flags);
#endif
}
Could anyone help in fixing what could be the reason of the error?
The problem in your case is that data is sent too fast to the socket that has been set to non-blocking mode. You have several options:
Accept the fact that data may be lost. If you do not want to delay the processing on the main server, this is your only option.
Don't set the socket to non-blocking mode. The default mode, blocking, seems like a better fit for your application if you don't want data to be lost. However, this will also mean that the system may be slowed down.
Use poll(), select(), kqueue(), epoll(), /dev/poll or similar to wait until the socket has enough buffer space available. However, when using this, you should consider why you set the socket to non-blocking mode in the first place if you nevertheless want to block on it. This also leads to slowdown of the system.
I am using the preforking concept.
When I establish a socket in the server, the file descriptor value returned is 7. I know that it will allocate a available number for the file descriptor.
When I made the same child process to accept for new connections on the same socket, It wont accept() the connection.
But when I reset the value of file descriptor to 7, then It starts accepting the connections.
I am not finding the reason behind it. Can anyone through some light on this.
My code looks similar to this
for (;;)
{
int session_fd=accept(server_fd,0,0);
if (session_fd==-1)
{
if (errno==EINTR) continue;
die("failed to accept connection (errno=%d)",errno);
}
handle_session(session_fd);
close(session_fd);
server_fd = 7;
}
When I do some read and write operations, the value of file descriptor goes on increasing.
This doesn't make sense, reading or writing should be performed on the same socket for which the system returned a handle.
So I am just curious to know about the significance of value of socket file descriptor.
There is no significance, it's a process specific handle which is represented by an integer, usually it increases by 1 each time you open and/or create a new socket, etc.
I'm going to go out on a limb and assume your code looks something like this, where you bind server_fd and then fork off a child to deal with each connection.
If so, the value 7 doesn't mean anything. It just so happens that at that stage in your program, the lower numbers are already in use. If you opened some other files or sockets before calling listen, server_fd would have a different number.
A socket is just its number, so when you reset the value of the file descriptor, you're reading from your original, bound socket, not the new one that results from the accept call. And I suspect you don't want to be listening on the same socket twice, though I've never tried.
if (listen(server_fd,SOMAXCONN)) {
die("failed to listen for connections (errno=%d)",errno);
}
for (;;) {
int session_fd=accept(server_fd,0,0);
if (session_fd==-1) {
if (errno==EINTR) continue;
die("failed to accept connection (errno=%d)",errno);
}
pid_t pid=fork();
if (pid==-1) {
die("failed to create child process (errno=%d)",errno);
} else if (pid==0) {
close(server_fd);
handle_session(session_fd);
close(session_fd);
_exit(0);
} else {
close(session_fd);
}
}
Sorry for bothering you all. I found the reason behind the issue with change of session_fd value.
session_fd is a global variable. Since it's a huge legacy code, I was not aware that the value of session_fd was changed by some other socket.
Thanks all for providing your inputs :)
I wrote in C a server - client chatroom.
The server creates a new pthread for every new connection to a client, this pthread waits for a message to receive, and sends this message to all the other clients (using a array of all the file descriptors). If a client wants to quit he informs the server and he will terminate the pthread and delete the file descriptor from the array
This works fine !, but:
if a client disconnects unexpected, by closing the terminal for example, the server won't delete the file descriptor from the array and when an other client wants to send a message i have an error because the pthread tries to send the message to a fd which isn't a connection any more
Now my question:
How can in test if the file descriptor of a client's socket is still active before i send a message ?
the part of my code (from the pthread):
for(i=0; i<*p_Nbr_Clients; i++){ // send the message to all the other clients
if (fd_array[i] != fd){ // <- i want to test the fd here
if ( send(fd_array[i], msg, strlen(msg), 0) == -1 ){
perror("Serveur: send");
}
}
}
Check the return value of the recv().
If the user terminated abnormally then return value should be zero 0.
Based on that you can close fd easily.
if(recv(fd,buffer,length,flag) == 0)
close(fd);
There is no standalone api to check whether socket is closed. Just try to send data to that socket.
send will return -1 if you write to a closed socket. and errno will be set to appropriately. You may got EBADF or ECONNRESET i guess. Check (Check connection open or closed ?(in C in Linux)) and (How to find out if a socket is closed)
for(i=0; i<*p_Nbr_Clients; i++){ // send the message to all the other clients
if (fd_array[i] != fd){ // <- i want to test the fd here
if ( send(fd_array[i], msg, strlen(msg), 0) == -1 ){
//perror("Serveur: send");
// something wrong, check errno to see more detail
// you need to include <errno.h> to use errno
close(fd_array[i]);
fd_array[i] = -1;// or something you define as not open
}
}
}
This is a question about socket programming for multi-client.
While I was thinking how to make my single client and server program
to multi clients,I encountered how to implement this.
But even if I was searching for everything, kind of confusion exists.
I was thinking to implement with select(), because it is less heavy than fork.
but I have much global variables not to be shared, so I hadn`t considered thread to use.
and so to use select(), I could have the general knowledge about FD_functions to utilize, but here I have my question, because generally in the examples on websites, it only shows multi-client server program...
Since I use sequential recv() and send() in client and also in server program
that work really well when it`s single client and server, but
I have no idea about how it must be changed for multi cilent.
Does the client also must be unblocking?
What are all requirements for select()?
The things I did on my server program to be multi-client
1) I set my socket option for reuse address, with SO_REUSEADDR
2) and set my server as non-blocking mode with O_NONBLOCK using fctl().
3) and put the timeout argument as zero.
and proper use of FD_functions after above.
But when I run my client program one and many more, from the second client,
client program blocks, not getting accepted by server.
I guess the reason is because I put my server program`s main function part
into the 'recv was >0 ' case.
for example with my server code,
(I`m using temp and read as fd_set, and read as master in this case)
int main(void)
{
int conn_sock, listen_sock;
struct sockaddr_in s_addr, c_addr;
int rq, ack;
char path[100];
int pre, change, c;
int conn, page_num, x;
int c_len = sizeof(c_addr);
int fd;
int flags;
int opt = 1;
int nbytes;
fd_set read, temp;
if ((listen_sock = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0)
{
perror("socket error!");
return 1;
}
memset(&s_addr, 0, sizeof(s_addr));
s_addr.sin_family = AF_INET;
s_addr.sin_addr.s_addr = htonl(INADDR_ANY);
s_addr.sin_port = htons(3500);
if (setsockopt(listen_sock, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(int)) == -1)
{
perror("Server-setsockopt() error ");
exit(1);
}
flags = fcntl(listen_sock, F_GETFL, 0);
fcntl(listen_sock, F_SETFL, flags | O_NONBLOCK);
//fcntl(listen_sock, F_SETOWN, getpid());
bind(listen_sock, (struct sockaddr*) &s_addr, sizeof(s_addr));
listen(listen_sock, 8);
FD_ZERO(&read);
FD_ZERO(&temp);
FD_SET(listen_sock, &read);
while (1)
{
temp = read;
if (select(FD_SETSIZE, &temp, (fd_set *) 0, (fd_set *) 0,
(struct timeval *) 0) < 1)
{
perror("select error:");
exit(1);
}
for (fd = 0; fd < FD_SETSIZE; fd++)
{
//CHECK all file descriptors
if (FD_ISSET(fd, &temp))
{
if (fd == listen_sock)
{
conn_sock = accept(listen_sock, (struct sockaddr *) &c_addr, &c_len);
FD_SET(conn_sock, &read);
printf("new client got session: %d\n", conn_sock);
}
else
{
nbytes = recv(fd, &conn, 4, 0);
if (nbytes <= 0)
{
close(fd);
FD_CLR(fd, &read);
}
else
{
if (conn == Session_Rq)
{
ack = Session_Ack;
send(fd, &ack, sizeof(ack), 0);
root_setting();
c = 0;
while (1)
{
c++;
printf("in while loop\n");
recv(fd, &page_num, 4, 0);
if (c > 1)
{
change = compare_with_pre_page(pre, page_num);
if (change == 1)
{
page_stack[stack_count] = page_num;
stack_count++;
}
else
{
printf("same as before page\n");
}
} //end of if
else if (c == 1)
{
page_stack[stack_count] = page_num;
stack_count++;
}
printf("stack count:%d\n", stack_count);
printf("in page stack: <");
for (x = 0; x < stack_count; x++)
{
printf(" %d ", page_stack[x]);
}
printf(">\n");
rq_handler(fd);
if (logged_in == 1)
{
printf("You are logged in state now, user: %s\n",
curr_user.ID);
}
else
{
printf("not logged in.\n");
c = 0;
}
pre = page_num;
} //end of while
} //end of if
}
} //end of else
} //end of fd_isset
} //end of for loop
} //end of outermost while
}
if needed for code explanation : What I was about to work of this code was,
to make kind of web pages to implement 'browser' for server.
I wanted to make every client get session for server to get login-page or so.
But the execution result is, as I told above.
Why is that?
the socket in the client program must be non-blocking mode too
to be used with non-blocking Server program to use select()?
Or should I use fork or thread to make multi client and manage with select?
The reason I say this is, after I considered a lot about this problem,
'select()' seems only proper for multi client chatting program... that many
'forked' or 'threaded' clients can pend to, in such as chat room.
how do you think?...
Is select also possible or proper thing to use for normal multi-client program?
If there something I missed to let my multi client program work fine,
please give me some knowledge of yours or some requirements for the proper use of select.
I didn`t know multi-client communication was not this much easy before :)
I also considered to use epoll but I think I need to understand first about select well.
Thanks for reading.
Besides the fact you want to go from single-client to multi-client, it's not very clear what's blocking you here.
Are you sure you fully understood how does select is supposed to work ? The manual (man 2 select on Linux) may be helpful, as it provides a simple example. You can also check Wikipedia.
To answer your questions :
First of all, are you sure you need non-blocking mode for your sockets ? Unless you have a good reason to do so, blocking sockets are also fine for multi-client networking.
Usually, there are basically two ways to deal with multi-clients in C: fork, or select. The two aren't really used altogether (or I don't know how :-) ). Models using lightweight threads are essentially asynchronous programming (did I mention it also depends on what you mean by 'asynchronous' ?) and may be a bit overkill for what you seem to do (a good example in C++ is Boost.Asio).
As you probably already know, the main problem when dealing with more than one client is that I/O operations, like a read, are blocking, not letting us know when there's a new client, or when a client has said something.
The fork way is pretty straighforward : the server socket (the one which accepts the connections) is in the main process, and each time it accepts a new client, it forks a whole new process just to monitor this new client : this new process will be dedicated to it. Since there's one process per client, we don't care if i/o operations are blocking or not.
The select way allows us to monitor multiple clients in one same process : it is a multiplexer telling us when something happens on the sockets we give it. The base idea, on the server side, is first to put the server socket on the read_fds FD_SET of the select. Each time select returns, you need to do a special check for it : if the server socket is set in the read_fds set (using FD_ISSET(...)), it means you have a new client connecting : you can then call accept on your server socket to create the connection.
Then you have to put all your clients sockets in the fd_sets you give to select in order to monitor any change on it (e.g., incoming messages).
I'm not really sure of what you don't understand about select, so that's for the big explaination. But long story short, select is a clean and neat way to do single-threaded, synchronous networking, and it can absolutely manage multiple clients at the same time without using any fork or threads. Be aware though that if you absolutely want to deal with non-blocking sockets with select, you have to deal extra error conditions that wouldn't be in a blocking way (the Wikipedia example shows it well as they have to check if errno isn't EWOULDBLOCK). But that's another story.
EDIT : Okay, with a little more code it's easier to know what's wrong.
select's first parameter should be nfds+1, i.e. "the highest-numbered file descriptor in any of the three sets, plus 1" (cf. manual), not FD_SETSIZE, which is the maximum size of an FD_SET. Usually it is the last accept-ed client socket (or the server socket at beginning) who has it.
You shouldn't do the "CHECK all file descriptors" for loop like that. FD_SETSIZE, e.g. on my machine, equal to 1024. That means once select returns, even if you have just one client you would be passing in the loop 1024 times ! You can set fd to 0 (like in the Wikipedia example), but since 0 is stdin, 1 stdout and 2 stderr, unless you're monitoring one of those, you can directly set it to your server socket's fd (since it is probably the first of the monitored sockets, given socket numbers always increase), and iterate until it is equal to "nfds" (the currently highest fd).
Not sure that it is mandatory, but before each call to select, you should clear (with FD_ZERO for example) and re-populate your read fd_set with all the sockets you want to monitor (i.e. your server socket and all your clients sockets). Once again, inspire yourself of the Wikipedia example.
I have a TCP connection. Server just reads data from the client. Now, if the connection is lost, the client will get an error while writing the data to the pipe (broken pipe), but the server still listens on that pipe. Is there any way I can find if the connection is UP or NOT?
You could call getsockopt just like the following:
int error = 0;
socklen_t len = sizeof (error);
int retval = getsockopt (socket_fd, SOL_SOCKET, SO_ERROR, &error, &len);
To test if the socket is up:
if (retval != 0) {
/* there was a problem getting the error code */
fprintf(stderr, "error getting socket error code: %s\n", strerror(retval));
return;
}
if (error != 0) {
/* socket has a non zero error status */
fprintf(stderr, "socket error: %s\n", strerror(error));
}
The only way to reliably detect if a socket is still connected is to periodically try to send data. Its usually more convenient to define an application level 'ping' packet that the clients ignore, but if the protocol is already specced out without such a capability you should be able to configure tcp sockets to do this by setting the SO_KEEPALIVE socket option. I've linked to the winsock documentation, but the same functionality should be available on all BSD-like socket stacks.
TCP keepalive socket option (SO_KEEPALIVE) would help in this scenario and close server socket in case of connection loss.
There is an easy way to check socket connection state via poll call. First, you need to poll socket, whether it has POLLIN event.
If socket is not closed and there is data to read then read will return more than zero.
If there is no new data on socket, then POLLIN will be set to 0 in revents
If socket is closed then POLLIN flag will be set to one and read will return 0.
Here is small code snippet:
int client_socket_1, client_socket_2;
if ((client_socket_1 = accept(listen_socket, NULL, NULL)) < 0)
{
perror("Unable to accept s1");
abort();
}
if ((client_socket_2 = accept(listen_socket, NULL, NULL)) < 0)
{
perror("Unable to accept s2");
abort();
}
pollfd pfd[]={{client_socket_1,POLLIN,0},{client_socket_2,POLLIN,0}};
char sock_buf[1024];
while (true)
{
poll(pfd,2,5);
if (pfd[0].revents & POLLIN)
{
int sock_readden = read(client_socket_1, sock_buf, sizeof(sock_buf));
if (sock_readden == 0)
break;
if (sock_readden > 0)
write(client_socket_2, sock_buf, sock_readden);
}
if (pfd[1].revents & POLLIN)
{
int sock_readden = read(client_socket_2, sock_buf, sizeof(sock_buf));
if (sock_readden == 0)
break;
if (sock_readden > 0)
write(client_socket_1, sock_buf, sock_readden);
}
}
Very simple, as pictured in the recv.
To check that you will want to read 1 byte from the socket with MSG_PEEK and MSG_DONT_WAIT. This will not dequeue data (PEEK) and the operation is nonblocking (DONT_WAIT)
while (recv(client->socket,NULL,1, MSG_PEEK | MSG_DONTWAIT) != 0) {
sleep(rand() % 2); // Sleep for a bit to avoid spam
fflush(stdin);
printf("I am alive: %d\n", socket);
}
// When the client has disconnected, this line will execute
printf("Client %d went away :(\n", client->socket);
Found the example here.
I had a similar problem. I wanted to know whether the server is connected to client or the client is connected to server. In such circumstances the return value of the recv function can come in handy. If the socket is not connected it will return 0 bytes. Thus using this I broke the loop and did not have to use any extra threads of functions. You might also use this same if experts feel this is the correct method.
get sock opt may be somewhat useful, however, another way would to have a signal handler installed for SIGPIPE. Basically whenever you the socket connection breaks, the kernel will send a SIGPIPE signal to the process and then you can do the needful. But this still does not provide the solution for knowing the status of the connection. hope this helps.
You should try to use: getpeername function.
now when the connection is down you will get in errno:
ENOTCONN - The socket is not connected.
which means for you DOWN.
else (if no other failures) there the return code will 0 --> which means UP.
resources:
man page: http://man7.org/linux/man-pages/man2/getpeername.2.html
On Windows you can query the precise state of any port on any network-adapter using:
GetExtendedTcpTable
You can filter it to only those related to your process, etc and do as you wish periodically monitoring as needed. This is "an alternative" approach.
You could also duplicate the socket handle and set up an IOCP/Overlapped i/o wait on the socket and monitor it that way as well.
#include <sys/socket.h>
#include <poll.h>
...
int client = accept(sock_fd, (struct sockaddr*)&address, (socklen_t*)&addrlen);
pollfd pfd = {client, POLLERR, 0}; // monitor errors occurring on client fd
...
while(true)
{
...
if(not check_connection(pfd, 5))
{
close(client);
close(sock[1]);
if(reconnect(HOST, PORT, reconnect_function))
printf("Reconnected.\n");
pfd = {client, POLLERR, 0};
}
...
}
...
bool check_connection(pollfd &pfd, int poll_timeout)
{
poll(&pfd, 1, poll_timeout);
return not (pfd.revents & POLLERR);
}
you can use SS_ISCONNECTED macro in getsockopt() function.
SS_ISCONNECTED is define in socketvar.h.
For BSD sockets I'd check out Beej's guide. When recv returns 0 you know the other side disconnected.
Now you might actually be asking, what is the easiest way to detect the other side disconnecting? One way of doing it is to have a thread always doing a recv. That thread will be able to instantly tell when the client disconnects.