I'm writing a client server application and I'm using poll to multiplex between several client sockets and stdin, where I can insert commands (example: stop the server). I believe the structure (the "logic") of my code is correct, however it's not behaving the way I expect it to:
struct pollfd pfd[NSERVER]; //defined as 10
pfd[0].fd = fileno(stdin);
pfd[0].events = POLLIN;
pfd[1].fd = socktfd; //server bind, listen socket
pfd[1].events = POLLIN;
struct sockaddr_storage remoteaddr; // client address
socklen_t addrlen;
char remoteIP[INET6_ADDRSTRLEN];
addrlen = sizeof remoteaddr;
char buf[1024]; // buffer
int pos=2;
while(poll(pfd,1,0) >= 0)
{
if(pfd[0].revents & POLLIN) { //stdin
//process input and perform command
}
if(pfd[1].revents & POLLIN) {
/* new connection */
int connsockfd = accept(socktfd, (struct sockaddr *)&remoteaddr,&addrlen);
pfd[pos].fd=connsockfd;
}
int i=2;
//Loop through the fd in pfd for events
while (i<=NSERVER)
{
if (pfd[i].revents & POLLIN) {
int c=recv(pfd[i].fd, buf, sizeof buf, 0);
if(c<=0) {
if (c==0)
{
/* Client closed socket */
close(pfd[i].fd);
}
}else
{//Client sent some data
c=send(pfd[i].fd,sbuff,z,0);
if (c<=0)
{
Error;
}
free(sbuff);
}
}
i++;
}
}
I've removed some code inside the recv and send to make the code easier to read.
It fails to behave (it just hangs, doesn't accept connections or reacts to input from stdin).
Note: I would prefer to use poll over select, so please don't point to select :-).
Thanks in advance for any assistance.
you should set every pfd[i].fd = -1, so they get ignored initially by poll().
poll(pfd, 1, 0) is wrong and should at least be poll(pfd, 2, 0) or even poll(pfd, NSERVER, 0).
while(i<=NSERVER) should be while(i<NSERVER)
Your program probably hangs, because you loop through the pfd array, which is not initialized and containes random values for .fd and .revents, so it wants to send() or recv() on some random FD which might block. Do if(pdf[i].fd < 0) {i++; continue;} in the i<NSERVER loop.
You also don't set pfd[pos].events = POLLIN on newly accepted sockets. Don't set POLLOUT unless you have something to send, because it will trigger almost every time.
Related
I have implemented a server using epoll, in what I believe to be the standard way, in fact, when I implemented it using the example from the epoll man page I got the same behavior.
This leads me to believe that there must be a problem with my client, and that I'm making assumptions somehow where I shouldn't. The main method of my client forks n number of clients, which then connect to the server. What I'm seeing is that usually a subset of these clients don't trigger the epoll, and never hit the
accept() call. The three-way-handshake completes because there is a listening socket, so the client behaves as if it where accepted, but it never gets served, as the server doesn't know about it. I can't figure out why this is happening, and haven't been able to find similar questions online. Thoughts?
Here's the relevant server code:
// wrapper which binds to port and exits on error
listenFD = tcpSocket(host, port);
SetNonblocking(listenFD);
// wrapper which listens and exits on error
tcpListen(listenFD);
epollFD = epoll_create(EPOLL_QUEUE_LEN);
if (epollFD == -1)
errSystem("epoll_create");
// Add the server socket to the epoll event loop
event.events = EPOLLIN | EPOLLERR | EPOLLHUP | EPOLLET;
event.data.fd = listenFD;
if (epoll_ctl (epollFD, EPOLL_CTL_ADD, listenFD, &event) == -1)
errSystem("epoll_ctl");
while(TRUE){
//struct epoll_event events[MAX_EVENTS];
numFDs = epoll_wait(epollFD, events, EPOLL_QUEUE_LEN, -1);
for (i = 0; i < numFDs; i++){
// Case 1: Error condition
if (events[i].events & (EPOLLHUP | EPOLLERR)){
errMessage("epoll: EPOLLERR");
Close(events[i].data.fd);
printf("Closed connection to %d\n", events[i].data.fd);
fflush(stdout);
continue;
}
// Case 2: Server is receiving a connection request
if (events[i].data.fd == listenFD){
// socketlen_t
clientLen = sizeof(client);
newFD = Accept (listenFD, (SA *)&client, &clientLen);
SetNonblocking(newFD);
// Set receive low water mark to message size
SetSockOpt(newFD, SOL_SOCKET, SO_RCVLOWAT, &lowWater, sizeof(lowWater));
// Add the new socket descriptor to the epoll loop
event.data.fd = newFD;
event.events = EPOLLIN | EPOLLET;
if (epoll_ctl (epollFD, EPOLL_CTL_ADD, newFD, &event) == -1)
errSystem ("epoll_ctl");
printf("Connected to client on socket %d\n", newFD);
// tell the client we're connected and ready
// (this is an attempt to fix my issue. I'd rather not do this...)
Writen(newFD, buffer, CLIENT_MESSAGE_SIZE);
continue;
}
if (events[i].events & EPOLLIN){
//serve the client
}
}
}
And this is the client code. One instance of these works, but if I fork more than 5 or so (sometimes just 2) a number of them won't get accepted.
int Client(const char *host, const int port, const int timeLen, const int clientNum){
long double delta;
PTSTRUCTS ptstructs = (PTSTRUCTS) malloc(sizeof(TSTRUCTS));
size_t result;
stop = FALSE;
cNum = clientNum;
Signal(SIGINT, closeFD);
Signal(SIGALRM, sendMessage);
nsockets = 0;
// wrapper which calls connect() and exits with message on error
connectFD = tcpConnect(host, port);
printf("%d connected to server:\n", clientNum);
fflush(stdout);
bzero(sbuf, CLIENT_MESSAGE_SIZE);
// initialize client message
strcpy(sbuf, CLIENT_MESSAGE);
// get the start time
getStartTime(ptstructs);
getEndTime(ptstructs);
while((delta = getTimeDelta(ptstructs)) < timeLen){
// One or more clients blocks here for ever
if ((result = receiveMessage()) < CLIENT_MESSAGE_SIZE){
break;
}
//sendMessage();
alarm(1);
//delay(USER_DELAY);
getEndTime(ptstructs);
}
stop = TRUE;
Close (connectFD);
getEndTime(ptstructs);
delta = getTimeDelta(ptstructs);
printf("Client %d served %ld bytes in %.6Lf seconds.\n", clientNum, byteCount, delta);
fflush(stdout);
// free heap memory
free(ptstructs);
return (1);
}
I should probably note that I'm seeing the same behavior if I don't set EPOLLET. I originally thought this might be the result of edge-triggering behavior, but nope.
Does your backlog argument of listen() greater than the number of clients?
#some-programmer-dude, sorry for not commentting, the closed fd will be removed from epoll event set automatically http://man7.org/linux/man-pages/man7/epoll.7.html
I have a small problem, in practice I have to let two clients communicate (which perform different functions), with my concurrent server,
I discovered that I can solve this using the select, but if I try to implement it in the code it gives me a segmentation error, could someone help me kindly?
I state that before with a single client was a fable, now unfortunately implementing the select, I spoiled a bit 'all,
I should fix this thing, you can make a concurrent server with select ()?
can you tell me where I'm wrong with this code?
int main (int argc , char *argv[])
{
int list_fd,conn_fd;
int i,j;
struct sockaddr_in serv_add,client;
char buffer [1024];
socklen_t len;
time_t timeval;
char fd_open[FD_SETSIZE];
pid_t pid;
int logging = 1;
char swi;
fd_set fset;
int max_fd = 0;
int waiting = 0;
int compat = 0;
sqlite3 *db;
sqlite3_open("Prova.db", &db);
start2();
start3();
printf("ServerREP Avviato \n");
if ( ( list_fd = socket(AF_INET, SOCK_STREAM, 0) ) < 0 ) {
perror("socket");
exit(1);
}
if (setsockopt(list_fd, SOL_SOCKET, SO_REUSEADDR, &(int){ 1 }, sizeof(int)) < 0)
perror("setsockopt(SO_REUSEADDR) failed");
memset((void *)&serv_add, 0, sizeof(serv_add)); /* clear server address */
serv_add.sin_family = AF_INET;
serv_add.sin_port = htons(SERVERS_PORT2);
serv_add.sin_addr.s_addr = inet_addr(SERVERS_IP2);
if ( bind(list_fd, (struct sockaddr *) &serv_add, sizeof(serv_add)) < 0 ) {
perror("bind");
exit(1);
}
if ( listen(list_fd, 1024) < 0 ) {
perror("listen");
exit(1);
}
/* initialize all needed variables */
memset(fd_open, 0, FD_SETSIZE); /* clear array of open files */
max_fd = list_fd; /* maximum now is listening socket */
fd_open[max_fd] = 1;
//max_fd = max(conn_fd, sockMED);
while (1) {
FD_ZERO(&fset);
FD_SET(conn_fd, &fset);
FD_SET(sockMED, &fset);
len = sizeof(client);
if(select(max_fd + 1, &fset, NULL, NULL, NULL) < 0){exit(1);}
if(FD_ISSET(conn_fd, &fset))
{
if ( (conn_fd = accept(list_fd, (struct sockaddr *)&client, &len)) <0 )
perror("accept error");
exit(-1);
}
/* fork to handle connection */
if ( (pid = fork()) < 0 ){
perror("fork error");
exit(-1);
}
if (pid == 0) { /* child */
close(list_fd);
close(sockMED);
Menu_2(db,conn_fd);
close(conn_fd);
exit(0);
} else { /* parent */
close(conn_fd);
}
if(FD_ISSET(sockMED, &fset))
MenuMED(db,sockMED);
FD_CLR(conn_fd, &fset);
FD_CLR(sockMED, &fset);
}
sqlite3_close(db);
exit(0);
}
I cannot understand how you are trying to use select here, and why you want to use both fork to let a child handle the accepted connection socket, and select.
Common designs are:
multi processing server:
The parent process setups the listening socket and loops on waiting actual connections with accept. Then it forks a child to process the newly accepted connection and simple waits for next one.
multi threaded server:
A variant of previous one. The master thread starts a new thread to process the newly accepted connection instead of forking a new process.
asynchronous server:
The server setups a fd_set to know which sockets require processing. Initially, only the listening socket is set. Then the main loop is (in pseudo code:
loop on select
if the listening socket is present in read ready sockets, accept the pending connection and add is to the `fd_set`, then return to loop
if another socket is present in read ready socket
read from it
if a zero read (closed by peer), close the socket and remove it from the `fd_set`
else process the request and return to loop
The hard part here is that is processing takes a long time, the whole process is blocked, and it processing involves sending a lot of data, you will have to use select for the sending part too...
I have the following code excerpt (heavily redacted to remove unimportant details) which fails under a rare and particular set of circumstances.
struct epoll_event *events = calloc(MAXEVENTS+1, sizeof(struct epoll_event));
struct sockaddr_in in_addr;
socklen_t in_len = sizeof in_addr;
while(1)
{
int n = epoll_wait(efd,events, MAXEVENTS, -1);
for(int i=0; i<n; i++)
{
struct epoll_event *evI = &events[i];
uint64 u64 = evI->data.u64;
int type = u64>>32, fd=u64, fdx = fd;
if(type == -1)
{
while((fd = accept4(fdx, &in_addr, &in_len, SOCK_NONBLOCK|SOCK_CLOEXEC))>-1)
{
setNew_Connection(efd, fd);
storeAddrPort(fd, &in_addr, &in_len);
}
}
else
{
if(evI->events&(EPOLLERR|EPOLLHUP|EPOLLRDHUP)){closeConnection(fd);}
if(evI->events&EPOLLOUT) //process out data stuff
else if(evI->events&EPOLLIN) //process in data stuff and possibly close a different connection.
}
}
}
Listening sockets are differentiated by -1 in the upper part of evI->data.u64
setNew_Connection does the usual accepting stuff like adding the new socket to epoll etc
EPOLLET is used.
Now it all works brilliantly except under the following circumstances it fails because events is only updated in the epoll_wait so a connection closure does not affect the n events until after returning to the top of the while(1) loop.
epoll_wait unblocks with 3 events queued in the events struct table.
First event (n=0), is incoming data after which code decides to close a connection (e.g. file descriptor 8) as it is no longer needed.
2nd event (n=1) is an incoming new connection. accept4 assigns fd:8 as it has recently become available. setNew_Connection adds it to the epoll list.
3rd event is incoming data for the connection closed in step 2. i.e. fd:8 but it is no longer valid as the original fd:8 connection was closed and the current fd:8 is for a different connection.
I hope I have explained the problem adequately. The issue is that queued events in the events table are not updated when a connection is closed until the code returns to epoll_wait. How can I code around this problem?
Orel gave me the answer but I thought I would post the complete code solution. Instead of
close(fd)
I use
shutdown(fd,SHUT_RDWR);
FDS[FDSL++] = fd;
The shutdown prevents anymore data being read or written but doesn't actually close the socket. FDS[FDSL++] = fd; stores the fd so that later after the n events are done, it can be closed with while(FDSL)close(FDS[--FDSL];
int FDS[MAXEVENTS],FDSL=0;
struct epoll_event *events = calloc(MAXEVENTS+1, sizeof(struct epoll_event));
struct sockaddr_in in_addr;
socklen_t in_len = sizeof in_addr;
while(1)
{
int n = epoll_wait(efd,events, MAXEVENTS, -1);
for(int i=0; i<n; i++)
{
struct epoll_event *evI = &events[i];
uint64 u64 = evI->data.u64;
int type = u64>>32, fd=u64, fdx = fd;
if(type == -1)
{
while((fd = accept4(fdx, &in_addr, &in_len, SOCK_NONBLOCK|SOCK_CLOEXEC))>-1)
{
setNew_Connection(efd, fd);
storeAddrPort(fd, &in_addr, &in_len);
}
}
else
{
if(evI->events&(EPOLLERR|EPOLLHUP|EPOLLRDHUP)){closeConnection(fd);}
if(evI->events&EPOLLOUT) //process out data stuff
else if(evI->events&EPOLLIN) //process in data stuff and possibly close a different connection.
}
}
while(FDSL)close(FDS[--FDSL];
}
i've a problem wiht my multi-threads project: i create some threads, these must send and receive udp messages with each other, so each thread is listening on multiple sockets with an select(). The problem is: only last socket inserted in a fd_set variable, receive messages. I don't konw why
this is code:
initial part of thread:
int fdmax;
fd_set read_fd_set, service_fd_set;
/* allocate memory for receive msg */
msg=malloc(sizeof(char)*(SIZEBUF));
FD_ZERO(&read_fd_set);
FD_ZERO(&service_fd_set);
fdmax=0;
if (param->n_port != 0){
for (x=0; x<(param->n_port); x++){
/* take the port from array */
local_port_number = param->l_port_in[x];
/* create socket udp and bind on localhost */
socketfd=create_socket(local_port_number);
/* save socket_fd in my data struct */
param->sock_fd_local[x]=socketfd;
/* add socket in fd_set */
FD_SET(socketfd,&service_fd_set);
if (socketfd > (fdmax-1)){
fdmax=socketfd + 1;
}
secondo part of thread:
for(;;){
read_fd_set=service_fd_set;
ris=select(fdmax,&read_fd_set,NULL,NULL,NULL);
if(ris<0){
if (errno!=EINTR){
printf(_KRED "Error in select: errno different from EINTR \n" _KNRM);
}
}
if (ris>0){
for(p=0; p<fdmax; p++){
if((FD_ISSET(p,&read_fd_set))!=0){
for( x=1; x<=5; x++){
if( p == param->sock_fd_local[x]){
/* setup datagram to receive */
memset(&From, 0, sizeof(From));
Fromlen=sizeof(struct sockaddr_in);
/* RECVFROM */
msglen = recvfrom ( p, msg, (int)SIZEBUF, 0, (struct sockaddr*)&From, &Fromlen);
if (msglen<0){
...
}else{
sprintf((char*)string_remote_ip_address,"%s",inet_ntoa(From.sin_addr));
remote_port_number = ntohs(From.sin_port);
print_msg(...);
}
}
}
}
}
}
}
anyone can help me?
The entire project is here: https://github.com/bonfi/SpanningTreeUDP
(sorry for my english, i'm italian, the project is commented in italian)
I believe there are is at least one if not two issues. The first one is in the second part, in the recvfrom function, you are using socketfd but not setting it to the socket from which you want to receive data. The variable socketfd contains the last socket you created from the initial code. I suspect that is why you are only getting data from one thread which was the last socket created.
The other item which I don't quite understand is the loop after the if statement with teh FD_ISSET in the second part. I'm not sure why you are looping with x and then setting p or why you are doing 4 times.
I would propose the code in the second part should look like the follow starting at the FD_ISSET if statement.
if((FD_ISSET(p,&read_fd_set))!=0){
socketfd == param->sock_fd_local[p]){
/* setup datagram to receive */
memset(&From, 0, sizeof(From));
Fromlen=sizeof(struct sockaddr_in);
/* RECVFROM */
msglen = recvfrom ( socketfd, msg, (int)SIZEBUF, 0, (struct sockaddr*)&From, &Fromlen);
if (msglen<0){
...
}else{
sprintf((char*)string_remote_ip_address,"%s",inet_ntoa(From.sin_addr));
remote_port_number = ntohs(From.sin_port);
print_msg(...);
}
}
I removed the for loop with the x variable and I am using p to index the structure to set the socketfd for receiving data based on if the FD_ISSET returns true. That way you are setting the socketfd variable based on the file descriptor if it is set.
I'm new to socket programming and I've been introduced to the select() system call. My question is, lets say I'm writing a server in C (which I am attempting to do) and I want to use the select() call in my implementation for practice. I'm trying to write a server that receives information from a client, so my approach is to use select(), followed by read() and just output the information.
According to the documentation I've read select() returns the number of file descriptors in the input set which are ready for i/o. My question is, how do know which file descriptors in the original set are the ones that are ready for i/o? I can't seem to find this in my searches or examples I've looked at for the past while.
Let's say my code looks like the below:
int main() {
/* Create socket/server variables */
int select_value;
int this_socket;
int maxfd;
struct sockadder_in address;
fd_set allset;
/* Bind the socket to a port */
main_socket = socket(AF_INET, SOCK_STREAM, 0);
if (main_socket < 0) {
perror("socket()");
exit(1);
}
Connect(main_socket, (struct sockaddr *)&address, sizeof(address));
/* Add the socket to the list of fds to be monitored */
FD_ZERO(&allset);
FD_SET(main_socket, &allset);
fd_set read_ready = allset;
fd_set write_ready = allset;
while (1) {
/* Listen for a connection */
/* Accept a connection */
select_value = Select(maxfd+1, &read_ready, &write_ready, NULL, NULL);
if (select_value == -1) {
perror("select()");
exit(1);
}
else if(select_value > 0) {
/* How to access i/o ready file descriptors
now that we know there are some available? */
}
}
}
One can do this using the FD_ISSET macro that is part of <sys/select.h>.
When your select unblocks and a file descriptor is ready, you can test all of your file descriptors using the FD_ISSET macro in a simple loop. This can be translated to the following sample :
for (i = 0; i < FD_SETSIZE; ++i) {
if (FD_ISSET (i, &read_fd_set)) {
if (i == bound_socket) {
// A new client is waiting to be accepted
new = accept(sock, (struct sockaddr *) &clientname, &size);
// ...
FD_SET (new, &active_fd_set);
}
else {
// There is something to be read on the file descriptor.
data = read_from_client_on(i);
}
}
}
Of course, this is just sample which is obviously lacking any error handling, which you should handle in your application.