Implementing poll() on a TCP server's read/write - c

I need this server to be able to listen for and establish new connections with clients while simultaneously writing to existing connections.. ie. Asynchronous non-blocking i/o. I've been told to use poll() but after spending an inordinate amount of time simply trying to grasp socket programming, I'm still unsure how implement the poll() function.
int sockfd;
int main(int argc, char *argv[])
{
int newsockfd, portno;
socklen_t clilen;
char buffer[256];
struct sockaddr_in serv_addr, cli_addr;
int n;
if (argc < 2) {
fprintf(stderr,"ERROR, no port provided\n");
exit(1);
}
sockfd = socket(AF_INET, SOCK_STREAM, 0);
if (sockfd < 0)
error("ERROR opening socket");
bzero((char *) &serv_addr, sizeof(serv_addr));
portno = atoi(argv[1]);
serv_addr.sin_family = AF_INET;
serv_addr.sin_addr.s_addr = INADDR_ANY;
serv_addr.sin_port = htons(portno);
if (bind(sockfd, (struct sockaddr *) &serv_addr,
sizeof(serv_addr)) < 0)
error("ERROR on binding");
listen(sockfd,5);
clilen = sizeof(cli_addr);
while(1){
newsockfd = accept(sockfd,
(struct sockaddr *) &cli_addr,
&clilen);
if (newsockfd < 0)
error("ERROR on accept");
// READ READ READ READ READ READ READ READ READ READ READ READ READ READ READ READ
bzero(buffer,256);
n = read(newsockfd,buffer,255);
if (n < 0) error("ERROR reading from socket");
printf("Here is the message: %s\n",buffer);
// WRITE WRITE WRITE WRITE WRITE WRITE WRITE WRITE WRITE WRITE WRITE WRITE WRITE
n = write(newsockfd,"I got your message",18);
if (n < 0) error("ERROR writing to socket");
close(newsockfd);
}
return 0;
}
My understanding is that I need to build something like this:
// Set up array of file descriptors for polling
struct pollfd ufds[2];
ufds[0].fd = sockfd;
ufds[0].events = POLLIN;
ufds[1].fd = newsockfd;
ufds[1].events = POLLOUT;
and use poll(ufds,2,2000); inside the loop to check whether sockfd or newsockfd have any activity, in which case I use the appropriate read or write.. If anybody could give me some guidance I'd be very appreciative.

The kernel will fill in the events that occurred in the revents field of your struct pollfd array.
From the manual page:
The field revents is an output parameter, filled by the kernel with the events that actually occurred. The bits returned in revents can include any of those specified in events, or one of the values POLLERR, POLLHUP, or POLLNVAL. (These three bits are meaningless in the events field, and will be set in the revents field whenever the corresponding condition is true.)
If you want event notifications for accepted connections, then you need to either reserve space in advance or resize the struct pollfd array for every connection.
You'll need some way to differentiate the listening socket. You could store it in index zero of your array.
int i, n;
n = poll(ufds, num_fds_in_array, timeout_value);
/* errors or timeout? */
if (n < 1)
;
for (i = 0; i < num_fds_in_array; i++) {
/* were there any events for this socket? */
if (!ufds[i].revents)
continue;
/* is it our listening socket? */
if (!i) {
if (ufds[0].revents & POLLIN)
/* call accept() and add the new socket to ufds */
else
/* error */
continue;
}
/* is there incoming data on the socket? */
if (ufds[i].revents & POLLIN)
/* call recv() on the socket and decide what to do from there */
}
The POLLOUT flag is used to signal when the sending data on the socket will not block the caller.
For non-blocking I/O, I'd use a more powerful API since it requires more bookkeeping to do reliably. See the next paragraph.
Unfortunately, there's no room for auxiliary per-connection data to store state when using poll. There are alternatives available depending on your platform, e. g. epoll for Linux, kqueue for *BSD, and a handful of options for Windows. If you want to use poll with context data, you'd have to use a data structure that can be searched using the file descriptor or array index.

Why don't u use libevent? It totally asynchronous and non-blocking.
http://libevent.org/

Related

Socket connection refused due to a burst of connections

I am using sockets in order to synchronize several remote processes.
The idea is that a process creates a pthread which manages the server side, just like that:
void *listener(void * in) {
int sockfd;
socklen_t clilen;
struct sockaddr_in serv_addr, cli_addr;
int n = *((int *) in);
sockfd = socket(AF_INET, SOCK_STREAM, 0);
if (sockfd < 0)
error("ERROR opening socket");
int option = 1;
setsockopt(sockfd, SOL_SOCKET, (SO_REUSEPORT | SO_REUSEADDR), (char*) &option, sizeof (option));
bzero((char *) &serv_addr, sizeof (serv_addr));
serv_addr.sin_family = AF_INET;
serv_addr.sin_addr.s_addr = INADDR_ANY;
serv_addr.sin_port = htons(PORT);
if (bind(sockfd, (struct sockaddr *) &serv_addr, sizeof (serv_addr)) < 0)
error("ERROR on binding");
if (listen(sockfd, n) < 0)
error("ERROR when listening");
clilen = sizeof (cli_addr);
int cnt = 0;
while (cnt < n) {
int newsockfd = accept(sockfd, (struct sockaddr *) &cli_addr, &clilen);
if (newsockfd < 0) {
error("ERROR on accept");
}
cnt++;
}
close(sockfd);
return 0;
}
Meanwhile, the other processes would execute:
int sockfd;
struct sockaddr_in serv_addr;
struct hostent *server;
sockfd = socket(AF_INET, SOCK_STREAM, 0);
if (sockfd < 0)
error("ERROR opening socket");
server = gethostbyname(_managementHost); //managementHost);
if (server == NULL)
error("ERROR, no such host\n");
bzero((char *) &serv_addr, sizeof (serv_addr));
serv_addr.sin_family = AF_INET;
bcopy((char *) server->h_addr, (char *) &serv_addr.sin_addr.s_addr, server->h_length);
serv_addr.sin_port = htons(PORT);
if (connect(sockfd, (struct sockaddr *) &serv_addr, sizeof (serv_addr)) < 0)
error("ERROR connection");
close(sockfd);
Now, the problem I have is when I have a lot of processes trying to connect at the same time to the server, some of them are throwing connection refused errors.
I guess that is because the accepts may not be ready... In fact, I have read that it could happen, but I have not found my specific case.
Could anyone shed light on the matter?
A possible solution that occurs to me, is to create a thread for each accept, but I would prefer to avoid it.
Thank you.
EDIT: corrected double initialization of socket in the server. Thanks to #Remy Lebeau.
Now, the problem I have is when I have a lot of processes trying to connect at the same time to the server, some of them are throwing connection refused errors.
A listening TCP socket has a backlog of pending connections. The 2nd parameter of listen() specifies how many connections are allowed to be in the backlog queue before they are accepted. If a new client tries to connect when the backlog is full, the client is refused. There is nothing the client or server can do about that. It is the client's responsibility to detect the error and re-connect at a later time.
Your listener() is accepting clients until it reaches a designated number of connections, but you are also using that same number for the listen backlog queue size. The number of active connections and the number of pending connections are two different things. If you are expecting a lot of clients to connect at the same time, you need a large backlog size to avoid the refusal errors. But that backlog should be sized proportional to the traffic you are expecting. If you have 1000 clients, but they connect only 20 at a time, you would set the backlog to, say, 25, not 1000.
Your listener() has some other logic bugs. It is calling socket() twice and saving the two sockets to the same sockfd variable, so it is leaking the first socket. You need to remove the second call to socket() (the one just before setsockopt()). You are also leaking the sockets that accept() returns. You need to close() an accepted socket after you are doing using it.
Just make your server concurrent by calling fork to spawn a child for each client. Easiest way to do it, IMO. Avoids threading and will stop the connection refused errors.
EDIT: You could also look into pre-forking your server. You'd have to research how you want to handle locking around accept, though (if you're locking at all).

Unix Network Programming Clarification

I was going through the classic book Unix Network Programming, when I stumbled upon this program (Section 6.8, page 179-180)
#include "unp.h"
int
main(int argc, char **argv)
{
int i, maxi, maxfd, listenfd, connfd, sockfd;
int nready, client[FD_SETSIZE];
ssize_t n;
fd_set rset, allset;
char buf[MAXLINE];
socklen_t clilen;
struct sockaddr_in cliaddr, servaddr;
listenfd = Socket(AF_INET, SOCK_STREAM, 0);
bzero(&servaddr, sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
servaddr.sin_port = htons(SERV_PORT);
Bind(listenfd, (SA *) &servaddr, sizeof(servaddr));
Listen(listenfd, LISTENQ);
maxfd = listenfd; /* initialize */
maxi = -1; /* index into client[] array */
for (i = 0; i < FD_SETSIZE; i++)
client[i] = -1; /* -1 indicates available entry */
FD_ZERO(&allset);
FD_SET(listenfd, &allset);
for ( ; ; ) {
rset = allset; /* structure assignment */
nready = Select(maxfd+1, &rset, NULL, NULL, NULL);
if (FD_ISSET(listenfd, &rset)) { /* new client connection */
clilen = sizeof(cliaddr);
connfd = Accept(listenfd, (SA *) &cliaddr, &clilen);
for (i = 0; i < FD_SETSIZE; i++)
if (client[i] < 0) {
client[i] = connfd; /* save descriptor */
break;
}
if (i == FD_SETSIZE)
err_quit("too many clients");
FD_SET(connfd, &allset); /* add new descriptor to set */
if (connfd > maxfd)
maxfd = connfd; /* for select */
if (i > maxi)
maxi = i; /* max index in client[] array */
if (--nready <= 0)
continue; /* no more readable descriptors */
}
for (i = 0; i <= maxi; i++) { /* check all clients for data */
if ( (sockfd = client[i]) < 0)
continue;
if (FD_ISSET(sockfd, &rset)) {
if ( (n = Read(sockfd, buf, MAXLINE)) == 0) {
/*4connection closed by client */
Close(sockfd);
FD_CLR(sockfd, &allset);
client[i] = -1;
} else
Writen(sockfd, buf, n);
if (--nready <= 0)
break; /* no more readable descriptors */
}
}
}
}
The author mentions that this program is not safe against DOS attack. Quoting from the book,
"Unfortunately, there is a problem with the server that we just showed. Consider what happens if a malicious client connects to the server, sends one byte of data (other than a newline), and then goes to sleep. The server will call read (system call), which will read the the single byte of data from the client and then block in the next call to read, waiting for more data from this client. The server is then blocked by this one client, and will not service any other clients until malicious client either sends a newline or terminates"
I am not sure if I understand this correctly. Why will the read system call be called the second time for this malicious client, since it only sent 1 byte of data, that gets notified by the first call to select. The subsequent calls to select will never have this malicious file descriptor set as there is no activity. Am I missing something here?
My guess here is that there is a typo in the code, instead of Read, it should be some version of Readline method mentioned at other places in the book.
Note: The code contains Read and Select (with capital R and S), which are nothing but error handled wrappers of read and select system call
Yes, it seems likely that it was intended to be Readline.
In the downloadable source code that file is tcpcliserv/tcpservselect01.c and there is a corresponding .lc file (with line number annotations) which uses Readline instead of Read, and it was Readline in the second edition of the book (source code). About the only way to make sense of the parenthetic comment "(other than a newline)" is to assume that the intended read function reads up to a newline.
Oddly, it hasn't been reported in the errata. Maybe you should do so.
I think that the problem that he was pointing out was that, as you noted in your NOTE, this code uses Read which is a wrapper of read. My guess, since I'm not about to dig out my copy of the book right now, is that Read will try to call read a second time to finish receiving the data that is never coming.

C http server stops after unpredictable time

I am writing a custom HTTP server in C for my OpenWrt router. It makes use of the uclibc library. I am only using one static buffer which I am very careful not to overflow, and no multi-threading is involved. It doesn't contain any complex data structures, and what it does is that:
it listen() s on the socket
reads the request after accept() ing
gets an html page by sending an http request to a predefined remote server (not a proxy)
sends the result through the accepted connection, and closes both.
The program would just stop running after some time, (it can be on receiving the first request, or after working under heavy strain for more that 2 hours). I am getting no error messages through the console, or anything, the program just stops. I have watched it and, as expected it doesn't consume more and more memory as it runs...
Is it possible that the kernel stops it if it thinks its abusing the CPU? How do I overcome that?
Are there some quirks to watch for in socket programming in C that are known to cause such crashes?
Can the stability issues be caused by using the Barrier Bracker (bleeding edge) branch of OpenWrt? Although the router itself never stops working...
Where do I start to look for the source of the problem?
Ok, first, I would like to thank everybody for helping. After writing a lot of netcat testers, I have pinpointed the problem. The program would crash - end without a single error message, if the connection is closed by the client before the last write or read occurs.
The write or read would raise a SIGPIPE signal which by default crashed the program if not handled manually... More info here: How to prevent SIGPIPE or prevent the server from ending?
This seems to be similar to what your trying to do,
as shown on http://www.cs.rpi.edu/~moorthy/Courses/os98/Pgms/socket.html
Is this socket setup the same/similar to what your performing in your code?
/* A simple server in the internet domain using TCP
The port number is passed as an argument */
#include <stdio.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
void error(char *msg)
{
perror(msg);
exit(1);
}
int main(int argc, char *argv[])
{
int sockfd, newsockfd, portno, clilen;
char buffer[256];
struct sockaddr_in serv_addr, cli_addr;
int n;
if (argc < 2) {
fprintf(stderr,"ERROR, no port provided\n");
exit(1);
}
sockfd = socket(AF_INET, SOCK_STREAM, 0);
if (sockfd < 0)
error("ERROR opening socket");
bzero((char *) &serv_addr, sizeof(serv_addr));
portno = atoi(argv[1]);
serv_addr.sin_family = AF_INET;
serv_addr.sin_addr.s_addr = INADDR_ANY;
serv_addr.sin_port = htons(portno);
if (bind(sockfd, (struct sockaddr *) &serv_addr, sizeof(serv_addr)) < 0)
error("ERROR on binding");
listen(sockfd,5);
clilen = sizeof(cli_addr);
newsockfd = accept(sockfd, (struct sockaddr *) &cli_addr, &clilen);
if (newsockfd < 0)
error("ERROR on accept");
bzero(buffer,256);
n = read(newsockfd,buffer,255);
if (n < 0) error("ERROR reading from socket");
printf("Here is the message: %s\n",buffer);
n = write(newsockfd,"I got your message",18);
if (n < 0) error("ERROR writing to socket");
return 0;
}

print value of fds in FD_SET

Is there anyway of printing the state of a socket in a fd_set?
Say i have this code:
int main(int argc, char * argv[]) {
int sockfd, newfd, i;
struct sockaddr_un sv_addr, cli_addr;
int sv_len, cli_len;
fd_set testmask, mask;
if ((sockfd = socket(AF_UNIX,SOCK_STREAM,0))<0) {
perror("Error creating socket");
exit(-1);
}
bzero((char*)&sv_addr,sizeof(sv_addr));
sv_addr.sun_family = AF_UNIX;
strcpy(sv_addr.sun_path,UNIXSTR_PATH);
sv_len=sizeof(sv_addr.sun_family)+strlen(sv_addr.sun_path);
unlink(UNIXSTR_PATH);
if(bind(sockfd,(struct sockaddr*)&sv_addr,sv_len)<0) {
perror("Error binding socket");
exit(-1);
}
listen(sockfd, 15);
FD_ZERO(&testmask);
FD_SET(sockfd,&testmask);
for(;;) {
mask = testmask;
select(MAXSOCKS,&mask,0,0,0);
if(FD_ISSET(sockfd,&mask)) {
cli_len = sizeof(cli_addr);
newfd = accept(sockfd, (struct sockaddr*)&cli_addr, &cli_len);
echo(newfd);
close(newfd);
}
for(i=0;i<MAXSOCKS;i++) {
if (FD_ISSET(i, &mask)) {
close(i);
FD_CLR(i, &mask);
}
}
}
close(sockfd);
return 0;
}
Everything is working in my program (its an echo server, the client sends a line and the server just echos it back).
I would like to, after the select call, print in the server terminal something like;
00011011011
This means, print the socks that are ready to be handled.
Is there anyway i could do this?
Also, what should i do in the end of the for loop? I know i have to somehow clear the fd_set. The way i did it (the small for loop closing and FD_CLR the fd_set) its correct? Or i should i do it another way?
PS: Sorry for my english or any mistakes. :)
[This does not answer your question, but refers to a comment to the OP and is too long for another comment]
From man select:
nfds is the highest-numbered file descriptor in any of the three sets, plus 1.
nfds is not a constant! The man-pages does not read:
[...] the highest-possible-numbered file descriptor [...]
nfds dynamically has to describe the fd_sets passed to select().
int nfds = sockfd + 1;
for(;;) {
mask = testmask;
select(nfds, &mask, 0, 0, 0);
if(FD_ISSET(sockfd,&mask)) {
cli_len = sizeof(cli_addr);
newfd = accept(sockfd, (struct sockaddr*)&cli_addr, &cli_len);
echo(newfd);
close(newfd);
}
for(i = 0; i < nfds; ++i) {
if (FD_ISSET(i, &mask)) {
close(i);
FD_CLR(i, &mask);
}
}
}
Adjust nfds for every socket descriptor being add to fd_set passed to select().
After select call you checked the sockfd. If that's true, means that a client try to connect your server. Then you accept the connection.
newfd = accept(sockfd, (struct sockaddr*)&cli_addr, &cli_len);
newfd is the fd number you know, between client and server. Here, you don't still read the clientfd(newfd) data. After the connection accepted, you read data on the clentfd like that
read(newfd,buffer,sizeof(buffer))
Your data sent from client, now in buffer. Then maybe you can echo or write() in clientfd.
Your code sent to client , fd number between server and client.
Also if listen always the client(s), after the accept connection, you have to set your client fd in readfds(mask in your code) like FDSET(newfd,&mask)
Then you can listen the client(s) always

Why is this custom network-service crashing (written in C)?

I wrote this program which listens on a given port and then, once a connection is received, outputs a single line of text and disconnects. It runs for days, processing thousands of queries, but then (inevitably) crashes and I have to go restart it. Wondering if anyone sees anything wrong with it, or (alternatively) if anyone can suggest a way to make it more robust.
int main(int argc, char *argv[])
{
srand(time(0));
int sockfd, newsockfd, portno;
socklen_t clilen;
struct sockaddr_in serv_addr, cli_addr;
int n;
if (argc < 2) {
fprintf(stderr,"ERROR, no port provided\n");
exit(1);
}
sockfd = socket(AF_INET, SOCK_STREAM, 0);
if (sockfd < 0)
error("ERROR opening socket");
bzero((char *) &serv_addr, sizeof(serv_addr));
portno = atoi(argv[1]);
serv_addr.sin_family = AF_INET;
serv_addr.sin_addr.s_addr = INADDR_ANY;
serv_addr.sin_port = htons(portno);
if (bind(sockfd, (struct sockaddr *) &serv_addr,
sizeof(serv_addr)) < 0)
error("ERROR on binding");
listen(sockfd,5);
clilen = sizeof(cli_addr);
while (1)
{
unsigned char write_val;
unsigned char y[BYTES];
int i, j;
newsockfd = accept(sockfd,
(struct sockaddr *) &cli_addr,
&clilen);
if (newsockfd < 0)
error("ERROR on accept");
fill_buffer(y); // fills buffer y with a 128-bit string; not included here
for (i=BYTES-1; i >= 0; i--)
{
const void* ZERO = (void *)"0";
const void* ONE = (void *)"1";
for (j=7; j >= 0; j--)
write(newsockfd, (y[i] >> j) & 1 ? ONE : ZERO, 1);
}
write(newsockfd, "\n", 1);
close(newsockfd);
}
close(sockfd);
return 0;
}
anyone sees anything wrong with it
While this code could be made more efficient (by writing all the bytes in one single pass for example), there's no obvious flaw there.
That makes the unpublished part of your code a decent candidate for the problem:
fill_buffer(y); // fills buffer y with a 128-bit string; not included here
If you read more bytes than y[]'s size then you will crash.
or (alternatively) if anyone can suggest a way to make it more robust
Try enlarging the size of this y[] buffer (doubling it can't hurt).
And make sure that fill_buffer() can't read more than BYTES characters.
Publish this missing code in case of doubt.
You could also compile your code with debug symbols and dump a backtrace (with symbols) in a file from your signal handler. This way, if your program crashes, you will know where and why.
The code looks good with some comments.
One somewhat important comment:
portno should be declared as unsigned short. This works OK with an Intel-like (little endian) processor but it won't be portable to a processor with different endianness.
Anyway it was not the reason for your process crashing.
Obviously the crash occurs while executing within the 'while', and by looking at the code, if it crashes for a buffer overflow, the only possibility is within fill_buffer.
If you show the definition of BYTES and fill_buffer it will be easier to help you.
Now, if it is not a buffer overflow, there is the possibility that it aborts in the write if the client closed the connection before the server writes into the socket. In that case the process will receive a signal SIGPIPE and it will abort if the code does not handle that signal.
You can also ignore SIGPIPE with:
signal(SIGPIPE, SIG_IGN);
Other possibility is if you are doing something weird with write_val and you're not showing that code.
I hope this helps.

Resources