I have a server that has 2 ports with a socket for each port, each of them doing different thigs. I made this work with epoll but I have a question, can 2 different clients connect to each port and use the functionality offered by them at the same time? For example: client1 connects to port X and wants to read data and client2 connects at the same time and wants to write data at the port Y. Is this possible? Thanks!
// create the 2 sockets for each port
// bind(), listen() for each
// tried to do this 'parallelism' using epoll but after someone connects
// to a port that port takes over, the other can't be used
epollfd = epoll_create1(0);
if (epollfd == -1) {
perror("epoll_create1");
exit(EXIT_FAILURE);
}
ev.events = EPOLLIN;
ev.data.fd = sd;
if (epoll_ctl(epollfd, EPOLL_CTL_ADD, sd, &ev) == -1) {
perror("epoll_ctl: listen_sock");
exit(EXIT_FAILURE);
}
ev2.events = EPOLLIN;
ev2.data.fd = sd2;
if (epoll_ctl(epollfd, EPOLL_CTL_ADD, sd2, &ev2) == -1) {
perror("epoll_ctl: listen_sock");
exit(EXIT_FAILURE);
}
for (;;) {
nfds = epoll_wait(epollfd, events, MAX_EVENTS, -1);
if (nfds == -1) {
perror("epoll_wait");
exit(EXIT_FAILURE);
}
for (int n = 0; n < nfds; ++n) {
if (events[n].data.fd == sd2) {
// handle the clients at PORT2 where I just show data from
// database, I'm using processes to handle clients
}
else if (events[n].data.fd == sd) {
// handle the clients at PORT1 where they can update/add/delete
// from data base, I'm using processes to handle clients
}
Related
Recently, I am study system call epoll. I already have basic concept about this topic, but I stuck in example given by manual.
/* Code to set up listening socket, 'listen_sock',
(socket(), bind(), listen()) omitted */
epollfd = epoll_create1(0);
if (epollfd == -1) {
perror("epoll_create1");
exit(EXIT_FAILURE);
}
ev.events = EPOLLIN;
ev.data.fd = listen_sock;
if (epoll_ctl(epollfd, EPOLL_CTL_ADD, listen_sock, &ev) == -1) {
perror("epoll_ctl: listen_sock");
exit(EXIT_FAILURE);
}
for (;;) {
nfds = epoll_wait(epollfd, events, MAX_EVENTS, -1);
if (nfds == -1) {
perror("epoll_wait");
exit(EXIT_FAILURE);
}
for (n = 0; n < nfds; ++n) {
if (events[n].data.fd == listen_sock) {
conn_sock = accept(listen_sock,
(struct sockaddr *) &addr, &addrlen);
if (conn_sock == -1) {
perror("accept");
exit(EXIT_FAILURE);
}
setnonblocking(conn_sock);
ev.events = EPOLLIN | EPOLLET;
ev.data.fd = conn_sock;
if (epoll_ctl(epollfd, EPOLL_CTL_ADD, conn_sock,
&ev) == -1) {
perror("epoll_ctl: conn_sock");
exit(EXIT_FAILURE);
}
} else {
do_use_fd(events[n].data.fd);
}
}
}
Following is my assumption:
Above is server code.
listen_sock is file descriptor return from socket() in server.
My question is:
What did file descriptor events[n].data.fd mean? Is server's fd or client's?
Why if events[n].data.fd == listen_sock , then we need to create a new connection?
You can look at the man page for epoll_wait for details on struct epoll_event and its data member and to help make sense of the code.
Any given events[n].data.fd refers to an fd that had some kind of event on it. It could either be the listening socket, or it could be a client socket if there are active client connections.
If the fd with activity is the listen_sock listening socket, this means that a new client is attempting a connection. At this point we accept the connection and add the new conn_sock fd to the set of epoll's fds.
I am building a server/client application. The server will need to deal with less than 1000+ clients. I currently have a simple EPOLL implementation to be able to receive information from the clients to the server. However, I will need to be able to send tasks to specific clients as well. My question is there a way to use EPOLL to identify which clients I need to send a task too (potentially using the EPOLLOUT flag) and send the message out. I've attached the snippet of what I currently have. If possible, how would I go about implementing this. It would be great to be able to have one epoll for all the sending an the receiving if it could work like that.
Thanks for any help/recommendations!
void epoll(int listening_port)
{
char buffer[500]; //buffer for message
int listen_sock = 0; //file descriptor (fd) for listening socket
int conn_sock = 0; //fd for connecting socket
int epollfd = 0; // fd for epoll
int nfds = 0; //number of fd's ready for i/o
int i = 0; //index to which file descriptor we are lookng at
int curr_fd = 0; //fd for socket we are currently looking at
bool loop = 1; //boolean value to help identify whether to keep in loop or not
socklen_t address_len;
struct sockaddr_in serv_addr;
struct epoll_event ev, events[EPOLL_MAX_EVENTS];
ssize_t result = 0;
bzero(buffer, sizeof(buffer));
bzero(&serv_addr, sizeof(serv_addr));
serv_addr.sin_family = AF_INET;
serv_addr.sin_port = listening_port;
serv_addr.sin_addr.s_addr = INADDR_ANY;
listen_sock = create_socket();
if(bind(listen_sock, SA &serv_addr, sizeof(serv_addr)) != 0)
{
perror("Bind failed");
}
else
{
printf("Bind successful\n");
}
set_socket_nonblocking(listen_sock);
listen_on_socket(listen_sock, SOMAXCONN); //specifying max connections in backlog
epollfd = initialize_epoll();
ev.events = EPOLLIN | EPOLLOUT | EPOLLET | EPOLLRDHUP;
ev.data.fd = listen_sock;
if(epoll_ctl(epollfd, EPOLL_CTL_ADD, listen_sock, &ev) == ERROR)
{
perror("Epoll_ctl: listen sock");
}
else
{
printf("Successfully added listen socket to epoll\n");
}
while (RUN_EPOLL)
{
nfds = epoll_wait(epollfd, events, EPOLL_MAX_EVENTS, 0); //waiting for incoming connection;
if(nfds == ERROR)
{
perror("EPOLL_Wait");
}
//printf("Finished waiting\i");
for(i = 0; i < nfds; ++i)
{
curr_fd = events[i].data.fd;
loop = true; //reset looping flag
//Notification from Listening Socket - Process Incoming Connections
if (curr_fd == listen_sock) {
while(loop)
{
conn_sock = accept(listen_sock, SA &serv_addr, &address_len); //accept incoming connection
printf("Accepted new incoming connection - socket fd: %d\n", conn_sock);
if (conn_sock > 0) //if successful set socket nonblocking and add it to epoll
{
set_socket_nonblocking(conn_sock);
ev.events = EPOLLIN | EPOLLOUT| EPOLLET | EPOLLRDHUP; //setting flags
ev.data.fd = conn_sock; //specify fd of new connection in event to follow
if (epoll_ctl(epollfd, EPOLL_CTL_ADD, conn_sock, &ev) == ERROR) //add fd to monitored fd's
{
perror("epoll_ctl: conn_sck");
}
else
{
printf("Added %d to monitor list\n", conn_sock);
}
}
else if (conn_sock == ERROR)
{
if ((errno == EAGAIN) || (errno == EWOULDBLOCK))
{
printf("All incoming connections processed\n");
loop = false;
}
else
{
perror("Accept remote socket");
loop = false;
}
}
}
}
else if(events[i].events & EPOLLRDHUP) //detecting if peer shutdown
{
printf("Detected socket peer shutdown. Closing now. \n");
if (epoll_ctl(epollfd, EPOLL_CTL_DEL, curr_fd, NULL) == ERROR) {
perror("epoll_ctl: conn_sck");
}
close_socket(curr_fd);
}
else if(events[i].events & EPOLLIN)
{
while(loop)
{
result = recv(curr_fd, buffer, sizeof(buffer), 0);
//printf("Length of incoming message is %d\i", result);
if(result > 0) //
{
printf("File Descriptor: %d. Message: %s\n", curr_fd, buffer);
bzero(buffer, sizeof(buffer));
}
else if(result == ERROR) //Message is completely sent
{
if(errno == EAGAIN)
{
send_message(curr_fd, "Success", strlen("Success"));
loop = false;
}
}
else if(result == 0)
{
//Removing the fd from the monitored descriptors in epoll
if (epoll_ctl(epollfd, EPOLL_CTL_DEL, curr_fd, NULL) == ERROR) {
perror("epoll_ctl: conn_sck");
}
close_socket(curr_fd); //Closing the fd
loop = false;
}
}
}
}
}
close_socket(listen_sock);
//need to develop way to gracefully close out of epoll
return;
}
I will elaborate as much as possible without being too lengthy about the issue I would like help with, if possible:
I'm writing a program that communicates with two sockets, I listen on the multicast socket, while I delegate to the other socket "Unicast" important information stemming from the communicated data coming from the first socket "Multicast".
There are two issues that I think they are related:
1- I run the program in one comuputer "Linux" communicating to another "Linux", and the program performs as expected. But when i take it to another computer "running both my program and the other programs, all in one host" with similar Multicast configuration, I get the following error:
Select : Interrupted system call
This is a perror message, but i am not sure if it is due to error in select() or my multicast configuration.
2- As a result of the first issue, I am unable to delegate to the "Unicast" client socket, but the Unicast works because there is some periodic checking between "Unicast client" and my program running all the time.
My code is as a follow:
struct ConfigStruct
{
struct sockaddr_in Hinfo1, Hinfo2;
struct sockaddr_in Rinfo;
int sock1, sock2;
};
int main()
{
ConfigStruct StructArg;
int fd1, fd2;
int POS(1);
/****************** Network parameters declaration *************************/
// Declaration for socket addresses
struct sockaddr_in Host_info1, Host_info2;
struct sockaddr_in Remote_info;
struct in_addr localInterface;
struct ip_mreq Group;
memset((char *)&Host_info1,0,sizeof(Host_info1));
memset((char *)&Host_info2,0,sizeof(Host_info2));
memset((char *)&Remote_info,0,sizeof(Remote_info));
memset((char *)&Group,0,sizeof(Group));
//**** Reads configuration file****************
cout<<"Reading configuration file..........."<<endl;
std::string input1 ="192.***.**.**";
std::string input2 = "8888";
std::string input3 ="192.***.**.**";
std::string input4 = "8889";
const char* addr_input = input1.data();
const char* port_input = input2.data();
const char* addr_input2 = input3.data();
const char* port_input2 = input4.data();
Remote_info.sin_addr.s_addr=inet_addr(addr_input);
Remote_info.sin_port = htons((uint16_t)stoi(port_input,nullptr,0));
Remote_info.sin_family=AF_INET;
Host_info1.sin_addr.s_addr=inet_addr(addr_input2);//htonl(INADDR_ANY);
Host_info1.sin_port = htons((uint16_t)stoi(port_input2,nullptr,0));
Host_info1.sin_family=AF_INET;
//***** First socket *******
fd1= socket(AF_INET,SOCK_DGRAM,IPPROTO_UDP);
if (fd1 == -1)
{
std::cout<<"A problem occured"<<endl;
cease("socket", wd) ;
}
if (setsockopt(fd1,SOL_SOCKET,SO_REUSEADDR, &POS, sizeof(POS)) == -1)
{
perror(" Error in setsockopt");
exit(1);
}
// **** I'M NOT SURE IF THIS NECESSARY **************
int opts;
opts = fcntl(fd1,F_GETFL);
if (opts < 0)
{
perror("fcntl(F_GETFL)");
exit(EXIT_FAILURE);
}
opts = (opts | O_NONBLOCK);
if (fcntl(fd1,F_SETFL,opts) < 0)
{
perror("fcntl(F_SETFL)");
exit(EXIT_FAILURE);
}
//*****************************************************
if (bind(fd1,(struct sockaddr *)&Host_info1,sizeof(Host_info1)) < 0)
{
cease("Bind",wd);
}
else
{
cout<<" Socket ID number "<<fd1<<endl;
cout<<" Bound socket..."<<endl;
}
//********** The multicast network setup ***********************
std::string input5 ="230.*.**.**";
std::string input6 = "192.***.***"; // The same host IP address as above
std::string input7 = "1500" ; // The port number to listen to for Multicast message
const char* Group_Multi_Addr = input5.data();
const char* Group_Interface_Addr = input6.data();
const char* Host_port_input = input7.data();
Group.imr_multiaddr.s_addr = inet_addr(Group_Multi_Addr);
Group.imr_interface.s_addr = inet_addr(Group_Interface_Addr);
Host_info2.sin_family = AF_INET;
Host_info2.sin_addr.s_addr = INADDR_ANY;
Host_info2.sin_port = htons((uint16_t)stoi(Host_port_input,nullptr,0));
//***** The second socket *******
fd2 = socket(AF_INET, SOCK_DGRAM, 0);
if(fd2 < 0)
{
perror("Opening datagram socket error");
exit(1);
}
else
printf("Opening the datagram socket...OK.\n");
int reuse = 1;
if(setsockopt(fd2, SOL_SOCKET, SO_REUSEADDR, (char *)&reuse, sizeof(reuse)) < 0)
{
close(fd2);
cease("Setting SO_REUSEADDR error", wd);
}
else
printf("Setting SO_REUSEADDR...OK.\n");
if(bind(fd2, (struct sockaddr*)&Host_info2, sizeof(Host_info2)))
{
close(fd2);
cease("Binding datagram socket error",wd);
}
else
printf("Binding datagram socket...OK.\n");
if(setsockopt(fd2, IPPROTO_IP, IP_ADD_MEMBERSHIP,(char *)&Group,sizeof(Group)) < 0)
{
perror("Adding multicast group error");
close(fd2);
exit(1);
}
else
printf("Adding multicast group...OK.\n");
StructArg.Hinfo1= Host_info1;
StructArg.Hinfo2= Host_info2 ;
StructArg.Rinfo= Remote_info ;
StructArg.sock1=fd1;
StructArg.sock2=fd2;
fd_set readfds ,rd_fds;
struct timeval tv;
// clear the set ahead of time
FD_ZERO(&readfds);
// add our descriptors to the set
FD_SET(StructArg.sock1, &readfds);
FD_SET(StructArg.sock2, &readfds);
nls = StructArg.sock2 + 1;
tv.tv_sec = 0;
tv.tv_usec = 50;
char Recv_buffer[125];
char TX_buffer[125];
memset((char *)&Recv_buffer,'0',sizeof(Recv_buffer));
memset((char *)&TX_buffer,'0',sizeof(TX_buffer));
int lenremote(sizeof(StructArg.Rinfo));
ssize_t rs, rs2;
uint8_t MsgSize;
uint8_t MsgID;
do
{
rd_fds=readfds;
if (select(nls, &rd_fds, NULL, NULL, &tv) < 0)
{
perror("select"); // error occurred in select()
}
else
{
// one or both of the descriptors have data
if (FD_ISSET(StructArg.sock1, &rd_fds))
{
rs = recvfrom(StructArg.sock1,....,...,0,...,...) ;
if ( rs > 0 )
{
Do bunch of routines
}
}
if (FD_ISSET(StructArg.sock2, &rd_fds))
{
rs2 = recv(StructArg.sock2,&Recv_buffer,sizeof(Recv_buffer),0);
if ( rs2 > 0 )
{
send some data to StructArg.sock1
}
}
// I do some work here , i send somethings to Sock 1 (Is this appropriate ??)
}
while(1);
return 0;
}
So most importantly, why do I get Select : System interrupt call in one computer but not another?
I am not sure if it is due to error in Select() or my multicast configuration.
Neither. It is due to a signal being caught during the system call. Just loop around this condition, eg:
do
{
// ...
rc = select(nls, &rd_fds, NULL, NULL, &tv);
} while (rc == -1 && errno == EINTR);
if (rc == -1)
{
perror("select");
}
else
{
// ...
}
most importantly, why do i get , Select : System interrupt call in one computer but not another
Because you got a signal on one computer and not the other. And it isn't important.
how about the timeouts, because the man page, says that timeout
becomes undefined when an error arise, should i put the tv.sec and
tv.usec inside the loop.
man 2 select:
On Linux, select() modifies timeout to reflect the amount of time not
slept; most other implementations do not do this. (POSIX.1-2001
permits either behavior.)
On Linux thus, the code shown in the question probably doesn't act as intended, because the timeout is set before the loop only once and zeroed if it expires - from then on, it remains zero. Here, it would be appropriate to set the desired timeout inside the main loop, but outside the proposed loop … while (rc == -1 && errno == EINTR).
On systems that don't update timeout accordingly, there is in the presence of interruptions no such easy way to maintain the correct timeout.
I have got this code and I'm trying to use it as a part of my project. The details of the project are not important for now, but what I'm trying to do is to use this port forwarding proxy as proxy between the browser and the local http server.
So if I type http: //127.0.0.1:8999/ in my browser I want to get web page from 127.0.0.1:8888 back. This only works for small web pages (no images, small html files,...). When I try to do this on webpage with a few images, they don't get transmitted or are transmitted only partially.
I have also tested with telnet and a web site with a big html file. I have connected to the proxy and send the HEAD method request. I got all the meta-data, as expected. When I tried GET method I got only part of the web page's html file back.
Could anyone point me in the right direction? I'm really not sure what I am doing wrong.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netdb.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/epoll.h>
#include <errno.h>
#include <signal.h>
#include <assert.h>
#define FALSE 0
#define TRUE 1
#define EPOLL_QUEUE_LEN 256
#define SERVER_PORT 7000
#define LISTEN_PORT 8667 // Proxy Server Listens for this port
#define FORWARD_PORT 8888 // Proxy Server forwards all port LISTEN_PORT data to this port
#define BUFLEN 1024
//Globals
int fd_server;
int sent = 0; // for how many requests were processed to client
int forwardSockets[EPOLL_QUEUE_LEN];
int internalSockets[EPOLL_QUEUE_LEN];
// Function prototypes
static void SystemFatal (const char* message);
static int forwardData (int fd);
void close_fd (int);
// This is the main function which handles the epoll loop and
// accepts connections
// Also handles the data processing back to the client.
int main (int argc, char* argv[])
{
int i, arg, src_port, forwardSD, dest_port;
int num_fds, fd_new, epoll_fd;
int linenum=0;
static struct epoll_event events[EPOLL_QUEUE_LEN], event;
struct sockaddr_in addr, remote_addr;
socklen_t addr_size = sizeof(struct sockaddr_in);
struct sigaction act;
struct hostent *hp;
struct sockaddr_in server_fwd;
char *host;
char line[256], ip[256];
FILE *fp;
fp=fopen("port_forward_config", "r");
//src_port = LISTEN_PORT; // Use the default listen port
while(fgets(line, 256, fp) != NULL)
{
linenum++;
if(line[0] == '#') continue;
sscanf(line, "%s %d %d", &ip, &src_port, &dest_port);
{
fprintf(stderr, "Syntax error, line %d\n", linenum);
continue;
}
}
printf("Reading Config File...\n");
printf("IP %s LPORT %d DPORT %d\n", ip, src_port, dest_port);
host = ip;
// set up the signal handler to close the server socket when CTRL-c is received
act.sa_handler = close_fd;
act.sa_flags = 0;
if ((sigemptyset (&act.sa_mask) == -1 || sigaction (SIGINT, &act, NULL) == -1))
{
perror ("Failed to set SIGINT handler");
exit (EXIT_FAILURE);
}
//--------------------------------------------------------------------------------------------
// Create the listening socket
fd_server = socket (AF_INET, SOCK_STREAM, 0);
if (fd_server == -1)
{
SystemFatal("socket");
}
// set SO_REUSEADDR so port can be resused imemediately after exit, i.e., after CTRL-c
arg = 1;
if (setsockopt (fd_server, SOL_SOCKET, SO_REUSEADDR, &arg, sizeof(arg)) == -1)
{
SystemFatal("setsockopt");
}
// Make the server listening socket non-blocking
if (fcntl (fd_server, F_SETFL, O_NONBLOCK | fcntl (fd_server, F_GETFL, 0)) == -1)
{
SystemFatal("fcntl");
}
// Bind to the specified listening port
memset (&addr, 0, sizeof (struct sockaddr_in));
addr.sin_family = AF_INET;
addr.sin_addr.s_addr = htonl(INADDR_ANY);
addr.sin_port = htons(src_port);
if (bind (fd_server, (struct sockaddr*) &addr, sizeof(addr)) == -1)
{
SystemFatal("bind");
}
// Listen for fd_news; SOMAXCONN is 128 by default
if (listen (fd_server, SOMAXCONN) == -1)
{
SystemFatal("listen");
}
//---------------------------------------------------------------------------------------------
// Create the epoll file descriptor
epoll_fd = epoll_create(EPOLL_QUEUE_LEN);
if (epoll_fd == -1)
{
SystemFatal("epoll_create");
}
// Add the server socket to the epoll event loop
event.events = EPOLLIN | EPOLLERR | EPOLLHUP | EPOLLET;
event.data.fd = fd_server;
if (epoll_ctl (epoll_fd, EPOLL_CTL_ADD, fd_server, &event) == -1)
{
SystemFatal("epoll_ctl");
}
// Execute the epoll event loop
while (TRUE)
{
//struct epoll_event events[MAX_EVENTS];
num_fds = epoll_wait (epoll_fd, events, EPOLL_QUEUE_LEN, -1);
if (num_fds < 0)
{
SystemFatal ("Error in epoll_wait!");
}
for (i = 0; i < num_fds; i++)
{
// Case 1: Error condition
if (events[i].events & (EPOLLHUP | EPOLLERR))
{
fputs("epoll: EPOLLERR", stderr);
//close(events[i].data.fd);
continue;
}
assert (events[i].events & EPOLLIN);
//-----------------------------------------------------------------------------------------
// Case 2: Server is receiving a connection request
if (events[i].data.fd == fd_server)
{
//socklen_t addr_size = sizeof(remote_addr);
fd_new = accept (fd_server, (struct sockaddr*) &remote_addr, &addr_size);
if (fd_new == -1)
{
if (errno != EAGAIN && errno != EWOULDBLOCK)
{
perror("accept");
}
continue;
}
//------------------------------------------------------------------------------------------------
// Make the fd_new non-blocking
if (fcntl (fd_new, F_SETFL, O_NONBLOCK | fcntl(fd_new, F_GETFL, 0)) == -1)
{
SystemFatal("fcntl");
}
// Add the new socket descriptor to the epoll loop
event.data.fd = fd_new;
if (epoll_ctl (epoll_fd, EPOLL_CTL_ADD, fd_new, &event) == -1)
{
SystemFatal ("epoll_ctl");
}
//+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
printf(" Remote Address: %s\n", inet_ntoa(remote_addr.sin_addr));
//close(fd_new);
dest_port = FORWARD_PORT; // Used the default forward port
// create internal connection
printf("Trying to create forward socket\n");
if ((forwardSD = socket(AF_INET, SOCK_STREAM, 0)) == -1) {
perror("Cannot create forward socket.");
exit(1);
}
printf("Binding...\n");
bzero((char *)&server_fwd, sizeof(struct sockaddr_in));
server_fwd.sin_family = AF_INET;
server_fwd.sin_port = htons(dest_port);
//host = "192.168.0.10";
if ((hp = gethostbyname(host)) == NULL) {
printf("Failed to get host name");
}
bcopy(hp->h_addr, (char *)&server_fwd.sin_addr, hp->h_length);
printf("Connecting to interal machine.\n");
printf("Server Forward Port: %d\n", ntohs(server_fwd.sin_port));
printf("Server Forward IP: %s\n", inet_ntoa(server_fwd.sin_addr));
// Connecting to interal machine
if (connect (forwardSD, (struct sockaddr *)&server_fwd, sizeof(server_fwd)) == -1) {
perror("connect failed");
exit(1);
}
// Add the new socket descriptor to the epoll loop
event.data.fd = forwardSD;
if (epoll_ctl (epoll_fd, EPOLL_CTL_ADD, forwardSD, &event) == -1)
{
SystemFatal ("epoll_ctl");
}
printf ("Connected: Server: %s\n", hp->h_name);
forwardSockets[fd_new] = forwardSD;
internalSockets[forwardSD] = fd_new;
// end internal connection
//+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
continue;
}
// Case 3: One of the sockets has read data
if (!forwardData(events[i].data.fd))
{
// epoll will remove the fd from its set
// automatically when the fd is closed
close (events[i].data.fd);
}
}
}
close(fd_server);
exit (EXIT_SUCCESS);
}
/*
This function clears a socket if it has data waiting to be processed.
It takes the string sent from the client and parses it. After it has
done that it will send back to the client the amount of data it requested
for however many requests it defined as well.
*/
static int forwardData (int fd)
{
int n, bytes_to_read;
char *bp, buf[BUFLEN];
int forwardData;
printf ("Forwarding :\n");
//check if internal or external connection to send data back to.
if(forwardSockets[fd] != 0){
forwardData = forwardSockets[fd];
}
if(internalSockets[fd] != 0){
forwardData = internalSockets[fd];
}
bp = buf;
bytes_to_read = BUFLEN;
while ((n = recv (fd, bp, bytes_to_read, 0)) > 0) {
bp += n;
bytes_to_read -= n;
send (forwardData, buf, n, 0);
return TRUE;
}
return TRUE;
}
// Prints the error stored in errno and aborts the program.
static void SystemFatal(const char* message)
{
perror (message);
exit (EXIT_FAILURE);
}
// close fd
void close_fd (int signo)
{
close(fd_server);
exit (EXIT_SUCCESS);
}
It seems like you're using the EPOLLET (edge triggered) flag in all of your epoll events. This flag will cause epoll to only return events when a file descriptor changes from unavailable to available.
Furthermore, in your forwardData function, you receive BUFLEN bytes and then return TRUE. If there are more bytes available, you will never forward them, as your events are edge triggered and will not be resent.
Try modifying forwardData (perhaps remove the return TRUE in the loop) so that all readable data is forwarded before returning.
All below is from man epoll page:
The function do_use_fd() uses the new
ready file descriptor
until EAGAIN is returned by either read(2) or write(2).
Code example for ET triggered :
for(;;) {
nfds = epoll_wait(kdpfd, events, maxevents, -1);
for(n = 0; n < nfds; ++n) {
if(events[n].data.fd == listener) {
client = accept(listener, (struct sockaddr *) &local,
&addrlen);
if(client < 0){
perror("accept");
continue;
}
setnonblocking(client);
ev.events = EPOLLIN | EPOLLET;
ev.events = EPOLLIN | EPOLLET;
ev.data.fd = client;
if (epoll_ctl(kdpfd, EPOLL_CTL_ADD, client, &ev) < 0) {
fprintf(stderr, "epoll set insertion error: fd=%d\n",
client);
return -1;
}
}
else
do_use_fd(events[n].data.fd);
}
}
So for read/write operation,we should do it by looping until a EAGAIN is received;but why it's not the case for accept?
IMO the above code will miss some requests when there're multiple client sockets waiting to be accepted, as it accepts only 1 client socket, we should also wrap it in a loop until EAGAIN received.
Or maybe is there something I'm missing?
Look at how the listener socket is added to the epollfd:
ev.events = EPOLLIN; // this is the crucial bit
ev.data.fd = listen_sock;
It's not added in edge-triggered, it's added in level-triggered. So no need for a loop until EAGAIN on that one.