In C: Proxy HTTP requests to another server - c

I'm trying to proxy HTTP requests to another HTTP server. The hostname and port number of the upstream HTTP server, respectively, are server_proxy_hostname and server_proxy_port.
The first step is to do a DNS lookup of the server_proxy_hostname.
Secondly, I create a network socket and connet it to the IP address I got from DNS.
Last step: Wait for new data on both sockets. When data arrives, I immediately read it to a buffer and then write it to the other
socket. This maintains a 2-way communication between the HTTP client and
the upstream HTTP server.
If any of the socket is closed, I close the other one.
The problem right now is that it is not working. (It times out)
I believe that the way I'm getting my IP addresses is correct, so the problem has to be in either my while loop (it never terminates?) or the part where I called connect(). I tried adding error termination for read() and select() but that didn't work either.
void handle_proxy_request(int fd) {
char * targetHostName = server_proxy_hostname;
int targetPort = server_proxy_port;
struct hostent *info;
info = gethostbyname(targetHostName);
struct in_addr ** ipAddresslist;
ipAddresslist = (struct in_addr **) (info -> h_addr_list);
struct in_addr * ipAddress = ipAddresslist[0];
printf("ip address is %s\n", inet_ntoa(*ipAddress));
/*ip for in_addr struct*/
unsigned long ip = inet_addr(inet_ntoa(*ipAddress));
struct in_addr addressIp = {ip};
struct sockaddr_in address = {PF_INET, htons(targetPort), addressIp};
int socket_num = socket(PF_INET, SOCK_STREAM, 0);
connect(socket_num, (struct sockaddr *)&address, sizeof(address));
/*portion for select()*/
char buf[10000];
int nfds = (fd > socket_num)?fd:socket_num;
fd_set readSet;
fd_set writeSet;
while (1) {
FD_ZERO(&readSet);
FD_ZERO(&writeSet);
FD_SET(fd, &readSet);
FD_SET(socket_num, &readSet);
FD_SET(fd, &writeSet);
FD_SET(socket_num, &writeSet);
int selectReturn = select(nfds, &readSet, &writeSet, NULL, NULL);
if (selectReturn == -1){
break;
}
if(FD_ISSET(fd, &readSet)){
int readStat = read(fd, buf, sizeof(buf));
int status = write(socket_num, buf, sizeof(buf));
if (status == -1 || readStat == -1){
close(socket_num);
close(fd);
break;
}
/*memset(buf, 0, sizeof(buf));*/
}
if(FD_ISSET(socket_num, &readSet)){
int readStat2 = read(socket_num, buf, sizeof(buf));
int status2 = write(fd, buf, sizeof(buf));
if (status2 == -1 || readStat2 == -1){
close(socket_num);
close(fd);
break;
}
}
}
}

int socket_num = socket(PF_INET, SOCK_STREAM, 0);
Unchecked. Check this for errors.
connect(socket_num, (struct sockaddr *)&address, sizeof(address));
Ditto.
FD_SET(fd, &writeSet);
FD_SET(socket_num, &writeSet);
Remove. This is poor practice. Sockets are almost always ready to write, so you shouldn't use the writeSet unless you have previously encountered a case where a socket wasn't ready to write, i.e. write() returned -1 with errno == EAGAIN/EWOULDBLOCK.
int readStat = read(fd, buf, sizeof(buf));
int status = write(socket_num, buf, sizeof(buf));
That should be
int status = write(socket_num, buf, readStat);
in both socket cases.
and it should be preceded by tests for readStat == 0, indicating end of stream, and readStat == -1, indicating an error, which you should trace.
You can't get a timeout in this code, as you haven't set any.
There's a wrinkle. If you get end of stream reading a socket you should shutdown the other socket for output. If you get end of stream on a socket you've already shutdown for output, close them both. This correctly propagates FINs in both directions at the correct times.
When you get any error from any system call, you must immediately call perror() or log it with the result strerror(), before you call any other system calls.

Related

Weird order after select() (with FD_SET())

I am developing a multi-client Unix Domain Socket to transfer data through multiple processes. I found some code that implements chat between every client and stuff but I want that once a client send something to the server, the server reply back and the client disconnect.
Having that said, I don't want while(fgets()) but I want (on client side):
int main() {
int sockfd;
struct sockaddr_un remote;
fd_set readfds;
char buf[1024];
char buf2[1024];
int len;
sockfd = socket(AF_UNIX, SOCK_STREAM, 0);
remote.sun_family = AF_UNIX;
strcpy(remote.sun_path, SOCK_PATH);
len = strlen(remote.sun_path) + sizeof(remote.sun_family);
if(connect(sockfd, (struct sockaddr*)&remote, len) == -1)
/* handle error */
FD_ZERO(&readfds);
FD_SET(0, &readfds);
FD_SET(sockfd, &readfds);
if(select(sockfd+1, &readfds, NULL, NULL, NULL) == -1)
/* handle error */
if(FD_ISSET(0, &readfds)) {
fgets(buf, 1024, stdin);
if(write(sockfd, buf, 1024) <= 0)
/* handle error */
}
if(FD_ISSET(sockfd, &readfds)) {
if(read(sockfd, &buf2, 1024) <= 0)
/* handle error */
}
printf("%s\n", buf2);
close(sockfd);
}
In this order, it works if I do everything after connect() twice (with a loop) but I want to do it only once. Without this loop, my server (which is a daemon) crash and I don't know why.
Furthermore, I added printf() from the code above to understand how it works:
(...)
printf("before select\n");
fflush(stdout);
if(select(sockfd+1, &readfds, NULL, NULL, NULL) == -1)
/* handle error */
printf("before select\n");
fflush(stdout);
if(FD_ISSET(0, &readfds)) {
fgets(buf, 1024, stdin);
if(write(sockfd, buf, 1024) <= 0)
/* handle error */
}
(...)
And I have this output:
before select
"input to fgets"
after select
And I don't understand why I have the input BEFORE "after select". It doesn't make any sense to me since I call fgets() after printf().
I hope this is understandable enough.
What's wrong with my code ? Did I miss something ?
The first time through, you call select() before the server has responded. The result is that sockfd won't be ready for reading.
In your case, the client might not need select() on the sockfd. You know that if you wrote something to the server you want to wait for the reply, right?

Handling TCP out-of-band data correctly

I wrote a simple client and server to work with out-of-band data. The client just sends a single out of band data to the server and the server uses SIGURG to handle this single byte. The server also should handle normal traffic in an infinite loop. The code has a race condition which does not work as expected. Sometimes I get an "Invalid argument" from a call to recv() in the SIGURG handler. Another question I have is that should I block SIGURG signal when calling accept? Also, which one is the preferred scenario:
install SIGURG handler and set the socket owner for the listening socket before calling accept.
install SIGURG handler and set the socket owner for the connected socket after calling accept.
if none of the above, please write your suggestion.
My last question is, since the client sends the out-of-band data immediately, is there a chance for the server to receive the SIGURG just after the completion of three-way handshake, but before returning from accept? If so, I think the "clifd" var can has an invalid value when it is used in the SIGURG handler.
the code for the client:
#include "myheader.h"
int main(int argc, char *argv[])
{
struct sockaddr_in saddr;
int sockfd;
const char c = 'a';
if (2 != argc)
{
fprintf(stderr, "Usage: %s ipaddr\n", argv[0]);
exit(EXIT_FAILURE);
}
if (-1 == (sockfd = socket(AF_INET, SOCK_STREAM, 0)))
die("sockfd()");
(void)memset(&saddr, 0, sizeof(saddr));
saddr.sin_family = AF_INET;
saddr.sin_port = htons(PORT);
if (-1 == inet_pton(AF_INET, argv[1], &saddr.sin_addr))
die("inet_pton()");
if (-1 == connect(sockfd, (struct sockaddr*)&saddr, sizeof(saddr)))
die("connect()");
// if (-1 == send(sockfd, "HELLO\n", 6, 0))
// die("send()");
if (-1 == send(sockfd, &c, 1, MSG_OOB))
die("send()");
close(sockfd);
return 0;
}
and the code for the server:
#include "myheader.h"
void sigurg_handler(int);
char oob;
int sockfd, clifd;
int main(void)
{
struct sockaddr_in myaddr;
char buf[BUFSIZ];
ssize_t nbytes;
sigset_t sset, oset;
sigemptyset(&sset);
sigaddset(&sset, SIGURG);
if (-1 == (sockfd = socket(AF_INET, SOCK_STREAM, 0)))
die("socket()");
(void)memset(&myaddr, 0, sizeof(myaddr));
myaddr.sin_family = AF_INET;
myaddr.sin_port = htons(PORT);
myaddr.sin_addr.s_addr = htonl(INADDR_ANY);
if (-1 == bind(sockfd, (struct sockaddr*)&myaddr, sizeof(myaddr)))
die("bind()");
if (-1 == listen(sockfd, BACKLOG))
die("listen()");
if (-1 == fcntl(sockfd, F_SETOWN, getpid()))
die("fcntl()");
if (SIG_ERR == signal(SIGURG, sigurg_handler))
die("signal()");
for (;;)
{
/* block SIGURG when accepting the connection */
// sigprocmask(SIG_SETMASK, &sset, &oset);
printf("bloking in accept()\n");
if (-1 == (clifd = accept(sockfd, NULL, NULL)))
die("accept()");
/* unblock SIGURG */
// sigprocmask(SIG_SETMASK, &oset, NULL);
printf("recv()ing normal data\n");
nbytes = recv(clifd, buf, sizeof(buf), 0);
buf[nbytes] = 0; /* null-terminate */
printf("%s", buf);
}
close(sockfd);
}
void
sigurg_handler(int signo)
{
char buff[100];
ssize_t nbytes;
printf("SIGURG received\n");
if (clifd != 0)
{
if (-1 == (nbytes = recv(clifd, buff, sizeof(buff) - 1, MSG_OOB)))
die("recv() in sigurg_handler()");
buff[nbytes] = 0;
printf("from sigurg_handler: received \"%s\"\n", buff);
}
else
{
printf("clifd = %d\n", clifd);
exit(1);
}
}
Example:
> ./serv
bloking in accept() /* first client */
SIGURG received
from sigurg_handler: received "a"
recv()ing normal data
bloking in accept() /* second client */
SIGURG received
recv() in sigurg_handler(): Invalid argument
> ./serv /* third client */
bloking in accept()
SIGURG received
clifd = 0
>
I heard select() third parameter can handle tcp OOB
ret = select(connfd+1,&read_fds,NULL,&exceptions_fds,NULL);
https://blog.csdn.net/ty_laurel/article/details/52164669
https://github.com/ty92/OOB
select() exceptional
use select really can avoid the signal setup step,
so that you not miss the oob (that before signal setup).
https://man7.org/linux/man-pages/man7/tcp.7.html#:~:text=out%2Dof%2Dband%20data%20is%20present
man 2 select_tut has a demo code
https://man7.org/linux/man-pages/man2/select_tut.2.html#:~:text=read%20OOB%20data,-before
limitation
but if you did't read the oob byte in time, when new oob arrive, old oob byte become normal data (inserted as normal data into the stream), even if SO_OOBINLINE not set (on linux)
//that behavior may difference in various tcp stack.
https://man7.org/linux/man-pages/man7/tcp.7.html#:~:text=limited%20support%20for%20out%2Dof%2Dband
PS: you'd better copy links with :~:text= manually, it'll highlight the keyword in chrome.
// or click in edit preview mode.
// in normal page stackoverflow always encode ~ in url, which will invalidate the anchor
// those man pages still not support anchors to this day, it's pity.

socket is set with O_NONBLOCK, but it seems the calling thread still gets into sleep when sending data sometimes

Client
In fact, my client doesn't recv and process data send from server, just connects to my server.
int netif_msg_client_socket_create(char *sockpath)
{
int addrlen, retval;
int sockfd;
struct sockaddr_un serv;
sockfd = socket(AF_UNIX, SOCK_STREAM, 0);
if(sockfd < 0) {
PR_ERROR(NETIF_MSG_M, " fatal failure, client msg socket, error is %s, %s %u\n", strerror(errno), __FILE__, __LINE__);
return -1;
}
/* Make client socket. */
memset (&serv, 0, sizeof (struct sockaddr_un));
serv.sun_family = AF_UNIX;
strncpy (serv.sun_path, sockpath, strlen(sockpath));
addrlen = sizeof (serv.sun_family) + strlen(serv.sun_path);
retval = connect(sockfd, (struct sockaddr*)&serv, addrlen);
if(retval < 0)
{
PR_ERROR(NETIF_MSG_M, " fatal failure, client msg connect, error is %s, %s %u\n", strerror(errno), __FILE__, __LINE__);
close(sockfd);
return -1;
}
fcntl(sockfd, F_SETFL, O_NONBLOCK);
return sockfd;
}
2.Server
But my server will try to send some data to the client continuously.
int netif_msg_server_socket_create(char *sockpath)
{
int addrlen, retval;
int sockfd;
struct sockaddr_un serv;
/* First of all, unlink existing socket */
unlink (sockpath);
sockfd = socket(AF_UNIX, SOCK_STREAM, 0);
if(sockfd < 0)
return -1;
fcntl(sockfd, F_SETFL, O_NONBLOCK);
/* Make server socket. */
memset (&serv, 0, sizeof (struct sockaddr_un));
serv.sun_family = AF_UNIX;
strncpy (serv.sun_path, sockpath, sizeof(serv.sun_path)-1);
addrlen = sizeof (serv.sun_family) + strlen(serv.sun_path);
//printf("sizeof(serv) == %d, addrlen == %d.\r\n", sizeof(serv), addrlen);
retval = bind (sockfd, (struct sockaddr *) &serv, addrlen);
if (retval < 0)
{
close (sockfd); /* Avoid sd leak. */
return -1;
}
retval = listen (sockfd, 20);
if (retval < 0)
{
close (sockfd); /* Avoid sd leak. */
return -1;
}
return sockfd;
}
My server uses select and accepts the connection from my client successfully.
After my server sent 412 packets(96 Bytes each), it seems the server sleeps on send.
Key codes:
printf("Try to send packet(%d bytes) to clientfd %d.\n", MSGCB_DLEN(msgcb), client->acpt_fd);
retval = send(client->acpt_fd, msgcb->data_ptr, MSGCB_DLEN(msgcb), 0);
if(retval != MSGCB_DLEN(msgcb))
{
printf("Send netif notify msg failed[%d].\n", retval);
} else {
printf("Send netif notify msg succeeded.\n");
}
After 412 packets sent to my client and "Try to ..." outputed, nothing goes on, neither "...failed" nor "...succeeded" outputs.
I use getsockopt to fetch the SO_RCVBUF and SO_SNDBUF, there are about 100000Bytes for each of them.
I don't know why, need your help, thanks!
If you want the server socket that is connected to the client to be non-blocking, then you must specifically set the new socket that is returned from accept() to be non-blocking. Your code only sets the listening socket to non-blocking.
You can perform non-blocking I/O with send using the MSG_DONTWAIT flag in the last parameter.
retval = send(client->acpt_fd, msgcb->data_ptr, MSGCB_DLEN(msgcb),
MSG_DONTWAIT);
When performing non-blocking I/O, you need to detect when the return value is signalling you to retry the operation.
if (retval < 0) {
if (errno == EAGAIN) {
/* ... handle retry of send laster when it is ready ... */
} else {
/* ... other error value cases */
}
}

client socket unable to receive data using poll/select

/* SEND FUNC. */
int mysend(unsigned char *buffer, int len) {
int sock,ret;
int status,flags;
struct sockaddr_in6 servaddr;
int opt = 1;
char *addr = "1101::1";
sock = socket(AF_INET6,SOCK_DGRAM,0);
if (sock < 0)
return -1;
if( setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, (char *)&opt, sizeof(opt)) < 0 )
return -1;
flags = fcntl(sock, F_GETFL, 0);
fcntl(sock, F_SETFL, flags|O_NONBLOCK);
servaddr.sin6_family = AF_INET6;
servaddr.sin6_port = htons(61616);
status = inet_pton(AF_INET6, addr, &servaddr.sin6_addr);
if (status <= 0) {
perror("inet_pton");
return -1;
}
/* send message to server */
status = sendto(sock, buffer, len, 0, (struct sockaddr *)&servaddr, sizeof(servaddr));
if (status < 0) {
perror("sendto");
return -1;
}
close(sock);
printf("MESSAGE SENT SUCCESSFULLY\n");
return 0;
}
/* RECEIVE FUNC. */
int myrcv() {
int sock,ret;
int status,len,rx_bytes;
int timeout,nfds =1;
struct sockaddr_in6 servaddr;
struct timeval wait;
unsigned char rxbuff[1024];
char *rcv;
char *addr = "1101::1";
fd_set rd;
struct pollfd *fds;
sock = socket(AF_INET6,SOCK_DGRAM,0);
if (sock < 0)
return -1;
servaddr.sin6_family = AF_INET6;
servaddr.sin6_port = htons(61616);
status = inet_pton(AF_INET6, addr, &servaddr.sin6_addr);
if (status <= 0)
return -1;
bind(sock,(struct sockaddr *)&servaddr,sizeof(servaddr));
timeout = (1* 1000);
wait.tv_sec = 10;
wait.tv_usec = 0;
len = sizeof(servaddr);
fds->fd = sock;
fds->events = POLLIN;
for(;;) {
//FD_ZERO(&rd);
//FD_SET(sock,&rd);
printf("Waiting for data....\n");
ret = poll(fds,nfds,timeout);
//ret = select(1,&rd,NULL,NULL,&wait);
if(ret < 0)
break;
if(fds->revents == 0)
printf("revents 0 %d\n",ret);
if(ret == 0)
continue;
memset(rxbuff,0,1024);
//if(FD_ISSET(sock,&rd)) {
printf("receiving message\n");
rx_bytes = recvfrom(sock,rxbuff,1024,0,(struct sockaddr *)&servaddr,&len);
memcpy(rcv,rxbuff,rx_bytes);
//}
}
close(sock);
return 0;
}
int main()
{
/* call mysend() periodically using sigaction() */
/* create a thread that continuously monitors(calls myrcv()) for incoming data */
return 0;
}
I'm unable to receive the packets from the server, but I could see the packets in the tcpdump output. Above are the sample client code snippets, which tries to receive and send the data from/to the server. The scenario is: the client needs to send data periodically to server and should also be able to receive any data from the server.
I have tried using both poll and select methods but failed to receive. Please let me know if I'm missing anything. Thanks for your support.
The problem you have with receiving is that you need to bind the receiving socket to the local port.
You also have other things that can be improved, like creating a single socket for both sending and receiving and using SO_REUSEADDR on the sending socket (not needed on a write-only socket).
What you should do is:
Create socket
Set socket options
Bind to local address (Use IN6ADDR_ANY_INIT to bind to all interfaces)
Write to server
Poll for reply
Several things:
Your receive function (myrcv) isn't specifying a listen port via the bind() call. That's the most likely problem. Ditto for your send function, although a port is chosen randomly for you.
In you myrcv() function, I don't see where you have actually initialized fds or nfsd prior to calling poll().
Re-opening and closing the socket on each call to mysend() looks problematic. If you are expecting the server to send back to the same client on the same port it received the message on, chances are high you have already closed the socket. You should just open one socket for both sending and receiving. You can share the same socket between your send thread and your receive thread.

How can I set up sockets to do either "send/receive" or "receive/send"

How can I setup my sockets routine to either "send" (first) or (switch) to "receive" if data is "sent" from another computer (first)?
thanks
general code:
-(void) TcpClient{
char buffer[128];
struct sockaddr_in sin;
struct hostent *host;
int s;
host = gethostbyname("10.0.0.3");
memcpy(&(sin.sin_addr), host->h_addr,host->h_length);
sin.sin_family = host->h_addrtype;
sin.sin_port = htons(4000);
s = socket(AF_INET, SOCK_STREAM, 0);
connect(s, (struct sockaddr*)&sin, sizeof(sin));
while(1){//this is the Client sequence:
send(s, buffer, strlen(buffer), 0);//but what if the Server sends first ?? Client needs to receive here first
recv(s, buffer, sizeof(buffer), 0);
}
close(s);
}
A socket is bi-directional. It can be read from and written to at any time. If you want to write a single routine that decides when to read and when to write, you need to use the select() function. It will tell you when a socket has data available for reading, and when the socket can accept data for sending. If the socket receives data before you have data to send, your routine can detect that and perform a "receive/send" operation. If you have data to send before the socket receives data, your routine can detect that and perform a "send/receive" operation instead. For example:
while (1)
{
fd_set fd;
FD_ZERO(&fd);
FD_SET(s, &fd);
timeval tv;
tv.tv_sec = 0;
tv.tv_usec = 0;
int ret;
if (select(s+1, &fd, NULL, NULL, &tv) > 0)
{
ret = recv(s, buffer, sizeof(buffer), 0);
if (ret > 0)
send(s, buffer, ret, 0);
}
else
{
ret = send(s, buffer, strlen(buffer), 0);
if (ret > 0)
recv(s, buffer, ret, 0);
}
}
You can use the select() system call to deal with multiple sockets and to trigger actions when data is available for reading or writing. The internet is full of information about socket programming in general, maybe start here, which includes some links to other good information.
Also this one.
And just about any book on network programming should have some good examples, too.

Resources