C proxy server dropping requests - c

I have a tiny C proxy server which I just want to get one request at a time from the client and send back the response from the server. No pipelining, no anything advanced, just a persistent http connection.
Structs:
typedef struct http_request {
char* h_data; // Header raw data
int h_size; // Header size
char host[5000]; // Host to connect to
char resource[5000]; // Resource to get
} http_request;
typedef struct http_response {
char* h_data; // Header raw data
int h_size; // Header size
char* b_data; // Body raw data
int b_size; // Content-length of the body
} http_response;
Code:
while(1){
// Waiting for user to connect
int sock_user = accept(sock, (struct sockaddr*)NULL, NULL);
int sock_host=-1;
// Accept 1 request at a time and respond
while(1){
http_request req;
http_response resp;
// 1. Client ==> Proxy Server
http_parse_request(sock_user, &req); // uses recv(sock_user)
// 2. Client Proxy ==> Server
if (sock_host < 0)
sock_host=proxy_connect_host(req.host);
write(sock_host, req.h_data, req.h_size);
// 3. Client Proxy <== Server
http_parse_response(sock_host, &resp); // uses send(sock_host)
// 4. Client <== Proxy Server
write(sock_user, resp.h_data, resp.h_size);
write(sock_user, resp.b_data, resp.b_size);
}
}
Now this works good for a few first pages. Then the program blocks at step 1 and the browser just shows Waiting for www.calcoolate.com... all the time.
Firebug:
All of those GET, are requests sent to my proxy. I however receive only the first two of them. I double checked the return value of each write() and recv() and they seem to match exactly with what is expected. I checked for both -1s and 0s.
There must be something wrong with the logic of my proxy.. Any ideas?

Related

C standard library HTTP 301 Redirect using sockets on Objective-C

I'm a very beginner on using sockets with C standard library in Objective-C. I'm trying to make HTTP requests to fetch pages on a self-developed browser, to this we're using a query request such as follows:
+(NSData*)fetchWebPageForURL:(NSURL*)url error:(NSError **)anError {
//[...]
const char* query = [[NSString stringWithFormat:#"GET /%s HTTP/1.1\r\nHost: %s\r\nUser-Agent: %#\r\nConnection: close\r\n\r\n",
page, hostname, userAgentAsNSString] cStringUsingEncoding:NSUTF8StringEncoding];
//[...]
}
Where page is the path (if exists), hostname is the URL, and user agent is some device information such as iOS version, device name and so on.
The socket has been created with the following protocols and types:
+(NSData*)fetchWebPageForURL:(NSURL*)url error:(NSError **)anError {
//socket creation with TCP protocol and IPv4.
int sockfd = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
//[...]
//send request query to the server
size_t sentSoFar = 0;
do {
size_t bytesSent = write(sockfd, query + sentSoFar, strlen(query) - sentSoFar);
if(bytesSent == -1)
{
//send error message
close(sockfd);
return nil;
}
else if (bytesSent == 0)
{
// All done.
break;
}
sentSoFar += bytesSent;
} while (sentSoFar < strlen(query));
//[...]
// Read the response from the server.
int BUFFER_SIZE = 1024;
char response[BUFFER_SIZE];
fd_set rfds;
struct timeval tv;
bool htmlStarted = false;
int done = 0;;
size_t bytesRead = read(sockfd, response, BUFFER_SIZE);
//parsing response
}
The problem is that no matter what kind of URL we sent (with http or https scheme) if the page does not support HTTP/1.1 the response status is always HTTP/1.1 301 Moved Permanently. Our solution shows the response body (using stackoverflow's url as example: stackoverflow.com):
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
here.
</BODY></HTML>
When we click on the link and send the new URL with HTTPS scheme, the response with status 301 still happens. We already tried to change our query to HTTP/2 but the same response (with HTTP/1.1) is still received.
I think that can be a problem with my sockets configuration, but here enters the part that I'm not familiar with, so I don't know what to do.

paho.mqtt.c message loss in a request/reply server

I'm testing a simple MQTT client-server model where a client is repeatedly sending requests to a server and awaits replies to each request. The flow is like this:
Client subscribes to topic 'server/client' with QoS 1
Server subscribes to topic 'client/server' with Qos 1
Client sends 1000 messages to server, on message delivery, client increments its messagesOut counter by 1.
Server receives a new message and invoke a registered callback. In the callback, the server sends a simple message to 'server/client'. On message delivery, the server increments its messagesOut counter.
On new messages to topic 'server/client', the client will increment its messagesIn counter by 1.
Both server and client will stop when client's messagesIn and server's messagesOut are 1000.
With QoS level 1, I expect the server to fully receives 1000 messages sent from the client, and sends 1000 replies back. However, my program is always stuck when the server has received and sent out about 200-300 messages. When I comment out the call to MQTTAsync_sendMessage(), the server can receive all 1000 messages. I suspect some kind of race condition or threads limit is causing this, but I'm not too sure.
The callback for new messages in the server looks like this:
#define TOPIC_IN "client/server"
#define TOPIC_OUT "server/client"
#define PAYLOAD "hello client\n"
unsigned long long messagesIn = 0, messagesOut = 0;
int on_message_arrival (void *context, char *topic_name,
int topic_length, MQTTAsync_message *message) {
if (!strcmp(topic_name, TOPIC_IN)) {
MQTTAsync_freeMessage(&message);
MQTTAsync_free(topic_name);
messagesIn++;
MQTTAsync_message pub_message = MQTTAsync_message_initializer;
MQTTAsync_responseOptions opts = MQTTAsync_responseOptions_initializer;
pub_message.payload = PAYLOAD;
pub_message.payloadlen = strlen(PAYLOAD);
pub_message.qos = QOS1;
pub_message.retained = 0;
delivered_token = 0;
opts.context = &message;
MQTTAsync_sendMessage((MQTTAsync)context, TOPIC_OUT, &pub_message, &opts);
} else {
MQTTAsync_freeMessage(&message);
MQTTAsync_free(topic_name);
}
return 1;
}
Any guide to help me debug this is appreciated.

Eclipse Paho MQTT Client: How to check for existing connection?

On the Eclipse Paho MQTT website, the developers provide a client example (http://www.eclipse.org/paho/files/mqttdoc/MQTTClient/html/pubsync.html) that does the following:
Create a client object with the specified parameters
Connect the client with the specified connection options
Publish a MQTT message
Disconnect the client
Destroy the client object
This works well if all you want is to publish one single message.
In my code, I have a function that contains pretty much the same code as in the aforementioned example, however, the function is called repeatedly from main() as I need to publish a large number of messages one after another. The problem is, if I use the code exactly as in the example, every time my function is called a new connection is created and shortly after destroyed. This happens again and again and again as long as the function is called repeatedly, causing a huge overhead.
Is there a way to check whether a client object has already been created, and if so, don't do it again but use the existing one?
In my understanding, the MQTTClient_isConnected() function is supposed to do that: https://www.eclipse.org/paho/files/mqttdoc/MQTTClient/html/_m_q_t_t_client_8h.html#ad9e40bdb7149ee3e5d075db7f51a735f
But if I try it like this, I get a Segmentation fault:
if (!MQTTClient_isConnected(client)) {
MQTTClient_create(&client, mqtt.addr, CLIENT_ID, MQTTCLIENT_PERSISTENCE_NONE, NULL);
conn_opts.keepAliveInterval = 20;
conn_opts.cleansession = 1;
conn_opts.username = TOKEN;
if (MQTTClient_connect(client, &conn_opts) != MQTTCLIENT_SUCCESS) {
printf("\n==> Connection to MQTT Broker failed.\n");
MQTTClient_destroy(&client);
exit(EXIT_FAILURE);
}
}
[EDIT]
Here is a simple demo code that better illustrates what I'm trying to accomplish:
#include <stdio.h>
#include <MQTTClient.h>
MQTTClient client;
void publish_MQTT() {
MQTTClient_connectOptions conn_opts = MQTTClient_connectOptions_initializer;
MQTTClient_message pubmsg = MQTTClient_message_initializer;
MQTTClient_deliveryToken token;
char *payload = (char *)calloc(1024, sizeof(char));
strcpy(payload, "hello");
printf("DEBUG_BEFORE >> MQTTClient_isConnected(client) = %d\n", MQTTClient_isConnected(client)); // DEBUG OUTPUT
if (!MQTTClient_isConnected(client)) {
MQTTClient_create(&client, addr, CLIENT_ID, MQTTCLIENT_PERSISTENCE_NONE, NULL);
conn_opts.keepAliveInterval = 20;
conn_opts.cleansession = 1;
conn_opts.username = TOKEN;
if (MQTTClient_connect(client, &conn_opts) != MQTTCLIENT_SUCCESS) {
fprintf(stderr, RED "\n==> Connection to MQTT Broker failed.\n" RESET_CL);
MQTTClient_destroy(&client);
free(payload);
exit(EXIT_FAILURE);
}
}
printf("DEBUG_AFTER >> MQTTClient_isConnected(client) = %d\n", MQTTClient_isConnected(client)); // DEBUG OUTPUT
pubmsg.payload = payload;
pubmsg.payloadlen = strlen(payload);
pubmsg.qos = QOS;
pubmsg.retained = 0;
MQTTClient_publishMessage(client, TOPIC, &pubmsg, &token);
MQTTClient_waitForCompletion(client, token, TIMEOUT);
//MQTTClient_disconnect(client, 10000);
//MQTTClient_destroy(&client);
free(payload);
}
int main(void) {
for (i=0; i<1000; i++) {
publish_MQTT();
}
return 0;
}
Please ignore the fact that the addr parameter is never specified (in my real code it is) or that it is pretty useless specifying a message in the publish_MQTT() function (in my real code, data is passed from main() to that function).
I figured it out: Apparently, there is absolutely nothing wrong with the example codes in the original posting.
It turns out I was appending the port of the MQTT server to the addr parameter again and again (in a section of the code not shown here as I didn't suspect the source of the error to be there), every time the publish_MQTT() function was called. This made the addr char string grow and eventually exceed the specified length, thus causing the SegFault.
This way everything works just as intended:
printf("\nADDR = %s\n\n", addr); // DEBUG OUTPUT
if (!MQTTClient_isConnected(client)) {
strcat(strcat(addr, ":"), pt); // This line needed to be placed here, not before that if block
MQTTClient_create(&client, addr, CLIENT_ID, MQTTCLIENT_PERSISTENCE_NONE, NULL);
conn_opts.keepAliveInterval = 20;
conn_opts.cleansession = 1;
conn_opts.username = TOKEN;
if (MQTTClient_connect(client, &conn_opts) != MQTTCLIENT_SUCCESS) {
printf("\n==> Connection to MQTT Broker failed.\n");
MQTTClient_destroy(&client);
free(payload);
exit(EXIT_FAILURE);
}
}
Probably you are setting up "clean session flag", what's mean: "
If the ClientId represents a Client already connected to the Server then the Server MUST disconnect the existing Client [MQTT-3.1.4-2]." (from mqtt standard). So you client is disconnected (the existing one).
Code from example seems to be reasobable. It looks like there is problem with passing function argument. For example if function needed address, and you are giving objects itself.
Morze from standard:
"3.2.2.2 Session Present
Position: bit 0 of the Connect Acknowledge Flags.
If the Server accepts a connection with CleanSession set to 1, the Server MUST set Session Present to 0 in the CONNACK packet in addition to setting a zero return code in the CONNACK packet [MQTT-3.2.2-1].
If the Server accepts a connection with CleanSession set to 0, the value set in Session Present depends on whether the Server already has stored Session state for the supplied client ID. If the Server has stored Session state, it MUST set Session Present to 1 in the CONNACK packet [MQTT-3.2.2-2]. If the Server does not have stored Session state, it MUST set Session Present to 0 in the CONNACK packet. This is in addition to setting a zero return code in the CONNACK packet".

post json data through https in c

I found the following code snippet and did few changes to post json data to a re_url . But it is showing error failed connection ...the server listens on the 8888 port for json data.
#include <stdio.h>
#include <openssl/ssl.h>
#include <openssl/err.h>
#include <openssl/bio.h>
#define APIKEY "YOUR_API_KEY"
#define HOST "https://YOUR_WEB_SERVER_URI"
#define PORT "8888"
char *s = "somejson";
int main() {
//
// Initialize the variables
//
BIO* bio;
SSL* ssl;
SSL_CTX* ctx;
//
// Registers the SSL/TLS ciphers and digests.
//
// Basically start the security layer.
//
SSL_library_init();
//
// Creates a new SSL_CTX object as a framework to establish TLS/SSL
// or DTLS enabled connections
//
ctx = SSL_CTX_new(SSLv23_client_method());
//
// -> Error check
//
if (ctx == NULL)
{
printf("Ctx is null\n");
}
//
// Creates a new BIO chain consisting of an SSL BIO
//
bio = BIO_new_ssl_connect(ctx);
//
// Use the variable from the beginning of the file to create a
// string that contains the URL to the site that you want to connect
// to while also specifying the port.
//
BIO_set_conn_hostname(bio, HOST ":" PORT);
//
// Attempts to connect the supplied BIO
//
if(BIO_do_connect(bio) <= 0)
{
printf("Failed connection\n");
return 1;
}
else
{
printf("Connected\n");
}
//
// The bare minimum to make a HTTP request.
//
char* write_buf = "POST / HTTP/1.1\r\n"
"Host: " HOST "\r\n"
"Authorization: Basic " APIKEY "\r\n"
"Connection: close\r\n"
"\r\n";
//
// Attempts to write len bytes from buf to BIO
//
if(BIO_write(bio, write_buf, strlen(write_buf)) <= 0)
{
//
// Handle failed writes here
//
if(!BIO_should_retry(bio))
{
// Not worth implementing, but worth knowing.
}
//
// -> Let us know about the failed writes
//
printf("Failed write\n");
}
//
// Variables used to read the response from the server
//
int size;
char buf[1024];
//
// Read the response message
//
for(;;)
{
//
// Get chunks of the response 1023 at the time.
//
size = BIO_read(bio, buf, 1023);
//
// If no more data, then exit the loop
//
if(size <= 0)
{
break;
}
//
// Terminate the string with a 0, to let know C when the string
// ends.
//
buf[size] = 0;
//
// -> Print out the response
//
printf("%s", buf);
}
//
// Clean after ourselves
//
BIO_free_all(bio);
SSL_CTX_free(ctx);
return 0;
}
The code above will explain in details how to establish a TLS connection with a remote server.
Important note: this code doesn't check if the public key was signed by a valid authority. Meaning I don't use root certificates for validation. Don't forget to implement this check otherwise you won't know if you are connecting the right website
When it comes to the request itself. It is nothing more then writing the HTTP request by hand.
You can also find under this link an explanation how to instal openSSL in your system, and how to compile the code so it uses the secure library.
From BIO_set_conn_hostname:
BIO_set_conn_hostname() uses the string name to set the hostname. The
hostname can be an IP address. The hostname can also include the port
in the form hostname:port . It is also acceptable to use the form
"hostname/any/other/path" or "hostname:port/any/other/path".
You do not provide a valid host name:
#define HOST "https://YOUR_WEB_SERVER_URI"
...
// Use the variable from the beginning of the file to create a
// string that contains the URL to the site that you want to connect
// to while also specifying the port.
BIO_set_conn_hostname(bio, HOST ":" PORT);
The comment correctly refers to a URL but instead you provide a URI which is most probably wrong.
If you replace YOUR_WEB... with the real host name, it is still a URI and not a host name.
Try to remove the "https://" part and only provide the host name.

Getting Host field from TCP packet payload

I'm writing a kernel module in C, and trying to get the Host field from a TCP packet's payload, carrying http request headers.
I've managed to do something similar with FTP (scan the payload and look for FTP commands), but I can't seem to be able to do the same and find the field.
My module is connected to the POST_ROUTING hook.
each packet that goes to that hook, if it has a dst port of 80, is being recognized as an HTTP packet, and so my module starts to parse it.
for some reason, I can't seem to be able to get the HOST line (matter of fact, I only see the server HTTP 200 ok)
are these headers always go on the packets that use port 80?
if so, what is the best way to parse those packt's payload? seems like going char by char is a lot of work. is there any better way?
Thanks
EDIT:
Got some progress.
every packet I get from the server, I can read the payload with no problem. but every packet I send - it's like the payload is empty.
I thought it's a problem of skb pointer, but i'm getting the TCP ports fine. just can't seem to read this damn payload.
this is how i parse it:
unsigned char* user_data = (unsigned char *)((int)tcphd + (int)(tcphd->doff * 4));
unsigned char *it;
for (it = user_data; it != tail; ++it) {
unsigned char c = *(unsigned char *)it;
http_command[http_command_index] = c;
http_command_index++;
}
where tail:
tail = skb_tail_pointer(skb);
The pointer doesn't advance at all on the loop. it's like it's empty from the start or something, and I can't figure out why.
help, please.
I've managed to solve this.
using this
, I've figured out how to parse all of the packet's payload.
I hope this code explains it
int http_command_offset = iphd->ihl*4 + tcphd->doff*4;
int http_command_length = skb->len - http_command_offset;
http_command = kmalloc(http_command_length + 1, GFP_ATOMIC);
skb_copy_bits(skb, http_command_offset , (void*)http_command, http_command_length);
skb_cop_bits, just copies the payload entirely into the buffer i've created. parsing it now is pretty simple.

Resources