C HTTP Proxy, the browser shows "waiting for" indefinetely - c

Question

I have a hard time implementing a proxy server in C. It works for a few first webpages but then I get blocked while waiting for a new request.
Design:
Firefox -> Proxy -> Webserver --.
Firefox <- Proxy <- Webserver <-'
So each request is a round-trip from the browser to the proxy and the server and then back. Until the response from the request comes back, nothing should happen. I use no pipelining, threads or anything like that but rather only recv() and send() in a linear serial manner (for simplicity and intuition). I also don't close any sockets as I want to have a persistent connection.
I expect to be able and fetch one whole webpage including subrequested resources like css, img, js, etc..
In my implementation I manage to fetch a few first requests for a webpage. Then it hangs at step 1.
Implementation:
puts("Waiting for user to connect..");
int sock_user = accept(sock, (struct sockaddr*)NULL, NULL);
int sock_host = -1;
printf("User connected.\n");
// Accept requests
while(1){
http_request req;
http_response resp;
// 1. Client ==> Proxy Server
http_parse_request(sock_user, &req); // uses recv()
// 2. Client Proxy ==> Server
if (sock_host < 0)
sock_host = proxy_connect_host(req.header->host);
write(sock_host, req.header->raw_data, req.header->raw_size);
// 3. Client Proxy <== Server
http_parse_response(sock_host, &resp); // uses recv()
// 4. Client <== Proxy Server
write(sock_user, resp.header->raw_data, resp.header->raw_size);
write(sock_user, resp.body ->first_block->data, resp.body ->first_block->size);
}
Log:
---- ......................................... ----
---- after succesfully responded to 4 requests ----
Client ==> Proxy Server
Received 389
Client Proxy ==> Server
Sending.. 389
Sent 389
Client Proxy <== Server
Got header 312
Got body 1437
Response total 1749
Client <== Proxy Server
Sending header.. 312
Sent 312
Sending body.. 1437
Sent 1437
Client ==> Proxy Server
---- Hangs/blocks here ----
Firebug:
Wireshark:
I have no intuition as to what the reason for that block is and I have spent a whole week trying to resolve this problem without a breakthrough.
Among things tried to resolve problem:
Sending some extra CRLF for each response body
Checked the return value from each recv() and send(). (in the log above,
the values getting printed are the return values from recv and
send)
I hope someone could give at least some direction as to how to troubleshoot this or my brain will soon explode :)

Answer 1

You have to be very careful not to read too much data. E.g. make sure that:
header is read only up to the double CRLF; store extra data and sent them with the body
sending of body starts not before the server has sent the complete header (does not apply to this GET case but is important for POST or CONNECT)
there are received and sent only Content-Length bytes from the body
This applies to both the client -> proxy request and the server -> proxy response.
Your sample code stays in an endless loop (while (1) ...). How do you abort this? Do you honor the "Proxy-Connection" header?