HTTP protocol: end of a message body - c

I built a program that parses the header and I would like to read the message body in case I receive a POST.
For headers, I have been able to look for to determine when the header ends. I am having more issues for the message body. Am I supposed to look at "Content-Length" field to know when to stop reading input? In my current code (below), it will not stop until I hit the red cross (stop loading page) in Firefox.
Here is the code:
size_t n;
unsigned char newChar;
int index = 0;
int capacity = 50;
char *option = (char *) malloc(sizeof(char) * capacity);
while ( ( n = read( req->socket, &newChar, sizeof(newChar) ) ) > 0 ) {
if (newChar == '\0' || newChar == '\n') break; // This is not working
if (index == capacity) {
capacity *= 2;
option = (char *) realloc(option, sizeof(char) * capacity);
assert(option != NULL);
}
option[index++] = newChar;
fprintf(stderr, "%c", newChar);
}
if (index == capacity) {
capacity *= 2;
option = (char *) realloc(option, sizeof(char) * capacity);
assert(option != NULL);
}
option[index] = '\0';
The correct input gets printed, but I wonder why it won't stop until the stop loading button get pressed. I'd like to know if there is any other solution or if I please need to use the "Content-Length" field in the header.
Thank you very much,
Jary

There are a few things to consider. You'll want to consider how you want to handle all of these cases perhaps?
For HTTP protocol 1.0 the connection closing was used to signal the end of data.
This was improved in HTTP 1.1 which supports persistant connections. For HTTP 1.1 typically you set or read the Content-Length header to know how much data to expect.
Finally with HTTP 1.1 there is also the possibility of "Chunked" mode, you get the size as they come and you know you've reached the end when a chunk Size == 0 is found.
Also do you know about libcurl? It will certainly help you having to re-implement the wheel.

This code blocks on the read() waiting for another character which never comes.
Additionally, RFC2616, 3.7.1 states "HTTP applications MUST accept CRLF, bare CR, and bare LF as being representative of a line break in text media received via HTTP. In addition, if the text is represented in a character set that does not use octets 13 and 10 for CR and LF respectively, as is the case for some multi-byte character sets, HTTP allows the use of whatever octet sequences are defined by that character set to represent the equivalent of CR and LF for line breaks."
So you're going to need to catch more than just "\n".

Related

Sending Image Data via HTTP Websockets in C

I'm currently trying to build a library similar to ExpressJS in C. I have the ability to send any text (with res.send() functionality) or textually formatted file (.html, .txt, .css, etc.).
However, sending image data seems to cause a lot more trouble! I'm trying to use pretty much the exact same process I used for reading textual files. I saw this post and answer which uses a MAXLEN variable, which I would like to avoid. First, here's how I'm reading the data in:
// fread char *, goes 64 chars at a time
char *read_64 = malloc(sizeof(char) * 64);
// the entirety of the file data is placed in full_data
int *full_data_max = malloc(sizeof(int)), full_data_index = 0;
*full_data_max = 64;
char *full_data = malloc(sizeof(char) * *full_data_max);
full_data[0] = '\0';
// start reading 64 characters at a time from the file while fread gives positive feedback
size_t fread_response_length = 0;
while ((fread_response_length = fread(read_64, sizeof(char), 64, f_pt)) > 0) {
// internal array checker to make sure full_data has enough space
full_data = resize_array(full_data, full_data_max, full_data_index + 65, sizeof(char));
// copy contents of read_64 into full_data
for (int read_data_in = 0; read_data_in < fread_response_length / sizeof(char); read_data_in++) {
full_data[full_data_index + read_data_in] = read_64[read_data_in];
}
// update the entirety data current index pointer
full_data_index += fread_response_length / sizeof(char);
}
full_data[full_data_index] = '\0';
I believe the error is related to this component here. Likely something with calculating data length with fread() responses perhaps? I'll take you through the HTTP response creating as well.
I split the response sending into two components (as per the response on this question here). First I send my header, which looks good (29834 seems a bit large for image data, but that is an unjustified thought):
HTTP/1.1 200 OK
Content-Length: 29834
Content-Type: image/jpg
Connection: Keep-Alive
Access-Control-Allow-Origin: *
I send this first using the following code:
int *head_msg_len = malloc(sizeof(int));
// internal header builder that builds the aforementioned header
char *main_head_msg = create_header(status, head_msg_len, status_code, headers, data_length);
// send header
int bytes_sent = 0;
while ((bytes_sent = send(sock, main_head_msg + bytes_sent, *head_msg_len - bytes_sent / sizeof(char), 0)) < sizeof(char) * *head_msg_len);
Sending the image data (body)
Then I use a similar setup to try sending the full_data element that has the image data in it:
bytes_sent = 0;
while ((bytes_sent = send(sock, full_data + bytes_sent, full_data_index - bytes_sent, 0)) < full_data_index);
So, this all seems reasonable to me! I've even taken a look at the file original file and the file post curling, and they each start and end with the exact same sequence:
Original (| implies a skip for easy reading):
�PNG
�
IHDR��X��d�IT pHYs
|
|
|
RU�X�^Q�����땵I1`��-���
#QEQEQEQEQE~��#��&IEND�B`�
Post using curl:
�PNG
�
IHDR��X��d�IT pHYs
|
|
|
RU�X�^Q�����땵I1`��-���
#QEQEQEQEQE~��#��&IEND�B`
However, trying to open the file that was created after curling results in corruption errors. Similar issues occur on the browser as well. I'm curious if this could be an off by one or something small.
Edit:
If you would like to see the full code, check out this branch on Github.

Unable to send characters after null from a file to the server in c

OS: Ubuntu(20.04)
MCU: ESP WROOM 32
Coding language: C
I have a file of 120KB with lot of null characters in between. For example:
char[50] = "welcome\0 my name\0 is";
is my string, I am trying to send entire text(including \0) to server via GSM module. I am sending this data to GSM via UART and sending the data to server using AT command "AT+HTTPDATA" and later doing HTTPPOST. The problem that I am facing now is, I am able to send the data only till first null character is encountered.
For ex:
In the above string "welcome\0 my name\0 is"
I am able to send only till "welcome", but nothing after that.
the sample code is given here:
sent_bytes = uart_write_bytes(UART_NUM_1, dummy_data, size_of_data); /*data that I am sending
with '\0' in between*/
sent_bytes = uart_write_bytes(UART_NUM_1, "\x1A", 2); //ctrl+z for httpdata to terminate
I have tried different ways of sending it, but no luck till now.
For example:
for(int f=0;f<=500;f++){
char_ptr = dummy_data[f];
int l=0;
if(char_ptr != '\0' && f <= 500){
printf("char_ptr is %c\n", char_ptr);
temp[f] = char_ptr;
sent_bytes = uart_write_bytes(UART_NUM_1, temp, sizeof(temp));
printf("temp value inside condition is %s\n", temp);
temp[1000] = "";
l++;
if(l == 500){
printf("500 reached \n");
goto c2;
}else{
printf("500 has not reached, going to c1\n");
// goto c1;
}
}
}
Still this code doesn't work and I am able to receive only the string till the first null
i.e., till "welcome" in the above case.
So, if anyone knows the solution for this please let me know. Thanks in advance for any leads.

Split ogg vorbis stream without BOS

Input: a stream of ogg/vorbis coming from an encoder chip of an embedded system.
Problem: create output chunks of one second without transcoding.
Issue: the stream is being read "in the middle", so the first page with BOS (Beginning of Stream) is not available. Since the encoder chip has always the same parameters, I'd like to recreate the BOS page using the BOS page of a stream that was read from the start (reference stream).
I am trying to use vcut. I modified it so that it creates infinite chunks of one second. It was easy, and it works with files and streams with BOS.
I also hacked it so that I wrote to a file the first pages of the reference stream and then read them before reading the production stream with no BOS. In this way, vs->headers are populated. When I detect a page serial number change, I change it so that vcut and libogg do not freak:
int process_page(vcut_state *s, ogg_page *page) {
...
else if(vs->serial != ogg_page_serialno(page))
{
// fprintf(stderr, _("Multiplexed bitstreams are not supported.\n"));
vs->stream_in.serialno = ogg_page_serialno(page);
vs->serial = ogg_page_serialno(page);
vs->granulepos = -1;
vs->initial_granpos = 0;
// ogg_stream_init(&vs->stream_in, vs->serial);
// vorbis_info_init(&vs->vi);
// vorbis_comment_init(&vs->vc);
s->vorbis_init = 1;
}
However, this gigantic hack does not work. How to solve this issue?
It actually works: see VS1053 split ogg.
What I needed to do was to consider that starting reading in the middle of the stream, granulepos was naturally high. So it was mine logical mistake.
In process_audio_packet, I added:
int process_audio_packet(vcut_state *s,
vcut_vorbis_stream *vs, ogg_packet *packet)
{
...
if(packet->granulepos >= 0)
{
if (!firstNonZeroGranule) { // my addition
firstNonZeroGranule = 1;
vs->initial_granpos = packet->granulepos - bs;
if(vs->initial_granpos < 0)
vs->initial_granpos = 0;
} else if(vs->granulepos == 0 && packet->granulepos != bs) {
...

what is wrong with the following while loop?

I have a c winsock code section where a client receives a comma delimited stream of file fingerprints as shown below. I need to extract the fingerprints from the stream using strtok_s() in a while loop. My problem is most of the time the client does not extract the exact number of fingerprints sent from the server, even though the data received(observed by debugging) is exactly what the server sent.
What am I missing here?
recv_size = recv(clnt_sock, fp_buf, BUF_LEN, 0);
received_fp_size += recv_size;
if (0 != (last_string_len = recv_size % 33))
strncpy(last_string, &fp_buf[(recv_size - last_string_len)], last_string_len);//
while (recv_size > 0)
{
unique_fp = strtok_s(fp_buf, ",", &strtk);
k:
while (unique_fp != NULL)
{
memcpy(unique_fp_buf[unique_files_count], unique_fp, 32);
unique_fp = strtok_s(NULL, ",", &strtk);
unique_files_count++;
}
recv_size = recv(clnt_sock, fp_buf, BUF_LEN, 0);
received_fp_size += recv_size;
if (last_string_len > 0)
{
unique_fp = strtok_s(fp_buf, ",", &strtk);
strncat_s(last_string, unique_fp, strlen(unique_fp));
memcpy(unique_fp, last_string, 32);
last_string_len = 0;
goto k;
}
}
The reason behind the if (0 != (last_string_len = recv_size % 33)) line is; The server sends a multiple of 33 byte strings(32 for the fingerprint and 1 for the coma demlimiter)
One problem is that you never check that fp_buf actually contains a complete token. For instance if the first call only receives 20 bytes, your code will fail by copying a partial fingerprint.
I think another problem is here:
memcpy(unique_fp, last_string, 32);
Seems you are copying into the receive buffer and therefore overwrites some data that you haven't processed yet. Further, you may overwrite a token.
Maybe you actually wanted:
memcpy(unique_fp_buf[unique_files_count], last_string, 32);
^^^^^^^^^^^
unique_fp = strtok_s(NULL, ",", &strtk);
unique_files_count++;
Besides that I think you are making the code much more complicated than needed. The use of a goto kind of tell you that your design is wrong.
Instead of using a last_string you could do:
1) Call recv
2) Process all complete fingerprints
3) Copy the remainder (i.e. the last partial fingerprint) to the start of `fp_buf`
4) Call `recv` with an offset into `fp_buf`
5) Repeat from step 2 (i.e. use a while loop - don't use goto
Step 3 could be something like:
recv_size = recv(clnt_sock, fp_buf + length_of_remainder , BUF_LEN - length_of_remainder, 0);
In that way you don't have to handle the last_string stuff

Extracting JSON text from char response buffer

I receive a large chunk of text from my Wi-Fi module.
Which is saved in my response buffer.
char wifiResponseBuffer[500];
The contents can be seen below :
AT+CIPSEND=84
> GET http://api.noteu.co.uk/v1/poll/get/?seria
SEND OK
+IPD,308:{"data":[{"line1":" Facebook Note ","line2":"Nathan Weighill also","line3":" commented on Harry ","line4":" Bailey's photo.","beep":1,"received_time":1424976639},{"line1":" Gmail Message ","line2":"","line3":"Noteu Error","line4":"","beep":1,"received_time":1424976640}],"summary":{"note_count":2}}
OK
OK
Unlink
I have a JSON parser library however need to extract the actual JSON text from the response before it can be parsed. This is at the first occurrence of { and last occurrence of }.
What combination of string functions can I use in C to find the indexes of these characters and then extract the JSON text.
Any help is greatly appreciated,
Jack
strchr is used to find the first occurrence of a character in a string. strrchr finds the last.
Here's a brief example of how you might use these:
int test(void)
{
char wifiResponseBuffer[500];
int wifi_len;
// get the WiFi data, leaving 1 byte for a NUL terminator
wifi_len = get_wifi_response(wifiResponseBuffer, sizeof(wifiResponseBuffer)-1);
if (wifi_len < 0)
return -1; // error
// NUL-terminate to use with strxxx functions
wifiResponseBuffer[wifi_len] = '\0';
// Find start of JSON data
const char *json_start = strchr(wifiResponseBuffer, '{');
if (json_start == NULL)
return -1;
json_start++; // advance past {
// Find end of JSON data
const char *json_end = strchr(json_start, '}');
if (json_end == NULL)
return -1;
// Pass the JSON data to the library
size_t json_len = json_end - json_start;
do_something_with_json_data(json_start, json_len);
}

Resources