I'm trying to use sockets to get a small JSON test file, which is hosted on my website (http://a-cstudios.com/text.json). When I do this
long numbytes;
char *request = malloc(sizeof(char) * 300);
sprintf(request, "GET %s \r\nHOST:%s \r\n\r\n", restOfURL, baseServer);
// restOfURL = "/text.json" baseServer = "www.a-cstudios.com"
send(sockfd, request, strlen(request) + 1, 0);
char buf[1024];
if ((numbytes = recv(sockfd, buf, 1024-1, 0)) == -1) {
perror("recv");
}
I get recv: connection reset by peer. But if I use the same code, where restOfURL is /index.html and baseServer is www.google.com, this works fine, and buf will contain the text of index.html. Why won't this work for the file on my website?
Since you didn't post full code, I am going to take a stab at it and make an assumption:
You populate the format string of "GET %s \r\nHOST:%s \r\n\r\n" with restOfURL and baseServer
However, during the time of the sprintf call restOfURL isn't initilized so you're pushing garbage data into the first %s
Either post more of your code or make sure you initialize resOfURL
As #Kninnug pointed out, you need the HTTP version field (e.g., HTTP/1.1) at the end of the first line of the request. I just want to point out that you should not include the null terminator when you send the request. That is, change the send statement to
send(sockfd, request, strlen(request), 0);
Also, it is a good practice to always use snprintf instead of sprintf to prevent buffer overflow, although to be really safe you still need to check for truncation.
Related
I wrote a simple HTTP Client that can request data from a host.
I am using getaddrinfo(3). With "GET / HTTP/1.1" request I am able to download HTML page of a given http Host.
Here is a part of my code:
struct addrinfo hints, *ai;
memset(&hints, 0, sizeof(hints));
hints.ai_family = AF_INET; // Internet Protocol (IP) socket
hints.ai_socktype = SOCK_STREAM; // TCP
int res = getaddrinfo("example.com", "http", &hints, &ai);
if (res != 0)
ERROR_EXIT("getaddrinfo: %s\n", gai_strerror(res));
int sockfd = socket(ai->ai_family, ai->ai_socktype, ai->ai_protocol);
if (sockfd < 0)
ERROR_EXIT("socket: %s\n", strerror(errno));
if (connect(sockfd, ai->ai_addr, ai->ai_addrlen) < 0)
ERROR_EXIT("connect: %s\n", strerror(errno));
FILE *sockfile = fdopen(sockfd, "r+");
if (sockfile == NULL)
ERROR_EXIT("fdopen: %s\n", strerror(errno));
// send a GET request to the server:
if (fputs("GET / HTTP/1.1\r\n\r\n", sockfile) == EOF)
ERROR_EXIT("fputs: %s\n", strerror(errno));
if (fflush(sockfile) == EOF)
ERROR_EXIT("fflush: %s\n", strerror(errno));
char buf[1024];
// print the reply:
while (fgets(buf, sizeof(buf), sockfile) != NULL)
fputs(buf, stdout);
fclose(sockfile);
return 0;
Downloading a HTML page works ok, but downloading PNG image for example "GET /image.png HTTP/1.1\r\n\r\n" gives something like this:
???????ݹh??DHDZ?yW]%?9a??J?6F?Ѧ?E???ݐTd?US?:)??I??M,?-????=??U??&???Nr? ???б???
b??]??8?6+?;??i䂢d?G?WA?rԺ?H[??]?Z5????g?{8??i\?qAC?#c??v.?rb??'<?T?????O?z?
q,yĜ?ŷZI???X??fM?l?Z??l:;M???ۦ?????c?\\?W6+???o?}_???紈A??GvG?p??6{??{%?????0?{?
%??ژ??l?$r<?????ft*'W?N?m߂Ҿ4??E?:^?#?&?%%
????Dw??Z?$??1?4?l%&2?f-5!?? ?E? 8...
I understand that this is a byte transfer and that I have to do byte ordering, but don't know where to begin.
I know that I need to use ntohl(3) and Big-endian order for PNG images. Can Someone give me directions what to look up and how to approach this?
Do I save this output to .png file and then do byte order or do I do before I create a .png file?
The problem is more complex than just "byte order".
The Good News is that byte order is probably irrelevant. The code you've got now should work just fine sending and receiving text strings with the server.
The Bad News is that you need to connect differently if you want to read binary data from the server. Several considerations:
It sounds like you probably don't need to worry about "POST", "PUT" ... or anything but "GET".
It also sounds like you don't need to worry about "MIME Types" or "uuencoding/decoding" - you'll just be reading the data as-is.
You should definitely read the Content-Type and Content-Length headers from the server's HTTP response.
Perhaps the simplest approach is just call fgetc() in a loop if you're reading binary data. Use fgets() for strings, and fgetc() for Content-Length bytes to read the image.
You'll probably want to do an fopen() to write the image bytes to a file.
But calling fgets() eliminates the need to worry about "byte order".
Look here, here and here for examples.
'Hope that helps...
It wouldn't make a whole lot of sense to write out the image's bytes before deciding which order they should be in. What would be your next line be, fopen()?
If you're doing something architecture dependent like this, it's best to guard the case-specific sections with #if/#elif, and use #else to emit #error with a diagnostic message.
If everything is supposed to still work, it can just be #warning... but do put the warning in. For later, when it stops working.
Configuration macros you can test should include #ifdef __BIG_ENDIAN__ and #if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__; you can see what else is available by 'compiling' with -E -dD. [there will be a lot; grep is your friend]
Oh, and "big endian" means if you're reading bytes in address order (a[0], a[1], a[2], ...), then the byte at this end is the most significant/highest 'digit'. Hence "big-endian".
edit: this may also be of concern...
FILE *sockfile = fdopen(sockfd, "r+"); // "r+"
if (sockfile == NULL)
ERROR_EXIT("fdopen: %s\n", strerror(errno));
if (fputs("GET / HTTP/1.1\r\n\r\n", sockfile) == EOF) // fputs...
This question already has an answer here:
Differ between header and content of http server response (sockets)
(1 answer)
Closed 4 years ago.
I'm writing a wrapper for Berkley sockets on Windows and Linux. The test program got problem here:
char buf[BUFSIZE];
int res = 0;
while((res = NetRecv(sock, buf, BUFSIZE, 0)) > 0) // 'NetRecv' is pointing to 'recv'
{
buf[res-1] = '\0';
printf("%s", buf);
}
The response is to a HTTP-Get request of a web-page content. The socket is streaming.
The 'NetRecv' is initialized correctly - that is, no type mismatch of the functions' pointers there is, I've checked it.
So, Windows version works flawlessly, the Linux one is stuck after reading all page. Namely, the previous to the last 'NetRecv' call accepts last chunk of the response, outputs it, and the next (last) call just blocks. Closing the terminal causes 'SIGHUP' signal.
Looks like the Linux version just doesn't realize, that it received the last chunk of data and waits for more.
Is it as it should be? Don't understand then, for what reason there is blocking call possibility.
Now, I surely could make non-blocking call and use 'select', but do I really have to?
Thanks in advance)
EDIT: Minimal working example (all checks are omitted and net functions are the standard ones, which also were tested):
int sock = socket(AF_INET, SOCK_STREAM, 0);
// Here getting good IP address of google.com - no problem here
char serv_ip[IPADDR_BUFSIZE];
GetHostAddrByName(AF_INET, "www.google.com", serv_ip, IPADDR_BUFSIZE);
// ip ver site out buf out buf size
// The routine above is made with 'getaddrinfo', to be precise
printf("Current IP of '%s' is '%s'.\n", SERV_URL, serv_ip);
// Copying IP string to address struct
struct sockaddr_in addr;
NetIpFromStr(AF_INET, serv_ip, &addr.sin_addr);
addr.sin_family = AF_INET;
addr.sin_port = NetHtons(80);
connect(sock, (const struct sockaddr*)&addr, sizeof(addr));
const char* msg = "GET / HTTP/1.1\r\n\r\n";
send(sock, msg, strlen(msg), 0);
char buf[BUFSIZE];
int res = 0;
while((res = recv(sock, buf, BUFSIZE-1, 0)) > 0)
{
buf[res] = '\0';
printf("%s", buf);
}
EDIT 2: Important notice: the Windows version also blocks the call, when all the data is read. Closing the terminal just doesn't crash the program, like it happens in Linux. Therefore, the whole question is such: How to realize that all data is read?
The problem is that you are blindly reading from the socket in a loop until an error occurs. Once you have received the entire response, you go back to the socket and keep reading, which then blocks because there is nothing left to read. The only error that can occur at this point is when the connection is closed (or lost), which the server is likely not doing since you are sending an HTTP 1.1 request, where keep-alive is the default behavior for 1.1 (see RFC 2616 Section 8.1 Persistent Connections)
The correct solution is to parse the HTTP response and stop reading from the socket when you reach the end of the response, NOT simply relying on the server to close the socket. Read RFC 2616 Section 4.4 Message Length for how to detect when you have reached the end of the response. DO NOT read more than the response indicates! Once you stop reading, then you can decide whether to close your end of the socket, or reuse it for a new request.
Have a look at this pseudo code for the type of parsing and reading logic you need to use.
Also, your HTTP request is malformed, as you are not sending a required Host header, so no matter what, you will always receive a 400 Bad Request response from any HTTP 1.1 compliant server:
const char* msg = "GET / HTTP/1.1\r\n"
"Host: www.google.com\r\n" // <-- add this!
"\r\n";
The solution was to shutdown the socket for reading, both in Windows and Linux:
// after sending a request:
shutdown(sock, SD_SEND); // or 'SHUT_WR' in Linux
// now read loop
Curiously, 'shutdown' was called in Winsock tutorials too, but I thought that was unnecessary.
I am trying to learn client server model in Linux and I have setup two C files namely server.c and client.c. These are the code snippets that I seem to have problems with.
server.c code snippet
char* message = "<query>\n";
write(client_socket_filedescriptor, message, sizeof(message));
client.c code snippet
char* message = "<query>\n";
read(socket_filedescriptor, buffer, sizeof(buffer));
printf("%s", buffer);
printf("\n\n");
printf("%s", message);
Now when I run my server and then when I run my client, I expect the printf statements to print the same strings that is <query>\n, but I keep getting different outputs for buffer and message variables.
The output looks a bit like this when I run client code.
Output image
As you see, these two strings are different. I am trying to simulate a typical TCP handshake and I want to make sure that these two strings are same and then client will start writing or doing something with that server. But I am having this trivial problem. Could anyone tell my how to resolve it? I plan to use strcmp to compare buffer and message variables, but as it stands now, strcmp doesn't return 0 since these are different strings afterall.
You are ignoring the count returned by read(). It can be -1, indicating an error, or zero, indicating end of stream, or a positive number, indicating how many bytes were received. You cannot assume that read() fills the buffer, or that a single send() or write() corresponds to a single recv() or read().
In detail:
write(client_socket_filedescriptor, message, sizeof(message));
You are only sending four bytes, the size of the pointer. And you're ignoring the return value. It should be
int count = write(client_socket_filedescriptor, message, strlen(message));
if (count == -1)
perror("write"); // or better
char* message = "<query>\n";
read(socket_filedescriptor, buffer, sizeof(buffer));
That should be
int count = read(socket_filedescriptor, buffer, sizeof(buffer));
if (count == -1)
perror("read"); // or better
else if (count == 0)
; // end of stream: the peer has disconnected: close the socket and stop reading
else
Back to your code:
printf("%s", buffer);
That should be
printf("%.*s", count, buffer);
I plan to use strcmp()
You should plan to use strncmp(), with count above as the length parameter. In any case you can't assume the input ends with a null unless you (a) ensure you send the null, which you aren't, and (b) write a read loop that stops when you've read it.
I have some problems when trying to receive http response message of a website.
This is my function:
void Reveive_response(char *resp, SSL *ssl) {
const int BUFFER_SIZE = 1024;
char response[1048576];
char *buffer = NULL; // to read from ssl
char *check = (char *) malloc(BUFFER_SIZE*sizeof(char));
int bytes; // number of bytes actually read
int received = 0; // number of bytes received
buffer = (char *) malloc(BUFFER_SIZE*sizeof(char)); // malloc
memset(response, '\0', sizeof(response)); // response
assign = '\0'
do{
memset(buffer, '\0', BUFFER_SIZE); // empty buffer
bytes = SSL_read(ssl, buffer, BUFFER_SIZE);
if (bytes < 0) {
printf("Error: Receive response\n");
exit(0);
}
if (bytes == 0) break;
received += bytes;
printf("Received...%d bytes\n", received);
strncat(response, buffer, bytes); // concat buffer to response
} while (SSL_pending(ssl)); // while pending
response[received] = '\0';
printf("Receive DONE\n");
printf("Response: \n%s\n", response);
free(buffer);
strcpy(resp, response); // return via resp
}
When I call the function, it seems like the response message is not complete. Like this:
Received...1014 bytes
Received...1071 bytes
Receive DONE
Response:
HTTP/1.1 200 OK
<... something else....>
Vary: Accept-Encoding
Content-Type: text/html
Conne
Then if i call the function again, it returns:
Received...39 bytes
Receive DONE
Response:
ction: keep-alive
Content-Length: 0
The field Connection was split. Why my function didn't receive all the response message? I used do while loop inside. Please tell me where did i go wrong? Thank you.
There is nothing wrong. This is simply how TCP works. It is a streaming transport, it has no concept of message boundaries. There is no 1-to-1 relationship between the number of bytes sent and the number of bytes read. Your reading receives arbitrary bytes, which you are then responsible for processing as needed. Keep reading, buffering and parsing the HTTP data as you go, until you discover the end of the response (see RFC 2616 Section 4.4 Message Length for details). Looping on SSL_pending() is not sufficient (or correct).
In this case, you have to read CRLF-delimited lines one at a time until you reach a CRLF/CRLF pair indicating the end of the response headers, then you need to analyze the headers you have received to know whether a response body is present and how to read it, as it may be in one of several different encoded formats. If present, you can then read the body (decoding it as you go along) until you reach the end of the body as specified by the headers.
See the pseudo-code I posted in my answer to the following question:
Receiving Chunked HTTP Data With Winsock
That said, you really should not be implementing HTTP (let alone HTTPS) manually to begin with. HTTP is not trivial to implement from scratch, and neither is SSL/TLS for that matter. You have dived head-first into a deep well without understand some important basics of network programming and OpenSSL programming. You should use an existing HTTP/S library instead, such as libcurl, and let it handle the details for you so you can focus on your code's business logic and not its communications logic.
I have a very big problem... I'm working with sockets in C. I send a request to the server which sends me many responses. The problem is that the client receives the first response and then the connection is closed. What can I do? I tried with setsockopt()... SO_KEEPALIVE or SO_LINGER but I haven't resolved the problem. Can you help me? Thanks a lot
To be more clear here is the code. The socket is automatically closed after a certain amount of time or after the client received the first response... I'm not sure.
char* demarre_client( client_args * c_args,char* message, /*char* SERVEUR_PORT*/int port){
int socket_client=socket(PF_INET,SOCK_STREAM,0);
memset(&(c_args->adresse_serveur),0,sizeof(c_args->adresse_serveur));
c_args->adresse_serveur.sin_family=AF_INET;
c_args->adresse_serveur.sin_addr.s_addr=inet_addr(SERVEUR_IP);
//int port=APHash(SERVEUR_PORT,strlen(SERVEUR_PORT));
c_args->adresse_serveur.sin_port=htons(port);
int err=0;
if ((err=connect(socket_client, (struct sockaddr *) &(c_args->adresse_serveur), sizeof(c_args->adresse_serveur)))<0)
{
printf("CONNECT error %d\n", errno);
exit(-1);
}
if (send(socket_client, message, strlen(message), 0)!=strlen(message))
{
printf("send error!!\n");
exit(-2);
}
char* raspFin=(char* )malloc(sizeof(char)*1024);
strcpy(raspFin,"");
int num=-1;
int nn=0;
char* rasp=(char* )malloc(sizeof(char)*1024);
memset (rasp, 0, 1024 * sizeof(char ));
/* here it reads the first response and after he get out of while */
while ((num=recv(socket_client, rasp,1024,MSG_WAITALL))>0)
{
printf("recu %s mun=%d\n" , rasp,num);
strcat(raspFin,rasp);
strcat(raspFin,"\n");
rasp=(char* )malloc(sizeof(char)*1024);
memset (rasp, 0, 1024 * sizeof(char ));
}
if (num<0)
{
printf("rasp error!!\n");
exit(-3);
}
Are you sure you don't get all responses on first call?
TCP/IP is stream protocol without flow control built-in so different messages, sent using separate send() calls, can be received in one recv(). Because you use printf(), it prints the buffer until it sees null-terminator - maybe other responses beyond the terminator?
Try to use some flow control, like sending message length prefix or using some special characters (like STX/ETX, but make sure your message doesn't contain such characters). You'd need to implement some flow-control anyway if you plan to use this software.
For now try replacing your printf() with
char *ptr;
for (ptr = buffer; ptr <= buffer + num; ptr += strlen(ptr)+1;)
printf("%s\n", ptr);
It will print all strings from your response buffer.
And you don't need malloc() inside the loop - you leak memory.
BTW SO_KEEPALIVE and SO_LINGER have nothing to do with this problem.
My suggestion would be to fire up Wireshark network analyzer and see what's happening packet-wise. In filters set
tcp.srcport == <insert_server_port> || tcp.dstport == <insert_server_port>
You should see what data actually gets sent to and who closes the connection (sends FIN/RST packets).