I'm trying to read in an integer which is going to let the server know the message length. I read until I reach sizeof(int) bytes using a while loop. I'm following the same convention for the message and length using a while loop to call recv multiple times. If all I'm doing is reading a int can I just call recv directly and expect all the bytes?
If not then how should I read in a integer using a while loop.
struct CONN_STAT {
int size; // length function should return the length into this field
int nRecv; // bytes sent of message
int nSent; // bytes received of message
int lRecv; // bytes received of length
int lSent; // bytes received of length
};
server : How I'm reading length
I copied the logic from my message function it reads similar to this
but info is replaced by a char array and the (info + pStat->lRecv) works for it
int readLength(int sockfd, int * info, struct CONN_STAT * pStat){
int infoSize = sizeof(int);
// I copied the logic from my message function it reads similar to this
// but info is replaces by a char array and the (info + pStat->lRecv) works for
while(pStat->lRecv < infoSize){
int n = recv(sockfd, info + pStat->lRecv, infoSize - pStat->lRecv, 0);
if (n > 0) {
pStat->lRecv += n;
}
else if (n == 0 || (n < 0 && errno == ECONNRESET)) {
close(sockfd);
return -1;
}else if (n < 0 && (errno == EWOULDBLOCK)) {
//The socket becomes non-readable. Exit now to prevent blocking.
//OS will notify us when we can read
return 0;
}else {
printf("Unexpected recv error.");
}
}
return 0;
}
Calling it like this
readLength(sockfd, (int*)pStat->size, pStat);
error: warning: cast to 'int *' from smaller integer type 'int' [-Wint-to-pointer-cast]
If all I'm doing is reading a int can I just call recv directly and expect all the bytes?
Generally speaking, no. TCP is a byte-streaming protocol, so it doesn't guarantee anything about how many bytes will be delivered by any one call to recv(). It's entirely possible (and therefore, given enough time, inevitable) that you'll recv() only the first part of the integer from a given recv() call, and you'll need to save the bytes you've received into a buffer somewhere and plan to append the rest of the bytes to that buffer later on. You can only actually parse/use the received integer after you've collected the whole set of bytes that were used to represent it.
If not then how should I read in a integer using a while loop.
Pretty much the same way you are (presumably) reading in the data-payload that follows the integer: write any received bytes into an array until the array has the number of bytes in it that are required to parse it. (In this case, you need to have sizeof(int) bytes in your array before you can read the integer as an integer... and don't forget that sizeof(int) may be a different value on different machines, and that an int may be represented in either big-endian or little-endian form. You might want to use int32_t instead of int, and htonl() and ntohl() to handle any necessary endian-conversion)
Since you're using non-blocking I/O, I suggest putting your collection-buffer into the CONN_STAT struct, so that a given call to readLength() can update the array with any received bytes and then a subsequent call can update the array some more, and so on.
The way I think about is to just see it as receiving two data-buffers: The first buffer I can assume the size of -- it will always be sizeof(int) bytes long. The second buffer I will know the size of as soon as I have received the entire first buffer and can read what it contains. So I can use (almost) the same logic for both of the two buffers, and then repeat as necessary.
Related
I've recieved this assignment where I have to read from a file.txt(max size 4096B) four times, basically splitting it in 4 strings of equal size. I have to fill this structure(just consider field 'msg', i think the problem is there):
struct message {
long mtype
int nclient;
int pid;
char path[151];
char msg[1025];
};
I used an array of 4 struct message to store all 4 parts
This is my read:
struct message msgs[4];
for (int i = 0; i < 4; i++) {
msgs[i].nclient=pos+1;
msgs[i].mtype = 42;
msgs[i].pid = getpid();
strcpy(msgs[i].path, filespath[pos]);
if (read(fd, msgs[i].msg, nMsgSize[i]) == -1)
ErrExit("read failed");
printf("I've read: %s\nMSGSize: %d\nPath: %s\n",msgs[i].msg, nMsgSize[i], msgs[i].path);
}
I tested it on a file "sendme_5.txt" that has this text in it:
ABCD
And this is my output:
I've read: A MSGSize: 1 Path:
/home/luca/Desktop/system_call_meh/myDir/joe_bastianich/bruno_barbieri/sendme_5.txt
I've read: BP"�> MSGSize: 1 Path:
/home/luca/Desktop/system_call_meh/myDir/joe_bastianich/bruno_barbieri/sendme_5.txt
I've read: C#��;�U MSGSize: 1 Path:
/home/luca/Desktop/system_call_meh/myDir/joe_bastianich/bruno_barbieri/sendme_5.txt
I've read: D�.�>� MSGSize: 1 Path:
/home/luca/Desktop/system_call_meh/myDir/joe_bastianich/bruno_barbieri/sendme_5.txt
If i try to read the full file without dividing it in 4(with only one read), it displays it correctly.
The problem started when i changed the field char path[151]. We had to set the max size to 151 from PATH_MAX(4096) after a change in the assignment, but i dont know if it's related.
What is the problem here?
As stated above, read does not know what a null-terminated string is. It deals with raw bytes, making no assumptions about the data it reads.
As is, your strings are possibly not null-terminated. printf("%s", msgs[i].msg) might continue past the the end of the read data, possibly past the end of the buffer, searching for a null-terminating byte. Unless the data read happens to contain a null-terminating byte, or the buffer was zeroed-out beforehand (and not completely filled by read), this is Undefined Behaviour.
On success, read returns the number of bytes read into the buffer. This may be less than requested. The return value is of type ssize_t.
When using this system call to populate string buffers, the return value can be used to index and place the null-terminating byte. An additional byte should always be reserved for this case (that is, always read at most the size of the buffer minus one: char buf[256]; read(fd, buf, 255)).
Always check for error, or the return value of -1 will index the buffer out-of-bounds.
Assuming nMsgSize[i] is the exact size of the msgs[i].msg buffer:
ssize_t n;
if (-1 == (n = read(fd, msgs[i].msg, nMsgSize[i] - 1)))
ErrExit("read failed");
msgs[i].msg[n] = 0;
printf("READ:%zd/%d expected bytes, MSG:<<%s>>\n", n, nMsgSize[i] - 1, msgs[i].msg);
I was running into a problem i couldnt really solve so I restarted.
I had a problem with Data encapsulation or more specific with no encapsulation. So after I figured out, that encapsulation is useful, I started rewriting the code.
Now I run into a different Problem. Somehow my send and recv calls are not working as I want them to be.
Here is the part where I send:
char to_send[] = "hello. I am the Data.";
// get size of data
int len = strlen(to_send);
char slen[len];
sprintf(slen,"%d",len);
printf("%s\n",slen);
// send size of data
if(send(comm_fd,slen,len,0)<0){perror("Error on send"); exit(1);}
// send data
if(send(comm_fd,to_send,len,0)<0){perror("Error on send"); exit(1);}
And here Part where I recv:
// getting size of bytes to recv
char buf[1000];
bzero(buf,1000);
int rec = recv(comm_fd, buf, 100,0);
printf("rec\n: %i",rec);
printf("buf\n: %s\n", buf);
int buffsize;
buffsize = atoi(buf);
bzero(buf,1000);
printf("buffsize: %i\n",buffsize);
// recv the bytes
bzero(buf,1000);
rec = recv(comm_fd, buf, buffsize,0);
printf("rec\n: %i",rec);
printf("%s",buf);
So my problem now is: I can recv the size of the next Data and print it. But the Data itself is not showing up.
Can someone help me? I think I'm doing major things wrong (I'm new to C and to Network programming)
Thanks in advance
Two things with that first send call:
if(send(comm_fd,slen,len,0)<0){perror("Error on send"); exit(1);}
Here you send len number of bytes, but len is the length of to_send and not the length of slen. You will most likely send data from outside the initialized parts of slen which leads to undefined behavior
The second problem is that you send the length of to_send as a variable-length string, so the received doesn't actually know how much to receive. In your case you could actually (and probably do) receive the length and the string in a single recv call. At least if you're using TCP (streaming) sockets.
Both of these problems can be solved by making slen a fixed-size array, big enough to hold the largest numbers you can think of (ten digits is usually enough), and then send this fixed-length array using sizeof slen .
Perhaps something like this:
// Ten digits, plus string terminator
char slen[10 + 1];
// Prefix length with zeroes, and don't overflow the buffer
snprintf(slen, sizeof(slen), "%010d", strlen(to_send));
// Send the whole array, including terminator
send(comm_fd, slen, sizeof slen, 0);
Then on the receiving side, you could do
// Ten digits, plus string terminator
char slen[10 + 1];
// Receive the whole string, including terminator
recv(comm_fd, slen, sizeof(slen), 0);
// Convert to a number
size_t len = strtoul(slen, NULL, 10);
// Now receive `len` bytes
Note that I have no error checking, which you should have.
EDIT: It has been proven in the comments that defining the length instead should produce the same results and would not use any significant extra data. If you are looking for a way to send data between machines running your program(s), sending the length is better than reading until a terminating character. BonzaiThePenguin has some very good points you should look at.
But for educational purposes: I never found good example code that does this for standard C sockets that handles situations where the data is not all received in one packet, or multiple separate messages are contained within one packet. Simply calling recv repeatedly will not work in all cases.
This is one of those questions where I've answered it myself below, but I'm not 100% confident in my response.
It isn't 'dangerous to allow the client to specify the size of the message it is sending'. Most of the protocols in the word do that, including HTTP and SSL. It's only dangerous when implementations don't bounds-check messages properly.
The fatal flaw with your suggestion is that it doesn't work for binary data: you have to introduce an escape character so that the terminating character can appear within a message, and then of course you also need to escape the escape. All this adds processing and data copying at both ends.
Here is what I came up with. I cannot guarantee that this is perfect because I am not a professional, so if there are any mistakes, I (and anyone else looking for help) would greatly appreciate it if someone would point them out.
Context: socket is the socket, buffer is the array that stores all network input, line is the array that stores just one message extracted from buffer (which is what the rest of your program uses), length is the length of both inputted arrays, and recvLength is a pointer to an integer stored outside of the function that is meant to be 0 initially and should not be freed or modified by anything else. That is, it should persist across multiple calls to this function on the same socket. This function returns the length of the data outputted in the line array.
size_t recv_line(int socket, char* buffer, char* line, size_t length, size_t* recvLength){ //receives until '\4' (EOT character) or '\0' (null character)
size_t readHead = 0;
size_t lineIndex = 0;
char currentChar = 0;
while (1){
for (; readHead < *recvLength; readHead = readHead + 1){
currentChar = buffer[readHead];
if (currentChar=='\4' || currentChar=='\0'){ //replace with the end character(s) of your choice
if (DEBUG) printf("Received message===\n%s\n===of length %ld\n", line, lineIndex+1);
memcpy(buffer, buffer + readHead + 1, length-(readHead)); //shift the buffer down
*recvLength -= (readHead + 1); //without the +1, I had an "off by 1" error before!
return lineIndex+1; //success
}
if (readHead >= length){
if (DEBUG) printf("Client tried to overflow the input buffer. Disconnecting client.\n");
*recvLength = 0;
return 0;
}
line[lineIndex] = currentChar;
lineIndex++;
}
*recvLength = recv(socket, buffer + readHead, length, 0);
}
printf("Unknown error in recv_line!\n");
return 0;
}
Simple example usage:
int function_listening_to_network_input(int socket){
char netBuffer[2048];
char lineBuffer[2048];
size_t recvLength = 0;
while (1==1){
size_t length = recv_line(socket, netBuffer, lineBuffer, 2048, &recvLength);
// handle it…
}
return 0;
}
Note that this does not always leave line as a null-terminated string. If you want it to, it's easy to modify.
How can I store the result of Serial.readBytesUntil(character, buffer, length) in a buffer while I don't know the length of the incoming message ?
Here is a little code that makes use of realloc() to keep growing your buffer. You will have to free() when you're done with buf.
int length = 8;
char * buf = malloc(length);
int total_read = 0;
total_read = Serial.readBytesUntil(character, buf, length);
while(length == total_read) {
length *= 2;
buf = realloc(buf, length);
// Bug in this line:
// total_read += Serial.readBytesUntil(character, buf+total_read, length);
// Should be
total_read += Serial.readBytesUntil(character, buf+total_read, length-total_read);
}
*Edit: fixed a bug where readBytesUntil would have read off the end of buf by reading length bytes instead of length-total_read bytes.
make the buffer big enough for the message. Don't know the maximum length of the message? Use length to control the characters read, then continue reading until character encountered.
int bytesRead = Serial.readBytesUntil(character, buffer, length);
You could create a buffer that is just smaller than the remaining RAM and use that. The call to find the remaining ram (as I've posted elsewhere) is:
int freeRam () {
extern int __heap_start, *__brkval;
int v;
int fr = (int) &v - (__brkval == 0 ? (int) &__heap_start : (int) __brkval);
Serial.print("Free ram: ");
Serial.println(fr);
}
Regardless, you should make sure you only read into as much RAM as you actually have.
One answer is that when a program reads serial bytes it typically does NOT store them verbatim. Rather, the program examines each byte and determines what action to take next. This logic is typically implemented as Finite State Machine.
So, what does your specific serial stream represent? Can it be analyzed in sequential chunks? For example: "0008ABCDEFGH" says that 8 chars follow the 4 character length field. In this silly example your code would read 4 chars, then know how much space to allocate for the rest of the serial stream!
I wonder about a buffer overflow in my app.
For example I have this code:
enum { BUFSIZE = 1024};
char username[this->BUFSIZE];
memset(username,0,BUFSIZE);
char password[this->BUFSIZE];
memset(password,0,BUFSIZE);
send(client_fd, "Login: ", BUFSIZE, 0);
recv(client_fd,username,sizeof(username),0)
Can a malicious attacker type more than 1024 chars and do a bof?
send(client_fd, "Login: ", BUFSIZE, 0);
-- For this statement, send will try to send BUFSIZE bytes though your string is only "Login: ". send doesn't inherently try to understand a 'C' string. It just recognizes a byte stream. So, your statement for send is incorrect.
recv(client_fd,username,sizeof(username),0)
-- In case of recv, though you have mentioned "sizeof(username)", it doesn't mean it will return "sizeof(username)" bytes, the number of bytes returned can be found out using the return value of recv. Never try to interpret the contents of the buffer passed to recv without checking the return value of recv. But, specifying sizeof(username) ensures that recv call will not return more than sizeof(username) bytes back even if more number of bytes are present in the network buffer.