sending avro(avro c) encoded data through a socket
I'm trying to send an avro encoded data through a socket after converting it to byte array(using memcpy).What I have done is as given below
/client side : client.c/
avro_datum_t t_msg = avro_record(temp_schema);
avro_record_set(t_msg, "TIME", time_datum) // encoded like this
...
...
unsigned char *temp=(unsigned char *) malloc(sizeof(t_msg));
memcpy(temp,&t_msg,sizeof(t_msg));//converting to byte array
sendto(sock, &temp, strlen(temp), 0, (struct sockaddr *)&server_addr, sizeof(struct sockaddr));//send to socket
/server side :server.c/
avro_datum_t time_datum;
..
..
unsigned char *recv_data=(unsigned char *) malloc(sizeof(t_msg));
avro_datum_t t_msg;
bytes_read = recvfrom(sock,recv_data,1024,0,(struct sockaddr *)&client_addr, &addr_len);
memcpy(bgpmsg1,recv_data,sizeof(recv_data));
if (avro_record_get(t_msg, "TIME", &time_datum) == 0) {
printf("\nHi...0\n");
avro_string_get(time_datum, &p);
fprintf(stdout, "TIME: %s ", p)
}
//trying to decode and print message like this
but it's not getting anything...when I try to decode and print in clent.c itself it is getting printed.But after sending through socket it is not able to retrieve..
The data pointed to by temp is not actually a string, and so you can't use string functions like strlen with it.
Think about what happens if one byte in the middle is zero (i.e. the string terminator '\0'), the functions look for that to know when a string ends, and so will stop their processing there.
On the receiving side you have another problem, and is your use of sizeof(recv_data), which will always be 4 or 8 bytes depending on if you're on a 32 or 64 bit platform. The sizeof operator is a compile-time only operator, and for pointers it returns the size of the pointer and not what it points to.
On the receiving side you have a potentially even worse problem, which is that you tell recvfrom to receive up to 1024 bytes, instead of e.g. sizeof(t_msg).This might cause recvfrom to write beyond the allocated data if sizeof(t_msg) is less than 1024.
Related
I'm trying to read in an integer which is going to let the server know the message length. I read until I reach sizeof(int) bytes using a while loop. I'm following the same convention for the message and length using a while loop to call recv multiple times. If all I'm doing is reading a int can I just call recv directly and expect all the bytes?
If not then how should I read in a integer using a while loop.
struct CONN_STAT {
int size; // length function should return the length into this field
int nRecv; // bytes sent of message
int nSent; // bytes received of message
int lRecv; // bytes received of length
int lSent; // bytes received of length
};
server : How I'm reading length
I copied the logic from my message function it reads similar to this
but info is replaced by a char array and the (info + pStat->lRecv) works for it
int readLength(int sockfd, int * info, struct CONN_STAT * pStat){
int infoSize = sizeof(int);
// I copied the logic from my message function it reads similar to this
// but info is replaces by a char array and the (info + pStat->lRecv) works for
while(pStat->lRecv < infoSize){
int n = recv(sockfd, info + pStat->lRecv, infoSize - pStat->lRecv, 0);
if (n > 0) {
pStat->lRecv += n;
}
else if (n == 0 || (n < 0 && errno == ECONNRESET)) {
close(sockfd);
return -1;
}else if (n < 0 && (errno == EWOULDBLOCK)) {
//The socket becomes non-readable. Exit now to prevent blocking.
//OS will notify us when we can read
return 0;
}else {
printf("Unexpected recv error.");
}
}
return 0;
}
Calling it like this
readLength(sockfd, (int*)pStat->size, pStat);
error: warning: cast to 'int *' from smaller integer type 'int' [-Wint-to-pointer-cast]
If all I'm doing is reading a int can I just call recv directly and expect all the bytes?
Generally speaking, no. TCP is a byte-streaming protocol, so it doesn't guarantee anything about how many bytes will be delivered by any one call to recv(). It's entirely possible (and therefore, given enough time, inevitable) that you'll recv() only the first part of the integer from a given recv() call, and you'll need to save the bytes you've received into a buffer somewhere and plan to append the rest of the bytes to that buffer later on. You can only actually parse/use the received integer after you've collected the whole set of bytes that were used to represent it.
If not then how should I read in a integer using a while loop.
Pretty much the same way you are (presumably) reading in the data-payload that follows the integer: write any received bytes into an array until the array has the number of bytes in it that are required to parse it. (In this case, you need to have sizeof(int) bytes in your array before you can read the integer as an integer... and don't forget that sizeof(int) may be a different value on different machines, and that an int may be represented in either big-endian or little-endian form. You might want to use int32_t instead of int, and htonl() and ntohl() to handle any necessary endian-conversion)
Since you're using non-blocking I/O, I suggest putting your collection-buffer into the CONN_STAT struct, so that a given call to readLength() can update the array with any received bytes and then a subsequent call can update the array some more, and so on.
The way I think about is to just see it as receiving two data-buffers: The first buffer I can assume the size of -- it will always be sizeof(int) bytes long. The second buffer I will know the size of as soon as I have received the entire first buffer and can read what it contains. So I can use (almost) the same logic for both of the two buffers, and then repeat as necessary.
I was running into a problem i couldnt really solve so I restarted.
I had a problem with Data encapsulation or more specific with no encapsulation. So after I figured out, that encapsulation is useful, I started rewriting the code.
Now I run into a different Problem. Somehow my send and recv calls are not working as I want them to be.
Here is the part where I send:
char to_send[] = "hello. I am the Data.";
// get size of data
int len = strlen(to_send);
char slen[len];
sprintf(slen,"%d",len);
printf("%s\n",slen);
// send size of data
if(send(comm_fd,slen,len,0)<0){perror("Error on send"); exit(1);}
// send data
if(send(comm_fd,to_send,len,0)<0){perror("Error on send"); exit(1);}
And here Part where I recv:
// getting size of bytes to recv
char buf[1000];
bzero(buf,1000);
int rec = recv(comm_fd, buf, 100,0);
printf("rec\n: %i",rec);
printf("buf\n: %s\n", buf);
int buffsize;
buffsize = atoi(buf);
bzero(buf,1000);
printf("buffsize: %i\n",buffsize);
// recv the bytes
bzero(buf,1000);
rec = recv(comm_fd, buf, buffsize,0);
printf("rec\n: %i",rec);
printf("%s",buf);
So my problem now is: I can recv the size of the next Data and print it. But the Data itself is not showing up.
Can someone help me? I think I'm doing major things wrong (I'm new to C and to Network programming)
Thanks in advance
Two things with that first send call:
if(send(comm_fd,slen,len,0)<0){perror("Error on send"); exit(1);}
Here you send len number of bytes, but len is the length of to_send and not the length of slen. You will most likely send data from outside the initialized parts of slen which leads to undefined behavior
The second problem is that you send the length of to_send as a variable-length string, so the received doesn't actually know how much to receive. In your case you could actually (and probably do) receive the length and the string in a single recv call. At least if you're using TCP (streaming) sockets.
Both of these problems can be solved by making slen a fixed-size array, big enough to hold the largest numbers you can think of (ten digits is usually enough), and then send this fixed-length array using sizeof slen .
Perhaps something like this:
// Ten digits, plus string terminator
char slen[10 + 1];
// Prefix length with zeroes, and don't overflow the buffer
snprintf(slen, sizeof(slen), "%010d", strlen(to_send));
// Send the whole array, including terminator
send(comm_fd, slen, sizeof slen, 0);
Then on the receiving side, you could do
// Ten digits, plus string terminator
char slen[10 + 1];
// Receive the whole string, including terminator
recv(comm_fd, slen, sizeof(slen), 0);
// Convert to a number
size_t len = strtoul(slen, NULL, 10);
// Now receive `len` bytes
Note that I have no error checking, which you should have.
In case someone gets this issue in the future I'll leave this up.
*Note This approach wasn't going to work when going from C client to C server. This would have only worked with the Java client to C server. So I had to abandoned this approach.
Ok, I've been fighting with C for too long now. I'm passing some info with UDP from a java client to a C server. I can get the info there, but I'm not sure how to break apart the message to store into a struct like so
struct __attribute__((__packed__)) clientMessage
{
short tml;
short rid;
char op;
char message[MAXBUFLEN-5];
};
I recieve the message like this
Where test is a char test[MAXBUFLEN-5];
if ((numbytes = recvfrom(sockfd, test, MAXBUFLEN-1, 0,
(struct sockaddr *)&their_addr, &addr_len)) == -1) {
perror("recvfrom");
exit(1);}
So, I need to take the message "7 2 1Yo" (two 2 byte shorts and a char followed by an unknown message length) and store it into it's appropriate parts in the struct. The message gets sent correctly, I just can't break it into the bits of info that I need. I'm currently attempting
memcpy(&cm.rid, &test, 2);
memcpy(&cm.tml, &test[1], 2);
memcpy(&cm.op, &test[4], 1);
memcpy(&cm.message, &test[5], MAXBUFLEN-5);
But my resutls end up being
Message: Yo
OP: 1Yo
RID: 7 1Yo
TML: 2 7 1Yo
it Should be
Message: Yo
OP: 1
RID: 2
TML: 7
I successfully get the message, but nothing else. I'm relatively new to C so forgive my ignorance. I'm guessing this is really easy but idk.
The first line should be memcpy(&cm.rid, &test[0], 2);, for the address is supposed to be the address of the first byte.
And the rest:
memcpy(&cm.tml, &test[2], 2); // you want to get the third and forth byte, begin with index 2.
memcpy(&cm.op, &test[4], 1); // the fifth byte, begin with index 4.
memcpy(&cm.message, &test[5], MAXBUFLEN-5); // the rest bytes.
If you are using a binary protocol, it would be better to use e.g. int16_t instead of short in the declaration of clientMessage because the size of short is not specified within the c standard.
To access the message stored in test you could do simply something like this:
struct clientMessage *cm = (struct clientMessage *) test;
You should also notice, that the endianness of network protocols is different from the endianness of a x86 and amd64 architecture, so instead of 42 you might recieve 10752.
To fix this you can use the ntohs() (Network TO Host Short) function to acces the tm and rid.
Use sscanf() for the first part and memcpy() for the text.
const char *message = "7 2 1Yo";
struct clientMessag cM;
int offset = 0;
int n;
n = sscanf(message, "%hd %hd %c%n", &cM.tml, &cM.rid, &cM.op, &offset);
size_t SuffixLength = strlen(&message[offset]);
if ((n != 3) || (SuffixLength >= sizeof(cM.message))) {
exit(1); // handle syntax error;
}
memcpy(cM.message, &message[offset], SuffixLength + 1);
// Additional field checks like IsOKOpCode(cM.op)
I avoided %s as it does not store whitespace into your message.
Error checking is always good to do. Recommend additional field checks.
BTW: its not clear if the cM.op field should be treat as text or a number. Does the OP want a 1 or '1' stored? The above assume text. Alternatively one could use "%hd%hd%hhd%n".
Since test contains the string "7 2 1Yo", you have to parse the string into the values you want. You can do this with sscanf():
char fmt[256];
snprintf(fmt, sizeof(fmt), "%%hd %%hd %%c%%%ds", MAXBUFLEN-5-1);
sscanf(test, fmt, &cm.rid, &cm.tml, &cm.op, cm.message);
In this code fragment above, we create a format string for the parsing. This is necessary to prevent sscanf() from scanning past the end of the message in the case that it is not \0 terminated. The result of the snprintf() call with a MAXBUFLEN of 128 is:
"%hd %hd %c%122s"
Which tells sscanf() to scan for two short decimal numbers, a char, and a string no longer than 122 characters.
I wonder about a buffer overflow in my app.
For example I have this code:
enum { BUFSIZE = 1024};
char username[this->BUFSIZE];
memset(username,0,BUFSIZE);
char password[this->BUFSIZE];
memset(password,0,BUFSIZE);
send(client_fd, "Login: ", BUFSIZE, 0);
recv(client_fd,username,sizeof(username),0)
Can a malicious attacker type more than 1024 chars and do a bof?
send(client_fd, "Login: ", BUFSIZE, 0);
-- For this statement, send will try to send BUFSIZE bytes though your string is only "Login: ". send doesn't inherently try to understand a 'C' string. It just recognizes a byte stream. So, your statement for send is incorrect.
recv(client_fd,username,sizeof(username),0)
-- In case of recv, though you have mentioned "sizeof(username)", it doesn't mean it will return "sizeof(username)" bytes, the number of bytes returned can be found out using the return value of recv. Never try to interpret the contents of the buffer passed to recv without checking the return value of recv. But, specifying sizeof(username) ensures that recv call will not return more than sizeof(username) bytes back even if more number of bytes are present in the network buffer.
I'm writing simple server/client in c, where server temporary stores message from client and retrieve it when client request it.
The problem is when client receives message from server, the buffer acts kinda weird.
All i did is read as much as receive from server and print it on the screen, but somehow buffer was overwrited more than maximum size of buffer
in client
while((byteRead = recv(ssock, buffer, MAXBUF, 0)) > 0)
{
if(byteRead <= 0)
break;
printf("%s", buffer);
}
where MAXBUF is 256. It keep contains some garbages so i examined the string size in buffer
and surprisingly
printf("%d READ vs %d buffer strlen \n", byteRead, strlen(buffer))
show me that byteRead is 256 but string length of buffer is 262.
Any idea??
P.s on server side, it reads file correctly and send it onto socket.
recv does not place a null terminator at the end of the string (whilst printf %s assumes there is one).
You must use byteRead to determine the length of the string. Add a null terminator if you want to use a function like printf, but ensure your buffer has the space for it even on a maximum-size read.
The problem here is that buffer is not NULL-terminated by recv(). In fact, recv only puts the raw socket data into the buffer. If it recieves 256 bytes of data, whatever comes after that might be null characters (e.g. as it is on your server) or it might be something else (as it is on your client). It's an artifact of program execution, not of how you programmed it.
The easiest and fastest way to fix this:
Allocate buffer with size MAXBUF + 1. The +1 will be for an extra NULL character.
Immediately before the printf, add a null character at buffer[bytesRead].
So all-told:
buffer = malloc((MAXBUF + 1) * sizeof(char)); // NEW
while((byteRead = recv(ssock, buffer, MAXBUF, 0)) > 0)
{
if(byteRead <= 0)
break;
else {
buffer[bytesRead] = '\0'; // NEW
printf("%s", buffer);
}
}
Yes.
strlen() looks for the nearest NULL terminator, as in a conventional C string.
recv() has nothing to do with null terminator and would not add one. So, the strlen call is wrong and may even crash your program by unauthorized read.