I'm in the middle of of reading Internetworking with TCP/IP Vol III, by Comer.
I am looking at a some sample code for a "TIME" client for UDP.
The code gets to the point where it does the read of the response, and it takes what should be a 4 bytes and converts it to a 32 bit unsigned integer, so it can be converted to UNIX time.
"n" is a file descriptor that points to a socket that listens for UDP.
n = read (s, (char *)&now, sizeof(now));
if (n < 0)
errexit("read failed: %s\n", strerror(errno));
now = ntohl((u_long)now); /* put in host byte order */
What I am wondering is:
Are there some assumptions that should be checked before making the conversion? This is in C, and I am wondering if there are situations where read would pass a number of bytes that is not 4. If so, it seems like "now" would be a mess.
"Now" is defined as:
time_t now; /* 32-bit integer to hold time */
So maybe I don't understand the nature of "time_t", or how the bytes are passed around in C, or what situations UDP would return the wrong number of bytes to the file descriptor...
Thanks in advance.
With UDP, as long as the recieve buffer you pass to read is long enough, a single UDP packet won't be broken up between read calls.
However, there's no guarantee that the other side sent a packet of at least 4 bytes - you're quite right, if a server sent only a 2 byte response then that code would leave now containing garbage.
That probably doesn't matter too much in this precise situation - after all, the server is just as free to send 4 bytes of garbage as it is to send only 2 bytes. If you want to check for it, just check that the n returned by read is as long as you were expecting.
Related
I am trying to send a packet over a TCP socket. The structure that I am sending through the socket is the following:
struct requestPacket {
uint32_t INDEX;
uint32_t KEYSIZE;
uint8_t *KEY;
};
I send the packet (after having it placed in a buffer) in one go through a TCP socket with send:
send(sockfd, buff, size, 0);
Without knowing the size beforehand how can the server process this message? In the recv function, you have to define a buffer of a set size but you only know of this size after you process the beginning of the packet. Any tips? Is it possible to parse a packet twice? I know that the first 8 bytes will contain the index and the size of the message.
A very simple method is to first read 8 bytes from the socket to get the size (appropriately converted to the host endianness), then allocate size bytes for the key and read size bytes.
Remember however that the packet may have been sliced and diced on its way from the client to the server, so multiple calls to recv or read may be required to read both the initial 8 bytes and the remaining size bytes.
You won't be reading the same packet twice. A TCP socket is a stream of bytes, you can read it in multiple chunks, just like a file, except you cannot seek. As a matter of fact, there is no need for the client to construct a buffer, it can issue 2 writes, one for the 8 byte header, and another one for the key contents. Conversely, the server may need more than one read for both the header and the key: a read request for 8 bytes may return a smaller count. Wrapping the socket in a FILE with fdopen is recommended.
I'm writing a tcp server in C but I'm facing problems on send. I read local file and send data back to the client, when the file is small I have no problems, but when it becomes bigger I have this strange situation:
server tcp:
// create socket, bind, listen accept
// read file
fseek(fptr, 0, SEEK_SET);
// malloc for the sending buffer
ssize_t read = fread(sbuf, 1, file_size, fptr);
while(to_send>0) {
sent = send(socket, sbuf, buf_size, 0);
sbuf += sent;
to_send -= sent;
}
On huge files sent becomes equals to the max value of size_t, I think that I have a buffer overflow. How can I prevent this? What is the best practice to read from a file and send it back?
The problem is that you send buf_size bytes every time, even if there aren't that many left.
For example, pretend buf_size is 8 and you are sending 10 bytes (So initially, to_send is also 10). The first send sends 8 bytes, so you need to send 2 more. The second time, you also send 8 bytes (Which probably reads out of bounds). Then, to_send will be will be -6, which is the same as SIZE_MAX - 5.
Simple fix is to send to_send if it is smaller:
sent = send(socket, sbuf, to_send < buf_size ? to_send : buf_size, 0);
Also, send returns -1 if it is unsuccessful. This is the same as SIZE_MAX when it is assigned to a size_t. You would need some error handling to fix this.
On huge files sent becomes equals to the max value of size_t, I think
that I have a buffer overflow.
Since sent gets its value as the return value of send(), and send() returns ssize_t, which is a signed type unlikely to be wider than size_t, it is virtually certain that what is actually happening is that send() is indicating an error by returning -1. In that case, it will also be setting errno to a value indicative of the error. It cannot return the maximum value of size_t on any system I've ever had my hands on.
How can I prevent this?
In the first place, before you worry about preventing it, you should be sure to detect it by
declaring sent as a ssize_t to match the return type of send(), not a size_t, and
checking the value returned into sent for such error conditions.
Second, if you are really dealing with files longer than can be represented by a ssize_t (much less a size_t), then it is a poor idea to load the whole thing into memory before sending any of it. Instead, load it in (much) smaller blocks, and send the data one such block at a time. Not only will this tend to have lower perceived latency, but it will also avoid any risk associated with approaching the limits of the data types involved.
Additionally, when you do so, be careful to do it right. You have done well to wrap your send() call in a loop to account for short writes, but as #Artyer describes in his answer, you don't quite get that right because you do not reduce the number of bytes you try to send on the second and subsequent calls.
What is the
best practice to read from a file and send it back?
As above.
I read the documentation regarding send() function, when it was said that the third parameter (len) is "The length, in bytes, of the data in buffer pointed to by the buf parameter".
I can't seem to understand if it sends the number of bytes I pass, or I nedd to pass the size of the buffer and it sends all the data included there.
Exmaple:
#define X 256
// main :
char test[X] = {0};
memcpy(test, "hello", 6);
send(sockfd, test, 6, 0)
send(sockfd, test, 256,0)
// will the first opetion send only hello? or hello000000....?
Thanks!
The send function sends precisely the number of bytes you tell it to (assuming it's not interrupted and doesn't otherwise fail). The number of bytes you need to send is determined by the protocol you are implementing. For example, if the protocol says you should send "FOO\r\n", then you need to send 5 bytes. If the protocol specifies that integers are represented as 4 bytes in network byte order and you're sending an integer, the buffer should contain an integer in network byte order and you should send 4 bytes. The size of the buffer doesn't matter to send.
As a complement to David Schwartz proper answer:
Depending on if the socket is non-blocking,or not, it is NOT guaranteed that a single send will actually send all data. You must check return value and might have to call send again (with correct buffer offsets).
For instance if you want to send 10 bytes of data (len=10), you call send(sock, buf, len, 0). However lets say it only manages to send 5 bytes, then send(..) will return 5, meaning that you will have to call it again later like send(sock, (buf + 5), (len - 5), 0). Meaning, skip first five bytes in buffer, they're already sent, and withdraw five bytes from the total number of bytes (len) we want to send.
(Note that I used parenthesis to make it easier to read only, and it assumes that buf is a pointer to 1 byte type.)
I'm writing client and server programs and I'm looking for a way to ensure all bytes are read and all bytes are sent when using read() or write() to/from the sockets opened by the client/server.
I'm assuming I'll have to use a loop to check the number of bytes returned from the read or write functions.
Something like this probably:
#define BUFFER 20
char buffer[BUFFER];
while (I haven't read all bytes from the buffer){
int bytesRead = read(theSocket, myWord, BUFFER);
}
And how would I ensure that all the bytes I am trying to transmit using write() have been transmitted?
Thanks for any help!
Yes, exactly like that. Typical read logic goes like this:
1) Call read.
2) Did we get EOF or error? If so, return.
3) Did we receive all the bytes? If so, return.
4) Go to step 1.
Note that when you call read, you'll need to pass it a pointer to the buffer after the data that was already read, and you'll need to try to read an appropriate amount of bytes that won't overflow the buffer. Also, how you tell if you received all the bytes depends on the protocol.
To write:
1) Call write, passing it a pointer to the first unwritten byte and the number of unwritten bytes.
2) Did we get zero or error? If so, return.
3) Did we write all the bytes? If so, return.
4) Go to step 1.
Note that you have to adjust appropriately for blocking or non-blocking sockets. For example, for non-blocking sockets, you have to handle EWOULDBLOCK.
I'm lead to believe that write() can only send data buffers of byte (i.e. signed char), so how do I send an array of long integers using the C write() function in the sys/socket.h library?
Obviously I can't just cast or convert long to char, as any numbers over 127 would be malformed.
I took a look at the question, how to decompose integer array to a byte array (pixel codings), but couldn't understand it - please could someone dumb it down a little if this is what I'm looking for?
Follow up question:
Why do I get weird results when reading an array of integers from a TCP socket?
the prototype for write is:
ssize_t write(int fd, const void *buf, size_t count);
so while it writes in units of bytes, it can take a pointer of any type. Passing an int* will be no problem at all.
EDIT:
I would however, recomend that you also send the amount of integers you plan to send first so the reciever knows how much to read. Something like this (error checking omitted for brevity):
int x[10] = { ... };
int count = 10;
write(sock, &count, sizeof(count));
write(sock, x, sizeof(x));
NOTE: if the array is from dynamic memory (like you malloced it), you cannot use sizeof on it. In this case count would be equal to: sizeof(int) * element_count
EDIT:
As Brian Mitchell noted, you will likely need to be careful of endian issues as well. This is the case when sending any multibyte value (as in the count I recommended as well as each element of the array). This is done with the: htons/htonl and ntohs/ntohl functions.
Write can do what you want it to, but there's some things to be aware of:
1: You may get a partial write that's not on an int boundary, so you have to be prepared to handle that situation
2: If the code needs to be portable, you should convert your array to a specific endianess, or encode the endianess in the message.
The simplest way to send a single int (assuming 4-byte ints) is :
int tmp = htonl(myInt);
write(socket, &tmp, 4);
where htonl is a function that converts the int to network byte order. (Similarly,. when you read from the socket, the function ntohl can be used to convert back to host byte order.)
For an array of ints, you would first want to send the count of array members as an int (in network byte order), then send the int values.
Yes, you can just cast a pointer to your buffer to a pointer to char, and call write() with that. Casting a pointer to a different type in C doesn't affect the contents of the memory being pointed to -- all it does is indicate the programmer's intention that the contents of memory at that address be interpreted in a different way.
Just make sure that you supply write() with the correct size in bytes of your array -- that would be the number of elements times sizeof (long) in your case.
It would be better to have serialize/de-serialize functionality in your client /server program.
Whenever you want to send data, serialize the data into a byte buffer and send it over TCP with byte count.
When receiving data, de-serialize the data from buffer to your own interpretation .
You can interpret byte buffer in any form as you like. It can contain basic data type, objects etc.
Just make sure to take care of endianess and also alignment stuff.
Declare a character array. In each location of the array, store integer numbers, not characters.
Then you just send that.
For example:
char tcp[100];
tcp[0] = 0;
tcp[1] = 0xA;
tcp[2] = 0xB;
tcp[3] = 0xC;
.
.
// Send the character array
write(sock, tcp, sizeof(tcp));
I think what you need to come up with here is a protocol.
Suppose your integer array is:
100, 99, 98, 97
Instead of writing the ints directly to the buffer, I would "serialize" the array by turning it into a string representation. The string might be:
"100,99,98,97"
That's what would be sent over the wire. On the receiving end, you'd split the string by the commas and build the array back up.
This is more standardised, is human readable, and means people don't have to think about hi/lo byte orders and other silly things.
// Sarcasm
If you were working in .NET or Java, you'd probably encode it in XML, like this:
<ArrayOfInt><Int>100</Int><Int>99</Int><Int>98</Int><Int>97</Int></ArrayOfInt>
:)