I have a socket server and client, both written in C.
These clients (multiple clients and one server) send information to the server, but these pieces of information must be as small as possible.
Currently, every 'packet' from the client is about 40-60 bytes and my client sends about 10 packets/second.
My protocol sends 2 bytes with packet size + payload + 2 bytes for 'end of TX', then the server compares the size + 2 bytes of EOT.
The size and two bytes of EOT are necessary because I can have those two bytes in the payload too, so size is considered too. This is working without any problem.
Now, I need to 'protect' this payload and I don't know how.
OpenSSL is not an option since I've made some tests and, using a 2048 byte certificate, the smallest 'encrypted' packet size is 256 bytes, even using 50 bytes as original data.
I don't need such strong encryption... in fact, something simple will be even better, since my client could run on embedded systems with low processing power. Probably something with one fixed 'key' on both ends would be perfect, but I don't have any idea how to do this or if there is already some standard method.
I think that any solution will increase my packet size... this is fine, but not from 50 bytes to 256 bytes (like using openSSL).
How can I protect the data with at most a moderate increase in the packet size?
Related
I'm using libevent for creating a small p2p network.
Let's say that I want to send 10,000 bytes buffer from one node to another, due to libevent the max size that can be send in one time is 4,096 bytes (so 4,096 + 4,096 + 1,808).
Is it technically possible (in TCP protocol) for the node that receives the data, to know the total number of bytes that will be sent when receives the first 4,096 bytes ?
No. It is, however, common for the application protocol to tell the receiver how many bytes to expect. With HTTP 1.1, it's the Content-Length header. You can also use length prefix along the lines of <len><data>. PostgresSQL uses that protocol between the front and backend.
My question is mainly educational so please don't tell me you shouldn't do that or you should do that. Thanks.
This is basically about how TCP works.
I'm implementing a HTTP client in C that basically just sends a HTTP GET and reads the response from ths server. Now I want to separate the headers from the body. The question is, is it possible to read from the socket byte-by-byte:
while(recv(sockfd, buffer, 1, 0))
{
// do whatever with buffer[0]
}
or is it so that: once the server wrote to the socket, say, 1000 bytes, so once the client reads even 1 bytes than all this message is "wasted" and cannot be read anymore?
Because I remember that when dealing with sockets sometime in the past that was how I understood that it is working.
Yes, in TCP that is possible. TCP does not have messages, only bytes, like a file. You can send 1000 bytes and then receive 1 byte 1000 times.
In fact that is a convenient way to start out, because if you try to receive more than 1 byte, you might not get the same number of bytes you asked for, you could get any number down to 1 byte. If you only ask to receive 1 byte, then you always get exactly 1 byte (unless the connection is closed).
It is inefficient, however, because the receive function uses a certain amount of CPU time just to process the fact that you want to receive - i.e. the function call itself costs a bit of CPU time. If you ask for, say, 100000 bytes at once, then you don't have to call the receive function 100000 times to get it. It's probably fast enough for you, just not as fast as it could be.
Although it is not good practice, it isn't "wasted". Most socket implementations on OS's do use a socket buffer, ranging from 4K till 6MB. So yes you can, as long as you read it fast engough.
But still, it is saver to just copy the stuf to your own memory-managed buffer..
I'm trying to do a multiplayer game in c, but when I send multiple package like "ARV 2\n\0" and "POS 2 0 0\n\0" from the server to the client (with send()), when I try to read them with recv(), he only found 1 package that appear to be the 2 package in 1..
So I'm asking, is that normal ? And if yes, how could I force my client to read 1 by 1 the packages ? (or my server to send them 1 by 1 if the problem come from the call send)
Thanks !
Short answer: Yes, this is normal. You are using TCP/IP, I assume. It is a byte stream protocol, there are no "packets". Network and OS on either end may combine and split the data you send in any way that fits in some buffers, or parts of network. Only thing guaranteed is, that you get the same bytes in same order.
You need to use your own packet framing. For text protocol, separate packets with, for example, '\0' bytes or newlines. Also note that network or OS may give you partial packets per single "read", so you need to handle that in your code as well. This is easiest if packet separator is single byte.
Especially for a binary protocol where there are no "unused" byte values to mark packet boundaries, you could write length of packet as binary data, then that many data bytes, then again length, data, and so on. Note that the data stream may get split to different "read" calls even in the middle of the length info as well (unless length is single byte), so you may need a few lines more of code to handle receiving split packets.
Another option would be to use UDP protocol, which indeed sends packets. But UDP packets may get lost or delivered in wrong order (and have a few other problems), so you need to handle that somehow, and this often results in you re-inventing TCP, poorly. So unless you notice TCP/IP just won't cut it, stick with that.
I'm trying to count total send bytes from my program, but I can't get accurate value.
All my functions call a single function that send data to my server using send() function.
In this function, i get return of send() and sum into global counter. This is working fine.
But when I compare to 'iftop' utility (sudo iftop -f 'port 33755'), I'm getting more data on iftop then in my app....and my guess if because of tcp headers/protocol data. I really don't know how to calculate this. I'm sending packets using send() and variable data length, so I'm not sure if is possible to detect/calculate TCP packet size from there. I know that each TCP packet send TCP header, but I'm not sure how many packets is sent.
May I assume that every call to send(), if data length is less than 1518 (TCP packet size limite?), than it's only one TCP packet and I need to sum TCP Header length? Even if I sent one byte? If so, how much is these extra-bytes from TCP structure?!
For information: I'm using GCC on linux as compiler.
Tks!
How to count bytes using send() in C, including protocol size?
There is no reliable way to do so from within your program. You can compute a minimum total number of bytes required to transmit data of the total payload size you count, subject to a few assumptions, but you would need to monitor from the kernel side to determine the exact number of bytes.
May I assume that every call to send(), if data length is less than
1518 (TCP packet size limite?), than it's only one TCP packet and I
need to sum TCP Header length?
No, that would not be a safe assumption. The main problem is that the kernel does not necessarily match the data transferred by each send() call to its own sequence of packets. It may combine data from multiple send()s into a smaller number of packets. Additionally, however, it may use either a smaller MTU or a larger one than Ethernet's default of 1500 bytes, depending on various factors, and, furthermore, you need to fit packet headers into the chosen MTU, so the payload carried by one packet is smaller than that.
I suspect you're making this too hard. If this is a task that has been assigned to you -- a homework problem, for example -- then my first guess would be that it is intended that you count only the total payload size, not the protocol overhead. Alternatively, if you do need to account for the overhead, then my guess would be that you are supposed to estimate, based on measured or assumed characteristics of the network. If you've set this problem for yourself, then I can only say that people generally make one of the two computations I just described, not the one you asked about.
which is the beast approach to send packets that can be of different size using TCP sockets in C?
I wonder because we're trying to write a multiplayer games that needs a protocol which has many kinds of packets of different sizes.. according to recv documentation I can get how many bytes have been read but how should I manage to dispatch packets only when they are exaclty full?
Suppose that I have packets with a 5 bytes header that contains also the length of the payload.. should I use circular buffers or something like that to dispatch packets when ready and keep new partials?
Create a static variable which represents the packet header, this variable will be five bytes long. Create an associated integer which counts how many of those five bytes have yet been read. Create a second integer which counts how many bytes of the "content" have been read. Zero both those integers. Create an associated char * pointer which eventually will point to the received packet content.
As data arrives (e.g., select indicates so), read the five bytes of header. You may receive these bytes gradually, thus you need the first integer count variable. Account for the header bytes you have received here.
When you are done receiving the header, sanity check it. Are the size values possible to satisfy (e.g. not greater than 2^30)? If so, malloc a buffer of that size or that size plus the header. (If you want the header contiguous, allocate sufficient space, then memcpy it into your new buffer.)
Now, as data arrives, place it in your allocated buffer. Account for the received bytes in the second integer you created. When you have received all the bytes the header called for, then repeat all the above.
you can design a custom header for your packet transmission, which specifies packet length, indexing info (if packet fragmentation is implemented) and some hashing if you need.
some rough pseudocode as follows :
recv(socket, headerBuf, headerSize, MSG_WAITALL);
nPacketSize = headerBuf[16]; //sample
nByteRead = 0;
while (nByteRead != nPacketSize)
{
nByteToRead = nPacketSize - nByteRead;
nCurRead = recv(socket, someBuf, nByteToRead, MSG_PARTIAL);
nByteRead += nCurRead;
Sleep(0); // Yield processor
}
TCP is a stream based protocol, not a datagram one. That means that there isn't necessarily a one to one correspondence between the number and size of buffers passed to send and what comes out of receive. That means that you need to implement your own "protocol" on top of TCP.
Examples of such protocols include HTTP, where HTTP messages are delineated by two consecutive carriage return, line feed pairs. \r\n\r\n. The only concern with such delineation is to make sure that the pattern can't occur in the body of the message, either that or to make sure it is escaped. Other protocols create a header which contains the information necessary to correctly identify and read the next piece of information. I can't think of an application that does this off the top of my head. You could even go for a hybrid approach that combines the two.