I'm working on a project where I must send data to server and this client will run on different os's. I knwo the problem with endians on different machines, so I'm converting 'everything' (almost) to network byte order, using htonl and vice-versa on the other end.
Also, I know that for a single byte, I don't need to convert anything. But, what should I do for char*? ex:
send(sock,(char*)text,sizeof(text));
What's the best approach to solve this? Should I create an 'intermediate function' do intercept this 'send', then really send char-by-char of this char array? If so, do I need to convert every char to network byte order? I think no, since every char is only one byte.
Thinking of this, if I create this 'intermediate functions', I don't have to convert nothing more to network byte order, since this function will send char by char, thus don't need conversion of endians.
I any advice on this.
I am presuming from your question that the application layer protocol (more specifically everything above level 4) is under your design control. For single byte-wide (octet-wide in networking parlance) data there is no issue with endian ordering and you need do nothing special to accommodate that. Now if the character data is prepended with a length specifier that is, say 2 octets, then the ordering of the bytes must be treated consistently.
Going with network byte ordering (big-endian) will certainly fill the bill, but so would consistently using little-endian. Consistency of byte ordering on each end of a connection is the crucial issue.
If the protocol is not under your design control, then the protocol specification should offer guidance on the issue of byte ordering for multi-byte integers and you should follow that.
Related
I want to send some stuff over TCP, including int8_t, uint32_t, char, .... and I need to send it at once, so I was thinking about using one big byte array - especially since the receiver will treat the incoming packet as a byte array.
What's the proper way to do this in C? Since there's no "byte" datatype, I'm thinking about just using a int8_t array, filling it and sending it via FreeRTOS. What's stuff I should be aware of? Should I cast every value to int8_t (while splitting up larger ones with bit operators)? Anything I should take care of when trying to put unsigned data types in there?
Thanks for any help, haven't found much online surprisingly.
I need to receive a nested integer array on a socket, e.g.
[[1,2,3],[4,5,6],...]
The subarrays are always 3 values long, the length of the main array varries, but is known in advance.
Searching google has given me a lot of options, from sending each integer seperatly to just casting the buffer to what I think it should be (seems kind of unsafe to me), so I am looking for a safe and fast way to do this.
The "subarrays" don't matter, in the end you're going to be transmitting 3 n numbers and have the receiver interpret them as n rows of 3 numbers each.
For any external representation, you're going to have to pick a precision, i.e. how many bits you should use for each integer. The type int is not well-specified, so perhaps pick 32 bits and treat each number as an int32_t.
As soon as an external integer representation has multiple bytes, you're going to have to worry about the order of those bytes. Traditionally network byte ordering ("big endian") is used, but many systems today observe that most hardware is little-endian so they use that. In that case you can write the entire source array into the socket in one go (assuming of course you use a TCP/IP socket), perhaps prepended by either the number of rows or the total number of integers.
Assuming that bandwidth and data size isn't very critical I would propose, that (de-)serializing the array to a string is a safe and platform/architecture independent way to transfer such an array. This has the following advantages:
No issues with different sizes of the binary representations of integers between the communicating hosts
No issues with differing endiannesses
More flexible if the parameters change (length of the subarrays, etc)
It is easier to debug a text-protocol in contrast to a binary protocol
The drawback is, that more bytes have to be transmitted over the channel as minimal necessary with a good binary encoding.
If you want to go with a ready-to-use library for serializing/deserializing your array, you could take a look at one of the many JSON-libraries available.
http://www.json.org/ provides a list with several implementations.
Serialize it the way you want, two main possibilities:
encode as strings, and fix separators, etc.
encode with NBO, and send data to fix some parameters: first the length of your ints, then the length of the array and then the data; everything properly encoded.
In C, you can use XDR routines to encode properly your data.
This question is pure teoretical, and not about the right way of doing it, but do we need to convert char 'x' to network format? I'm intressted in all cases: always / sometimes / never
I personally think I should but i need to be sure, than you.
No, char is a single byte value so endianess doesn't matter
As you're thinking of it (endianess, ntohs ntohl, etc...), no.
Less basically, I should raise a non-network-bound warning : Any string not attached with its encoding is unreadable.
Say you're sending/storing, over network or not, the string "Français" the 'ç' will have to be encoded using a character encoding. Not specifying the character encoding mean it can be read "Français" if you encoded it using utf8 but your reader was thinking it was latin1.
So you have two solutions :
Write down a spec for you application to only use the same char encoding
Put a metadata on a header somewhere specifying the encoding for future strings (like HTTP).
I'm a bit new to C, but I've done my homework (some tutorials, books, etc.) and I need to program a simple server to handle requests from clients and interact with a db. I've gone through Beej's Guide to Network programming, but I'm a bit unsure how to piece together and handle different parts of the data getting sent back and forth.
For instance, say the client is sending some information that the server will put in multiple fields. How do I piece together that data to be sent and then break it back up on the server side?
Thanks,
Eric
If I understand correctly, you're asking, "how does the server understand the information the client sends it"?
If that's what you're asking, the answer is simple: it's mutually agreed upon ahead of time that the data structures each uses will be compatible. I.e. you decide upon what your communication protocol will be ahead of time.
So, for example, if I have a client-server application where the client connects and can ask for things such as "time", "date" and can say "settime " and "setdate ", I need to write my server in such a way that it will understand those commands.
Obviously, in the above case it's trivial, since it'd just be a text-based protocol. But let's say you're writing an application that will return a struct of information, i.e.
struct Person {
char* name;
int age;
int heightInInches;
// ... other fields ...
};
You might write the entire struct out from the server/client. In this case there are a few things to be aware of:
You need to hton/ntoh properly
You need to make sure that your client and server both can understand the struct in question.
You may or may not have to align on a 4B boundary (because if you don't, different C compilers may do different things, which may burn you between the client and the server, or it may not).
In general, though, when writing a client/server app, the most important thing to get right is the communication protocol.
I'm not sure if this quite answers your question, though. Is this what you were after, or were you asking more about how, exactly, do you use the send/recv functions?
First, you define how the packet will look - what information will be in it. Make sure the definition is in an architecture-neutral format. That means that you specify it in a sequence that does not depend on whether the machine is big-endian or little-endian, for example, nor on whether you are compiling with 32-bit long or 64-bit long values. If the content is of variable length, make sure the definition contains the information needed to tell how long each part is - in particular, each variable length part should be preceded by a suitable count of its length.
When you need to package the data for transmission, you will take the raw (machine-specific) values and write them into a buffer (think 'character array') at the appropriate positions, in the appropriate format.
This buffer will be sent across the wire to the receiver, which will read it into another buffer, and then reverse the process to obtain the information from the buffer into local variables.
There are functions such as ntohs() to convert from a network ('n') to host ('h') format for a 'short' (meaning 16-bit) integer, and htonl() to convert from a host 'long' (32-bit integer) to network format - etc.
One good book for networking is Stevens' "UNIX Network Programming, Vol 1, 3rd Edn". You can find out more about it at its web site, including example code.
As already mentioned above what you need is a previously agreed means of communication. One thing that helps me is to use xmls to communicate.
e.g. You need time to send time to client then include it in a tag called time.
Then parse it on the client side and read the tag value.
The biggest advantage is that once you have a parser in place on client side then even if you have to send some new information them just have to agree on a tag name that will be parsed on the client side.
It helps me , I hope it helps you too.
In my program I have a few structs and a char array that I want to send as a single entity over UDP.
I am struggling to think of a good way to do this.
My first thought was to create a structure which contains everything I want to send but it would be of the wrong type for using sendto()
How would I store the two structs and a char array in another array so that it will be received in the way I intended?
Thanks
Since C allows you to cast to your heart's content, there's no such thing as a wrong type for sendto(). You simply cast the address of your struct to a void * and pass that as the argument to sendto().
However, a lot of people will impress on you that it's not advisable to send structs this way in the first place:
If the programs on either side of the connection are compiled by different compilers or in different environments, chances are your structs will not have the same packing.
If the two hosts involved in the transfer don't have the same endinanness, part of your data will end up backwards.
If the host architectures differ (e.g. 32 bit vs. 64 bits) then sizes of structs may be off as well. Certainly there will be size discrepancies if the sizes of your basic data types (int, char, long, double, etc.) differ.
So... Please take the advice of the first paragraph only if you're sure your two hosts are identical twins, or close enough to it.
In other cases, consider converting your data to some kind of neutral text representation, which could be XML but doesn't need to be anything that complicated. Strings are sent as a sequence of bytes, and there's much less that can go wrong. Since you control the format, you should be able to parse that stuff with little trouble on the receiving side.
Update
You mention that you're transferring mostly bit fields. That means that your data essentially consists of a bunch of integers, all of them less than (I'm assuming) 32 bits.
My suggestion for a "clean" solution, therefore, would be to write a function to unpack all those bit fields, and to ship the whole works as an array of (perhaps unsigned) integers. Assuming that sizeof(int) is the same across machines, htons() will work successfully on the elements (each individually!) of those arrays, and you can then wrap them back into a structure on the other side.
You can send multiple pieces of data as one with writev. Just create the array of struct iovec that it needs, with one element for each data structure you want to send.