I'm working on small client-server application written on C.
For sending unsigned 32-bit number to remote side used this code:
u_int32_t number = 123;
send(client_socket_, &number, 4, CLIENT_SOCKET_FLAGS);
For receiving this number on remote side used this code:
u_int32_t number;
recv(acceptor, &number, 4, SERVER_DATA_SOCKET_FLAGS);
printf("Number is: %u\n", number);
If client and server run on same architecture of processors (amd64) - all works fine.
But if my client app launched on different processor architecture (mips) - I getting invalid number on server (bytes order are inverted).
How I can architecturally independent serialize my number in binary format?
An important feature is the good performance of the binary serializer/deserializer.
Also the solution should not be tied to a library.
Serialization
static void Ser_uint32_t(uint8_t arr [sizeof(uint32_t)], uint32_t value)
{
arr[0] = (value >> 24u) & 0xFFu;
arr[1] = (value >> 16u) & 0xFFu;
arr[2] = (value >> 8u) & 0xFFu;
arr[3] = (value >> 0u) & 0xFFu;
}
And de-serialization
static uint32_t DeSer_uint32_t(const uint8_t arr [sizeof(uint32_t)])
{
uint32_t x = 0;
x |= (uint32_t)(arr[0]) << 24u;
x |= (uint32_t)(arr[1]) << 16u;
x |= (uint32_t)(arr[2]) << 8u;
x |= (uint32_t)(arr[3]) << 0u;
return x;
}
Depending on the endianness you want you can correct the posted functions
I prefer to use hexadecimal encoding.
Eg: 0xDEADBEEF will literally be send as: <stx>DEADBEEF<etx>.
This method has the advantage of not losing the ability to use ASCII control characters.
This method has the disadvantage that each number byte is doubled.
Conversion is easy, you can use a lookup table after masking a nibble.
It's a bit like using something like json, but more lightweight in execution.
Related
I just finished an easy SPI implementation that is used for 2 MCUs to communicate on a PCB.
This communication is supposed to pass 16 bit Values from the master to a virtual register on the slave.
The frame length is 64 bit.
The communication frame is build as followed:
bit0: read or write register. (0 == READ; 1 == WRITE)
bit1-bit15: register address
bit16-31: 0xFFFF to accomodate computation time on slave to provide data
bit32-47: register Value
bit 48-63: crc
The communication works flawlessly.
However, and this is what i do not understand, i am retrieving the address by taking the first 2 transmitted bytes and converting them to a uint16_t like this:
register_address = (uint16_t)(((byte0) & 0xFF) << 8 | ((byte1) & 0xFF));
This address is then used in a function that retrieves the value from a global register struct. As argument it should only accept values from the enum type "virtual_register_address_t"
typedef enum virtual_register_address_t
{
//READ/WRITE
REGISTER_ONE_ADD = 0,
REGISTER_TWO_ADD,
REGISTER_THREE_ADD,
//READ ONLY
ERROR_REGISTER_ADD
}virtual_register_address_t;
uint16_t get_virtual_register(virtual_register_address_t address)
{
uint16_t value = 0;
switch(address)
{
case(REGISTER_ONE_ADD):
value = virtual_register.registerOne;
break;
case(REGISTER_TWO_ADD):
value = virtual_register.registerTwo;
break;
case(REGISTER_THREE_ADD):
value = virtual_register.registerThree;
break;
case(ERROR_REGISTER_ADD):
value = virtual_register.errorRegister;
break;
default:
value = 0xFF;
break;
}
return value;
}
void set_virtual_register(virtual_register_address_t address, uint16_t data)
{
switch(address)
{
case(REGISTER_ONE_ADD):
virtual_register.registerOne = data;
break;
case(REGISTER_TWO_ADD):
virtual_register.registerTwo = data;
break;
case(REGISTER_THREE_ADD):
virtual_register.registerThree = data;
break;
case(ERROR_REGISTER_ADD):
break;
default:
break;
}
}
However, as some of you may already have recognized, i made a mistake by copying bit 0-15 from the spi frame instead of bit 1-15.
So the address copied in a write case (first bit 1) should always be >=32768. The enum "virtual_register_address_t" is only defined up to 8. However, the code works flawlessly. It takes the parameter as "virtual_register_address_t" type even if the value is not regarded in the enum definition. It does not go to default state wihtin the switch even if the value is not regarded in the switch. It changes the values reliantly and sends them back just as recieved if i read the address after writing it.
I changed the getting of the register_address to
register_address = (uint16_t)(((byte0) & 0x7F) << 8 | ((byte1) & 0xFF));
and it still works.
To my understanding, the function "set_virtual_register" should do nothing if presented with values not in the switch case. But it reliantly set the values.
My question is, does this always happen if enums are taken as function parameter? How did it work when it shouldn't?
EDIT:
A user asked to add the function calls that use register address:
void spi_serialize(spi_handle_t* handle, virtual_register_address_t address, SPI_State read_or_write)
{
uint16_t crc = 0;
uint16_t data = 0;
switch(read_or_write)
{
case(READ):
data = get_virtual_register(address);
handle->dataTx[4] = (uint8_t)((data >> 8) & 0xff);
handle->dataTx[5] = (uint8_t)(data & 0xff);
break;
case(WRITE):
handle->dataTx[4] = (0xFF);
handle->dataTx[5] = (0xFF);
break;
default:
handle->dataTx[4] = (0xAA);
handle->dataTx[5] = (0xBB);
break;
}
//crc
crc = calculateCRC(handle->dataTxBase, SPI_FRAMESIZE-2);
handle->dataTx[SPI_FRAMESIZE-2] = ((crc >> 8) & 0XFF);
handle->dataTx[SPI_FRAMESIZE-1] = (crc & 0xFF);
}
void spi_deserialize(spi_handle_t* handle)
{
uint16_t register_address = 0;
uint16_t data = 0;
register_address = (uint16_t)(((handle->dataRx[0]) & 0xFF) << 8 | ((handle->dataRx[1]) & 0xFF));
data = (uint16_t)(((handle->dataRx[4]) & 0xFF) << 8 | ((handle->dataRx[5]) & 0xFF));
set_virtual_register(register_address, data);
}
case does not need any parenthesises
The error is somewhere else
register_address is not used anywhere in the code so it makes no difference what is there.
The switch(...) case will work exactly as you wrote it ie it will assign or get the value if the address is 0,1 or 2 (three case has only break)(
void spi_deserialize(spi_handle_t* handle)
{
uint16_t register_address = 0;
uint16_t data = 0;
register_address = (uint16_t)(((handle->dataRx[0]) & 0xFF) << 8 | ((handle->dataRx[1]) & 0xFF));
data = (uint16_t)(((handle->dataRx[4]) & 0xFF) << 8 | ((handle->dataRx[5]) & 0xFF));
set_virtual_register(register_address, data);
}
If the code above sets the registers it means that only two LS bits are set in those two bytes.
Okay, i dived a little into the ARM Keil compiler.
An enum is either a char or an int. This depends on the range of values that are defined within.
https://www.keil.com/support/man/docs/c51/c51_le_enum.htm
https://www.keil.com/support/man/docs/c51/c51_ap_1bytescalar.htm
since i have way below 256 different definitions it is handled as a char.
However, since the function parameter is an enum and not a uint8_t or char, calling the function with an uint16_t as parameter is not thrown as a warning or error at compile time. It accepts an uint16_t as parameter.
If a value greater than 255 is passed into the function it is truncated. Leaving only the lowest 8 bit. Therefore, it worked, even if the address passed to the function was not part of the enum. Because the lowest 8 bit were.
I tested the input and like described before, all values passed to the function are handled like
(parameter = value%256)
Thanks to all for taking the time. And sorry for not finding the answer before posting here.
If you do not think my answer is correct please elaborate. I would hate to have come to a wrong conclusion.
I am trying to transmit values between architectures, by creating a uint8_t[] buffer and then sending that. To ensure they are transmitted correctly, the spec is to convert all values to little-endian as they go into the buffer.
I read this article here which discussed how to convert from one endianness to the other, and here where it discusses how to check the endianness of the system.
I am curious if there is a method to read bytes from a uint64 or other value in little-endian order regardless of whether the system is big or little? (ie through some sequence of bitwise operations)
Or is the only method to first check the endianness of the system, and then if big explicitly convert to little?
That's actually quite easy -- you just use shifts to convert between 'native' format (whatever that is) and little-endian
/* put a 32-bit value into a buffer in little-endian order (4 bytes) */
void put32(uint8_t *buf, uint32_t val) {
buf[0] = val;
buf[1] = val >> 8;
buf[2] = val >> 16;
buf[3] = val >> 24;
}
/* get a 32-bit value from a buffer (little-endian) */
uint32_t get32(uint8_t *buf) {
return (uint32_t)buf[0] + ((uint32_t)buf[1] << 8) +
((uint32_t)buf[2] << 16) + ((uint32_t)buf[3] << 24);
}
If you put a value into a buffer, transmit it as a byte stream to another machine, and then get the value from the received buffer, the two machines will have the same 32 bit value regardless of whether they have the same or different native byte oridering. The casts are needed becuase the default promotions will just convert to int, which might be smaller than a uin32_t, in which case the shifts could be out of range.
Be careful if you buffers are char rather than uint8_t (char might or might not be signed) -- you need to mask in that case:
uint32_t get32(char *buf) {
return ((uint32_t)buf[0] & 0xff) + (((uint32_t)buf[1] & 0xff) << 8) +
(((uint32_t)buf[2] & 0xff) << 16) + (((uint32_t)buf[3] & 0xff) << 24);
}
You can always serialize an uint64_t value to array of uint8_t in little endian order as simply
uint64_t source = ...;
uint8_t target[8];
target[0] = source;
target[1] = source >> 8;
target[2] = source >> 16;
target[3] = source >> 24;
target[4] = source >> 32;
target[5] = source >> 40;
target[6] = source >> 48;
target[7] = source >> 56;
or
for (int i = 0; i < sizeof (uint64_t); i++) {
target[i] = source >> i * 8;
}
and this will work anywhere where uint64_t and uint8_t exists.
Notice that this assumes that the source value is unsigned. Bit-shifting negative signed values will cause all sorts of headaches and you just don't want to do that.
Deserialization is a bit more complex if reading byte at a time in order:
uint8_t source[8] = ...;
uint64_t target = 0;
for (int i = 0; i < sizeof (uint64_t); i ++) {
target |= (uint64_t)source[i] << i * 8;
}
The cast to (uint64_t) is absolutely necessary, because the operands of << will undergo integer promotions, and uint8_t would always be converted to a signed int - and "funny" things will happen when you shift a set bit into the sign bit of a signed int.
If you write this into a function
#include <inttypes.h>
void serialize(uint64_t source, uint8_t *target) {
target[0] = source;
target[1] = source >> 8;
target[2] = source >> 16;
target[3] = source >> 24;
target[4] = source >> 32;
target[5] = source >> 40;
target[6] = source >> 48;
target[7] = source >> 56;
}
and compile for x86-64 using GCC 11 and -O3, the function will be compiled to
serialize:
movq %rdi, (%rsi)
ret
which just moves the 64-bit value of source into target array as is. If you reverse the indices (7 ... 0; big-endian), GCC will be clever enough to recognize that too and will compile it (with -O3) to
serialize:
bswap %rdi
movq %rdi, (%rsi)
ret
Most standardized network protocols specify numbers in big-endian format. In fact, big-endian is all referred to as network byte order, and there are functions specifically for translating integers of various sizes between host and network byte order.
These function are htons and ntohs for 16 bit values and htonl and ntohl` for 32 bit values. However, there is no equivalent for 64 bit values, and you're using little-endian for the network protocol, so these won't help you.
You can still however translate between the host byte order and the network byte order (little-endian in this case) without knowing the host order. You can do this by bit shifting the relevant values in to or out of the host numbers.
For example, to convert a 32 bit value from host to little endian and back to host:
uint32_t src_value = *some value*;
uint8_t buf[sizeof(uint32_t)];
int i;
for (i=0; i<sizeof(uint32_t); i++) {
buf[i] = (src_value >> (8 * i)) & 0xff;
}
uint32_t dest_value = 0;
for (i=0; i<sizeof(uint32_t); i++) {
dest_value |= (uint32_t)buf[i] << (8 * i);
}
For two systems that must communicated, you specify an "intercomminication-byte order". Then you have functions that convert between that and the native architecture byte order of each system.
There are three approaches to this problem. In order of efficiency:
Compile time detection of endianess
Run time detection of endianness
Endian agnostic code (corresponding to "sequence of bitwise operations" in your question).
Compile time detection of endianess
On architectures whose byte order is the same as the intercomm byte order, these functions do no transformation, but by using them, the same code becomes portable between systems.
Such functions may already exist on your target platform, for example:
Linux's endian.h be64toh() et-al
POSIX htonl, htons, ntohl, ntohs
Windows' winsock.h (same as POSIX but adds 64 bit htonll() and ntohll()
Where they don't exist creating them with cross-platform support is trivial. For example:
uint16_t intercom_to_host_16( uint16_t intercom_word )
{
#if __BIG_ENDIAN__
return intercom_word ;
#else
return intercom_word >> 8 | intercom_word << 8 ;
#endif
}
Here I have assumed that the intercom order is big-endian, that makes the function compatible with network byte order per ntohs() et al. The macro __BIG_ENDIAN__ is a predefined macro on most compilers. If not simply define it as a command line macro when compiling e.g. -D __BIG_ENDIAN__.
Run time detection of endianness
It is possible to detect endianness at runtime with minimal overhead:
uint16_t intercom_to_host_16( uint16_t intercom_word )
{
static const union
{
uint16_t word ;
uint8_t bytes[2] ;
} test = {.word = 0xff00u } ;
return test.bytes[0] == 0xffu ?
intercom_word :
intercom_word >> 8 | intercom_word << 8 ;
}
Of course you might wrap the test in a function for use in similar functions for other word sizes:
#include <stdbool.h>
bool isBigEndian()
{
static const union
{
uint16_t word ;
uint8_t bytes[2] ;
} test = {.word = 0xff00u } ;
return test.bytes[0] == 0xffu ;
}
Then simply have :
uint16_t intercom_to_host_16( uint16_t intercom_word )
{
return isBigEndian() ? intercom_word :
intercom_word >> 8 | intercom_word << 8 ;
}
Endian agnostic code
It is entirely possible to use endian agnostic code, but in that case all participants in the communication or file processing have the software overhead imposed even if the native byte order is already the same as the intercom byte order.
uint16_t intercom_to_host_16( uint16_t intercom_word )
{
uint8_t host_word [2] = { intercom_word >> 8,
intercom_word << 8 } ;
return *(uint16_t*)host_word ;
}
I am using GCC struct bit fields in an attempt interpret 8 byte CAN message data. I wrote a small program as an example of one possible message layout. The code and the comments should describe my problem. I assigned the 8 bytes so that all 5 signals should equal 1. As the output shows on an Intel PC, that is hardly the case. All CAN data that I deal with is big endian, and the fact that they are almost never packed 8 bit aligned makes htonl() and friends useless in this case. Does anyone know of a solution?
#include <stdio.h>
#include <netinet/in.h>
typedef union
{
unsigned char data[8];
struct {
unsigned int signal1 : 32;
unsigned int signal2 : 6;
unsigned int signal3 : 16;
unsigned int signal4 : 8;
unsigned int signal5 : 2;
} __attribute__((__packed__));
} _message1;
int main()
{
_message1 message1;
unsigned char incoming_data[8]; //This is how this message would come in from a CAN bus for all signals == 1
incoming_data[0] = 0x00;
incoming_data[1] = 0x00;
incoming_data[2] = 0x00;
incoming_data[3] = 0x01; //bit 1 of signal 1
incoming_data[4] = 0x04; //bit 1 of signal 2
incoming_data[5] = 0x00;
incoming_data[6] = 0x04; //bit 1 of signal 3
incoming_data[7] = 0x05; //bit 1 of signal 4 and signal 5
for(int i = 0; i < 8; ++i){
message1.data[i] = incoming_data[i];
}
printf("signal1 = %x\n", message1.signal1);
printf("signal2 = %x\n", message1.signal2);
printf("signal3 = %x\n", message1.signal3);
printf("signal4 = %x\n", message1.signal4);
printf("signal5 = %x\n", message1.signal5);
}
Because struct packing order varies between compilers and architectures, the best option is to use a helper function to pack/unpack the binary data instead.
For example:
static inline void message1_unpack(uint32_t *fields,
const unsigned char *buffer)
{
const uint64_t data = (((uint64_t)buffer[0]) << 56)
| (((uint64_t)buffer[1]) << 48)
| (((uint64_t)buffer[2]) << 40)
| (((uint64_t)buffer[3]) << 32)
| (((uint64_t)buffer[4]) << 24)
| (((uint64_t)buffer[5]) << 16)
| (((uint64_t)buffer[6]) << 8)
| ((uint64_t)buffer[7]);
fields[0] = data >> 32; /* Bits 32..63 */
fields[1] = (data >> 26) & 0x3F; /* Bits 26..31 */
fields[2] = (data >> 10) & 0xFFFF; /* Bits 10..25 */
fields[3] = (data >> 2) & 0xFF; /* Bits 2..9 */
fields[4] = data & 0x03; /* Bits 0..1 */
}
Note that because the consecutive bytes are interpreted as a single unsigned integer (in big-endian byte order), the above will be perfectly portable.
Instead of an array of fields, you could use a structure, of course; but it does not need to have any resemblance to the on-the-wire structure at all. However, if you have several different structures to unpack, an array of (maximum-width) fields usually turns out to be easier and more robust.
All sane compilers will optimize the above code just fine. In particular, GCC with -O2 does a very good job.
The inverse, packing those same fields to a buffer, is very similar:
static inline void message1_pack(unsigned char *buffer,
const uint32_t *fields)
{
const uint64_t data = (((uint64_t)(fields[0] )) << 32)
| (((uint64_t)(fields[1] & 0x3F )) << 26)
| (((uint64_t)(fields[2] & 0xFFFF )) << 10)
| (((uint64_t)(fields[3] & 0xFF )) << 2)
| ( (uint64_t)(fields[4] & 0x03 ) );
buffer[0] = data >> 56;
buffer[1] = data >> 48;
buffer[2] = data >> 40;
buffer[3] = data >> 32;
buffer[4] = data >> 24;
buffer[5] = data >> 16;
buffer[6] = data >> 8;
buffer[7] = data;
}
Note that the masks define the field length (0x03 = 0b11 (2 bits), 0x3F = 0b111111 (16 bits), 0xFF = 0b11111111 (8 bits), 0xFFFF = 0b1111111111111111 (16 bits)); and the shift amount depends on the bit position of the least significant bit in each field.
To verify such functions work, pack, unpack, repack, and re-unpack a buffer that should contain all zeros except one of the fields all ones, and verify the data stays correct over two roundtrips. It usually suffices to detect the typical bugs (wrong bit shift amounts, typos in masks).
Note that documentation will be key to ensure the code remains maintainable. I'd personally add comment blocks before each of the above functions, similar to
/* message1_unpack(): Unpack 8-byte message to 5 fields:
field[0]: Foobar. Bits 32..63.
field[1]: Buzz. Bits 26..31.
field[2]: Wahwah. Bits 10..25.
field[3]: Cheez. Bits 2..9.
field[4]: Blop. Bits 0..1.
*/
with the field "names" reflecting their names in documentation.
I'm trying to learn how to write drivers for GPIO pins in STM32F4 Discovery. I was going through the tutorials and I came across this line:
static void hal_gpio_configure_pin_mode (GPIO_TypeDef *GPIOx, uint16_t pin_no, uint32_t mode)
{
GPIOx->MODER |= (mode << (2* pin_no));
}
I am relatively new to embedded systems and don't know what GPIOX->MODER |= (mode << (2* pin_no)); does. Also, I don't know why pin_no is 16 bit while mode is 32 bit. I know that << is the left shift bitwise operator. But I still don't get the complete context.
Can anybody explain all this?
Trying to explain the line GPIOx->MODER |= (mode << (2* pin_no));:
GPIOx is a pointer to a structure
GPIOx->MODER accesses the member MODER inside this structure
x |= y is an abbreviation of x = x | y, which means "perform a logic OR of x and y, and write the result back to x
mode << (2* pin_no) means left-shift the content of variable mode by twice pin_no bits.
Hence, the line means "take the content of GPIOx->MODER, bitwise-OR it with the left-shiftet content of mode.
I want to read and write from/to an unsigned char according to the table below:
for example I have following variables:
unsigned char hsi_div = 0x01; /* HSI/2 */
unsigned char cpu_div = 0x05; /* Fmaster/32 */
I want to write hsi_div to bits 4,3 and cpu_div to bits 2,1,0 (imagine the whole char is named CLK_DIVR):
CLK_DIVR |= hsi_div << 4; //not correct!
CLK_DIVR |= cpu_div << 2; //not correct!
And lets say I want to read the register back to make sure I did it correct:
if( ((CLK_DIVR << 4) - 1) & hsi_div) ) { /* SET OK */ }
if( ((CLK_DIVR << 2) - 1) & cpu_div) ) { /* SET OK */ }
Is there something wrong with my bitwise operations!? I do not get correct behaviour.
I assume CLK_DIVR is a hardware peripheral register which should be qualified volatile. Such registers should be set up with as few writes as possible. You change all write-able bits, so just
CLK_DIVR = (uint8_t)((hsi_div << 3) | (cpu_div << 0));
Note using fixed width type. That makes mentioniong it is an 8 bit register unnecessary. According to the excerpt, the upper bits are read-only, so they are not changed when writing. The cast keeps the compiler from issuing a truncation warning which is one of the recommended warnings to always enable (included in -Wconversion for gcc).
The shift count is actually the bit the field starts (the LSbit). A shift count of 0 means "no shifting", so the shift-operator is not required. I still use it to clarify I meant the field starts at bit 0. Just let the compiler optimize, concentrate on writing maintainable code.
Note: Your code bit-or's whatever already is in the register. Bit-or can only set bits, but not clear them. Addiionally the shift counts were wrong.
Not sure, but if the excerpt is for an ARM Cortex-M CPU (STM32Fxxxx?), reducing external bus-cycles becomes more relevant, as the ARM can take quite some cycles for an access.
For the HSIDIV bit fields you want:
hw_register = (hw_register & 0x18) | (hsi_value & 0x03) << 0x03;
This will mask the value to 2 bits wide then shift to bit position 3 and 4.
The CPUDIV fields are:
hw_register = (hw_register & 0x7) | (cpu_value & 7);
Reading the register:
hsi_value = (hw_register & 0x18) >> 3;
cpu_value = hw_register & 0x07;
Just
CLK_DIVR |= hsi_div << 3;
CLK_DIVR |= cpu_div << 0;
Since hsi_div is a 2-digit binary, you have to move it three positions to skip the CPUDIV field. And the cpu_div is already at the end of the field.