C a reference manual states that "The precise manner in which components( and especially bit fields) are packed into a structure is implementation dependent but is predictable for each implementation".
I read that some compilers pack bit fields left to right ( MSB to LSB) in Big endian machines whereas right to left(LSB to MSB) in Little endian machines.
is there a reason/advantage about representing bitfields in two diffrent ways depends on the endianness?
I've not implemented this, but I can imagine that it has to do with working with bit fields in registers, and reading/writing entire words to/from the structure when possible. If you implement it that way, instead of doing byte-level accesses, you will of course "feel" the endianness as the word is byte-swapped in memory.
So if you have
struct color {
uint32_t red : 8;
uint32_t green : 8;
uint32_t blue : 8;
uint32_t alpha : 8;
};
When you do
struct color orange = { .red = 255, .green = 127, .blue = 0, .alpha = 0 };
It might be implemented (since the fields are conveniently sized) as
struct color orange;
uint32_t *tmp = *(uint32_t *) &orange;
*tmp = 0xff7f0000; /* The field values, mapping red to the MSBs. */
Now, since the above does one single uint32_t-sized memory write, the value will be byte-swapped on a little-endian machine but not on a big-endian one, i.e. when viewed byte by byte, the representations are different.
Layout of bit fields inside a structure is implementation defined. It is not a good idea to use them if you need portable code.
Related
I have the following memory structure:
struct {
uint16_t MSB_VALUE : 8;
uint16_t : 8;
uint16_t LSB_VALUE;
} BIG_VALUE;
This structure, all together, represents a 32-bit section of memory that is fixed by hardware. The value of BIG_VALUE can be represented using Verilog concatenation notation thus:
BIG_VALUE = { MSB_VALUE[7:0], LSB_VALUE[15:0] }
I would like to be able to write a union (or something) such that I can access the value of BIG_VALUE using dot notation. Maybe something stupid like this:
union {
uint32_t val;
struct {
uint16_t MSB_VALUE : 8;
uint16_t : 8;
uint16_t LSB_VALUE;
} sub;
} BIG_VALUE;
But, the issue is that the MSB comes before the LSB in memory (with an 8-bit gap too), and so calling BIG_VALUE.val isn't going to get the hoped-for value.
I have a vague idea of something to try, but I'm just confusing myself. Is there a way to do this within the union/struct formalism, or should I give up now? Giving up, I guess, means having to manually split up the 24-bit value and then to store those into the appropriate fields. Maybe I could write a function to do that later, if it makes sense.
Having this work means that I could store a 24-bit value using dot notation and have the data go into the appropriate locations in memory. For example:
BIG_VALUE.val = 0x0031FFFE
Then
BIG_VALUE.MSB_VALUE == 0x31
and
BIG_VALUE.LSB_VALUE == 0xFFFE
But the memory layout would be
addr : 0x0031
addr +4 : 0xFFFE
I have an embedded project with a USART HAL. This USART can only transmit or receive 8 or 16 bits at a time (depending on the usart register I chose i.e. single/double in/out). Since it's a 32-bit MCU, I figured I might as well pass around 32-bit fields as (from what I have been lead to understand) this is a more efficient use of bits for the MPU. Same would apply for a 64-bit MPU i.e. pass around 64-bit integers. Perhaps that is misguided advice, or advice taken out of context.
With that in mind, I have packed the 8 bits into a 32-bit field via bit-shifting. I do this for both tx and rx on the usart.
The code for the 8-bit only register is as follows (the 16-bit register just has half the amount of rounds for bit-shifting):
int zg_usartTxdataWrite(USART_data* MPI_buffer,
USART_frameconf* MPI_config,
USART_error* MPI_error)
{
MPI_error = NULL;
if(MPI_config != NULL){
zg_usartFrameConfWrite(MPI_config);
}
HPI_usart_data.txdata = MPI_buffer->txdata;
for (int i = 0; i < USART_TXDATA_LOOP; i++){
if((USART_STATUS_TXC & usart->STATUS) > 0){
usart->TXDATAX = (i == 0 ? (HPI_usart_data.txdata & USART_TXDATA_DATABITS) : (HPI_usart_data.txdata >> SINGLE_BYTE_SHIFT) & USART_TXDATA_DATABITS);
}
usart->IFC |= USART_STATUS_TXC;
}
return 0;
}
EDIT: RE-ENTERTING LOGIC OF ABOVE CODE WITH ADDED DEFINES FOR CLARITY OF TERNARY OPERATOR IMPLICIT PROMOTION PROBLEM DISCUSSED IN COMMENTS SECTION
(the HPI_usart and USART_data structs are the same just different levels, I have since removed the HPI_usart layer, but for the sake of this example I will leave it in)
#define USART_TXDATA_LOOP 4
#define SINGLE_BYTE_SHIFT 8
typedef struct HPI_USART_DATA{
...
uint32_t txdata;
...
}HPI_usart
HPI_usart HPI_usart_data = {'\0'};
const uint8_t USART_TXDATA_DATABITS = 0xFF;
int zg_usartTxdataWrite(USART_data* MPI_buffer,
USART_frameconf* MPI_config,
USART_error* MPI_error)
{
MPI_error = NULL;
if(MPI_config != NULL){
zg_usartFrameConfWrite(MPI_config);
}
HPI_usart_data.txdata = MPI_buffer->txdata;
for (int i = 0; i < USART_TXDATA_LOOP; i++){
if((USART_STATUS_TXC & usart->STATUS) > 0){
usart->TXDATAX = (i == 0 ? (HPI_usart_data.txdata & USART_TXDATA_DATABITS) : (HPI_usart_data.txdata >> SINGLE_BYTE_SHIFT) & USART_TXDATA_DATABITS);
}
usart->IFC |= USART_STATUS_TXC;
}
return 0;
}
However, I now realize that this is potentially causing more issues than it solves because I am essentially internally encoding these bits which then have to be decoded almost immediately when they are passed through to/from different data layers. I feel like it's a clever and sexy solution, but I'm now trying to solve a problem that I shouldn't have created in the first place. Like how to extract variable bit fields when there is an offset i.e. in gps nmea sentences where the first 8 bits might be one relevant field and then the rest are 32bit fields. So it ends up being like this:
32-bit array member 0:
bits 24-31 bits 15-23 bits 8-15 bits 0-7
| 8-bit Value | 32-bit Value A, bits 24-31 | 32-bit Value A, bits 16-23 | 32-bit Value A, bits 8-15 |
32-bit array member 1:
bits 24-31 bits 15-23 bits 8-15 bits 0-7
| 32-bit Value A, bits 0-7 | 32-bit Value B, bits 24-31 | 32-bit Value B, bits 16-23 | 32-bit Value B, bits 8-15 |
32-bit array member 2:
bits 24-31 15-23 8-15 ...
| 32-bit Value B, bits 0-7 | etc... | .... | .... |
The above example requires manual decoding, which is fine I guess, but it's different for every nmea sentence and just feels more manual than programmatic.
My question is this: bitshifting vs array indexing, which is more appropriate?
Should I just have assigned each incoming/outgoing value to a 32-bit array member and then just index that way? I feel like that is the solution since it would not only make it easier to traverse the data on other layers, but I would be able to eliminate all this bit-shifting logic and then the only difference between an rx or tx function would be the direction the data is going.
It does mean a small rewrite of the interface and the resulting gps module layer, but that feels like less work and also a cheap lesson early on in my project.
Also any thoughts and general experience on this would be great.
Since it's a 32-bit MCU, I figured I might as well pass around 32-bit fields
That's not really the programmer's call to make. Put the 8 or 16 bit variable in a struct. Let the compiler add padding if needed. Alternatively you can use uint_fast8_t and uint_fast16_t.
My question is this: bitshifting vs array indexing, which is more appropriate?
Array indexing is for accessing arrays. If you have an array, use it. If not, then don't.
While it is possible to chew through larger chunks of data byte by byte, such code must be written much more carefully, to prevent running into various subtle type conversion and pointer aliasing bugs.
In general, bit shifting is preferred when accessing data up to the CPU's word size, 32 bits in this case. It is fast and also portable, so that you don't have to take endianess in account. It is the preferred method of serialization/de-serialization of integers.
I read about Endianness and understood squat...
so I wrote this
main()
{
int k = 0xA5B9BF9F;
BYTE *b = (BYTE*)&k; //value at *b is 9f
b++; //value at *b is BF
b++; //value at *b is B9
b++; //value at *b is A5
}
k was equal to A5 B9 BF 9F
and (byte)pointer "walk" o/p was 9F BF b9 A5
so I get it bytes are stored backwards...ok.
~
so now I thought how is it stored at BIT level...
I means is "9f"(1001 1111) stored as "f9"(1111 1001)?
so I wrote this
int _tmain(int argc, _TCHAR* argv[])
{
int k = 0xA5B9BF9F;
void *ptr = &k;
bool temp= TRUE;
cout<<"ready or not here I come \n"<<endl;
for(int i=0;i<32;i++)
{
temp = *( (bool*)ptr + i );
if( temp )
cout<<"1 ";
if( !temp)
cout<<"0 ";
if(i==7||i==15||i==23)
cout<<" - ";
}
}
I get some random output
even for nos. like "32" I dont get anything sensible.
why ?
Just for completeness, machines are described in terms of both byte order and bit order.
The intel x86 is called Consistent Little Endian because it stores multi-byte values in LSB to MSB order as memory address increases. Its bit numbering convention is b0 = 2^0 and b31 = 2^31.
The Motorola 68000 is called Inconsistent Big Endian because it stores multi-byte values in MSB to LSB order as memory address increases. Its bit numbering convention is b0 = 2^0 and b31 = 2^31 (same as intel, which is why it is called 'Inconsistent' Big Endian).
The 32-bit IBM/Motorola PowerPC is called Consistent Big Endian because it stores multi-byte values in MSB to LSB order as memory address increases. Its bit numbering convention is b0 = 2^31 and b31 = 2^0.
Under normal high level language use the bit order is generally transparent to the developer. When writing in assembly language or working with the hardware, the bit numbering does come into play.
Endianness, as you discovered by your experiment refers to the order that bytes are stored in an object.
Bits do not get stored differently, they're always 8 bits, and always "human readable" (high->low).
Now that we've discussed that you don't need your code... About your code:
for(int i=0;i<32;i++)
{
temp = *( (bool*)ptr + i );
...
}
This isn't doing what you think it's doing. You're iterating over 0-32, the number of bits in a word - good. But your temp assignment is all wrong :)
It's important to note that a bool* is the same size as an int* is the same size as a BigStruct*. All pointers on the same machine are the same size - 32bits on a 32bit machine, 64bits on a 64bit machine.
ptr + i is adding i bytes to the ptr address. When i>3, you're reading a whole new word... this could possibly cause a segfault.
What you want to use is bit-masks. Something like this should work:
for (int i = 0; i < 32; i++) {
unsigned int mask = 1 << i;
bool bit_is_one = static_cast<unsigned int>(ptr) & mask;
...
}
Your machine almost certainly can't address individual bits of memory, so the layout of bits inside a byte is meaningless. Endianness refers only to the ordering of bytes inside multibyte objects.
To make your second program make sense (though there isn't really any reason to, since it won't give you any meaningful results) you need to learn about the bitwise operators - particularly & for this application.
Byte Endianness
On different machines this code may give different results:
union endian_example {
unsigned long u;
unsigned char a[sizeof(unsigned long)];
} x;
x.u = 0x0a0b0c0d;
int i;
for (i = 0; i< sizeof(unsigned long); i++) {
printf("%u\n", (unsigned)x.a[i]);
}
This is because different machines are free to store values in any byte order they wish. This is fairly arbitrary. There is no backwards or forwards in the grand scheme of things.
Bit Endianness
Usually you don't have to ever worry about bit endianness. The most common way to access individual bits is with shifts ( >>, << ) but those are really tied to values, not bytes or bits. They preform an arithmatic operation on a value. That value is stored in bits (which are in bytes).
Where you may run into a problem in C with bit endianness is if you ever use a bit field. This is a rarely used (for this reason and a few others) "feature" of C that allows you to tell the compiler how many bits a member of a struct will use.
struct thing {
unsigned y:1; // y will be one bit and can have the values 0 and 1
signed z:1; // z can only have the values 0 and -1
unsigned a:2; // a can be 0, 1, 2, or 3
unsigned b:4; // b is just here to take up the rest of the a byte
};
In this the bit endianness is compiler dependant. Should y be the most or least significant bit in a thing? Who knows? If you care about the bit ordering (describing things like the layout of a IPv4 packet header, control registers of device, or just a storage formate in a file) then you probably don't want to worry about some different compiler doing this the wrong way. Also, compilers aren't always as smart about how they work with bit fields as one would hope.
This line here:
temp = *( (bool*)ptr + i );
... when you do pointer arithmetic like this, the compiler moves the pointer on by the number you added times the sizeof the thing you are pointing to. Because you are casting your void* to a bool*, the compiler will be moving the pointer along by the size of one "bool", which is probably just an int under the covers, so you'll be printing out memory from further along than you thought.
You can't address the individual bits in a byte, so it's almost meaningless to ask which way round they are stored. (Your machine can store them whichever way it wants and you won't be able to tell). The only time you might care about it is when you come to actually spit bits out over a physical interface like I2C or RS232 or similar, where you have to actually spit the bits out one-by-one. Even then, though, the protocol would define which order to spit the bits out in, and the device driver code would have to translate between "an int with value 0xAABBCCDD" and "a bit sequence 11100011... [whatever] in protocol order".
On my AVR I have an array of chars that hold color intensity information in the form of {R,G,B,x,R,G,B,x,...} (x being an unused byte). Is there any simple way to write a long int (32-bits) to char myArray[4*LIGHTS] so I can write a 0x00BBGGRR number easily?
My typecasting is rough, and I'm not sure how to write it. I'm guessing just make a pointer to a long int type and set that equal to myArray, but then I don't know how to arbitrarily tell it to set group x to myColor.
uint8_t myLights[4*LIGHTS];
uint32_t *myRGBGroups = myLights; // ?
*myRGBGroups = WHITE; // sets the first 4 bytes to WHITE
// ...but how to set the 10th group?
Edit: I'm not sure if typecasting is even the proper term, as I think that would be if it just truncated the 32-bit number to 8-bits?
typedef union {
struct {
uint8_t red;
uint8_t green;
uint8_t blue;
uint8_t alpha;
} rgba;
uint32_t single;
} Color;
Color colors[LIGHTS];
colors[0].single = WHITE;
colors[0].rgba.red -= 5;
NOTE: On a little-endian system, the low-order byte of the 4-byte value will be the alpha value; whereas it will be the red value on a big-endian system.
Your code is perfectly valid. You can use myRGBGroups as regular array, so to access 10th pixel you can use
myRGBGroups[9]
Think of using C union, where the first field of the union is a int32 and the second a vector of 4*chars. But, not sure if this is the best way for you.
You need to account for the endianness of uint32_t on the AVR to make sure the components are being stored in the correct order (for later dereferencing via myLights array) if you're going to do this. A quick Google seems to indicate that AVRs store data in memory little-endian, but other registers vary in endianness.
Anyway, assuming you've done that, you can dereference myRGBGroups using array indexing (where each index will reference a block of 4 bytes). So, to set the 10th group, you can just do myRGBGroups[ 9 ] = COLOR.
if can use arithmetic on myRGBgroup, for example myRGBgroups ++ will give next group, similarly you can use plus, minus, etc. operators.those operators operate using type sizes, rather than single byte
myRGBgroups[10] // access group as int
((uint8_t*)(myRGBgroups + 10)) // access group as uint8 array
Union of a struct and uint32_t is a much better idea than making a uint8_t of size 4 * LIGHTS. Another fairly common way to do this is to use macros or inline functions that do the bitwise arithmetic necessary to create the correct uint32_t:
#define MAKE_RGBA32(r,g,b,a) (uint32_t)(((r)<<24)|((g)<<16)|((b)<<8)|(a))
uint32_t colors[NUM_COLORS];
colors[i] = MAKE_RGBA32(255,255,255,255);
Depending on your endianness the values may need to be placed into the int in a different order. This technique is common because for older 16bpp color formats like RGBA5551 or RGB565, it makes more sense to think of the colors in terms of the bitwise arithmetic than in units of bytes.
You can perform something similar using struct assignment - this gets around the endian problem:
typedef struct Color {
unsigned char r, g, b, a;
} Color;
const Color WHITE = {0xff, 0xff, 0xff, 0};
const Color RED = {0xff, 0, 0, 0};
const Color GREEN = {0, 0xff, 0, 0};
Color colors[] = {WHITE, RED};
Color c;
colors[1] = GREEN;
c = colors[1];
However comparison is not defined in the standard you can't use c == GREEN and you can't use the {} shortcut in assignment (only initialisation) so c = {0, 0, 0, 0} would fail.
Also bear in mind that if it's an 8 bit AVR (as opposed to an AVR32 say), then you most likely won't see any performance benefit from either technique.
I'm writing a program in C for Linux on an ARM9 processor. The program is to access network packets which include a sequence of tagged data like:
<fieldID><length><data><fieldID><length><data> ...
The fieldID and length fields are both uint16_t. The data can be 1 or more bytes (up to 64k if the full length was used, but it's not).
As long as <data> has an even number of bytes, I don't see a problem. But if I have a 1- or 3- or 5-byte <data> section then the next 16-bit fieldID ends up not on a 16-bit boundary and I anticipate alignment issues. It's been a while since I've done any thing like this from scratch so I'm a little unsure of the details. Any feedback welcome. Thanks.
To avoid alignment issues in this case, access all data as an unsigned char *. So:
unsigned char *p;
//...
uint16_t id = p[0] | (p[1] << 8);
p += 2;
The above example assumes "little endian" data layout, where the least significant byte comes first in a multi-byte number.
You should have functions (inline and/or templated if the language you're using supports those features) that will read the potentially unaligned data and return the data type you're interested in. Something like:
uint16_t unaligned_uint16( void* p)
{
// this assumes big-endian values in data stream
// (which is common, but not universal in network
// communications) - this may or may not be
// appropriate in your case
unsigned char* pByte = (unsigned char*) p;
uint16_t val = (pByte[0] << 8) | pByte[1];
return val;
}
The easy way is to manually rebuild the uint16_ts, at the expense of speed:
uint8_t *packet = ...;
uint16_t fieldID = (packet[0] << 8) | packet[1]; // assumes big-endian host order
uint16_t length = (packet[2] << 8) | packet[2];
uint8_t *data = packet + 4;
packet += 4 + length;
If your processor supports it, you can type-pun or use a union (but beware of strict aliasing).
uint16_t fieldID = htons(*(uint16_t *)packet);
uint16_t length = htons(*(uint16_t *)(packet + 2));
Note that unaligned access aren't always supported (e.g. they might generate a fault of some sort), and on other architectures, they're supported, but there's a performance penalty.
If the packet isn't aligned, you could always copy it into a static buffer and then read it:
static char static_buffer[65540];
memcpy(static_buffer, packet, packet_size); // make sure packet_size <= 65540
uint16_t fieldId = htons(*(uint16_t *)static_buffer);
uint16_t length = htons(*(uint16_t *)(static_buffer + 2));
Personally, I'd just go for option #1, since it'll be the most portable.
Alignment is always going to be fine, although perhaps not super-efficient, if you go through a byte pointer.
Setting aside issues of endian-ness, you can memcpy from the 'real' byte pointer into whatever you want/need that is properly aligned and you will be fine.
(this works because the generated code will load/store the data as bytes, which is alignment safe. It's when the generated assembly has instructions loading and storing 16/32/64 bits of memory in a mis-aligned manner that it all falls apart).