I have following code snippet.
#include<stdio.h>
int main(){
typedef struct{
int a;
int b;
int c;
char ch1;
int d;
} str;
printf("Size: %d \n",sizeof(str));
return 0;
}
Which is giving output as follows
Size: 20
I know that size of the structure is greater than the summation of the sizes of components of the structure because of padding added to satisfy memeory alignment constraints.
I want to know how it is decided that how many bytes of padding have to be added. On what does it depend ? Does it depends on CPU architecture ? And does it depends on compiler also ? I am using here 64bit CPU and gcc compiler. How will the output change if these parameters change.
I know there are similar questions on StackOverflow, but they do not explain this memory alignment constraints thoroughly.
It in general depends on the requirements of the architecture. There's loads over here, but it can be summarized as follows:
Storage for the basic C datatypes on an x86 or ARM processor doesn’t
normally start at arbitrary byte addresses in memory. Rather, each
type except char has an alignment requirement; chars can start on any
byte address, but 2-byte shorts must start on an even address, 4-byte
ints or floats must start on an address divisible by 4, and 8-byte
longs or doubles must start on an address divisible by 8. Signed or
unsigned makes no difference.
In your case the following is probably taking place: sizeof(str) = 4 (4 bytes for int) + 4 (4 bytes for int) + 1 ( 1 byte for char) + 7 (3 bytes padding + 4 bytes for int) = 20
The padding is there so that int is at an address that's a multiple of 4 bytes. This requirement comes from the fact that int is 4 bytes long (my assumption regarding the architecture you're using). But this will vary from one architecture to another.
On what does it depend ? Does it depends on CPU architecture ? And does it depends on compiler also ?
CPU, operating system, and compiler at least.
I know that it depends on the CPU architecture, I think you can find some interesting articles that talks about this on the net, wikipedia is not bad in my opinion.
For me I am using a 64 bit linux machine, and what I can say is that, every field is aligned so that it would be on a memory address divisible by its size (for basic types), for example :
int and float are aligned by 4 (must be in a memory adress divisible by 4)
char and bool by 1 ( which means no padding)
double and pointers are aligned by 8
Best way to avoid padding is to put your fields from largest size to smallest (when there is only basic fields)
When there is composed fields, it a little more difficult for me to explain here, but I think you can figure it out yourself in a paper
On the question 'why do we need to use bit-fields?', searching on Google I found that bit fields are used for flags.
Now I am curious,
Is it the only way bit-fields are used practically?
Do we need to use bit fields to save space?
A way of defining bit field from the book:
struct {
unsigned int is_keyword : 1;
unsigned int is_extern : 1;
unsigned int is_static : 1;
} flags;
Why do we use int?
How much space is occupied?
I am confused why we are using int, but not short or something smaller than an int.
As I understand only 1 bit is occupied in memory, but not the whole unsigned int value. Is it correct?
A quite good resource is Bit Fields in C.
The basic reason is to reduce the used size. For example, if you write:
struct {
unsigned int is_keyword;
unsigned int is_extern;
unsigned int is_static;
} flags;
You will use at least 3 * sizeof(unsigned int) or 12 bytes to represent three small flags, that should only need three bits.
So if you write:
struct {
unsigned int is_keyword : 1;
unsigned int is_extern : 1;
unsigned int is_static : 1;
} flags;
This uses up the same space as one unsigned int, so 4 bytes. You can throw 32 one-bit fields into the struct before it needs more space.
This is sort of equivalent to the classical home brew bit field:
#define IS_KEYWORD 0x01
#define IS_EXTERN 0x02
#define IS_STATIC 0x04
unsigned int flags;
But the bit field syntax is cleaner. Compare:
if (flags.is_keyword)
against:
if (flags & IS_KEYWORD)
And it is obviously less error-prone.
Now I am curious, [are flags] the only way bitfields are used practically?
No, flags are not the only way bitfields are used. They can also be used to store values larger than one bit, although flags are more common. For instance:
typedef enum {
NORTH = 0,
EAST = 1,
SOUTH = 2,
WEST = 3
} directionValues;
struct {
unsigned int alice_dir : 2;
unsigned int bob_dir : 2;
} directions;
Do we need to use bitfields to save space?
Bitfields do save space. They also allow an easier way to set values that aren't byte-aligned. Rather than bit-shifting and using bitwise operations, we can use the same syntax as setting fields in a struct. This improves readability. With a bitfield, you could write
directions.alice_dir = WEST;
directions.bob_dir = SOUTH;
However, to store multiple independent values in the space of one int (or other type) without bitfields, you would need to write something like:
#define ALICE_OFFSET 0
#define BOB_OFFSET 2
directions &= ~(3<<ALICE_OFFSET); // clear Alice's bits
directions |= WEST<<ALICE_OFFSET; // set Alice's bits to WEST
directions &= ~(3<<BOB_OFFSET); // clear Bob's bits
directions |= SOUTH<<BOB_OFFSET; // set Bob's bits to SOUTH
The improved readability of bitfields is arguably more important than saving a few bytes here and there.
Why do we use int? How much space is occupied?
The space of an entire int is occupied. We use int because in many cases, it doesn't really matter. If, for a single value, you use 4 bytes instead of 1 or 2, your user probably won't notice. For some platforms, size does matter more, and you can use other data types which take up less space (char, short, uint8_t, etc.).
As I understand only 1 bit is occupied in memory, but not the whole unsigned int value. Is it correct?
No, that is not correct. The entire unsigned int will exist, even if you're only using 8 of its bits.
Another place where bitfields are common are hardware registers. If you have a 32 bit register where each bit has a certain meaning, you can elegantly describe it with a bitfield.
Such a bitfield is inherently platform-specific. Portability does not matter in this case.
We use bit fields mostly (though not exclusively) for flag structures - bytes or words (or possibly larger things) in which we try to pack tiny (often 2-state) pieces of (often related) information.
In these scenarios, bit fields are used because they correctly model the problem we're solving: what we're dealing with is not really an 8-bit (or 16-bit or 24-bit or 32-bit) number, but rather a collection of 8 (or 16 or 24 or 32) related, but distinct pieces of information.
The problems we solve using bit fields are problems where "packing" the information tightly has measurable benefits and/or "unpacking" the information doesn't have a penalty. For example, if you're exposing 1 byte through 8 pins and the bits from each pin go through their own bus that's already printed on the board so that it leads exactly where it's supposed to, then a bit field is ideal. The benefit in "packing" the data is that it can be sent in one go (which is useful if the frequency of the bus is limited and our operation relies on frequency of its execution), and the penalty of "unpacking" the data is non-existent (or existent but worth it).
On the other hand, we don't use bit fields for booleans in other cases like normal program flow control, because of the way computer architectures usually work. Most common CPUs don't like fetching one bit from memory - they like to fetch bytes or integers. They also don't like to process bits - their instructions often operate on larger things like integers, words, memory addresses, etc.
So, when you try to operate on bits, it's up to you or the compiler (depending on what language you're writing in) to write out additional operations that perform bit masking and strip the structure of everything but the information you actually want to operate on. If there are no benefits in "packing" the information (and in most cases, there aren't), then using bit fields for booleans would only introduce overhead and noise in your code.
To answer the original question »When to use bit-fields in C?« … according to the book "Write Portable Code" by Brian Hook (ISBN 1-59327-056-9, I read the German edition ISBN 3-937514-19-8) and to personal experience:
Never use the bitfield idiom of the C language, but do it by yourself.
A lot of implementation details are compiler-specific, especially in combination with unions and things are not guaranteed over different compilers and different endianness. If there's only a tiny chance your code has to be portable and will be compiled for different architectures and/or with different compilers, don't use it.
We had this case when porting code from a little-endian microcontroller with some proprietary compiler to another big-endian microcontroller with GCC, and it was not fun. :-/
This is how I have used flags (host byte order ;-) ) since then:
# define SOME_FLAG (1 << 0)
# define SOME_OTHER_FLAG (1 << 1)
# define AND_ANOTHER_FLAG (1 << 2)
/* test flag */
if ( someint & SOME_FLAG ) {
/* do this */
}
/* set flag */
someint |= SOME_FLAG;
/* clear flag */
someint &= ~SOME_FLAG;
No need for a union with the int type and some bitfield struct then. If you read lots of embedded code those test, set, and clear patterns will become common, and you spot them easily in your code.
Why do we need to use bit-fields?
When you want to store some data which can be stored in less than one byte, those kind of data can be coupled in a structure using bit fields.
In the embedded word, when one 32 bit world of any register has different meaning for different word then you can also use bit fields to make them more readable.
I found that bit fields are used for flags. Now I am curious, is it the only way bit-fields are used practically?
No, this not the only way. You can use it in other ways too.
Do we need to use bit fields to save space?
Yes.
As I understand only 1 bit is occupied in memory, but not the whole unsigned int value. Is it correct?
No. Memory only can be occupied in multiple of bytes.
Bit fields can be used for saving memory space (but using bit fields for this purpose is rare). It is used where there is a memory constraint, e.g., while programming in embedded systems.
But this should be used only if extremely required because we cannot have the address of a bit field, so address operator & cannot be used with them.
A good usage would be to implement a chunk to translate to—and from—Base64 or any unaligned data structure.
struct {
unsigned int e1:6;
unsigned int e2:6;
unsigned int e3:6;
unsigned int e4:6;
} base64enc; // I don't know if declaring a 4-byte array will have the same effect.
struct {
unsigned char d1;
unsigned char d2;
unsigned char d3;
} base64dec;
union base64chunk {
struct base64enc enc;
struct base64dec dec;
};
base64chunk b64c;
// You can assign three characters to b64c.enc, and get four 0-63 codes from b64dec instantly.
This example is a bit naive, since Base64 must also consider null-termination (i.e. a string which has not a length l so that l % 3 is 0). But works as a sample of accessing unaligned data structures.
Another example: Using this feature to break a TCP packet header into its components (or other network protocol packet header you want to discuss), although it is a more advanced and less end-user example. In general: this is useful regarding PC internals, SO, drivers, an encoding systems.
Another example: analyzing a float number.
struct _FP32 {
unsigned int sign:1;
unsigned int exponent:8;
unsigned int mantissa:23;
}
union FP32_t {
_FP32 parts;
float number;
}
(Disclaimer: Don't know the file name / type name where this is applied, but in C this is declared in a header; Don't know how can this be done for 64-bit floating-point numbers since the mantissa must have 52 bits and—in a 32 bit target—ints have 32 bits).
Conclusion: As the concept and these examples show, this is a rarely used feature because it's mostly for internal purposes, and not for day-by-day software.
To answer the parts of the question no one else answered:
Ints, not Shorts
The reason to use ints rather than shorts, etc. is that in most cases no space will be saved by doing so.
Modern computers have a 32 or 64 bit architecture and that 32 or 64 bits will be needed even if you use a smaller storage type such as a short.
The smaller types are only useful for saving memory if you can pack them together (for example a short array may use less memory than an int array as the shorts can be packed together tighter in the array). For most cases, when using bitfields, this is not the case.
Other uses
Bitfields are most commonly used for flags, but there are other things they are used for. For example, one way to represent a chess board used in a lot of chess algorithms is to use a 64 bit integer to represent the board (8*8 pixels) and set flags in that integer to give the position of all the white pawns. Another integer shows all the black pawns, etc.
You can use them to expand the number of unsigned types that wrap. Ordinary you would have only powers of 8,16,32,64... , but you can have every power with bit-fields.
struct a
{
unsigned int b : 3 ;
} ;
struct a w = { 0 } ;
while( 1 )
{
printf("%u\n" , w.b++ ) ;
getchar() ;
}
To utilize the memory space, we can use bit fields.
As far as I know, in real-world programming, if we require, we can use Booleans instead of declaring it as integers and then making bit field.
If they are also values we use often, not only do we save space, we can also gain performance since we do not need to pollute the caches.
However, caching is also the danger in using bit fields since concurrent reads and writes to different bits will cause a data race and updates to completely separate bits might overwrite new values with old values...
Bitfields are much more compact and that is an advantage.
But don't forget packed structures are slower than normal structures. They are also more difficult to construct since the programmer must define the number of bits to use for each field. This is a disadvantage.
Why do we use int? How much space is occupied?
One answer to this question that I haven't seen mentioned in any of the other answers, is that the C standard guarantees support for int. Specifically:
A bit-field shall have a type that is a qualified or unqualified version of _Bool, signed int, unsigned int, or some other implementation defined type.
It is common for compilers to allow additional bit-field types, but not required. If you're really concerned about portability, int is the best choice.
Nowadays, microcontrollers (MCUs) have peripherals, such as I/O ports, ADCs, DACs, onboard the chip along with the processor.
Before MCUs became available with the needed peripherals, we would access some of our hardware by connecting to the buffered address and data buses of the microprocessor. A pointer would be set to the memory address of the device and if the device saw its address along with the R/W signal and maybe a chip select, it would be accessed.
Oftentimes we would want to access individual or small groups of bits on the device.
In our project, we used this to extract a page table entry and page directory entry from a given memory address:
union VADDRESS {
struct {
ULONG64 BlockOffset : 16;
ULONG64 PteIndex : 14;
ULONG64 PdeIndex : 14;
ULONG64 ReservedMBZ : (64 - (16 + 14 + 14));
};
ULONG64 AsULONG64;
};
Now suppose, we have an address:
union VADDRESS tempAddress;
tempAddress.AsULONG64 = 0x1234567887654321;
Now we can access PTE and PDE from this address:
cout << tempAddress.PteIndex;
Code 1:-
struct emp
{
char a;
double b;
};
int main()
{
struct emp e;
printf("%p %p", (void*)&e.a, (void*)&e.b);
}
Output on my computer:-
OO28FF00 0028FF08
As the size of char and double is '1' and '8' respectively and hence the 0028FF00 and 0028FF08 are multiples of '1' and '8' respectively.
Code 2:-
struct emp
{
char a;
long double b;
};
int main()
{
struct emp e;
printf("%p %p \n", (void*)&e.a,(void*)&e.b);
}
The output is :-
0028FF00 0028FF04
As the size of char and long double is '1' and '12' respectively but 0028FF04 is not a multiple of '12'.
Why padding is not applied in this case?
A long double is an 80 bit floating point so you need 10 bytes. 10 is really not a very good size though, thus Intel 32 bit processors decided on 12 bytes. 12 is a multiple of 4 which represents 32 bits (3 x 32 bits). This is considered aligned because a 32 bit processor only needs 4 bytes alignment, so the 12 bytes is aligned at any 4 bytes boundary. Obviously, the compiler knows what it's doing and it always tries to generate the smallest possible structure.
This being said, this is where you see that you cannot use a struct declaration and hope to save it as is in a file... at least not with the default C types (you can use int32_t, uint64_t, etc. to get exactly what you want, but there is no equivalent for floating point numbers...)
As someone commented, on a 64 bit architecture, long double is 16 bytes. A waste of 6 bytes... but it makes the type 64 bit aligned all the time.
Wikipedia has a table of typical alignments:
http://en.wikipedia.org/wiki/Data_structure_alignment#Typical_alignment_of_C_structs_on_x86
x86 has flexible memory access instructions, so there rules are up to the compiler designers' decision. We can only imagine why they thought there are the most reasonable.
I found #LưuVĩnhPhúc 's comment very educative. Although long double is 12 bytes on your platform (GCC?) it was 4 bytes aligned by the same reason why a 512 bytes struct will not be 512 bytes aligned; there will be too much waste in space. I guess the designers of GCC thought accessing double variables should be done in the least possible latency at the cost of some space (up to 7 bytes.)
Well, as far as I know padding and data alignment heavily depend on the target architecture, compiler optimization options and overall quality of the optimizator. Thus if you don't specifically care about alignment, you get a "quazi-optimal" structures and the compiler is free to decide what is better for this particular set of optimization options (whether -Os or -O is used and so on). If you wish to have a specific alignment, you should use compiler-specific options to tune the things. For GCC and CLang use __packed__ attribute, for MSVC use #pragma pack. See pragma pack(1) nor __attribute__ ((aligned (1))) works for more info
I do have a structure having bit-fields in it.Its comes out to be 2 bytes according to me but its coming out to be 4 .I have read some question related to this here on stackoverflow but not able to relate to my problem.This is structure i do have
struct s{
char b;
int c:8;
};
int main()
{
printf("sizeof struct s = %d bytes \n",sizeof(struct s));
return 0;
}
if int type has to be on its memory boundary,then output should be 8 bytes but its showing 4 bytes??
Source: http://geeksforgeeks.org/?p=9705
In sum: it is optimizing the packing of bits (that's what bit-fields are meant for) as maximum as possible without compromising on alignment.
A variable’s data alignment deals with the way the data stored in these banks. For example, the natural alignment of int on 32-bit machine is 4 bytes. When a data type is naturally aligned, the CPU fetches it in minimum read cycles.
Similarly, the natural alignment of short int is 2 bytes. It means, a short int can be stored in bank 0 – bank 1 pair or bank 2 – bank 3 pair. A double requires 8 bytes, and occupies two rows in the memory banks. Any misalignment of double will force more than two read cycles to fetch double data.
Note that a double variable will be allocated on 8 byte boundary on 32 bit machine and requires two memory read cycles. On a 64 bit machine, based on number of banks, double variable will be allocated on 8 byte boundary and requires only one memory read cycle.
So the compiler will introduce alignment requirement to every structure. It will be as that of the largest member of the structure. If you remove char from your struct, you will still get 4 bytes.
In your struct, char is 1 byte aligned. It is followed by an int bit-field, which is 4 byte aligned for integers, but you defined a bit-field.
8 bits = 1 byte. Char can be any byte boundary. So Char + Int:8 = 2 bytes. Well, that's an odd byte boundary so the compiler adds an additional 2 bytes to maintain the 4-byte boundary.
For it to be 8 bytes, you would have to declare an actual int (4 bytes) and a char (1 byte). That's 5 bytes. Well that's another odd byte boundary, so the struct is padded to 8 bytes.
What I have commonly done in the past to control the padding is to place fillers in between my struct to always maintain the 4 byte boundary. So if I have a struct like this:
struct s {
int id;
char b;
};
I am going to insert allocation as follows:
struct d {
int id;
char b;
char temp[3];
}
That would give me a struct with a size of 4 bytes + 1 byte + 3 bytes = 8 bytes! This way I can ensure that my struct is padded the way I want it, especially if I transmit it somewhere over the network. Also, if I ever change my implementation (such as if I were to maybe save this struct into a binary file, the fillers were there from the beginning and so as long as I maintain my initial structure, all is well!)
Finally, you can read this post on C Structure size with bit-fields for more explanation.
int c:8; means that you are declaring a bit-field with the size of 8 bits. Since the alignemt on 32 bit systems is normally 4 bytes (=32 bits) your object will appear to have 4 bytes instead of 2 bytes (char + 8 bit).
But if you specify that c should occupy 8 bits, it's not really an int, is it? The size of c + b is 2 bytes, but your compiler pads the struct to 4 bytes.
They alignment of fields in a struct is compiler/platform dependent.
Maybe your compiler uses 16-bit integers for bitfields less than or equal to 16 bits in length, maybe it never aligns structs on anything smaller than a 4-byte boundary.
Bottom line: If you want to know how struct fields are aligned you should read the documentation for the compiler and platform you are using.
In generic, platform-independent C, you can never know the size of a struct/union nor the size of a bit-field. The compiler is free to add as many padding bytes as it likes anywhere inside the struct/union/bit-field, except at the very first memory location.
In addition, the compiler is also free to add any number of padding bits to a bit-field, and may put them anywhere it likes, because which bit is msb and lsb is not defined by C.
When it comes to bit-fields, you are left out in the cold by the C language, there is no standard for them. You must read compiler documentation in detail to know how they will behave on your specific platform, and they are completely non-portable.
The sensible solution is to never ever use bit fields, they are a reduntant feature of C. Instead, use bit-wise operators. Bit-wise operators and in-depth documented bit-fields will generate the same machine code (non-documented bit-fields are free to result in quite arbitrary code). But bit-wise operators are guaranteed to work the same on any compiler/system in the world.
Are pointers on a 64-bit system still 4 byte aligned (similar to a double on a 32 bit system)? Or are they note 8 byte aligned?
For example, on a 64-bit system how big is the following data structure:
struct a {
void* ptr;
char myChar;
}
Would the pointer by 8 byte aligned, causing 7 bytes of padding for the character (total = 8 + 8 = 16)? Or would the pointer be 4 byte aligned (4 bytes + 4 bytes) causing 3 bytes of padding (total = 4 + 4 + 4 = 12)?
Thanks,
Ryan
Data alignment and packing are implementation specific, and can be usually changed from compiler settings (or even with pragmas).
However assuming you're using default settings, on most (if not all) compilers the structure should end up being 16 bytes total. The reason is because computers reads a data chunk with size of its native word size (which is 8 bytes in 64-bit system). If it were to pad it to 4 byte offsets, the next structure would not be properly padded to 64-bit boundary. For example in case of a arr[2], the second element of the array would start at 12-byte offset, which isn't at the native byte boundary of the machine.
I don't think you can rely on any hard-and-fast rules. I think it's a function of the compiler you use and the compilation options you choose.
Your best bet is to write a program that tests this and spits out a header file that codifies the alignment rules as #defines. You might also be able to just calculate what you're interested in right in the macros, too.
Generally on a 64-bit system:
struct a {
void* ptr; // size is 8 bytes, alignment is 8
char myChar; // size is 1 byte, alignment is 1
// padding of 7 bytes so array elements will be properly aligned
}
For a total size of 16 bytes.
But this is all implementation defined - I'm just giving an example that likely to be true for many (most?) 64-bit systems.
You would need to consult the documentation for the particular ABI you are interested in. For example, here is the System V ABI x86-64 architeture supplement - you can see on page 12 that pointers on this ABI are 8-byte aligned (so yes, the structure you show would be padded out to 16 bytes).
The language standard makes no statements about padding. The alignment rules are platform-specific (i.e., you have to align differently on e.g. a PowerPC CPU than on a x86_64 CPU), and they are implementation-defined, meaning your compiler can do whatever works (and might change that behaviour with different command-line options or after a version update).
I strongly believe that any recommendation along the lines of "this is usually this or that" is misleading, and possibly dangerous.
You could write a test program that executes a couple of sizeof() and/or offsetof()statements and writes a header for you containing some #defines stating the paddings used.
You can use autoconf to do that for you.
In the very least, you should add assert( sizeof( ... ) ) statements at the beginning of your main() function so you get informed when your assumptions are wrong.