I believe my question is different to this one. Here I am asking why we need to differentiate those two, the link only answer which one goes to which.
We know that:
Common section is for uninitialized global variables and
Bss section is for uninitialized static variables plus global variable initialized to 0.
But why differentiate BSS and COMMON section? Especially for global variables initialized to 0, can't we put them in .data section which is for initialized global variables? Isn't that initialize a variable to 0 is also a initialization?
Below is an explanation from my textbook:
in some cases the linker allows multiple modules to define global symbols with the same name. When the compiler is translating some module and encounters a weak global symbol, say, x, it does not know if other modules also define x, and if so, it cannot predict which of the multiple instances of x the linker might choose. So the compiler defers the decision to the linker by assigning x to COMMON. On the other hand, if x is initialized to zero, then it is a strong symbol, so the compiler can confidently assign it to bss.
I am really confused, it says “it does not know if other modules also define x”, but how can you define a variable twice? Is an example code available to illustrate?
.bss section is used for allocating zero-initialized data for optimization purposes, to allow
static linker to reduce executable size (but not storing zeros in it)
runtime linker (loader) to speed up the loading process: common data is efficiently initialized either by mapping a dedicated physical page filled with zeroes or by memseting memory at startup.
Common section is used (on some platforms e.g. Windows but not ELF) to implement so called "common symbols" i.e. symbols which may be duplicated in different object files ("translation units"). When such symbol falls into a common section, static linker will merge all separate definitions (with some platform-specific rules, e.g. merge only if identical, prefer largest definition, etc.).
On some targets common sections are used only for uninitialized data (which make them somewhat similar to .bss) and on others also for vague symbols. In general there are no logical reasons for why different platforms made different choices regarding usage of common sections, it's purely historical.
You can find some history behind common symbols in [Raymond Chen's article]
(https://devblogs.microsoft.com/oldnewthing/20161024-00/?p=94575).
Related
I realize there are ways to remove unreferenced symbols from the final binary by passing flags to the compiler and linker, but why doesn't this happen by default (static linking)?
Because there are some traditional practices that depend on unreferenced variables staying in the binary.
In particular, it has been common to declare a global string containing special sequences that are replaced by the version control system, e.g. something like this:
static char sccsid[] = "#(#)ls.c 8.1 (Berkeley) 6/11/93";
Standard(*) linking semantics for static libraries are that exactly those object files from an archive (static library) which are needed to resolve undefined symbols get pulled into the link, as if they were object files listed on the link command line. So, as long as you factor your libraries into indepedent translation units (and thus object files) well, unreferenced symbols "are optimized out" already, by never being pulled in to begin with.
If you want finer-grained optimizing-out, you need to leave the object files in a form where this is possible. Traditionally, object files contain a single text section for all code and a single data section for all data, and these are already flattened in a way that individual functions or data objects can't be subsequently removed. Modern tooling optionally supports using a separate section per function or data object, which the linker can then use for fine-grained dropping of unreferenced sections via --gc-sections. Arguably this should be default nowadays, but it does break certain custom linking setups using explicit placement of code or objects into sections without explicit referencing, which is probably the reason why it's still not default.
(*) Here "standard" is outside the scope of the C language standard, and is a matter of how the Unix-derived C language tooling has always worked and been specified (roughly equivalently) in various places like SysV, ELF, etc.
We just encountered and solved an issue in which our RAM spiked when we included some code that accessed a certain large array. This leads to this follow on question: I apparently was under the misconception that C compilers excluded functions that weren't called, but didn't exclude arrays declared at the file level but weren't touched. I guess it makes all the sense in the world that it would do this, but I'm sure I've seen different behavior, just created an array and watched RAM usage jump (without writing code that touches the array). This was esp. shocking since we are at zero optimization.
So to learn the right lesson here: are arrays that are not touched always, never, or sometimes excluded by compilers. Does it depend on the compiler and the optimization level, or is this somehow tied to a C standard requirement? And am I crazy, or do most compilers seem to not exclude them?
Thanks.
As far as the C standard is concerned, C allows optimization, but a compiler need not optimize the code at all to be compliant.
As for how most systems work in practice, file scope variables are allocated in .data or .bss sections and have static storage duration, meaning that they have to get initialized to a value by the compiler before main() is called. This is a C standard requirement.
A compiler with optimizations disabled may therefore very well include such variables as part of the initialization code, regardless of if those variables are used or not. And most compilers have optimizations disabled as default.
You can help the compiler to do a better job at spotting unused variables by declaring them static - meaning that the variable gets "internal linkage" and no other file can touch it. If you don't, then the compiler might not be able to tell if the variable is used before compiling all other files, perhaps getting forced to leave that decision to the linker.
But overall, it isn't meaningful to ponder about what a compiler will do with optimizations off. If the unused variable is still allocated with optimizations enabled, that's when you should start to worry.
You say "compiler" but this is a function of the linker: only the linker can know if an array is not used by any of the compilation units (object files).
The linker knows this, if no object file has a reference to the array (the data) that the linker has to resolve.
What remains is the compilation unit that declares the array (the data). That unit does not have (or does not need to have) a reference to the data because it is declared in the compilation unit (object file).
Taken together, there may be no way for the compiler or linker to know if some data is not used and consequently the linker will need to include it in the executable.
Note: if the array is declared static, then the compiler can decide because the data will have no visibility outside the current compilation unit.
I'm using the checkpatch.pl script from the linux kernel for my own firmware since I use the same coding style (which I like).
There is just an error that I don't quite understand about global variables:
do not initialise globals to 0
For sure I want to avoid using globals as much as possible, but don't know why this is a style error?
Is it because some compilers don't put such globals in .BSS?
(Usually they are smart enough)
First, it is redundant, and increases the size of the kernel (not what is finally loaded, but by having explicit instructions to the linker which are unnecessary).
It is part of a larger problem:
Supposing that you had two different object files to link together, with different ideas of how to initialize them. Then the linker has to detect that and produce a symbol conflict error. The script is concerned with that as well.
Further reading:
.bss vs COMMON: what goes where?
Uninitialised global data in C – .bss section vs common symbols
Shared Libraries Redux (Ian Lance Taylor)
I'm writing an embedded application and almost all of my RAM is used by global byte-arrays. When my firmware boots it starts by overwriting the whole BSS section in RAM with zeroes, which is completely unnecessary in my case.
Is there some way I can instruct the compiler that it doesn't need to zero-initialize certain arrays? I know this can also be solved by declaring them as pointers, and using malloc(), but there are several reasons I want to avoid that.
The problem is that standard C enforces zero initialization of static objects. If the compiler skips it, it wouldn't conform to the C standard.
On embedded systems compilers there is usually a non-standard option "compact startup" or similar. When enabled, no initialization of static/global objects will occur at all, anywhere in the program. How to do this depends on your compiler, or in this case, on your gcc port.
If you mention which system you are using, someone might be able to provide a solution for that particular compiler port.
This means that any static/global (static storage duration) variable that you initialize explicitly will no longer be initialized. You will have to initialize it in runtime, that is, instead of static int x=1; you will have to write static int x; x=1;. It is rather common to write embedded C programs in this manner, to make them compatible with compilers where the static initialization is disabled.
It turned out that the linker-script included in my toolchain has a special "noinit" section.
__attribute__ ((section (".noinit")))
/** Forces the compiler to not automatically zero the given global
variable on startup, so that the current RAM contents is retained.
Under most conditions this value will be random due to the
behaviour of volatile memory once power is removed, but may be used in some specific
circumstances, like the passing of values back after a system watchdog reset.
So all global variabeles marked with that attribute will not be zero-initialised during boot.
The C standard REQUIRES global data to be initialized to zero.
It is possible that SOME embedded system manufacturers provide a way to bypass this option, but there are certainly many typical applications that would simply fail if the "initialize to zero" wasn't done.
Some compilers also allow you to have further sections, which may have other characteristics than the 'bss' section.
The other alternative is of course to "make your own allocation". Since it's an embedded system, I suppose you have control over how the application and data is loaded into RAM, in particular, what addresses are used for that.
So, you could use a pointer, and simply use your own mechanism for assigning the pointer to a memory region that is reserved for whatever you need large arrays for. This avoids the rather complex usage of malloc - and it gives you a more or less permanent address, so you don't have to worry about trying to find where your data is later on. This will of course have a small effect on performance, since it adds another level of indirection, but in most cases, that disappears as soon as the array is used as an argument to a function, as it decays to a pointer at that point anyways.
There are a few workarounds like:
Deleting the BSS section from the binary or setting its size to 0 or 1. This will not work if the loader must explicitly allocate memory for all sections. This will work if the loader simply copies data to the RAM.
Declaring your arrays as extern in C code and defining the symbols (along with their addresses) either in assembly code in separate assembly files or in the linker script. Again, if memory must be explicitly allocated, this won't work.
Patching or removing the relevant BSS-zeroing code either in the loader or in the startup code that is executed in your program before main().
All embedded compilers should allow a noinit segment. With the IAR AVR compiler the variables you don't want to be initialised are simply declared as follows:
__no_init uint16_t foo;
The most useful reason for this is to allow variables to maintain their values over a watchdog or brown-out reset, which of course doesn't happen in computer-based C programs, hence its omission from standard C.
Just search you compiler manual for "noinit" or something similar.
Are you sure the binary format actually includes a BSS section in the binary? In the binary formats I've worked with BSS is simply a integer that tells the kernel/loader how much memory to allocate and zero out.
There definitely is no general way in C to get uninitialized global variables. This would be a function of your compiler/linker/runtime system and highly specific to that.
with gcc, -fno-zero-initialized-in-bss
Assume I have a 128KB memory region. In my linker directives I split this region into three sections:
.section_text
.section_data
.section_bss
The size of each section is unknown pre-compilation, but I have constrained .section_bss to use all remaining space within the memory region after .section_text and .section_data are allocated.
Is there any way I can declare a C array that uses up all available space in .region_bss? Assume it is the only thing using .region_bss so it can safely use the entire region. For example purposes but obviously wrong:
char entire_bss[sizeof(.region_bss)];
Here are my pre-answers to some anticipated responses. First, I know sizeof() doesn't work like this. I'm just using it to get an idea across. Second, assume this must be done with an array and not with pointers (solving with pointers is possible and fairly simple). Third, I'm aware I can get the start and end addresses of .region_bss, but I'm not aware of any way to use them to size my array. At least not any way that works in C.
There very well may be no way to do this, but I'm hoping some genius out there has figured it out. Extra credit if you can make it work with the Green Hills toolset.
Is there any way I can declare a C array that uses up all available space in .region_bss?
Short answer is "no."
If GreenHills is using GNU toolchain with binutils and allows customizing the linker script, then you can add into an application namespace a variable using PROVIDE to mark the end of the 128K block. (See an example how to access such variables here). You will not have a C array of fixed size that way, but program would be able to find the end of the array thus the size of the array what is generally sufficient for C programs.
If you want to accomplish that using pure C99, then you might be out of luck as it is highly unreliable as it is not covered by the standard.
Usually the way you do this is with something like:
extern char section_bss[];
With some extra stuff thrown in so that the compiler, assembler, and linker know that section_bss is an alias for .section_bss (or similar name).
To get the size you would probably be want to do the same thing for a symbol that is at the end of bss and find the difference.
No you can't. It's also easy to understand why. At the compilation phase there is no concept of .bss and .data these are pure linker conventions. The compiler knows some abstract concepts (const data, static data, code), but the mapping of them to the linker section is only a convention, and depending on linker, OS and processor or even memory model options. When building position independend code, often the constants are put in the code segment for example. On embedded platforms you sometime have to struggle with these things and even have to adapt your coding style because of that (we had once a model based in 80186 processors that were built with MS-C 5.1 compiler, were we couldn't use the .data section (global, initialized variables and statics) because the toolchain put them in the ROM.
We solved once a similar problem of yours by patching the generated binary at the end of the whole build process. We extracted the used memory of the different section from the map file generated by the linker (it's an option look it up), and subtracted it from the memory size of the given model (we had 256K, 512K and 896K models) and set one of the global constants with the right value. It was thus important that all references to this size work via this global variable. This memory pool was then used by our implementation of malloc/free.
What you are seeking to do will be very platform dependent. Your best bet is probably to declare an array as extern char the_array[],end_of_array[]; within your C file, and then, perhaps in an assembly-language source file, declare a couple of data sections for the_array and end_of_array. Then arrange the linker spec so that the_array will be at the end of stuff that's allocated bottom-up, and end_of_array will be at the start of stuff that's allocated top-down, and nothing else will be between them. It's possible--I've done it on some platforms--but it's important to read the linker documentation carefully to ensure that the linker won't rearrange things when you don't expect it.
What's wrong with
extern char __ghsbegin_region_bss[];
extern char __ghssize_region_bss[];
#define entire_bss __ghsbegin_region_bss
#define sizeof_entire_bss ((size_t)__ghssize_region_bss)
? I can think of at least one reason you might not be satisfied with this answer, but it involves C++ template metaprogramming magic (namely, the type of entire_bss wouldn't have the right size, so templates that depend on its size would resolve differently). As long as you don't use the size of the array for anything (and this is true, in C), this approach should be fine.
Note that __ghssize_region_bss is not a compile-time constant — it's not resolved until link-time — so there's 100% no way to get it inside the [...] of an array type at compile time.
See the chapter on "Beginning, End, and Size of Section Symbols" in the Green Hills manual.