__start_section and __stop_section symbols missing when linking to library - c

Question

I'm using custom elf headers in an autotools C project similar to this thread: How do you get the start and end addresses of a custom ELF section in C (gcc)?. The problem is that the c files that declare the custom sections are linked into a static library which is then linked to the final application.
In this configuration the symbols __start_custom_section and __stop_custom_section do not get generated. I define the elf section like this:
struct mystruct __attribute((__section__("custom_section"))) __attribute((__used__) = {
...
};
If I link to the object file instead of the library the symbols get created and everything works as expected. This isn't a scalable solution though because I'd like new modules to just work by compiling them into the modules library. Any idea why the linker doesn't create these special symbols when the section exists in a library vs a single object file?

Answer 1

I have done something similar to this recently, and my solution does not rely on any compiler specific implementations, internal undocumented symbols, etc. However, it does require a bit more work :)
Background
The ELF binary on disk can be loaded and parsed quite easily by knowing its format and using a couple structures provided to us: http://linux.die.net/man/5/elf. You can iterate through each of its segments and sections (segments are containers for sections). If you do this, you can calculate the the relative start/end virtual addresses of your section. By this logic, you would think that you can do the same thing at runtime by iterating through the segments and sections of the loaded, in-memory version of the ELF binary. But alas, you can only iterate through the segments themselves (via http://linux.die.net/man/3/dl_iterate_phdr), and all section metadata has been lost.
So, how can we retain the section metadata? Store it ourselves.
Solution
If you have a custom section named '.mycustom', then define a metadata struct that should at minimum store two numbers that will indicate the relative start address and the size of your '.mycustom' section. Create a global instance of this metadata struct that will live by itself in another custom section named '.mycustom_meta'.
Example:
typedef struct
{
unsigned long ulStart;
unsinged long ulSize;
} CustomSectionMeta;
__attribute((__section__(".mycustom_meta"))) CustomSectionMeta g_customSectionMeta = { 0, 0 };
You can see that our struct instance is initialized with zero for both start and size values. When you compile this code, your object file will contain a section named '.mycustom_meta' which will be 8 bytes in size for a 32-bit compilation (or 16 bytes for 64-bit), and the values will be all zeroes. Run objdump on it and you will see as much. Go ahead and put that into a static lib (.a) if you want, run readelf on it, and you will see exactly the same thing. Build it into a shared object (.so) if you want, run readelf on it, and again you will see the same thing. Build it into an executable program, run readelf on it, and voila its still there.
Now the trick comes in. You need to write a little executable (lets call it MetaWriter) that will update your ELF file on disk to fill in the start and size values. Here are the basic steps:
Open your ELF file (.o, .so, or executable) in binary mode and read it into a contiguous array. Or, you can mmap it into memory to achieve the same.
Read through the binary using header structures and instructions found in the ELF link I listed above.
Find your '.mycustom' section and read section.sh_addr and section.sh_size.
Find your '.mycustom_meta' section. Create an instance of CustomSectionMeta using the start and size values from step 3. memcpy() your struct over the top of the existing '.mycustom_meta' section data, which up to now was all zeroes.
Save you ELF data back to the original file. It should now be completely unmodified except for the few bytes you wrote into your '.mycustom_meta' section.
What I did was executed this MetaWriter program as part of the build process in my Makefile. So, you would build your .so or executable, then run MetaWriter on it to fill in the meta section. After that, its ready to go.
Now, when the code in your .so or executable runs, it can just read from g_customSectionMeta, which will be populated with the starting address offset of your '.mycustom' section, as well as the size of it, which can be used to easily calculate the end, of course. This start offset must be added to the base address of your loaded ELF binary. There are a couple ways to get this, but the easiest way I found was to run dladdr on a symbol that I know to exist in the binary (such as g_customSectionMeta!) and use the resulting value of dli_fbase to know the base address of the module.
Example:
#include <dlfcn.h>
Dl_info dlInfo;
if (dladdr(&g_customSectionMeta, &dlInfo) != 0)
{
void * vpBase = dlInfo.dli_fbase;
void * vpMyCustomStart = vpBase + g_customSectionMeta.ulStart;
void * vpMyCustomEnd = vpMyCustomStart + g_customSectionMeta.ulSize;
}
It would be a bit overboard to post the full amount of code required to do all this work, especially the parsing of the ELF binary in MetaWriter. However, if you need some help, feel free to reach out to me.

Answer 2

In my case, the variable was not referenced in the code and the section was optimised out in release mode (-O2). Adding used attribute solved the issue. Example:
static const unsigned char unused_var[] __attribute__((used, section("foo"))) = {
0xCA, 0xFE, 0xBA, 0xBE
};