I'm trying to figure out how to manually link an ELF file. I am aware of the file structure, but how do I determine the order in which the relocatable objects are to be linked?
Start with the object containing the entry point. Resolve its internal references. Then take first unresolved reference. Find the object defining an unresolved reference and link it in. Resolve all references you can now. If there are still unresolved references, repeat the process. Something like this...
Related
I have a .so library and while building it I didn't get any undefined reference errors.
But now I am building an executable using the .so file and I can see the undefined reference errors during the linking stage as shown below:
xy.so: undefined reference to `MICRO_TO_NANO_ULL'
I referred to this and this but couldn't really understand the dynamic linking.
Also reading from here lead to more confusion:
Dynamic linking is accomplished by placing the name of a sharable
library in the executable image. Actual linking with the library
routines does not occur until the image is run, when both the
executable and the library are placed in memory. An advantage of
dynamic linking is that multiple programs can share a single copy of
the library.
My questions are:
Doesn't dynamic linking means that when I start the executable using
./executable_name then if the linker not able to locate the .so
file on which executable depends it should crash?
What actually is dynamic linking if all external entity references are
resolved while building? Is it some sort of pre-check performed by dynamic linker? Else
dynamic linker can make use of
LD_LIBRARY_PATH to get additional libraries to resolve the undefined
symbols.
Doesn't dynamic linking means that when I start the executable using ./executable_name then if the linker not able to locate the .so file on which executable depends it should crash?
No, linker will exit with "No such file or directory" message.
Imagine it like this:
Your executable stores somewhere a list of shared libraries it needs.
Linker, think of it as a normal program.
Linker opens your executable.
Linker reads this list. For each file.
It tries to find this file in linker paths.
If it finds the file, it "loads" it.
If it can't find the file, it get's errno with No Such file or directory from open() call. And then prints a message that it can't find the library and terminates your executable.
When running the executable, linker dynamically searches for a symbol in shared libraries.
When it can't find a symbol, it prints some message and the executable teerminates.
You can for example set LD_DEBUG=all to inspect what linker is doing. You can also inspect your executable under strace to see all the open calls.
What actually is dynamic linking if all external entity references are resolved while
building?
Dynamic linking is when you run the executable then the linker loads each shared library.
When building, your compiler is kind enough to check for you, that all symbols that you use in your program exist in shared libraries. This is just for safety. You can for example disable this check with ex. --unresolved-symbols=ignore-in-shared-libs.
Is it some sort of pre-check performed by dynamic linker?
Yes.
Else dynamic linker can make use of LD_LIBRARY_PATH to get additional libraries to resolve the undefined symbols.
LD_LIBRARY_PATH is just a comma separated list of paths to search for the shared library. Paths in LD_LIBRARY_PATH are just processed before standard paths. That's all. It doesn't get "additional libraries", it gets additional paths to search for the libraries - libraries stay the same.
It looks like there is a #define missing when you compile your shared library. This error
xy.so: undefined reference to `MICRO_TO_NANO_ULL'
means, that something like
#define MICRO_TO_NANO_ULL(sec) ((unsigned long long)sec * 1000)
should be present, but is not.
The compiler assumes then, that it is an external function and creates an (undefined) symbol for it, while it should be resolved at compile time by a preprocessor macro.
If you include the correct file (grep for the macro name) or put an appropriate definition at the top of your source file, then the linker error should vanish.
Doesn't dynamic linking means that when I start the executable using ./executable_name then if the linker not able to locate the .so file on which executable depends it should crash?
Yes. If the .so file is not present at run-time.
What actually is dynamic linking if all external entity references are resolved while building? Is it some sort of pre-check performed by dynamic linker? Else dynamic linker can make use of LD_LIBRARY_PATH to get additional libraries to resolve the undefined symbols.
It allows for libraries to be upgraded and have applications still be able to use the library, and it reduces memory usage by loading one copy of the library instead of one in every application that uses it.
The linker just creates references to these symbols so that the underlying variables or functions can be used later. It does not link the variables and functions directly into the executable.
The dynamic linker does not pull in any libraries unless those libraries are specified in the executable (or by extension any library the executable depends on). If you provide an LD_LIBRARY_PATH directory with a .so file of an entirely different version than what the executable requires the executable can crash.
In your case, it seems as if a required macro definition has not been found and the compiler is using implicit declaration rules. You can easily fix this by compiling your code with -pedantic -pedantic-errors (assuming you're using GCC).
Doesn't dynamic linking means that when I start the executable using
./executable_name then if the linker not able to locate the .so file
on which executable depends it should crash?
It will crash. The time of crash does depend on the way you call a certain exported function from the .so file.
You might retrieve all exported functions via functions pointers by yourself by using dlopen dlysm and co. In this case the program will crash at first call in case it does not find the exported method.
In case of the executable just calling an exported method from a shared object (part of it's header) the dynamic linker uses the information of the method to be called in it's executable (see second answer) and crashes in case of not finding the lib or a mismatch in symbols.
What actually is dynamic linking if all external entity references are resolved while building? Is it some sort of pre-check performed by dynamic linker? Else dynamic linker can make use of LD_LIBRARY_PATH to get additional libraries to resolve the undefined symbols.
You need to differentiate between the actual linking and the dynamic linking. Starting off with the actual linking:
In case of linking a static library, the actual linking will copy all code from the method to be called inside the executable/library using it.
When linking a dynamic library you will not copy code but symbols. The symbols contain offsets or other information pointing to the acual code in the dynamic library. If the executable does invoke a method which is not exported by the dynamic library though, it will already fail at the actual linking part.
Now when starting your executable, the OS will at some point try to load the shared object into memory where the code actually resides in. If it does not find it or also if it is imcotable (i.e.: the executable was linked to a library using different exports), it might still fail at runtime.
Suppose I have a shared objects load-time dependency graph and symbol foo referenced in one of the .so. Suppose also this symbol foo is defined in several other shared objects. My question is: which definition will be found, what is a look up order and where it is defined (in what standard, man page)?
Example:
Consider a dependency graph
https://i.imgur.com/jdhD3V0.png
where libraries listed in ldd order e.g.
ldd a.so
b.so
d.so
Lets assume foo defined in c.so and d.so and first referenced in f.so. My experimentation shows that implementation from d.so will be taken by the linker. It looks like that libraries are searched in bfs order. Is this right? Does this coincide with libraries loading order? I could not find this in any documentation and must be sure it is not implementation defined.
Thank you!
Dynamic linking is specified in the ELF specfication. (Note that there are some really old PDFs and Postscript files floating around, but those are generally very outdated.) Symbol lookup is described in section Shared Object Dependencies:
When resolving symbolic references, the dynamic linker examines the symbol tables with a breadth-first search. That is, it first looks at the symbol table of the executable program itself, then at the symbol tables of the DT_NEEDED entries (in order), and then at the second level DT_NEEDED entries, and so on.
(There are various extensions which alter this behavior. The ELF specification itself defines the DF_SYMBOLIC flag.)
This means that your question cannot be answered because your graph does not show the main executable, and it is unclear in which order multiple dependencies are searched (top-to-bottom or bottom-to-top).
Whether the lookup order matches the object loading order is implementation-defined because merely loading an object (without executing its initialization functions) is not something that has an observable effect according to the ELF specification.
Initialization order (the order in which initialization functions are executed) is less constrained than symbol lookup order because the order of DT_NEEDED entries does not matter to that. So in theory, it is possible that an implementation loads an initializes d.so before b.so, but the symbols from b.so interpose that of d.so because it comes first in the symbol search order (due to the way the DT_NEEDED entries are ordered).
yes I actually want to get that error. I am using MSVC (the command prompt). I would like to have .lib that would require definition of external symbol that gets linked to it. I must understand something wrong about static linking, because to me my approach seems legit:
I have a file that looks roughly like this:
extern INFO_BLOCK user_setup;
int
crtInit()
{
SetInfoBlock(&user_setup);
return 0;
}
when I try to use the .obj of this file in compilation with the main module
cl main.c file.obj it says unresolved externals. That is desired behaviour.
Nevertheless, once I pack the file.obj with lib file.obj even using /include:user_data (which frankly I don't trust as being of any use in this case)
using the .lib with cl main.c /link file.lib does not generate missing externals and that is the problem. I need the programmer to define that symbol. Does extern get stripped away once you put your .obj in .lib? Where am I wrong?
If main.c does not contain any reference to crtInit there is no reason for the linker to pull that function into the generated binary - Thus it will not "see" the unresolved reference to user_setup at all.
When you mention an object file to the linker, you force it to include the object file into the binary, regardless of whether it is needed by the program or not.
Opposed to that, when you mention a library to the linker, it will only use the library to resolve unresolved references you already have at that point with object files that it pulls in from this library. In case nothing is unresolved up to this point (or not satisfied by any symbol in the library), the linker will not use anything from the library at all.
The above is also the reason why many linkers are a bit picky about the order of libraries when linking (typically from specific to generic - or "user" to "system"), because linkers are normally single pass and will only pull in what they "see" needed at this specific point in the linking process.
I have an executable which links to a static library. The library exposes a symbol create_widget(). This symbol is not linked into the executable by clang because it is not referenced by the executable.
The executable links to another library which, at runtime, can reference any symbol exposed by the executable. I need to let this library see create_widget().
Is there a good way to resolve this? If not, i'll share the source code for create_widget() with all of the executables that need it instead of bundling it in a library.
There are two ways you can do it:-
1) Link the object file that exports create_widget() explicitly, rather
than from a library. That way any symbols it exports will be linked whether
they are referenced or not.
2) Link the static library (say it's libwidget.a) with the options --whole-archive -lwidget --no-whole-archive.
That way, every object file in libwidget will be linked whether it contains any
referenced symbols or not.
Say I have 2 static libs
ex1.a
ex2.a
In both libs I will define 10 same functions
When Compiling a sample test code say "test.c" , I link with both static libs ex1.a and ex2.a
In "test.c" I will call only 3 functions, then I will get the
linker error "same symbols deifned in both ex1.a and ex2.a libraries" This is Ok.
My Question here is :
1. Why this error only display 3 functions as multiple defined.. Why not it list all 10 functions
In VC8 How can I list all multiple defined symbols without actualy calling that function in test code ...
Thanks,
Thats because, linker tries to resovle a symbol name, when it compiles and links a code which has the function call. Only when the code has some function calls, linker would try to resolve it in either the test code or the libraries linked along and thats when it would find multiple definitions. If no function called, then I guess no problem.
What you experience is the optimizing part of the linker: By default it won't include code that isn't referenced. The compiler will create multiple object files with most likely unresolved dependencies (calls that couldn't be satisfied by the code included). So the linker takes all object files passed and tries to find solutions for the unresolved dependencies. If it fails, it will check the available library files. If there are multiple options with the same exact name/signature it will start complaining cause it won't be able to decide which one to pick (for identical code this won't matter but imagine different implementations using different "behind the scenes" work on memory, such as debug and release stuff).
The only (and possibly easiest way) I could think of to detect all these multiple definitions would be creating another static library project including all source files used in both static libs. When creating a library the linker will include everything called or exported - you won't need specific code calling the stuff for the linker to see/include everything as long as it's exported.
However I still don't understand what you're actually trying to accomplish as a whole. Trying to find code shared between two libraries?