I know dlopen is used for loading a shared library compiled with "-shared" and possibly "-fPIC".
But since the executable and dynamic library share the same format, I'm wondering if it is possible to directly load an executable?
It is acceptable that we may need to re-construct a customized dlopen and dlsym, just want to know whether it is possible to do such things while being transparent to the binaries.
BTW, It is also acceptable that the loaded executable cannot be reused (since it is not compiled with -fPIC).
Thanks in advance.
This used to be possible to dlopen a PIE executable on Linux, but not anymore -- GLIBC-2.30 prohibits this.
Since it is possible to execute a .so file:
https://stackoverflow.com/a/68339111/14760867
Can you turn the question around to compile the executable (application) as a '.so' file? With -shared etc?
Related
My application needs to load one or more algorithms at run time and I use .so for this. The thing is that these libraries are not used by any other process but my applicaiton so there is no need to share the .text section with others. Some parts of the .so come from other static libraries that I compile beforehand.
In this case, do I still have to use -fpic flag for the static files?
EDIT
I found this article article. At page 7 it states this "So, if performance is important for a library or dynamically loadable module, you can compile it as non-PIC code. The primary downside to compiling the module as non-PIC is that loading time in-creases because the dynamic linker must make a large number of code patches when binding symbols."
Yes you do. Anything that will be loaded with dlopen must be compiled using -fpic (or -fPIC).
This is not about sharing the text segment, but about the different rules for accessing global data (including things that you might not realize are global data, such as the "procedure linkage table" trampolines used to call between global functions) in the main executable versus in shared libraries.
i am a little confused about how shared library and the OS works.
1st question : how the OS manages shared libraries?, how they are specified uniquely? by file name or some other(say an ID) things? or by full path?!
2nd question : i know first when we compile and link codes, the linker need to access the shared library(.so) to perform linking, then after this stage when we execute the compiled program the OS loads the shared library and this libraries may be in different locations(am I wrong?) BUT i do not understand how the OS knows where to look for shared library, is library information (name? path? or what?!) coded in the executable ?
When compiling a program, libraries (other than the language runtime) must be explicitly specified in the build, otherwise they will not be included. There are some standard library directories, so for example you can specify -lfoo, and it will automatically look for libfoo.a or libfoo.so in the various usual directories like /usr/lib, /usr/local/lib etc.
Note, however, that a name like libfoo.so is usually a symlink to the actual library file name, which might be something like libfoo.so.1. This way, if there needs to be a backward-incompatible change to the ABI (the layout of some structure might change, say), then the new version of the library becomes libfoo.so.2, and binaries linked against the old version remain unaffected.
So the linker follows the symlink, and inserts a reference to the versioned name libfoo.so.1 into the executable, instead of the unversioned name libfoo.so. It can also insert the full path, but this is usually not done. Instead, when the executable is run, there is a system search path, as configured in your systemwide /etc/ld.so.conf, that is used to find the library.
(Actually, ld.so.conf is just the human-readable source for your library search paths; this is compiled into binary form in /etc/ld.so.cache for speed, using the ldconfig command. This is why you need to run ldconfig every time you make changes to the shareable libraries on your system.)
That’s a very simplified explanation of what is going on. There is a whole lot more that is not covered here. Here and here are some reference docs that might be useful on the build process. And here is a description of the system executable loader.
I recently came across a code which loads a .so file with dl_open() and works with dlsym() etc. I understand that dl_open() this would load the dynamic library file. What is the difference between dynamically loading a library file and specifying .so path in Makefile?
Another question is that if I want to dynamically load a library file, do I need to compile it with -rdynamic option?
Aren't both these compiled with -fPIC flag?
Dynamically loading a library file is frequently used in implementing software plugins.
unlike specifying .so path in Makefile or static linking, dynamic linking will allow a computer program to startup in the absence of these libraries, to discover available libraries, and to potentially gain additional functionality.
link
If you statically link an .so file in your Makefile then you won't be able to build the application unless it is present. It has the advantage of no nasty surprises at run time.
When creating a shared object, assuming you are using gcc, then -fpic only means the code can be relocated at run-time, you need -shared as well. I don't know the -rdynamic option, but compilers differ.
Loading the module at run-time allows the module load to be optional. For example, say you have a huge application with 300 modules, each representing different functionality. Does it make sense to map all 300 when a user might only use 10% of them? (code is loaded on demand anyway) It can also be used to load versions from different libraries at runtime, giving flexibility. Downside is that you can end-up loading incompatible versions.
fakeroot seems to build some libfakeroot.a (but inside a .lib directory).
But I am not sure, that the static linker can indeed replace/rename symbols as the dynamic linker can.
Fakeroot uses the dynamic linker in order to do its magic (specifically, LD_PRELOAD). Unfortunately, the dynamic linker is not involved in loading statically linked binaries (which is how the dynamic linker itself is invoked: /lib/ld-linux.so.2 is statically compiled).
As answered above, your only option, as far as I'm aware, is to use fakeroot-ng, which uses a completely different mechanism to inject into the process, and is, thus, able to work on statically linked libraries without a problem.
In fact, statically linked libraries was part of the reason I set out to write fakeroot-ng in the first place. At the time, there was no way to tell ldconfig to run on a subtree, and ldconfig is statically linked.
Shachar
seems the solution is to use fakeroot-ng, which works for statically linked binaries.
I've encountered a few cases building projects which use shared libraries or dynamic-loaded modules where the module/library depends on another library, but doesn't check that a shared copy is available before trying to link. This causes object files from a static archive (.a file) to get pulled into the resulting .so, and since these object files are non-PIC, the resulting .so file either has TEXTRELs (very bad load performance and memory usage) or fails altogether (on archs like x86_64 that don't support non-PIC shared libraries).
Is there any way I can make the gcc compiler driver refuse to link static library code into shared library output? It seems difficult and complicated by the possible need to link minimal amounts from libgcc.a and the like...
As you know, you can use -static to only link against static libraries, but there doesn't appear to be a good equivalent to only linking against dynamic libraries.
The following answer may be useful...
How to link using GCC without -l nor hardcoding path for a library that does not follow the libNAME.so naming convention?
You can use -l:[libraryname].so to list the dynamic libraries you want to link against in your library search path. Specifying the .so ending will probably help with your dynamic library only case. You will probably have to specify the whole name with the 'lib' prefix instead of just the shortened version.