I have a program written in C that uses dlopen for loading plug-in modules. When the library is dynamically loaded, it runs constructor code which register pointer to structure with function implementations with the main application by use of exported function. I want to use absolute path for specifying the file to dlopen.
Then I have other part of the program with takes file, determine if it is ELF, then looks into the ELF header for specific ELF section, read this section and extract from it pertinent information. This way it filters only shared libraries which I have previously tagged as a plug-in module.
However, I am solving a problem how to discover them on the fly (in portable Linux way, i.e. it will run on Debian and on Fedora too and so on) from the main program. I have been thinking about using ldconfig for this. (As the modules will be installed by way of distro packaging system, APT for example.) Is there any way how to programmatically get the string list of known libraries from C program other than directly reading the /etc/ld.co.cache file? I was thinking that maybe there is some header library which will give char** when I ask.
Or, maybe is there any better solution to my problem?
(I am proponent of using standard system components that programming one-off solutions which will need support in the future.)
Related
I have a library stack that is not going to change, and an firmware that is going to use only this stack. Firmware will change alot along the way. I don't want to every time release the whole image(including library stack) because of limited memory and resources issue(This is an embedded application not a desktop or server).
I just want to release the application image and that automatically be able to use the library image. I am not sure how to do it. I know in Windows for example this is handled by dll's. But this is an embedded application and has no OS. Binary images loads to memory and processor is going to execute it.
Any experience/suggestions?
Toolchain: IAR 8051
This depends quite a bit on your tool-chain. Here's a possible high-view approach.
Compile your library into an executable image, setting your linker to use a particular portion of your flash memory space. You'll probably need a fake/stub entry function for the linker to be happy.
Once that is done, find all of the addresses of the symbols used by the library and instruct your linker as to those symbol locations when building your normal program, and do not instruct the link process to use the intermediary library objects when linking. Also instruct the linker to place the code into the section of flash that is update-able.
What you will then have is an image for the library, and the ability to build new versions of the main program image using at library.
This could probably be scripted if your linker output format is an unstripped elf (prior to converting to a binary for burning on the flash), and if your linker can accept a plain text file for instructions (both are true if you are using the gnu toolchains). I'd recommend scripting it for your sanity unless the library has very few externally visible functions and variables in it.
I do have to agree with some of the commentors; unless transferring the library is very hard, you should just build a single simple image that includes the library and push the whole thing. You might say the library will never change now, but inevitably something will come up that requires a change to the library code, and if you change the library and cannot keep the symbols in exactly the same spot, all of your application images will not be able to work with the new library. This is a recipe for a nightmare when dealing with compatible software (firmware) updates.
The background is following: there is 3'rd party provider that provides us with a libveryfancylib.so, in 32b. Softaware that uses the library has quite a load of other linux library dependencies (like QT) also, but they are open source, so no problem for statical linking. The target platform is 64b and running Debian 7.
We can ship the program with binary + dynamical libraries, no problem, but i would rather see single static binary with no dependencies.
So my question is: why i cannot link the dynamical library into static binary? I mean what bit of information is there missing, or is it just feature that is rarely needed -> not implemented.
We can ship the program with binary + dynamical libraries, no problem, but i would rather see single static binary with no dependencies.
What is the problem you are trying to solve?
You can follow the model most commercial applications on Linux do: put your executable, shared libraries and other resources in one directory (possibly with subdirectories). When linking your executable against those shared libraries pass -Wl,-rpath,'$ORIGIN' (in make use -Wl,-rpath,'$$ORIGIN') to the linker, so that when starting your application the runtime linker looks for required shared libraries in the same directory where executable is.
Then archive that directory and give it to your users.
There are programs for MS Windows that can do so, eg DLL to Lib and DLL to Static Lib.
In the open source world, there isn't really much of an incentive to develop such a tool as you can always recompile from source (but of course it's possible that someone somewhere did it anyway).
It's because dynamic libraries and static libraries are two different things. A static library is just an archive of object files (much like a zip archive). A dynamic library is more like an executable program.
So you can't really link anything into a static library, you can only add more object files.
Recently I tried to write a simple compiler on the linux platform by myself.
When it comes to the backend of the compiler, I decided to generate ELF-formatted binaries without using a third-party library, such as libelf.
Instead I want to try to write machine code directly into the file coresponding to the ELF ABI just by using the write() function and controlling all details of the ELF file.
The advantage of this approach is that I can control everything for my compiler.
But I am hesitating. Is that way feasible, considering how detailed the ELF ABI is?
I hope for any suggestions and pointers to good available resources available.
How easy/feasible this is depends on what features you want to support. If you want to use dynamic linking, you have to deal with the symbol table, relocations, etc. And of course if you want to be able to link with existing libraries, even static ones, you'll have to support whatever they need. But if your goal is just to make standalone static ELF binaries, it's really very easy. All you need is a main ELF header (100% boilerplate) and 2 PT_LOAD program headers: one to load your program's code segment, the other to load its data segment. In theory they could be combined, but security-hardened kernels do not allow a given page to be both writable and executable, so it would be smart to separate them.
Some suggested reading:
http://www.linuxjournal.com/article/1059
On Windows, it's more or less common to create "proxy DLLs" which take place of the original DLL and forward calls to it (after any additional actions as needed). You can read about it here and here for example.
However, shlib munging culture under Linux is quite different. It starts with the fact that LD_PRELOAD is the builtin feature with ld.so under Linux, which simply injects separate shlib into process and uses any symbols it defines as override. And that "injection" technique seems to define whole direction of thought - here's a typical ELF hacking tool or this question, where gentleman seems to have the same usecase as me, but starts with asking how he can patch existing binaries.
No, thanks. I don't want to inject into or modify something which is nor mine. All I want to do is to make a standalone proxy shlib which will call out to the original. Ideally, there would be a tool which can be fed with the original .so and create a C source code which would just redirect to original's functions, while letting me easily override anything I want. So, where's such tool? ;-) Thanks.
Using LD_PRELOAD doesn't really involve modifying something which isn't yours, and the injection isn't all that different from normal dynamic library loading. The “typical ELF hacking tool” from the ERESI project is unrelated to LD_PRELOAD. You should not be afraid of it. A good introduction to writing LD_PRELOAD-able “proxies” is here.
That being said, if you want to create a system-wide proxy for some library, you might argue that globally setting LD_PRELOAD (and thus loading your proxy into every binary that ever runs on your system) is undesirable. It is commonly used to override functions from glibc by tools such as libeatmydata or socksify, but if you're overriding a function in a library that is bigger and/or less widespread than glibc, it makes sense to try to find another approach, to really create a proxy for just that one library.
One such approach is to use patchelf --replace-needed or --add-needed to hardcode the full pathname of the original library and then make sure the proxy library is found first by setting LD_LIBRARY_PATH¹. So, the complete procedure is:
create an LD_PRELOAD-able library that overrides some functions of the original one (test that it works using only LD_PRELOAD before proceeding further!)
compile and link this library with the original library so that ldd libwrapper-foo.so includes something like:
libfoo.so.0 => /usr/lib/x86_64-linux-gnu/libfoo.so.0 (0x0000deadbeef0000)
hardcode the full path using patchelf:
patchelf --replace-needed libfoo.so.0 /usr/lib/x86_64-linux-gnu/libfoo.so.0 libwrapper-foo.so
symlink libwrapper-foo.so to libfoo.so.0
now LD_LIBRARY_PATH=. ldd $(which program-that-uses-libfoo) should include these lines:
libfoo.so.0 => ./libfoo.so.0 (0x0000dead56780000)
/usr/lib/x86_64-linux-gnu/libfoo.so.0 (0x0000dead1234000000)
set LD_LIBRARY_PATH to full path to the wrapper library in your .bashrc or somewhere
A real-life example of such proxy libary is my wrapper for libpango that enables subpixel positioning for all applications.
¹) It might also be possible to put this proxy library into /usr/local/lib, but ldconfig (the tool that updates shared libraries cache) refuses to use libraries with hardcoded absolute paths.
apitrace is a tool which covers detailed tracing of graphic libs (OpenGL, DirectX) calls for a number of platform. It's probably too detailed and complex for generic solution, but at least provides some reference and affinity.
If I just want to use the gsl_histogram.h library from Gnu Scientific Library (GSL), can I copy it from an existing machine (Mac OS Snow Leopard) that has GSL installed to a different machine (Linux CentOS 5.7) that doesn't have GSL installed, and just use an #include <gls_histogram.h> statement in my c program? Would this work?
Or, do I have to go through the full install of GSL on the Linux box, even though I only need this one library?
Just copying a header gsl_histogram.h is not enough. Header states merely the interface that is exposed by this library. You would need to copy also binaries like *.so and *.a files, but it's hard to tell which ones to copy. So I think the you'd better just install it on your machine. It's pretty easy, just use this tutorial to find and install GSL package.
So there are surely a lot of libraries out there. However the particular one is Gnuplot. Using it you even do not need to compile the code, however you do need to read a bit of documentation. But luckily there is already a question about how to draw a histogram with Gnuplot on Stackoverflow: Histogram using gnuplot? It worth noting that Gnuplot is actually very powerful tool, so invested time into reading its documentation will certainly pay off.
You cannot copy libraries from OS and expect them to work unchanged.
OS X uses the Mach-O object file format while modern Linux systems use the ELF object file format. The usual ld.so(8) linker/loader will not know how to load the Mach-O format object files for your executable to execute. So you would need the Apple-provided ld.so(8) -- or whatever they call their loader. (It's been a while.)
Furthermore, the object files from OS X will be linked against the Apple-supplied libc, and require the corresponding symbols from the Apple-supplied library. You would also need to provide the Apple-provided libc on the Linux system. This C library would try to make system calls using the OS X system call numbers and calling conventions. I guarantee the system call numbers have changed and almost certainly calling conventions are different.
While the Linux kernel's binfmt_misc generic object loader can be used to teach the kernel how to load different object file formats, and the kernel's personality(2) system call can be used to select between different calling conventions, system call numbers, and so on, the amount of work required to make this work is nothing short of immense: the WINE Project has been working on exactly this issue (but with the Windows format COFF and supporting libraries) since 1993.
It would be easier to run:
apt-get install libgs0-dev
or whatever the equivalent is on your distribution of choice. If your distribution does not make it easily available, it would still be easier to compile and install the library by hand rather than try to make the OS X version work.