I wrote a program, that uses a shared library installed on my system. This library is seldom installed on other systems. How do I compile my program so that the library doesn't need to be installed on other systems? I have the source code for the library available. What's the best way?
The other systems of course have the same architecture and OS.
Compile it as a static library and link that into the executable.
Though the OP had solved his problem by answering a different question, there are (at least) two ways to wedge a shared library into your binary in case
there is no source code available
there is no compiler (or build-chain) available
static link does not work or it's not obvious how do it
to preserve memory layout - static link will change it and may "wake-up" hidden bugs
for "permanent link" LD_PRELOAD library into executable
The first is statifier (open source but limited to x86 and x86_64 and only object code)
The second that I know of is magic ermine (by the same developer). It is closed source, but the developer is friendly to opensource projects and ermine has the advantage of supporting more platforms as well as the ability to include all necessary data files within its virtual file system.
http://statifier.sourceforge.net/ and http://www.magicermine.com/
Related
I know similar questions have been asked, but it's still unclear to me.
I have written a library with multiple drivers and modules for Zephyr RTOS. Now I would like to share part of that library with a company, but not as source code. The idea is to compile the relevant source code for the specific hardware they have, and then share it. This way I can control for which products it's used, and of course I don't want to share my source code with them.
At first I have tried just sharing a static library, but that didn't compile for them. Shared libraries are not yet supported by Zephyr's CMake extensions, hence I haven't tried that yet. If it's the way to go I will dive into it.
What are my options? Shared library vs. static library (+ object files?)? What is recommended?
More info
Zephyr uses Device Trees. Hence, the drivers / modules I provide are compiled for a specific hardware target. I would like the company to provide me with the relevant hardware definitions so that I can provide them a pre-compiled library of my drivers/modules that works for their specific target. This library might have to be updated sometimes to include bugfixes / new functionality.
As the binaries are compiled with application + library, what would the trade-off be for Static vs. Shared library?
I think shared libraries and header files is the way to go . As they offer advantages. Like smaller binary size and flexibility to update. You can find a nice description here.
https://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html
C++ recommends using shared libraries because it provides flexibility on Linux or Linux like systems
https://www.bogotobogo.com/cplusplus/libraries.php
My host application took over the ownership of e.g. a FILE object which came from a dynamic library. Can I call fclose() on this object safely even though my host application and the dynamic library are compiled with different versions of clang / gcc?
Background
On Windows (with different VS runtimes) it would be illegal and I have to first extract the fclose() function from the runtime library which is used by the dynamic library since all runtimes have their own pools and internal structures for file or memory objects.
An illustration for the situation in Windows would look like this:
Does this restriction apply for Linux and macOS as well?
The issue is not whether your application and the dynamic libraries were compiled with different versions of clang and/or gcc. The issue is whether, ultimately, there's one underlying C library that manipulates one kind of FILE * object and has one, compatible implementation of fclose().
Under MacOS and Linux, at least, the answer to all these questions is likely to be "yes". In my experience it's hard to get two different, incompatible C libraries into the mix; you'd have to really work at it.
Addendum: I suppose I should admit, however, that my experience may be getting dated. In my experience, on any Unix-like system, there's exactly one C library, generally /lib/libc.{a,so}. But I gather that "modern" compilers are tending to access their own compiler- and version-specific libraries off in special places, meaning that the scenario you're worried about could be a problem. To me, it seems, this way lies madness, but then again, it seems that more and more of the world seems to be embracing dependency hell, rather than trying to eliminate it.
It is not generally safe to use a library designed for one compiler with code compiled by a different compiler. A compiler may generate code that implements the nominal functions in the standard library using internal routines or interfaces, and those routines or interfaces may be different or missing in the library designed for another compiler.
Nor is it safe to take any pointer to some internal data structure from one library and use it with another library.
If the sources are just compiled with different versions of one compiler (e.g., clang 73 and clang 89), not different compilers (e.g., Apple clang versus GCC), the compiler might offer some guarantee about library compatibility. You would have to check its documentation. Or, if the compiler is intended to use the library provided with the operating system, that could work. Again, you would have to check its documentation.
On Linux, if both your code and the other library dynamically link to the same library (such as libc.so.6), both will get the same version and implementation of that library at runtime. You can check which libraries a given dynamic library links to with ldd.
If you were linking to a library that statically linked in a supporting library, you would need to be careful to pass any structures to or from it against the same version of the library. But this is more likely to come up in practice with libc++ and libstdc++ than with libc.
So, don't statically link your library to another and then pass a data structure that requires client code to separately link to the same library.
The background is following: there is 3'rd party provider that provides us with a libveryfancylib.so, in 32b. Softaware that uses the library has quite a load of other linux library dependencies (like QT) also, but they are open source, so no problem for statical linking. The target platform is 64b and running Debian 7.
We can ship the program with binary + dynamical libraries, no problem, but i would rather see single static binary with no dependencies.
So my question is: why i cannot link the dynamical library into static binary? I mean what bit of information is there missing, or is it just feature that is rarely needed -> not implemented.
We can ship the program with binary + dynamical libraries, no problem, but i would rather see single static binary with no dependencies.
What is the problem you are trying to solve?
You can follow the model most commercial applications on Linux do: put your executable, shared libraries and other resources in one directory (possibly with subdirectories). When linking your executable against those shared libraries pass -Wl,-rpath,'$ORIGIN' (in make use -Wl,-rpath,'$$ORIGIN') to the linker, so that when starting your application the runtime linker looks for required shared libraries in the same directory where executable is.
Then archive that directory and give it to your users.
There are programs for MS Windows that can do so, eg DLL to Lib and DLL to Static Lib.
In the open source world, there isn't really much of an incentive to develop such a tool as you can always recompile from source (but of course it's possible that someone somewhere did it anyway).
It's because dynamic libraries and static libraries are two different things. A static library is just an archive of object files (much like a zip archive). A dynamic library is more like an executable program.
So you can't really link anything into a static library, you can only add more object files.
I'm reading a book about gcc and the following paragraph puzzles me now:
Furthermore, shared libraries make it possible to update a library with-
out recompiling the programs which use it (provided the interface to the
library does not change).
This only refers to the programs which are not yet linked, right?
I mean, in C isn't executable code completely independent from the compiler? In which case any alteration to the library, whether its interface or implementation is irrelevant to the executable code?
A shared library is not linked until the program is executed, so the library can be upgraded/changed without recompiling (nor relinking).
EG, on Linux, one might have
/bin/myprogram
depending upon
/usr/lib64/mylibrary.so
Replacing mylibrary.so with a different version (as long as the functions/symbols that it exports are the same/compatible) will affect myprogram the next time that myprogram is started. On Linux, this is handled by the system program /lib64/ld-linux-x864-64.so.2 or similar, which the system runs automatically when the program is started.
Contrast with a static library, which is linked at compile-time. Changes to static libraries require the application to be re-linked.
As an added benefit, if two programs share the same shared library, the memory footprint can be smaller, as the kernel can “tell” that it's the same code, and not copy it into RAM twice. With static libraries, this is not the case.
No, this is talking about code that is linked. If you link to a static libary, and change the library, the executable will not pick up the changes because it contains its own copy of the original version of the library.
If you link to a shared library, also known as dynamic linking, the executable does not contain a copy of the library. When the program is run, it loads the current version of the library into memory. This allows you to fix the library, and the fixes will be picked up by all users of the library without needing to be relinked.
Libraries provide interfaces (API) to the outside world. Applications using the libraries (a .DLL for example) bind to the interface (meaning they call functions from the API). The library author is free to modify the library and redistribute a newer version as long as they don't modify the interface.
If the library authors were to modify the interface, they could potentially break all of the applications that depend on that function!
When I run a program (in linux) does it all get loaded into the physical memory? If so, is using shared libraries, instead of static libraries, help in terms of caching? In general, when should I use shared libraries and when should I use static libraries? My codes are either written in C or in C++ if that matters.
This article hits covers some decent ground on what you want. This article goes much deeper about the advantages of shared libraries
SO also has covered this topic in depth
Difference between static and shared libraries?
When to use dynamic vs. static libraries
Almost all the above mentioned articles are shared library biased. Wikipedia tries to rescue static libraries :)
From wiki,
There are several advantages to statically linking libraries with an
executable instead of dynamically linking them. The most significant
is that the application can be certain that all its libraries are
present and that they are the correct version. This avoids dependency
problems. Usually, static linking will result in a significant
performance improvement.
Static linking can also allow the application
to be contained in a single executable file, simplifying distribution
and installation.
With static linking, it is enough to include those
parts of the library that are directly and indirectly referenced by
the target executable (or target library).
With dynamic libraries, the
entire library is loaded, as it is not known in advance which
functions will be invoked by applications. Whether this advantage is
significant in practice depends on the structure of the library.
Shared libraries are used mostly when you have functionality that could be used and "shared" across different programs. In that case, you will have a single point where all the programs will get their methods. However, this creates a dependency problem since now your compiled programs are dependent on that specific version of the library.
Static libraries are used mostly when you don't want to have dependency issues and don't want your program to care which X or Y libraries are installed on your target system.
So, which one to use?. for that you should answer the following questions:
Will your program be used on different platforms or Linux distributions? (e.g. Red Hat, Debian, SLES11-SP1)
Do you have replicated code that is being used by different binaries?
Do you envision that in the future other programs could benefit from your work?
I think this is a case by case decision, and it is not a one size fits all kind of answer.