I have created a communication library which is linked statically into several different applications. The library provides support for communication via different kinds of hardware. Some of the hardware is supported from the vendor via a shared library. On systems without those kinds of hardware, the shared library is not available.
Previously we handled this by compiling dual versions of the communication library and the applications. This is however not very practical, so I thought about using a more dynamic communication library which tries to load the vendor library with dlopen()/dlsym() if it is available. This seems to work well. But a problem is that everyone who use my library will need to pass the -ldl option when linking their application with my library. Even if this is a minor nuisance I wonder on how this is normally solved.
Is it somehow possible to create a static library that will automatically (at compile-time or run-time) bring in the needed shared library?
Is it considered good practice to let static libraries have dependencies on shared libraries?
EDIT: I know that libtool could probably solve this, but that would still change the build process for all applications even more, which I would prefer to avoid.
EDIT 2: Target platforms are primarily Linux and Solaris. Gcc as compiler.
I don't know about Solaris, so assume everything in my answer here applies to Linux only (although pkg-config should be usable on Solaris too).
First off, there's no supported way for a static library to pull in a link-time dependency. Sorry. Most libraries use something like pkg-config for this - that is, when you build, you add to the compiler command line:
gcc `pkg-config --cflags your-library` [...]
and when you link:
gcc `pkg-config --libs your-library` [...]
Note that the --libs parameter, in this case, would produce something like -ldl -lyourlib as output. The --cflags parameter may produce no output, or it may add some include paths.
If you absolutely need it to work with just a -lyourlib, though, and you don't mind being tied to a unsupported and unstable interface in glibc... Well, libdl is just a thin wrapper over some routines in the dynamic linker, via an undocumented vtable. If you look at the source under the dlfcn/ directory of the version of glibc in use, you should be able to replicate what it does.
HOWEVER, the interface between libdl and ld-linux is PRIVATE and UNSTABLE - it may change at any glibc release, including minor and bugfix releases. ONLY do this if you control the version of glibc deployed, and are prepared to rebuild your application when necessary. Also note that if your library is not LGPL itself, then using a private API like this may have licensing issues; not sure how things stand with this for the LGPL.
Related
This may seem a little stupid:) But it's been bothering a while. When I include some header files which are written by others in my C++/C program, how does the compiler know where is the implementation of the class member function declared in the header files?
Say I want to write some program which takes advantage of the OpenCV library. Normally I would want to use:
#include <opencv2/core/core.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/highgui/highgui.hpp>
However, these are just header files which, as far as I can tell, only declares functions but without implementation. Then how does the compiler know where to find the implementation? Especially when I want to build a .so file.
There is a similar post. Basically it said thrid-party library, esp. commercial product don't release source code, so they ship the lib file with the header. However, it didn't make clear how does the compiler know where to find the lib file. In addition, The answer in that post mentioned if I want to compile the code of my own, I would need the source code of the implementation of those header files. Does that mean I cannot build a .so file without the source of the implementation?
In general, the implementation is distributed as form of pre-compiled libraries. You need to tell the compiler where they are located.
For example, for gcc, quoting the online manual
-llibrary
-l library
Search the library named library when linking. [...]
and,
-Ldir
Add directory dir to the list of directories to be searched for -l.
Note: you don't need to explicitly specify the standard libraries, they are automatically linked. Rather, if you don't want them to be linked with you binary, you need to inform the compiler by passing the -nostdlib option.
The exact answer is platform specific, but in general I'd say that some libraries are in fact header-only, and others include the implementation of the library's methods in binary object files. I believe OpenCV belongs to the second kind, i.e. provides the implementation in object files, for either dynamic or static linking, and your program links against them. If your build works, then it is already configured to link against those libraries. At this point the details become very much platform and build-system specific.
Note that for common platforms like Windows, Mac and Linux you seldom need to build popular libraries like OpenCV yourself. You mentioned .so files, which implies dynamic linking on Linux. This library is open-source so in theory you could build it yourself, but in practice I'd much rather use my distribution's package installation tool (e.g. apt-get or yum) to install opencv-dev (or something similar) from my distribution's repository.
As the others already explained, you need to tell your compiler where to look for the files.
This implies that you should know which path to specify for your compiler.
Some components provide a mechanism where you don't need to know the exact path but can automatically retrieve it from your system.
For example if you want to compile using GTK+3 you need to specify these flags for your compiler:
CFLAGS:= -I./ `pkg-config --cflags gtk+-3.0`
LIBS:= -lm `pkg-config --libs gtk+-3.0`
This will automatically result in the required include and library path flags for GCC.
The compiler toolchain contains at least two major tools: the compiler and the link editor (it is very common to name compiler the whole chain, but strictly speaking it is wrong).
The compiler is in charge of producing object code from the available source code. In that phase the compiler knows where to locate standard headers, and can be told to use non-standard dirs to locate headers. For example, gcc uses -I to let you specify some more alternate dirs that may contains headers.
The link editor is in charge of producing executable files (its basic common usage) from object codes. To produce an executable it needs to find every implementation of declared things at compile-time for which you didn't provide source code. These can be other object codes, object codes in libraries, etc. The link editor knows where are located standard libraries and can be told to let you specify non-standard dirs. For example you can tell the gcc toolchain to use alternate dirs with L that may contain libraries. You may be aware that link edition is now usually a two phase process: location-and-name-resolution at link-time and real link-edition at run-time (dynamic libraries are very common).
Basically a library is just a collection of object code. Consult internet to see how you can easily build libraries either from source code of from object code.
When a custom library is used in the code, it requires -l linker parameter to use:
gcc myprogram.c -lmylibrary
Is there a way to convince MinGW linker to check header files and automatically find and link a library in /lib folder? Or is there a reason why it would be a bad idea?
No.
The problem of looking at C source code and figuring out which libraries it uses is very hard. It feels kind of "AI complete" to me, which is why it's typically solved manually by the programmer pointing out the exact right libraries to satisfy the dependencies with.
Imagine for mylibrary, it's easy to imagine a system with both mylibrary 1.x and 2.x versions installed, and some calls are named exactly the same. Now try to imagine a computer program capable of deducing what you meant, which library to link with. It's not possible, since only the programmer knows.
The pkg-config tool helps with the mechanics of what each library requires in order to be used, but it's still up to you to tell it (via the module name argument(s)) which exact libraries to use.
I need to write a xinput driver for a virtual device, e.g. http://cgit.freedesktop.org/~whot/xf86-input-random. The device is connected to a LAN. The client for this device is written in C++. Is it possible to use C++ code in this driver or must the whole project be C only?
An Xorg driver is just an ELF shared object plugin following some documented convention. In principle, how you obtain that .so is your own business (you could in theory write it manually bit by bit if you have centuries of your time to lose).
In principle, you could link the libstdc++.so to your shared object (since one can link shared objects to other ones). I guess that you would compile and link your plugin with g++ and perhaps explicitly need to link with -lstdc++
However, I guess that it might be unsafe. Perhaps C++ ABI requires some specific things to be executed by the crt0.
So you might try, but I won't be surprised if something does not work as you want (e.g. exception handling). It could depend upon the version of the C++ library and the version of the C library and the version of the compiler.... I guess that it might work better with recent g++, recent libc, recent libstdc++ ....
Read Drupper's paper: How to Write Shared Libraries
Make your driver free software, and publish very quickly its source code, so you could get some help from the Xorg community (even when your driver is incomplete). Use probably a recent Xorg....
I have implemented semaphores in Linux last year. But for that I have to use -lpthread.
Now while implementing log10() function in C, I surfed the INTERNET and I saw that I have to use -lm.
I want to know why these kind of command line arguments are necessary in Linux.And Does this rule is compiler oriented?
(In windows Turboc compiler, I never used these kind of arguments.)
You are instructing the compiler to look for certain libraries and use them to try and produce a final object file.
When you were doing your threading code, you used threading primitives. These threading primitives are implemented in a library called pthread, -lpthread tells the linker to use the library pthread, without providing this switch the compiler will not be able to produce a valid object file as it is missing threading code implementation.
On the file system the libraries can be found in /usr/lib and lib (among others) when you look in these directories you will see files start with the lib prefix. for example libpthreadxxxxxx. You will have to do your own research to figure out what the xxxx means.
The development cycle using unix style tools is very granular on the surface, when you use heavyweight IDE's (read: visual studiio for C++), the IDE implicetly links against loads of standard libraries, so often you do not need to supply the name of the libraries you will commonly use. However, when you start doing more advanced programming you will probably have to install and configure your IDE to use external code libraries. If you were to use threading primitives in visual studio, you most likely will not have to provide the compiler with information on where to look for threading primitives, Microsoft considers this a common library and every new project will implicitly link against it.
A little discussion on GCC
GCC is a very diverse compiler producing code for various different usage scenarios. As such they try to be neutral and do not make assumptions. For example pthread is a particular threading primitives implementation. However, even through now on Linux at least it is the defacto standard, it is not the only one. Other Unix implementation have had different implementation. When such choices exist it is not fair for the compiler developers to implicitly link against libraries. They do however implicitly link against standard libraries; for example G++ is just a wrapper command to the internal compiler code, it is a C++ front-end so it implicitly links against an implementation of the C++ standard library. Similarly the C front end links against a the standard C library.
People often do not want to use certain standard library implementation, and instead they might want to use another implementation, in such cases you have to explicetly inform the compiler to use an implementation that you provide. Such use cases are very granular and are surface level issues with G++. In visual studio, you would have to tinker a lot to make such changes generally, since it is not an anticipated use-case anymore.
wikipedia will provide you with more information.
Edit: I'll fix the spelling and Grammatical issues later :D
The option -l indicates to gcc what libraries must be used for linking. -lpthread stands for "use the pthread library", and -lm stands for "use the m library" which is the math library. These commands are relative to gcc, not linux.
Because by default, gcc only links the C library (libc), which contains the well-known functions printf, scanf, and many more.
log10 exists in a different library called libm, and thus you need to explictly tell gcc to link that library, with -lm. The same logic applies for -lpthread.
This is purely a backwards, harmful practice. Separating parts of the standard library into separate .so files does nothing but increase load time and memory usage. Good luck getting anyone to change it though... Just accept that you have to do it (and that POSIX specifically allows, but does not require, that an implementation require -lm for using the math functions and -lpthread for using threads, etc.) and move on to more important things.
Or, go pester Drepper about it on the glibc bug tracker/mailing list. He won't change his mind, but if you enjoy flamewars you can get some kicks...
Is it possible to have glibc and uClibc based applications running side-by-side on one system?
Background: We have binary gcc based cross-compiler configured to link with uClibc. We have cross-compiled glibc with it. Now we want to build some applications so they will link with the glibc rather than uClibc. We don't want to rebuild the compiler.
There's no problem with glibc and uClibc living side-by-side with some programs linking to one and other programs linking to the other. However, there is a problem with additional libraries. Each shared library on your system will be built against either glibc or uClibc (using the corresponding headers, which define distinct ABIs for the standard library functions), so for example if both a glibc program and a uClibc program need ncurses, you'll need to have two versions of ncurses built, and have a way of ensuring that the correct one for the given program gets loaded at runtime. Alternatively, you could choose to only use one set of shared libraries, and use static libraries for programs linked to the other libc, but you'd still need to build your 2 sets of libraries.
Yes, it should be perfectly possible, but you might have to play around with LD_PRELOAD_PATH. If you are linking statically, change to dynamic linking.
It is nearly impossible to mix them in the same FHS, as the ABI and include dir are incompatible. However, you could install either of them in an directory offset, by tweaking dynamic-linker field in ELF and exploiting sysroot feature in gcc/binutils. An on going experiment is in Gentoo community[1], known as Prefix/libc.
http://wiki.gentoo.org/wiki/Prefix/libc