I compile a project and run it on my system, it does perfectly fine.
I uploaded this to git and asked my administrator to run it on server and it says undefined reference to 'clock_gettime'. This error can be resolved by linking a library with -lrt option. Similar problem happened to me earlier but that was related to linking math.h library with the help of -lm option.
Is there a way to find which all libraries to link so that ou do not run into such problems and not feel embarrassed in front of your administrator. How to decide what all libraries to link. Can there be a thought process for this?
Some man pages will mention both the headers and libraries that should be used. (This is not true of all pages unfortunately.)
For example, look at man 3 signal. On my system that page has a LIBRARY section at the top with Standard C Library (libc, -lc), followed by a SYNOPSIS section with #include <signal.h>.
Some other strategies include:
Blunt use of strings piped to egrep in a shell loop, to search for references to names among installed libraries. This can return a lot of false positives for common names, but it works very well for elaborate function names that don't match single English words.
Iterative compilation and linking. If you are trying to resolve a single function, write a tiny program that calls the function and use a shell or makefile loop to test each single library on your system (starting with a small list of most likely candidates from /usr/lib or wherever else). At some point the build will succeed and show you the right library. This method takes a little work to set up, but the infrastructure you create is also very easy to reuse in the future.
Inspect binaries iteratively using your linker's tools (e.g. otool on Mac OS X, or objdump or readelf on Solaris or Linux). For instance, try to disassemble a library's text section starting from a particular name (otool -v -t -p _symbol on Mac OS X).
For *nix based machines, ldd on the executable will print shared library dependencies.
Related
All of my programs tend to be rather rudimentary console applications in C. For example, I may write some code to parse a file header and then print some data from the header to the screen. To do this, I would just use functions/symbols from stdio.h stdlib.h, string.h, stdbool.h such as printf(), fopen(), fread(), etc... I usually get away with writing my code in the main.c file as well as several .h files and .c files to go along with them. When it comes time to compile, I will do something like: gcc main.c file1.c file2.c -g -Wall -o my_program
The program runs fine, and with my colleagues, I simply share the source code, or if they're on the same OS, I share the binary and they can typically just either build the code just as I did and run it, or run the binary directly. If my colleague is on a different OS, he/she will just build the source on that machine OR I will build it for them on a machine I have with that OS.
I've never really had to consider how my dependencies were being linked at all in fact. This is probably because I write mostly internal tools and am not releasing to large audiences. That being said, in which situations would the above method fail to run on a system? Is it possible that somebody who has the same version of gcc installed would not be able to just run my executable or build the code themselves, then run it, when I'm only using std C functionality? In fact, I've taken my very same C code from a linux box, copy/pasted it into Visual Studio and compiled with MSVC, and it still works fine with the standard functions... So even cross-compiler, I've not needed to think about the linking yet.
For Linux/Linux compatibility:
Usually, as long as you move to machine which has the same/better glibc (and other libraries that you use), you will not face problem. The glibc (C standard library/runtime) is very good at being backward compatible. It will usually work across distributions. In most cases, you can take your binary to a machine with a minor lower version of the library, and it will work (minor versions are suppose to have only bug fixes, so unless your code trigger bugs it should work).
For Linux/Windows compatibility: in most cases, you will need to recompile, as the libraries runtimes and executable formats are different. Disclaimer: I'm not an expert on this topic.
i am a little confused about how shared library and the OS works.
1st question : how the OS manages shared libraries?, how they are specified uniquely? by file name or some other(say an ID) things? or by full path?!
2nd question : i know first when we compile and link codes, the linker need to access the shared library(.so) to perform linking, then after this stage when we execute the compiled program the OS loads the shared library and this libraries may be in different locations(am I wrong?) BUT i do not understand how the OS knows where to look for shared library, is library information (name? path? or what?!) coded in the executable ?
When compiling a program, libraries (other than the language runtime) must be explicitly specified in the build, otherwise they will not be included. There are some standard library directories, so for example you can specify -lfoo, and it will automatically look for libfoo.a or libfoo.so in the various usual directories like /usr/lib, /usr/local/lib etc.
Note, however, that a name like libfoo.so is usually a symlink to the actual library file name, which might be something like libfoo.so.1. This way, if there needs to be a backward-incompatible change to the ABI (the layout of some structure might change, say), then the new version of the library becomes libfoo.so.2, and binaries linked against the old version remain unaffected.
So the linker follows the symlink, and inserts a reference to the versioned name libfoo.so.1 into the executable, instead of the unversioned name libfoo.so. It can also insert the full path, but this is usually not done. Instead, when the executable is run, there is a system search path, as configured in your systemwide /etc/ld.so.conf, that is used to find the library.
(Actually, ld.so.conf is just the human-readable source for your library search paths; this is compiled into binary form in /etc/ld.so.cache for speed, using the ldconfig command. This is why you need to run ldconfig every time you make changes to the shareable libraries on your system.)
That’s a very simplified explanation of what is going on. There is a whole lot more that is not covered here. Here and here are some reference docs that might be useful on the build process. And here is a description of the system executable loader.
This may seem a little stupid:) But it's been bothering a while. When I include some header files which are written by others in my C++/C program, how does the compiler know where is the implementation of the class member function declared in the header files?
Say I want to write some program which takes advantage of the OpenCV library. Normally I would want to use:
#include <opencv2/core/core.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/highgui/highgui.hpp>
However, these are just header files which, as far as I can tell, only declares functions but without implementation. Then how does the compiler know where to find the implementation? Especially when I want to build a .so file.
There is a similar post. Basically it said thrid-party library, esp. commercial product don't release source code, so they ship the lib file with the header. However, it didn't make clear how does the compiler know where to find the lib file. In addition, The answer in that post mentioned if I want to compile the code of my own, I would need the source code of the implementation of those header files. Does that mean I cannot build a .so file without the source of the implementation?
In general, the implementation is distributed as form of pre-compiled libraries. You need to tell the compiler where they are located.
For example, for gcc, quoting the online manual
-llibrary
-l library
Search the library named library when linking. [...]
and,
-Ldir
Add directory dir to the list of directories to be searched for -l.
Note: you don't need to explicitly specify the standard libraries, they are automatically linked. Rather, if you don't want them to be linked with you binary, you need to inform the compiler by passing the -nostdlib option.
The exact answer is platform specific, but in general I'd say that some libraries are in fact header-only, and others include the implementation of the library's methods in binary object files. I believe OpenCV belongs to the second kind, i.e. provides the implementation in object files, for either dynamic or static linking, and your program links against them. If your build works, then it is already configured to link against those libraries. At this point the details become very much platform and build-system specific.
Note that for common platforms like Windows, Mac and Linux you seldom need to build popular libraries like OpenCV yourself. You mentioned .so files, which implies dynamic linking on Linux. This library is open-source so in theory you could build it yourself, but in practice I'd much rather use my distribution's package installation tool (e.g. apt-get or yum) to install opencv-dev (or something similar) from my distribution's repository.
As the others already explained, you need to tell your compiler where to look for the files.
This implies that you should know which path to specify for your compiler.
Some components provide a mechanism where you don't need to know the exact path but can automatically retrieve it from your system.
For example if you want to compile using GTK+3 you need to specify these flags for your compiler:
CFLAGS:= -I./ `pkg-config --cflags gtk+-3.0`
LIBS:= -lm `pkg-config --libs gtk+-3.0`
This will automatically result in the required include and library path flags for GCC.
The compiler toolchain contains at least two major tools: the compiler and the link editor (it is very common to name compiler the whole chain, but strictly speaking it is wrong).
The compiler is in charge of producing object code from the available source code. In that phase the compiler knows where to locate standard headers, and can be told to use non-standard dirs to locate headers. For example, gcc uses -I to let you specify some more alternate dirs that may contains headers.
The link editor is in charge of producing executable files (its basic common usage) from object codes. To produce an executable it needs to find every implementation of declared things at compile-time for which you didn't provide source code. These can be other object codes, object codes in libraries, etc. The link editor knows where are located standard libraries and can be told to let you specify non-standard dirs. For example you can tell the gcc toolchain to use alternate dirs with L that may contain libraries. You may be aware that link edition is now usually a two phase process: location-and-name-resolution at link-time and real link-edition at run-time (dynamic libraries are very common).
Basically a library is just a collection of object code. Consult internet to see how you can easily build libraries either from source code of from object code.
What is the difference between LD_PRELOAD_PATH and LD_LIBRARY_PATH?
I understand what they do, but not the particulars of how they differ.
From
http://en.wikipedia.org/wiki/Dynamic_linker
The dynamic linker can be influenced into modifying its behavior
during either the program's execution or the program's linking.
Examples of this can be seen in the run-time linker manual pages for
various Unix-like systems. A typical modification of
this behavior is the use of the LD_LIBRARY_PATH and LD_PRELOAD
environment variables. These variables adjust the runtime linking
process by searching for shared libraries at alternate locations and
by forcibly loading and linking libraries that would otherwise not be,
respectively.
In particular, I am interested in the differences in Linux which has both LD_PRELOAD_PATH and LD_LIBRARY_PATH:
https://linuxgazette.net/issue48/tag/48.html
Update: The author of this 1999 Linux Gazette article notes in his 2013 comment below the accepted answer that LD_PRELOAD_PATH does not in fact exist.
LD_PRELOAD (not LD_PRELOAD_PATH) is a list of specific libraries (files) to be loaded before any other libraries, whether the program wants it or not. LD_LIBRARY_PATH is a list of directories to search when loading libraries that would have been loaded anyway. On linux you can read man ld.so for more information about these and other environment variables that affect the dynamic linker.
LD_PRELOAD is more powerful
The most important difference is that LD_PRELOAD can replace functions statically linked into a binary.
Example
If ever you find that LD_PRELOAD works for you, but LD_LIBRARY_PATH mysteriously does not, this, almost certainly, is the reason why.
For example, I was debugging GNU Readline in bash and was confused at first why my modified libreadline.so was loading with LD_PRELOAD but not LD_LIBRARY_PATH.
$ LD_PRELOAD=shlib/libreadline.so bash -
(worked)
$ LD_LIBRARY_PATH=shlib/ bash -
(failed)
Taking a look with the ldd command, which lists dependencies on dynamically linked libraries, gives the answer:
$ ldd /bin/bash
linux-vdso.so.1
libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
/lib64/ld-linux-x86-64.so.2
Sure enough, libreadline is not listed as one of the shared libraries. Therefore, /bin/bash on my machine must have been statically linked with its own version of libreadline at compile time. The reason LD_LIBRARY_PATH doesn't work is because the binary never asks the dynamic linker (ld.so) to load libreadline.
LD_PRELOAD, on the other hand, loads the library no matter what, allowing a library to override even statically linked functions.
After reading this question, my first reaction was that the user is not seeing the error because he specifies the location of the library with -L.
However, apparently, the -L option only influences where the linker looks, and has no influence over where the loader looks when you try to run the compiled application.
My question then is what's the point of -L? Since you won't be able to run your binary unless you have the proper directories in LD_LIBRARY_PATH anyway, why not just put them there in the first place, and drop the -L, since the linker looks in LD_LIBRARY_PATH automatically?
It might be the case that you are cross-compiling and the linker is targeting a system other than your own. For instance, MinGW can be used to compile Windows binaries on Linux. Here -L will point to the DLLs needed for linking and LD_LIBRARY_PATH will point to any libraries needed by linker to run. This allows compiling and linking of different architectures, OS ABIs, or processor types.
It's also helpful when trying to build special targets. I might be case that one links a static version of program against a different static library. This is the first step in Linux From Scratch, where one creates a separate mini-environment on the main system to become a chroot jail.
Setting LD_LIBRARY_PATH will affect all the commands you run to build your code (including the compiler itself).
That's not desirable in general (e.g. you might not want your compiler to run debug/instrumented libraries while it compiles - it might even go as far as breaking your compiles).
Use -L to tell the compiler where to look, LD_LIBRARY_PATH to influence runtime linking.
Building the binary and running the binary are two completely independent and unrelated processes. You seem to suggest that the running environment should affect the building environment, i.e. you seem to be making an assumption that the code build in some setup (account, machine) will be later run in the same setup. I find this assumption rather strange. I'd even say that in most cases the building and the running are done in different environments. I would actually prefer my compilers not to derive any assumptions about future running environment from the environment these compilers are invoked in. Looking onto the LD_LIBRARY_PATH of the building environment would be a major no-no.
The other answers are all good, but one nobody has mentioned yet is static libraries. Most of the time when you use -L it's with a static library built locally in your build tree that you don't intent to install, and it has nothing to do with LD_LIBRARY_PATH.
Compilers on Solaris support the -R /runtime/path/to/some/libs that adds to the path where libraries are to be searched by the run-time linker. On Linux the same could be achieved with -Wl,-rpath,/runtime/path/to/some/libs. It passes the -rpath /runtime/path/to/some/libs option to ld. GNU ld also supports the -R /path/to/libs for compatibility with other ELF linkers but this should be avoided as -R is normally used to specify symbol files to GNU ld.