I am using a program that has some shared library that are usually installed to /usr/lib.
However for some reason i have to have these libs locally. So to make my program run (which depends on the former) I need to export LD_LIBRARY_PATH= ... or add my local path permanently. This is ok for me but users of my software don't know this and for them this is too complicated. So my question: is there a way to automatically set the local path to my shared libs which are called by my program at runtime.
is there a way to automatically set the local path to my shared libs
Maybe.
If you are on Linux, and your application is installed together with shared library into say /some/prefix/bin/app and /some/prefix/lib/libsharedlib.so, then linking your application with:
gcc -o app -Wl,--rpath='$ORIGIN/../lib' main.o ... -lsharedlib
will achieve exactly the result you want (note: you can move both app and library into /another/dir, and it will still work so long as both the lib and bin directories are moved together).
Note: single quotes around $ORIGIN are required.
If you are on a platform that doesn't support $ORIGIN, the other common technique is to wrap the application in a shell script, which looks at $0, sets LD_LIBRARY_PATH appropriately, then execs the real application (which is often called app.bin, or app.exe).
Related
My question is in relation to .so shared libraries. I am building a project that uses cmake on one ubuntu machine but running the application on another ubuntu machine.
In the CMakeLists.txt file, I have the following lines:
project (clientapp)
add_executable(${PROJECT_NAME} ${SOURCES} ${WAKAAMA_SOURCES} ${SHARED_SOURCES})
LINK_DIRECTORIES(/home/user//mraa-master-built/build/src)
target_link_libraries (clientapp libmraa.so)
target_link_libraries(clientapp m)
These lines add two libraries libmraa.so and the math library to the executable and it runs successfully on the other machine.
My understanding of shared libraries is that they must be present at compile time, and when the application starts. But I do not have the libmraa.so file on the other machine and the application runs ok. I expected it not to work.
Is my assumption correct?
In general, gcc and clang support lazy linking/binding of symbols, but not for entire libraries. This means that all of the shared objects (ie: .so files) should be present at application startup, at a minimum. The one exception to this is if you modified your makefile to not link against these libraries, and you manually call library functions via dlopen()/dlsym(), etc.
The binding of individual symbols within those libraries can be postponed until they are needed, or you can force all the symbols to be resolved at startup, using -z lazy or -z now, respectively.
It is strange that your application runs without libmraa.so being present. The two most likely reasons your application is running in the absence of the library is:
Your application isn't using any symbols defined in the library, so the linker ignores the library at build time (try ldd app_name and see if your library is present in the list of libraries provided by ldd).
Something is amiss in your build script, and you are statically linking against a .a archive of the library.
Edit: In response to how the application knows how to find the library, your linker (ld in this case) will use rpath lookup to decide which directories to use in its search for the appropriate library. You can see how this works by doing something like LD_DEBUG=libs app_name from the command line. You can also add an extra path via LD_LIBRARY_PATH=/some/path app_name.
Is my assumption correct?
Yes.
There are two likely explanations for why the application runs anyway:
You are mistaken, and there is libmraa.so somewhere on the machine (though perhaps not in the place where you looked), or
Your compiler defaults to -Wl,--as-needed by default, and your binary does not in fact depend on libmraa.so despite the fact that it appears on your link line.
You can trivially confirm or disprove either of the above guesses.
To confirm guess 2, do this:
readelf -d clientapp | grep NEED | grep libmraa
# if there is no output, guess 2 is correct
If guess 2 is wrong, to confirm guess 1, do this (on machine without libmrra.so):
ldd clientapp | grep libmraa.so
# if guess 2 is incorrect, and this command produces no output, then
# your dynamic loader is broken, which is very unlikely.
Original
I am looking for a way to create a non-isolated development environment for a C-library.
I will most likely use cmake to build the library and my IDE is a simple text editor.
The problem now is that I do not only create the library but also some sample "applications" using the library.
Therefore I need to install the library's headers and the shared object (I'm using GNU/Linux) somewhere and I do not want to install it to /usr/local/lib or (the even worse) /usr/lib.
Is there a way to create a virtual environment similar to python's pyvenv (and similar) where I can install the everything to but still have access to the host libraries?
Also I do not want to rewrite my $PATH/$LD_LIBRARY_PATH, setup a VM, container, or chroot.
The usage would then look like:
# switch to environment somehow
loadenv library1
# for library
cd library
make && make install
# for application
cd ../application1
make && ./application1
Is this possible?
Edit 1
So basically my directory structure will look like this:
library/
library/src/
library/src/<files>.c
library/include/<files>.h
application/
application/src/
application/src/<files>.c
First I need to compile the library and install the binary and header files.
These should be installed in a fake system-location.
Then I can compile the application and run it.
Edit 2
I thought a bit about it and it seems all I need is a filesystem sandbox.
So basically I want to open up a shell where every write to disk is not committed to the filesystem but rather temporarily saved in e.g. a ramfs/tmpfs just to be dropped when the shell exits.
This way I can exactly test how everything would behave if compiled, deployed and executed on the real machine without any danger to existing files or directories and without accidentally creating files or directories without cleaning them up.
You don't really need to 'install' the library, you can work in the development tree.
(1) for compilation all you need to do is use -I flag to specify where the libraries header files are, and this can be a relative path, for example in your case you could do -I../../library/include
(2) for linking you need to tell the linker where the library is located at, you can use the -L flag append to the library search order.
(3) for testing the application, you are correct that the application needs to be able to find the library. You have a couple of options:
(a) make sure the library and the executable are in the same directory
(b) you can temporarily modify your LD_LIBRARY_PATH, in your current shell only, for testing:
export LD_LIBRARY_PATH=abs_path_to_library:$LD_LIBRARY_PATH
note that this will only effect the current shell (command terminal) you are working in. Any other shells you may have open, or open later will have your normal LD_LIBRARY_PATH. I know you specified that you don't want to modify your PATH or LD_LIBRARY_PATH, but being local to the shell that the command is executed it is a nice, easy way to do this.
(c) embed the path to the library in the client executable. To do this you need to pass an option to the linker. The command for gcc is:
-Wl,-rpath,$(DEFAULT_LIB_INSTALL_PATH)
see this how-to
I have a shared library that my application needs (a .so) and I am wondering what is the best way to distribute it?
It's not something that can be apt-get installed and I need it in the LD path's in order to run the application.
In the past I've needed to include a separate "launcher script" that the user would click on instead of clicking on the Linux executable directly. The launcher script would set up LD_LIBRARY_PATH to include the directory where the shared library was stored, and then launch the executable. Here's the script, for reference (it assumes that the executable and the shared library are hidden away in a sub-folder named "bin", and that the executable's name is the same as the script's name except without the ".sh" suffix):
#!/bin/bash
appname=$(basename "$0" .sh)
dirname=$(dirname "$0")
cd "$dirname/bin"
export LD_LIBRARY_PATH=$(pwd):$LD_LIBRARY_PATH
./$appname "$#"
Distribute it the same way you distribute the executable that depends upon it; bundle the two together.
If you didn't write the library, make sure you're complying with its license terms for redistribution.
I'm doing some C programming on a machine for which I don't have root access. I've compiled some shared libraries that I'm linking to, but because I cannot install the libraries in the typical location (/usr/local/lib) I have to explicitly specify the location of the libraries each time I compile and run. When compiling, this simply means adding the -L flag to the gcc command, but for program execution it's a lot more annoying. Either I must add the non-standard directory to LD_LIBRARY_PATH in each session, or I must add LD_PRELOAD=/path/to/libs to the beginning of the execute command.
Is there a better way to do this on a machine for which I don't have root access?
BTW, the machine is running Red Hat 4.1.
There are several solutions, from better to worse:
Use $ORIGIN, e.g. gcc main.o -L../lib -lfoo -Wl,-rpath='$ORIGIN'/../lib
Use target RPATH, e.g. gcc main.o -L../LIB -lfoo -Wl,-rpath=/home/user/lib
Set LD_LIBRARY_PATH from your .bashrc or .profile
Solution 1 allows you to install the binary anywhere, so long as you move the binary and the libraries together, e.g. my-app/bin/a.out and my-app/lib/{needed-shared-libs}.so. It also allows for multiple versions of the application and their set of shared libs.
Solution 2 works fine if you only need one set of shared libs, and never wish to move them around.
Solution 3 affects every application you run, and may cause some of them to bind to your shared libraries instead of the system ones. That may cause them to crash, fail with unresolved symbols, or cause you other pain. To exacerbate, the problem will only happen to you and nobody else, so you'll have hard time getting help for it.
You can add the environment variables to your .bashrc (or whatever file your shell sources when you log-in).
If you set the environment variable LD_RUN_PATH when you compile and link your program, then that search path will be baked in to the executable, and the dynamic linker will search it at runtime.
Using LD_LIBRARY_PATH or LD_PRELOAD is pretty much how to do this. To fix this, rename your program from myprog to myprog-exe, and create a shell script that looks like this called myprog:
#!/bin/sh
export LD_LIBRARY_PATH=/path/to/libs:$LD_LIBRARY_PATH
`dirname $0`/myprog-exe
This way, when someone runs myprog, it will really run the shell script which then runs myprog.
I,m using gcc compiler(MinGW) on Windows XP.I created a .dll library libdir.dll than I tried to build a program that is using that library.
I don't want to put that .dll file into System or System32 folder nor to set path to it in PATH variable, what i want is to give that information to the program itself.
I know there is a -R and -rpath switches available so i was gonna link it with one of them.
First -rpath:
gcc -L/path/to/lib -Wl,-rpath,/path/to/lib main.o -ldir -o prog
Than -R:
gcc -L/path/to/lib -Wl,-R,/path/to/lib main.o -ldir -o prog
This links successfully into prog but when i start the program Windows prints message that it cannot find libdir.dll.
So my question is what went wrong, why path to libdir.dll is not known in runtime even when I'm using appropriate switches?
Let's say i have prog1 and prog2 each containing their own copy of libdir.dll and both of them start to run at the same time loading code in the library.What happens in memory is there a two copies loaded or linker figures out that there is a copy and uses that for both programs?
Second question is about how libraries are loaded(any OS).Does linkers always load entire library or just parts needed?For example if program references function foo() which is in the library, does linker maps into memory only that function or entire library first?
There are only two real alternatives: put the DLL in the same folder as the EXE or put it in the working directory for the EXE. The latter being not much of an option since you'd have to create a shortcut to make the default working directory different from the directory that contains the EXE.
Not putting the DLL in the same directory as the EXE only makes sense if you want to share the DLL with other applications. To avoid the inevitable DLL hell this causes, you'd need to store the DLL in the side-by-side cache. The tooling you need to create the manifest and embed it in the EXE and the installer you'd need to deploy the DLL to the target machine are probably hard to come by with your tool chain. It is very rarely done anyway.
Part of this question is a duplicate of this one: Is there a Windows/MSVC equivalent to the -rpath linker flag?
The summary of the answer is that there is no direct equivalent of RPATH on Windows.
Since you precluded placing your DLLs in the default library search path (which on Windows includes the system directories you listed and the directories in the PATH environment variable), you are left with these options:
using batch files
placing all the DLLs and executables in the same directory
making OS-level calls in your program for adding to the DLL search path