Generate Static Executable with CMake - c

Please bear with this question, it is rather long.
TLDR: A CMake project with a subdirectory library links successfully, but creates a dynamic executable.
Code Available at: https://github.com/georcon/cmake-issue
Also Note: I have read all related questions/answers, and none answer this question.
I have created the following minimal example:
Create a statically-linked executable (Works correctly)
(Git Tag: SimpleExecutable)
main.c
#include <uuid/uuid.h>
#include <stdio.h>
int main(){
uuid_t some_uuid;
char uuid_str[40];
uuid_generate(some_uuid);
uuid_unparse(some_uuid, uuid_str);
printf("UUID: %s\n", uuid_str);
return 0;
}
CMakeLists.txt
project(cmake-issue VERSION 1.0 DESCRIPTION "Static target issue" LANGUAGES C)
add_executable(main-dynamic main.c)
target_link_libraries(main-dynamic uuid)
add_executable(main-static main.c)
target_link_libraries(main-static uuid -static)
Result
ldd main-dynamic
linux-vdso.so.1 (0x00007ffc406bb000)
libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f76781cd000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7677fdb000)
/lib64/ld-linux-x86-64.so.2 (0x00007f76781eb000)
ldd main-static
not a dynamic executable
Create a static executable with library
(Git Tag: ExecutableWithLibrary)
lib/lib.h
#ifndef LIB_H
#define LIB_H
void PrintUUID();
#endif //LIB_H
lib/lib.c
#include <uuid/uuid.h>
#include <stdio.h>
void PrintUUID(){
uuid_t some_uuid;
char uuid_str[40];
uuid_generate(some_uuid);
uuid_unparse(some_uuid, uuid_str);
printf("UUID: %s\n", uuid_str);
}
lib/CMakeLists.txt
cmake_minimum_required(VERSION 3.13)
project(testlibrary VERSION 1.0 DESCRIPTION "Static target issue - Library" LANGUAGES C)
add_library(testlib lib.c)
target_link_libraries(testlib uuid)
main.c
#include "lib/lib.h"
int main(){
PrintUUID();
return 0;
}
CMakeLists.txt
cmake_minimum_required(VERSION 3.13)
project(cmake-issue VERSION 1.0 DESCRIPTION "Static target issue" LANGUAGES C)
add_subdirectory(lib ./bin)
link_directories(./lib/ ./bin)
add_executable(main-dynamic main.c)
add_dependencies(main-dynamic testlib)
target_link_libraries(main-dynamic libtestlib.a uuid -static)
link_libraries("-static")
add_executable(main-static main.c)
target_link_libraries(main-static PUBLIC "-static" libtestlib.a uuid)
add_dependencies(main-static testlib)
#target_link_libraries(main-static libtestlib.a uuid -static)
Result
ldd main-static
linux-vdso.so.1 (0x00007ffe6b485000)
libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007fc67edd3000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc67ebe1000)
/lib/ld64.so.1 => /lib64/ld-linux-x86-64.so.2 (0x00007fc67edec000)
Looking at the linker command:
~/cmake_issue$ cat CMakeFiles/main-static.dir/link.txt /usr/bin/cc
CMakeFiles/main-static.dir/main.c.o -o main-static
-L/home/georcon/cmake_issue/./lib -L/home/georcon/cmake_issue/./bin -Wl,-rpath,/home/georcon/cmake_issue/./lib:/home/georcon/cmake_issue/./bin -static -static
-Wl,-Bstatic -ltestlib -Wl,-Bdynamic -luuid
Why does CMake not generate a statically-linked executable in this case?

Long story short: You need to tell CMake that you prefer static linking with the libraries.
This is done by setting property LINK_SEARCH_START_STATIC. Also you need to tell CMake to not reset static linkage at the end of the libraries list.
This is done by setting property LINK_SEARCH_END_STATIC:
set_target_properties(main-static PROPERTIES
LINK_SEARCH_START_STATIC ON
LINK_SEARCH_END_STATIC ON
)
See also that question: CMake and Static Linking.
What is going on
Actually, the linker option -static not only disables PIE, but also affects on further libraries listed in the command... unless -dynamic is specified.
CMake has a notion about "default linking type", which is applied for every library (uuid in your case) for which CMake cannot deduce its type. Moreover, CMake maintains that default linking type after each library it adds into the linker's command line. And CMake expects the same behavior from the user, who manually adds linker flags.
Your first example is wrong, but suddenly works:
You add -static which turns current linkage type to static. And thus you break CMake expectations about current linkage type.
When produce the linker option for link with uuid, CMake expects that current linkage is dynamic. So, CMake doesn't add -dynamic linker switch.
That time CMake expectations doesn't correspond to the reality, which in turn corresponds to your expectations: uuid is linked statically.
But the second example reveals the problem:
When linking with libtestlib.a library, CMake is perfectly aware that this is a static library, and thus adds Wl,-Bstatic option before that library. But CMake need to maintain default linkage after every option, so it adds -Wl,-Bdynamic after the library:
-Wl,-Bstatic -ltestlib -Wl,-Bdynamic
With such options CMake expectations about default dynamic linking corresponds to the reality: uuid is linked dynamically. But now that reality doesn't correspond to yours expectations.

Related

Add dynamic dependency to shared object in C++

I want to create shared object B that dynamically links to shared object A. I'm using the following command to compile shared object B:
g++ -fPIC -shared -L/path/to/directory -lA -o libB.so B.cpp
It is my understanding that -lA is what tells the linker that libB.so should dynamically link to /path/to/directory/libA.so. However, when I do ldd on the final product, the dependency is not listed (and loading libB.so fails because of these missing dependencies).
ldd libB.so
linux-vdso.so.1 => (0x00007ffd4233e000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f35072fe000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f35070e7000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f3506de1000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3506a1c000)
/lib64/ld-linux-x86-64.so.2 (0x00007f3507807000)
Am I wrong about what -l is supposed to do? I assume that the above is a minimal set of dynamic dependencies from C++.
Are there any gotchas to look for? For instance, does the linker simply ignore -l requests when it can't find the file or something (and I have to debug my paths more than I already have)?
Do I have to put something in my C++ code to indicate dependency (like "extern" functions or something)?
Update:
I have determined that the set of dynamic dependencies that ld reports does depend on my C++ code and does not appear to depend on any -L or -l flags I supply. The linker is automatically guessing which shared objects my libB.so should depend on, and it's not assuming enough.
For instance, I know that I'll need B to load A because I call some code that eventually calls code in libA.so. How do I provide this information to the linker?
Clarifications:
What I'm calling "shared object A" is a complex thing that may load some code dynamically. I want to include enough dependencies so that it will not fail with "missing symbols" when it tries to load this code dynamically. That's why I want to force dependencies in B, because the linker might not statically find them in the dependency tree.
Also, I'm using g++ 4.8.4 (Ubuntu 14.04). This is relevant because g++ started implicitly applying -Wl,--as-needed as of version 4.6.
Probably the dependency is not listed in the ldd's output because no symbols which are used in B.cpp are found in libA.so. This may happen because of C++ symbol names mangling: if libA.so has been compiled by the C compiler, it may store the symbol for function void foo() under pretty name foo, whereas C++ compiler will mangle it into something like _Z3foov. You can check it with the following commands:
$ # Replace "SomeSharedObject" with the actual name of symbol exported by libA.so.
$ strings libA.so | grep SomeSharedObject
$ strings libB.so | grep SomeSharedObject
To avoid this, one could put the declaration of the symbol foo into an extern "C" {} clause. Then compiler will not mangle this name, and linker will probably find this name in libA.so.
The -l option works as you expect in a laboratory environment:
$ cat bar.cpp
extern void foo();
void bar()
{
foo();
}
$ cat baz.cpp
extern void bar();
void baz()
{
bar();
}
$ # Link against libssl.so (OpenSSL).
$ # Obviously libssl.so is unnecessary in libbar.so.
$ g++ -fPIC -shared bar.cpp -o libbar.so -lssl
$ ldd libbar.so
statically linked
$ # Link against libbar.so in the current directory
$ g++ -fPIC -shared baz.cpp -o libbaz.so -L`pwd` -lbar
$ ldd libbaz.so
linux-vdso.so.1 => (0x00007fff7cfe2000)
libbar.so => not found
Here libbar.so depends on the function foo. But it hasn't been found in any library, including libssl.so. So ldd reports the shared object libbar.so as "statically linked". All symbols which were not found when libbar.so was produced, will be searched for when creating the final executable which depends on libbar.so.
In turn, libbaz.so depends on libbar.so, because the function void bar() was found in the above shared object which we have specified via -l option. If we omit the -L option, linker would report error like -lbar: not found. If we omit both -L and -l, libbaz.so would not depend on any shared object just as libbar.so.
You are compiling to a weird object name. Linking will happen when you create an executable. then you have to mention your various objects and libraries.

Are runtime libraries inherently dynamic libraries?

I am cross compiling for a system with an OpenMP parallelized program, but when I run on the target, I get the error:
can't load library 'libgomp.so.1'
After looking around, I see that it's an OpenMP runtime library. Is there any was to statically link the library it on the compiler host machine, or does it need to be present on the target machine? If it can be statically linked, then what makes a runtime library different from a dynamic library? Could one statically or dynamically link any library, provided the environment was correct?
You can selectively statically link certain libraries by providing certain linker options. For libgomp it would be something like:
gcc -o executable foo.o bar.o -Wl,-static -lgomp -Wl,-Bdynamic -lpthread -lother -llibs
Any library listed between -Wl,-static and -Wl,-Bdynamic will be linked in statically. -fopenmp should not be present in the linking command as it expands to linker flags that get appended after the user supplied options, and therefore libpthread should be listed explicitly. It also means that even simple OpenMP programs have to be compiled and linked in two separate steps for static linking to work.
Example:
// foo.c
#include <stdio.h>
#include <omp.h>
int main(void)
{
#pragma omp parallel
printf("Hello world from thread %d\n", omp_get_thread_num());
return 0;
}
Traditional compilation:
$ gcc -fopenmp -o foo foo.c
$ ldd foo
linux-vdso.so.1 => (0x00007ffff5661000)
libgomp.so.1 => /usr/lib64/libgomp.so.1 (0x0000003bcfa00000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003bc2600000)
libc.so.6 => /lib64/libc.so.6 (0x0000003bc1e00000)
librt.so.1 => /lib64/librt.so.1 (0x0000003bc3200000)
/lib64/ld-linux-x86-64.so.2 (0x0000003bc1a00000)
The program is linked against the DSO version of libgomp.
Fully static linking:
$ gcc -fopenmp -static -o foo foo.c
$ ldd foo
not a dynamic executable
With -static all libraries are linked in statically into the executable.
Linking only libgomp statically:
$ gcc -fopenmp -c foo.c
$ gcc -o foo foo.o -Wl,-static -lgomp -Wl,-Bdynamic -lpthread
$ ldd foo
linux-vdso.so.1 => (0x00007ffdaaf61000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003bc2600000)
libc.so.6 => /lib64/libc.so.6 (0x0000003bc1e00000)
/lib64/ld-linux-x86-64.so.2 (0x0000003bc1a00000)
It is important to maintain the correct order of statically linked objects in that case. If foo.o is placed after -lgomp, a link error results:
$ gcc -o foo -Wl,-static -lgomp -Wl,-Bdynamic foo.o -lpthread
foo.o: In function `main':
foo.c:(.text+0x14): undefined reference to `GOMP_parallel_start'
foo.c:(.text+0x23): undefined reference to `GOMP_parallel_end'
foo.o: In function `main.omp_fn.0':
foo.c:(.text+0x3b): undefined reference to `omp_get_thread_num'
collect2: ld returned 1 exit status
Any object file resulting from source code that contains OpenMP constructs should be placed before -lgomp.
The term "runtime library" is usually used for the standard library and environment needed to run your program. In the case of a C program, it's the C standard library, maybe some other libraries specific for your compiler, and some object files linked to your program to set up the standard C environment.
A "runtime library" can be a dynamic library, or even a collection of multiple dynamic libraries. But it can also be one or more static libraries as well.
Dynamic libraries are convenient way of providing runtime libraries, as many programs will potentially want to link against such a library. And this is why dynamic linking and dynamic libraries are out there - the first process that requires particular dynamic library will cause it to be loaded to memory. Later, this instance of dynamic library can be reused by many processes.
Imagine if you'd have tens of process running and each of them statically link against say C runtime library. I assume memory consumption would grow rather significantly compared to the case where each of these processes would link against single DLL.
In my opinion, if library will be used by many different processes (for example DirectX might be such library) it might be more efficient to provide it as dynamic library. Otherwise, static linking is preferable.

gcc static library linking vs dynamic linking

My build environment is CentOS 5. I have a third party library called libcunit. I installed it with autotools and it generates both libcunit.a and libcunit.so. I have my own application that links with a bunch of shared libraries. libcunit.a is in the current directory and libcunit.so and other shared libraries are in /usr/local/lib/. When I compile like:
gcc -o test test.c -L. libcunit.a -L/usr/local/lib -labc -lyz
I get a linkage error:
libcunit.a(Util.o): In function `CU_trim_left':
Util.c:(.text+0x346): undefined reference to `__ctype_b'
libcunit.a(Util.o): In function `CU_trim_right':
Util.c:(.text+0x3fd): undefined reference to `__ctype_b'
But when I compile with .so like:
gcc -o test test.c -L/usr/local/lib -lcunit -labc -lyz
it compiles fine and runs fine too.
Why is it giving error when linked statically with libcunit.a?
Why is it giving error when linked statically with libcunit.a
The problem is that your libcunit.a was built on an ancient Linux system, and depends on symbols which have been removed from libc (these symbols were used in glibc-2.2, and were removed from glibc-2.3 over 10 years ago). More exactly, these symbols have been hidden. They are made available for dynamic linking to old binaries (such as libcunit.so) but no new code can statically link to them (you can't create a new executable or shared library that references them).
You can observe this like so:
readelf -Ws /lib/x86_64-linux-gnu/libc.so.6 | egrep '\W__ctype_b\W'
769: 00000000003b9130 8 OBJECT GLOBAL DEFAULT 31 __ctype_b#GLIBC_2.2.5
readelf -Ws /usr/lib/x86_64-linux-gnu/libc.a | egrep '\W__ctype_b\W'
# no output
Didn't notice that the libcunit.a is actually found in your case and the problem with linakge is rather in the CUnit library itself. Employed Russian is absolutely right, and he's not talking about precompiled binary here. We understand that you've built it yourself. However, CUinit itself seems to be relying on the symbol from glibc which is not available for static linking anymore. As a result you have only 2 options:
File a report to the developers of CUnit about this and ask them to fix it;
Use dynamic linking.
Nevertheless, my recommendation about your style of static linkage still applies. -L. is in general bad practice. CUnit is a 3rd party library and should not be placed into the directory containing source files of your project. It should be rather installed in the same way as the dynamic version, i.e. like you have libcunit.so in /usr/local/lib. If you'd supply prefix to Autotools on the configure stage of CUnit, then Autotools would install everything properly. So if you want to link statically with CUnit, consider doing it in the following form:
gcc -o test test.c -L/usr/local/lib -Wl,-Bstatic -lcunit -Wl,-Bdynamic -labc -lyz

CMake: Static and dynamic linking based on BUILD_TYPE

I am developing a small simulation software that depends on two libraries, the GSL and the libconfig. As a build system, I use CMake. For both the GSL and libconfig, I found cmake files and copied them into the cmake/ directory of my project.
The scenario is the following: I want the project to have several different build types, like debug, release, etc., but also a custom one called cluster, which adds -static to the GCC flags and links against the .a libraries of the GSL and the libconfig, which I assume exist.
My CMakeLists.txt looks like this so far:
# version
SET(PACKAGE_VERSION "1.0")
SET(PACKAGE_NAME "INTERFACE")
PROJECT(interface C CXX)
CMAKE_MINIMUM_REQUIRED(VERSION 2.8)
# dirs -----------------------------------------------------
SET(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_SOURCE_DIR}/cmake")
FIND_PACKAGE(GSL REQUIRED)
INCLUDE_DIRECTORIES(${GSL_INCLUDE_DIRS})
SET(LIBS ${LIBS} ${GSL_LIBRARIES})
FIND_PACKAGE(LibConfig REQUIRED)
INCLUDE_DIRECTORIES(${LIBCONFIG_INCLUDE_DIRS})
SET(LIBS ${LIBS} ${LIBCONFIG_LIBRARIES})
CONFIGURE_FILE("res/config.h.in" "config.h")
INCLUDE_DIRECTORIES(${CMAKE_BINARY_DIR})
SET_DIRECTORY_PROPERTIES(PROPERTIES
ADDITIONAL_MAKE_CLEAN_FILES "config.h"
)
# compilation ----------------------------------------------
ADD_EXECUTABLE( interface
interface.c interface.h config.h
helpers.c
output.c
lattice.c
genetic.c
)
# optional CFLAGS
SET(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wall -Wextra -pedantic -std=c99")
SET(CMAKE_C_FLAGS_RELEASE "-O3 -ffast-math")
SET(CMAKE_C_FLAGS_CLUSTER "-O3 -ffast-math -static")
SET(CMAKE_C_FLAGS_DEBUG "-g")
SET(CMAKE_C_FLAGS_PROFILE "-g -ffast-math -pg")
TARGET_LINK_LIBRARIES(interface m ${LIBS})
# installation --------------------------------------------
INSTALL(TARGETS interface DESTINATION bin)
This adds the -static to the compiler, when I use -DCMAKE_BUILD_TYPE=cluster. The thing is, that it still links against the .so versions of the libs, which causes gcc to throw errors. At least the FindLibConfig.cmake scripts sets both a LIBCONFIG_LIBRARY and a LIBCONFIG_STATIC_LIBRARY variable, which I could use.
What is the most elegant or smart way to reach my goal?
Acording to cmake documentation, cmake uses the variable BUILD_SHARED_LIBS to determine the default for add_library().
If you set to ON cmake will assume all add_library() call will be as
add_library(target SHARED lib1 lib2 ...)
For example something like -DBUILD_SHARED_LIBS=ON on the cmake command line may do what you are asking.
I solved it like this:
The User can specify an additional variable -DSTATIC_LINKING=TRUE. Then, the script looks like this. (Only the important parts for the static linking and compilation are shown!)
# determine, whether we want a static binary
SET(STATIC_LINKING FALSE CACHE BOOL "Build a static binary?")
# do we want static libraries?
# When STATIC_LINKING is TRUE, than cmake looks for libraries ending
# with .a. This is for linux only!
IF(STATIC_LINKING)
SET(CMAKE_FIND_LIBRARY_SUFFIXES ".a")
ENDIF(STATIC_LINKING)
# set -static, when STATIC_LINKING is TRUE and set LINK_SEARCH_END_STATIC
# to remove the additional -bdynamic from the linker line.
IF(STATIC_LINKING)
SET(CMAKE_EXE_LINKER_FLAGS "-static")
SET_TARGET_PROPERTIES(surface PROPERTIES
LINK_SEARCH_END_STATIC 1)
ENDIF(STATIC_LINKING)

C program linking with shared library without setting LD_LIBRARY_PATH

I was reading Introduction to GCC and it says if a package has both .a and .so. gcc prefer the shared library. By default the loader searches for shared libraries only in a predefined set of system directories, such as /usr/local/lib and /usr/lib. If the library is not located in one of these directories it must be added to the load path, or you need to use -static option to force it to use the .a library. However, I tried the following:
vim hello.c:
#include <gmp.h>
#include <stdio.h>
int main() {
mpz_t x;
mpz_init(x);
return 0;
}
gcc hello.c -I/opt/include -L/opt/lib -lgmp (my gmp library is in opt)
./a.out
And it runs. The book says it should have the following error:
./a.out: error while loading shared libraries:
libgdbm.so.3: cannot open shared object file:
No such file or directory
(well, the book uses GDBM as example but I used GMP, but this won't matter right?)
However, I did not set LD_LIBRARY_PATH=/opt/lib, and as you can see I did not use -static option either, but a.out still runs.
Can you all tell me why and show me how to get the error described in the book? Yes I want the error so I will understand what I misunderstood.
From your response to my comment:
linux-gate.so.1 => (0xb7746000)
libgmp.so.10 => /usr/lib/i386-linux-gnu/libgmp.so.10 (0xb76c5000)
libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xb7520000)
/lib/ld-linux.so.2 (0xb7747000)
So, your program is picking up the lib from /usr/lib.
What you can try to do is rename the lib in your /opt/lib, and link against the new name.
mv /opt/lib/libgmp.so /opt/lib/libgmp-test.so
gcc hello.c -I/opt/include -L/opt/lib -lgmp-test
Then try running the program. Also, compare the result of ldd against the new a.out against what you got before.

Resources