I have a program which is linked (dynamically) with libm.
There are also several plugins for this program.
Plugins are loaded explicitely with dlopen().
Some of these plugins use round() from libm.
On one system (Linux Mint 19.1 gcc 7.5.0) the program
does not work because of unresolved round.
Here is simple example:
Library (lib.c)
#include <stdio.h>
#include <math.h>
void func(double a, double b)
{
double c;
c = round(a + b);
printf("c = %lf\n", c);
}
Main program (main.c)
#include <stdio.h>
#include <dlfcn.h>
void *dll;
void (*f)(double, double);
double a = 1.234, b = 4.321;
int main(void)
{
dll = dlopen("./lib.so", RTLD_LAZY);
f = dlsym(dll, "func");
f(a,b);
return 0;
}
Building (Makefile)
all:
gcc -Wall -Os -shared -fPIC lib.c -o lib.so
gcc -Wall -Os -rdynamic -fPIC main.c -o main -ldl -lm
Run on Debian 8, gcc 4.9.2
./main
c = 6.000000
Run on Linux Mint 19.1, gcc 7.5.0
./main
./main: symbol lookup error: ./lib.so: undefined symbol: round
Now, add -lm for dll compilation
gcc -Wall -Os -shared -fPIC lib.c -o lib.so -lm
./main
c = 6.000000
So, the question is - why on this particular system one must use -lm not only for main program but for plugin also?
Just like an executable program, shared libraries are linked entities (unlike static libraries which are archives of object files).
Since shared libraries are linked like executables, you also need to link with the libraries that your library depends on:
gcc -Wall -Os -shared -fPIC lib.c -o lib.so -lm
Related
I have a statically linked library, containing a global variable barvar. I can compile the library with no problems with either gcc-10 or clang (this is on macOS Catalina). Interestingly, the behavior differs between the two when I try to link it into a program that uses the library. Here's the code:
In globvars.h, int barvar is declared:
#ifndef H_GLOBVARS_H
#define H_GLOBVARS_H
extern int barvar;
#endif
In globvars.c, int barvar is defined:
#include "globvars.h"
int barvar;
In foo.c, the function foo sets and prints barvar:
#include <stdio.h>
#include "globvars.h"
void foo()
{
barvar = 10;
printf("barvar is: %d\n", barvar);
return;
}
Here's test.c, the program that uses the library:
void foo();
int main(int argc, char **argv)
{
foo();
return 0;
}
When I compile and link with gcc-10, no problems:
gcc-10 -c foo.c -o foo.o
gcc-10 -c globvars.c -o globvars.o
gcc-10 -c test.c -o test.o
gcc-ar-10 rcs liblinktest.a foo.o globvars.o
gcc -o testlinkrun test2.o -L. -llinktest
When I compile and link with clang, I get an undefined symbol error at the last step:
cc -c foo.c -o foo.o
cc -c globvars.c -o globvars.o
cc -c test.c -o test.o
ar rcs liblinktest.a foo.o globvars.o
cc -o testlinkrun test2.o -L. -llinktest
with error:
Undefined symbols for architecture x86_64:
"_barvar", referenced from:
_foo in liblinktest.a(foo.o)
Any ideas? Interestingly, it appears the only step that has to be done with gcc-10 is compiling globvars.c. I can use clang and the clang linker for all other steps, and everything is fine. Is it possible that clang is optimizing away all the variables in globvars.c? How can I prevent this?
As #EricPostpischil observed in this comment, the issue is that clang defaults to treating barvar as a common symbol. Either changing int barvar; to int barvar = 0;, or compiling with -fno-common, fix the issue.
Beginning with gcc-10, gcc's default behavior is -fno-common instead of -fcommon.
I am experimenting with externs and various methods of linking to better understand the linking process.
I have three files:
foo.c:
#include "foo.h"
int a = 4;
test.c:
#include <stdio.h>
#include "foo.h"
int main(int, char**);
int mymain();
int mymain() {
main(0, 0);
printf("test\r\n");
return 0;
}
int main(int argc, char** argv) {
printf("extern a has %d\r\n", a);
return 0;
}
foo.h:
extern int a; // defined in foo.c
If I build each file together and link at compile time using gcc like this:
gcc *.c -o final.bin
I can execute final.bin as:
./final.bin
and get expected output
extern a has 4
However, if I compile (but don't link) test.c and foo.c separately, then try and link the object files together at runtime to produce a binary, I get a segmentation fault 11 (which from what I can gather is some generic memory corruption bug like a normal segfault(?)
Here is my makefile I'm using to compile and link separately. Note I am specifying my own entry point and linking against libc to get printf()...
all: test.o foo.o
#echo "Making all..."
ld test.o foo.o -o together.bin -lc -e _mymain
test.o: test.c
#echo "Making test..."
gcc -c test.c -o test.o
foo.o: foo.c
#echo "Making foo..."
gcc -c foo.c -o foo.o
Output when running 'together.bin':
./together.bin
extern a has 4
test
Segmentation fault: 11
Perhaps my function signature for 'mymain' is wrong? My guess is that something is wrong with my 'myentry' usage.
Also, if anyone has any recommendations on good books for how linkers and loaders work, I am certainly in the market for one. I've heard mixed things about 'Linkers and Loaders', so I'm waiting on more opinions before I invest the time in that book in particular.
Thanks for any help on this... My understanding of linkers is sub-par to say the least.
Unless if you have a good reason to do so, just use gcc to link:
$ gcc test.o foo.o "-Wl,-e,_mymain" -o ./final.bin; ./final.bin
extern a has 4
test
gcc calls ld---though, with a few more arguments than you are providing in your example. If you want to know exactly how gcc invokes ld, use the -v option. Example:
$ gcc -v test.o foo.o "-Wl,-e,_mymain" -o ./final.bin
Apple LLVM version 8.0.0 (clang-800.0.38)
Target: x86_64-apple-darwin15.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -dynamic -arch x86_64 -macosx_version_min 10.12.0 -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk -o ./final.bin test.o foo.o -e _mymain -lSystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/8.0.0/lib/darwin/libclang_rt.osx.a
I want to link three files but in hierarchical way.
// a.c
int fun1(){...}
int fun2(){...}
// b.c
extern int parameter;
int fun3(){...//using parameter here}
// main.c
int parameter = 1;
int main(){...// use fun1 fun2 fun3}
So, I first compile three files separately into object file a.o, b.o and main.o. And then I want to combine a.o and b.o into another object file tools.o. And eventually use tools.o and main.o to generate executable file.
But, when I try to combine a.o and b.o like ld -o tools.o a.o b.o, the linker says undefined reference to 'parameter'. How could I link those object files into an intermediate object file?
You want the -r option to produce a relocatable object file (think 'reusable'):
ld -o tools.o -r a.o b.o
Working code
abmain.h
extern void fun1(void);
extern void fun2(void);
extern void fun3(void);
extern int parameter;
a.c
#include <stdio.h>
#include "abmain.h"
void fun1(void){printf("%s\n", __func__);}
void fun2(void){printf("%s\n", __func__);}
b.c
#include <stdio.h>
#include "abmain.h"
void fun3(void){printf("%s (%d)\n", __func__, ++parameter);}
main.c
#include <stdio.h>
#include "abmain.h"
int parameter = 1;
int main(void){fun1();fun3();fun2();fun3();return 0;}
Compilation and execution
$ gcc -Wall -Wextra -c a.c
$ gcc -Wall -Wextra -c b.c
$ gcc -Wall -Wextra -c main.c
$ ld -r -o tools.o a.o b.o
$ gcc -o abmain main.o tools.o
$ ./abmain
fun1
fun3 (2)
fun2
fun3 (3)
$
Proved on Mac OS X 10.11.6 with GCC 6.1.0 (and the XCode 7.3.0 loader, etc). However, the -r option has been in the ld command on mainstream Unix since at least the 7th Edition Unix (circa 1978), so it is likely to be available with most Unix-based compilation systems, even if it is one of the more widely unused options.
Context :
Trying to understand how lto (link time compilation) works
Code:
I have those files :
julia.h:
#ifndef JULIA_H
#define JULIA_H
#include <stdio.h>
int julian();
#endif // JULIA_H
julia.c :
#include "julia.h"
int julian()
{
printf("Hello Worldu!\n");
return 0;
}
compiled as a shared library like so :
gcc -O3 -fPIC -shared julia.c -o libjulia.so -L$PWD -I$PWD -flto
and my main program :
main.c
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include "julia.h"
int main()
{
julian();
return 0;
}
compiled with :
gcc -O3 main.c -I/path/to/inc -L/path/to/lib -Wl,-rpath=/path/to/lib -ljulia -flto
It compiles fines.
Question :
So, this is a hello world program but am I doing it right with LTO ?
Is is all it takes to optimize the linkage ?
Thanks
As keltar saied, LTO doesn't affect shared libraries. But...
LTO works with static libraries
Just replace ar by gcc-ar and add the option --plugin gccpath/liblto_plugin.so. This LTO plugin will copy the declarations, types, callgraph and GIMPLE representation from LTO-compiled objects into the static lib. (same for ranlib to be replaced by gcc-ranlib)
In your example
# First retrieve the GCC path
gccpath=$(gcc -print-search-dirs | awk '/install/{print $2}')
# Compile the static library
gcc julia.c -o julia.o -flto -ffat-lto-objects
gcc-ar rcs libjulia.a julia.o --plugin $gccpath/liblto_plugin.so
# Compile & link the executable
gcc main.c libjulia.a -flto -Ofast -march=native
Note: -Ofast introduced in GCC-4.6 [ref] (else use -03)
Update Makefile
GCCPATH = $(shell gcc -print-search-dirs | awk '/install/{print $$2}')
AR = gcc-ar
RANLIB = gcc-ranlib
ARFLAGS += --plugin $(GCCPATH)/liblto_plugin.so
RANLIBFLAGS += --plugin $(GCCPATH)/liblto_plugin.so
CFLAGS += -flto -ffat-lto-objects
CXXFLAGS += -flto -ffat-lto-objects
LDFLAGS += -flto=8 # 8 -> compiles using 8 threads
Do not forget, the real compilation will be done at link time. Therefore, move your optimization flags from CFLAGS (and CXXFLAGS) to LDFLAGS ;-) One more thing, debugging info and LTO is still experimental in GCC-4.9. GCC-5.0 should improve this point...
LTO doesn't affect shared libraries; they're being linked with by dynamic linker, which is not aware of LTO and can't modify code at runtime.
Moreover, LTO doesn't even work with static libraries, but some day it presumably will (it is TODO on gcc wiki).
But yes, what it takes to enable is using -flto on both compilation and linking phases.
In the following example, the program should print "foo called\n":
// foo.c
#include <stdio.h>
__attribute__((constructor)) void foo()
{
printf("foo called\n");
}
// main.c
int main()
{
return 0;
}
If the program is compiled like this, it works:
gcc -o test main.c foo.c
However, if foo.c is compiled into a static library, the program prints nothing.
gcc -c main.c
gcc -c foo.c
as rcs foo.a foo.o
gcc -o test foo.a main.o
Why does this happen?
The linker does not include the code in foo.a in the final program because nothing in main.o references it. If main.c is rewritten as follows, the program will work:
//main.c
void foo();
int main()
{
void (*f)() = foo;
return 0;
}
Also, when compiling with a static library, the order of the arguments to gcc (or the linker) is significant: the library must come after the objects that reference it.
gcc -o test main.o foo.a
As it was stated, unreferenced symbols from archive does not make it to the output binary, because linker discards them by default.
To override this behaviour when linking with static library, --whole-archive/--no-whole-archive options for the linker may be used, like this:
gcc -c main.c
gcc -c foo.c
ar rcs foo.a foo.o
gcc -o test -Wl,--whole-archive foo.a -Wl,--no-whole-archive main.o
This may lead to bloated binary, because all symbols from foo.a will be included by the linker to the output, but sometimes it is justified.