Consider the two files listed below:
file a.c
extern int foovar;
int foobarize() {
return foovar * foovar;
}
and file b.c
int foovar = 10;
I compile the static library liba.a and the shared library libb.so as follows:
# liba.a
gcc -fPIC -c a.c -o a.o
ar cr liba.a a.o
ranlib liba.a
# libb.so
gcc -fPIC -c b.c -o b.o
gcc -fPIC -shared -Wl,-soname,libb.so -o libb.so b.o liba.a
Note that the function foobarize defined in a.c is present in liba.a but it isn't present in libb.so. I can guarantee that by issuing the nm program:
$ nm liba.a
a.o:
0000000000000000 T foobarize
U foovar
U _GLOBAL_OFFSET_TABLE_
$ nm libb.so
000000000020088c B __bss_start
000000000020088c b completed.6617
w __cxa_finalize##GLIBC_2.2.5
0000000000000530 t deregister_tm_clones
00000000000005c0 t __do_global_dtors_aux
0000000000200650 t __do_global_dtors_aux_fini_array_entry
0000000000200880 d __dso_handle
0000000000200660 d _DYNAMIC
000000000020088c D _edata
0000000000200890 B _end
0000000000000630 T _fini
0000000000200888 D foovar
0000000000000600 t frame_dummy
0000000000200648 t __frame_dummy_init_array_entry
0000000000000640 r __FRAME_END__
0000000000200858 d _GLOBAL_OFFSET_TABLE_
w __gmon_start__
00000000000004d8 T _init
w _ITM_deregisterTMCloneTable
w _ITM_registerTMCloneTable
0000000000200658 d __JCR_END__
0000000000200658 d __JCR_LIST__
w _Jv_RegisterClasses
0000000000000570 t register_tm_clones
0000000000200890 d __TMC_END__
How can I get the foobarize function in the libb.so shared library?
What you need:
# force all symbols
gcc -fPIC -shared -Wl,-soname,libb.so -o libb.so b.o \
-Wl,--whole-archive liba.a -Wl,--no-whole-archive
#force just a specific symbol
gcc -fPIC -shared -Wl,-soname,libb.so -o libb.so b.o \
-u foobarize liba.a
Why you need it:
A static library is a simple collection of object files. The one major difference from a bunch of object files is the following: when an undefined symbol needs to be resolved, the library is searched and only the object file that actually defines the symbol is linked. No undefined symbols? Nothing is searched, nothing is linked. To override this default behaviour, the GNU linker implements --whole-archive. Most linkers implement -u to force a particular symbol to be treated as undefinmed.
Related
I witnessed this error which is reproducible in AppleClang 12.0.0 but not with gcc 7.5.0. The issue is that I have an extern variable defined in a static library which a different static library wants to use. The linker states that the symbol for the variable is undefined. I tried to come up with an as minimal as possible repro:
externs.h
extern int A;
a.c
int A;
b.h
void bar();
b.c
#include "externs.h"
#include <stdio.h>
void bar()
{
A = 100;
printf("%d\n", A);
}
main.c
#include "b.h"
int main()
{
bar();
}
makefile
liba.a: a.c
gcc -c a.c -o a.o
ar crs liba.a a.o
libb.a: b.c
gcc -c b.c -o b.o
ar crs libb.a b.o
program: main.c liba.a libb.a
gcc main.c -L. -lb -la -o program
On Linux, this program compiles, links and runs without issue. On macOS (where gcc is AppleClang) I get the following output :
gcc -c a.c -o a.o
ar crs liba.a a.o
warning: /Library/Developer/CommandLineTools/usr/bin/ranlib: archive library: liba.a the table of contents is empty (no object file members in the library define global symbols)
gcc -c b.c -o b.o
ar crs libb.a b.o
gcc main.c -L. -lb -la -o program
Undefined symbols for architecture x86_64:
"_A", referenced from:
_bar in libb.a(b.o)
ld: symbol(s) not found for architecture x86_64
AFAICT, the warning is a red herring. If I add a dummy function to a.c, the warning doesn't show. Further, if I inspect liba.a with nm I get the following output:
liba.a(a.o):
0000000000000004 C _A
If I do add a dummy function foo (which doesn't even refer to A) to liba.a and invoke it from main.c, the linking issue gets magically resolved. It's as if if and only if main.o is dependent on a symbol in liba.a, then all the symbols of liba.a become available to anything that may be dependent on them.
I am facing linking problem. I'll illustrate it:
a.c:
extern void b(void);
int main() {
a();
return 0;
}
void a() {
b();
}
b.S:
.extern a
b:
jmp a
No matter if I'll link
gcc a.o b.o -o c
or
gcc b.o a.o -o c
I'll get unresolved symbols. How do I link these files? I can't merge them. Example may be nonsensical, but that illustrates point, what do i try to archive.
Initial investigation:
a.c
extern void b(void);
void a(void);
int main() {
a();
return 0;
}
void a() {
b();
}
b.S
.extern a
b:
jmp a
b.c
void a(void);
void b(void)
{
a();
}
Output
$ gcc -c a.c
$ gcc -c b.c -o b_gcc.o
$ as b.S -o b_as.o
$ gcc a.o b_gcc.o -o test_gcc
$ gcc a.o b_as.o -o test_as
a.o: In function `a':
a.c:(.text+0x15): undefined reference to `b'
collect2: error: ld returned 1 exit status
So what gives? Why is it okay with GCC but not GAS?
$ objdump -t b_gcc.o > syms_gcc
$ objdump -t b_as.o > syms_as
$ diff syms_gcc syms_as
2c2
< b_gcc.o: file format elf64-x86-64
---
> b_as.o: file format elf64-x86-64
5d4
< 0000000000000000 l df *ABS* 0000000000000000 b.c
9,12c8
< 0000000000000000 l d .note.GNU-stack 0000000000000000 .note.GNU-stack
< 0000000000000000 l d .eh_frame 0000000000000000 .eh_frame
< 0000000000000000 l d .comment 0000000000000000 .comment
< 0000000000000000 g F .text 000000000000000b b
---
> 0000000000000000 l .text 0000000000000000 b
Okay, so gcc makes b a global symbol. Lets try .global b in b.S:
$ as b.S -o b_as2.o
$ gcc a.o b_as2.o
$
Success. So gcc/ld will do multi-pass symbol resolution for anything that is not in a static library. But it only looks for global symbols. Here's the final b.S:
.extern a
.global b
b:
jmp a
I am trying to create a static library and link it on MacOS X (several versions):
File foo.c:
char foo[111];
File bar.c:
#include <string.h>
extern char foo[];
int bar(char *src) {
strcpy(foo, src);
return strlen(foo);
}
Create a library:
$ cc -c foo.c bar.c
$ ar r libfoobar.a foo.o bar.o
ar: creating archive libfoobar.a
$ ranlib libfoobar.a
$ nm libfoobar.a
libfoobar.a(foo.o):
000000000000006f C _foo
libfoobar.a(bar.o):
U ___strcpy_chk
0000000000000000 T _bar
U _foo
U _strlen
Create a small test program:
File main.c:
#include <stdio.h>
int bar(char *);
int main(void) {
printf("foobarbar = %i\n", bar("123"));
return 0;
}
Compile and link:
$ cc -c main.c
$ cc -o m main.o -L. -lfoobar
Undefined symbols for architecture x86_64:
"_foo", referenced from:
_bar in libfoobar.a(bar.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Why is the symbol not found? It is defined in foo.c? Shouldn't at least ranlib create an index in the library that allows a random order of the files there?
The same code works well under Linux (gcc), and also when the symbol in foo.c is not a char array, but an int.
There is a similar question: Object files not properly added to archive on mac which has this answer:
Option 1:
ar -rs my_archive.a foo.o bar.o other_object_files.o
ranlib -c my_archive.a
Option 2:
libtool -c -static -o my_archive.a foo.o bar.o other_object_files.o
It is -c flag that makes a difference for both options on ranlib and libtool respectively:
-c
Include common symbols as definitions with respect to the table
of contents. This is seldom the intended behavior for linking
from a library, as it forces the linking of a library member
just because it uses an uninitialized global that is undefined
at that point in the linking. This option is included only
because this was the original behavior of ranlib. This option
is not the default.
I have a sample program like this:
#include <stdio.h>
#if 1
#define FOR_EXPORT __attribute__ ((visibility("hidden")))
#else
#define FOR_EXPORT
#endif
FOR_EXPORT void mylocalfunction1(void)
{
printf("function1\n");
}
void mylocalfunction2(void)
{
printf("function2\n");
}
void mylocalfunction3(void)
{
printf("function3\n");
}
void printMessage(void)
{
printf("Running the function exported from the shared library\n");
}
And compile it using
gcc -shared -fPIC -fvisibility=hidden -o libdefaultvisibility.so defaultvisibility.c
Now after compilation I do:
$ nm libdefaultvisibility.so
nm libdefaultvisibility.so
0000000000000eb0 t _mylocalfunction1
0000000000000ed0 t _mylocalfunction2
0000000000000ef0 t _mylocalfunction3
0000000000000f10 t _printMessage
U _printf
U dyld_stub_binder
Which means as far as I can tell that despite -fvisibility=hidden all symbols get exported. The book I was following claimed that only the function marked with FOR_EXPORT should be exported.
I looked oup several other resources, but for the simple test I'm doing -fvisibility=hidden should be sufficient.
My clang version:
$ clang -v
clang -v
Apple LLVM version 7.3.0 (clang-703.0.31)
Target: x86_64-apple-darwin15.0.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
You're misunderstanding the output of nm. Scroll through man nm and you'll you
read that the t flag means the symbol is a local (static) symbol in
the text section. The linker can't see it. If it were global (external)
the flag would be T. So all four of your functions are local.
Contrast:
$ clang -shared -fPIC -fvisibility=hidden -o libdefaultvisibility.so defaultvisibility.c
$ nm libdefaultvisibility.so | grep ' t '
0000000000000570 t deregister_tm_clones
0000000000000600 t __do_global_dtors_aux
0000000000200e08 t __do_global_dtors_aux_fini_array_entry
0000000000000640 t frame_dummy
0000000000200e00 t __frame_dummy_init_array_entry
0000000000000670 t mylocalfunction1
0000000000000690 t mylocalfunction2
00000000000006b0 t mylocalfunction3
00000000000006d0 t printMessage
00000000000005b0 t register_tm_clones
with dropping the -fvisibility=hidden:
$ clang -shared -fPIC -o libdefaultvisibility.so defaultvisibility.c
$ nm libdefaultvisibility.so | grep ' t '
0000000000000600 t deregister_tm_clones
0000000000000690 t __do_global_dtors_aux
0000000000200e08 t __do_global_dtors_aux_fini_array_entry
00000000000006d0 t frame_dummy
0000000000200e00 t __frame_dummy_init_array_entry
0000000000000700 t mylocalfunction1
0000000000000640 t register_tm_clones
$ nm libdefaultvisibility.so | grep ' T '
0000000000000780 T _fini
00000000000005b0 T _init
0000000000000720 T mylocalfunction2
0000000000000740 T mylocalfunction3
0000000000000760 T printMessage
Then only the explicitly hidden mylocalfunction1 remains local, and the
other three are now global.
You should not expect that a symbol marked with __attribute__ ((visibility("hidden")))
will be exported by a shared library in any circumstances. The attribute means precisely
that it will not be, whether it is applied explicitly to a symbol, as in this case,
or acquired by default in the presence of the linker option -fvisibility=hidden.
If you want to export just that one function in the example by means of a visibility attribution
you would have:
#define FOR_EXPORT __attribute__ ((visibility("default")))
Then:
$ clang -shared -fPIC -fvisibility=hidden -o libdefaultvisibility.so defaultvisibility.c
$ nm libdefaultvisibility.so | grep ' T '
0000000000000720 T _fini
0000000000000550 T _init
00000000000006a0 T mylocalfunction1
It is global, because the explicit attribition overrides the commandline option,
and all your other functions are local. Perhaps confusingly, default visibility
is always public.
And you could accomplish this without resorting to visibility attributions - which are
not portable - simply declaring all the functions that you don't want to export as static. Then the compiler
would not expose them to the linker in the first place:
foo.c
#include <stdio.h>
void mylocalfunction1(void)
{
printf("function1\n");
}
static void mylocalfunction2(void)
{
printf("function2\n");
}
static void mylocalfunction3(void)
{
printf("function3\n");
}
static void printMessage(void)
{
printf("Running the function exported from the shared library\n");
}
With which you get again:-
$ clang -shared -fPIC -o libfoo.so foo.c
$ nm libfoo.so | grep ' T '
00000000000006c0 T _fini
0000000000000550 T _init
00000000000006a0 T mylocalfunction1
Although the distinction does not make itself felt in your example you
should understand that while a local/static symbol is not seen by the linker and (therefore) is unavailable for dynamic linkage, a global/external symbol
may or may not be available for dynamic linkage. visibility
controls the availability of global symbols for dynamic linkage, only.
According to GCC Wiki on Visibility, you should:
Use nm -C -D on the outputted DSO [Dynamic Shared Object] to compare before and after to see
the difference it makes.
As stated on nm manual:
-D will display the dynamic symbols rather than the normal symbols
If I compile your code exactly as you did I get the following objects:
$ nm -C -D libdefaultvisibility.so
nm -C -D libdefaultvisibility.so
0000000000200a68 B __bss_start
w __cxa_finalize
0000000000200a68 D _edata
0000000000200a70 B _end
00000000000006c8 T _fini
w __gmon_start__
0000000000000518 T _init
w _ITM_deregisterTMCloneTable
w _ITM_registerTMCloneTable
w _Jv_RegisterClasses
U puts
And if I compile it without the -fvisibility=hidden option I get the objects:
$ nm -C -D libdefaultvisibility.so
nm -C -D libdefaultvisibility.so
0000000000200ae8 B __bss_start
w __cxa_finalize
0000000000200ae8 D _edata
0000000000200af0 B _end
0000000000000748 T _fini
w __gmon_start__
00000000000005a0 T _init
w _ITM_deregisterTMCloneTable
w _ITM_registerTMCloneTable
w _Jv_RegisterClasses
0000000000000712 T mylocalfunction2
0000000000000724 T mylocalfunction3
0000000000000736 T printMessage
U puts
I have a large code base which is mainly built as binary. I have changed the Makefile to create a static library and I am creating a binary linking the library.
When I use it as a static library, code doesn't run due to weak symbols undefined reference.
gcc test.c -L . -lasntc -ltc -lresolv -lnetlink -lutil -ltc -lm -o mytc
nm mytc | grep htb_qdisc_util
w htb_qdisc_util
I untared the archives which resulted in .o's and then using those object files, I created a binary and this however works as shown
gcc tc.o tc_qdisc.o tc_class.o tc_filter.o tc_util.o tc_monitor.o
m_police.o m_estimator.o m_action.o m_ematch.o emp_ematch.yacc.o
emp_ematch.lex.o asn_tc.o asn_global.o q_fifo.o q_sfq.o q_red.o q_prio.o q_tbf.o
q_cbq.o q_rr.o q_multiq.o q_netem.o f_rsvp.o f_u32.o f_route.o f_fw.o f_basic.o
f_flow.o f_cgroup.o q_dsmark.o q_gred.o f_tcindex.o q_ingress.o q_hfsc.o q_htb.o
q_drr.o q_qfq.o m_gact.o m_mirred.o m_nat.o m_pedit.o m_skbedit.o p_ip.o
p_icmp.o p_tcp.o p_udp.o em_nbyte.o em_cmp.o em_u32.o em_meta.o q_mqprio.o static-syms.o tc_core.o tc_red.o tc_cbq.o tc_estimator.o tc_stab.o -lresolv -L. -lnetlink -lutil -L. -lm -o tc
nm tc | grep htb_qdisc_util
0000000000641bc0 D htb_qdisc_util
Just looking at the object files symbol table, following is seen
nm *.o | grep htb_qdisc_util
0000000000000000 D htb_qdisc_util
w htb_qdisc_util
Weak symbols resulting due to
extern char hfsc_qdisc_util[] __attribute__((weak)); if (!strcmp(sym, "hfsc_qdisc_util")) return hfsc_qdisc_util;
extern char htb_qdisc_util[] __attribute__((weak)); if (!strcmp(sym, "htb_qdisc_util")) return htb_qdisc_util;
How do I create a static library and what is happening when I create a binary