I checked multiple questions like this, but none of the answers worked in my case.
I have a simple "hello world!" c program. When I tried to run it using clang, I get the following error:
/..//bin/ld: cannot find -lgcc
/..//bin/ld: cannot find -lgcc_s
clang: error: linker command failed with exit code 1 (use -v to see invocation)
when I do -v it gives me:
clang version 3.4 (branches/release_34)
Target: x86_64-unknown-linux-gnu
Thread model: posix
Selected GCC installation:
"/nfs/projects/zephyr/Ruturaj/softboundcets-34/softboundcets-llvm-clang34/Debug+Asserts/bin/clang" -cc1 -triple x86_64-unknown-linux-gnu -emit-obj -mrelax-all -disable-free -main-file-name agsd.c -mrelocation-model static -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -target-cpu x86-64 -target-linker-version 2.29.1 -v -resource-dir /nfs/projects/zephyr/Ruturaj/softboundcets-34/softboundcets-llvm-clang34/Debug+Asserts/bin/../lib/clang/3.4 -internal-isystem /usr/local/include -internal-isystem /nfs/projects/zephyr/Ruturaj/softboundcets-34/softboundcets-llvm-clang34/Debug+Asserts/bin/../lib/clang/3.4/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdebug-compilation-dir /projects/zephyr/Ruturaj -ferror-limit 19 -fmessage-length 96 -mstackrealign -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -vectorize-slp -o /tmp/agsd-ad6d7d.o -x c agsd.c
clang -cc1 version 3.4 based upon LLVM 3.4 default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
/usr/local/include
/nfs/projects/zephyr/Ruturaj/softboundcets-34/softboundcets-llvm-clang34/Debug+Asserts/bin/../lib/clang/3.4/include
/usr/include
End of search list.
"/..//bin/ld" --hash-style=gnu --no-add-needed --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o a.out /lib/../lib64/crt1.o /lib/../lib64/crti.o /lib/../lib64/crtbegin.o -L/lib/../lib64 -L/usr/lib/../lib64 -L/lib -L/usr/lib /tmp/agsd-ad6d7d.o -L/usr/lib/ -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed crtend.o /lib/../lib64/crtn.o
/..//bin/ld: cannot find -lgcc
/..//bin/ld: cannot find -lgcc_s
clang: error: linker command failed with exit code 1 (use -v to see invocation)
I tried to run it with:
clang -L /usr/lib/gcc/x86_64-redhat-linux/8
But, it is still not working.
I installed newer llvm (6.0) and it worked fine. But, I need this 3.4 version to run a project.
So what I am trying to do is on Ubuntu 14.04 (x86_64) I want to set up musl-libc based on the latest released 1.1.11 version which is available at this moment.
What I did was to:
Install multilib support for GCC: sudo apt-get --no-install-recommends install gcc-multilib
Configure the libraries for 32-bit and 64-bit respectively and install them into separate folders:
CFLAGS=-m32 ./configure --prefix=$HOME/bin/musl-32-bit --disable-shared --target=i386-linux-gnu && make && make install
CFLAGS=-m64 ./configure --prefix=$HOME/bin/musl-64-bit --disable-shared --target=x86_64-linux-gnu
Then in order to build a statically linked premake4, I invoke GNU make like this on the Makefile generated by premake4:
make -j 8 CC=$HOME/bin/musl-32-bit/bin/musl-gcc ARCH=-m32 LDFLAGS="-v -static" verbose=1
This appears to work up to the linking step, which bombs with:
Linking Premake4
$HOME/bin/musl-32-bit/bin/musl-gcc -o bin/release/premake4 intermediate/gmake__/premake.o intermediate/gmake__/os_uuid.o intermediate/gmake__/os_pathsearch.o intermediate/gmake__/os_match.o intermediate/gmake__/os_chdir.o intermediate/gmake__/os_mkdir.o intermediate/gmake__/os_stat.o intermediate/gmake__/os_getversion.o intermediate/gmake__/premake_main.o intermediate/gmake__/os_isdir.o intermediate/gmake__/string_endswith.o intermediate/gmake__/os_isfile.o intermediate/gmake__/scripts.o intermediate/gmake__/path_isabsolute.o intermediate/gmake__/os_rmdir.o intermediate/gmake__/os_getcwd.o intermediate/gmake__/os_is64bit.o intermediate/gmake__/os_copyfile.o intermediate/gmake__/lstate.o intermediate/gmake__/ltable.o intermediate/gmake__/lgc.o intermediate/gmake__/lobject.o intermediate/gmake__/lcode.o intermediate/gmake__/lmathlib.o intermediate/gmake__/lbaselib.o intermediate/gmake__/lmem.o intermediate/gmake__/lfunc.o intermediate/gmake__/lparser.o intermediate/gmake__/ldblib.o intermediate/gmake__/lzio.o intermediate/gmake__/lstrlib.o intermediate/gmake__/lvm.o intermediate/gmake__/lauxlib.o intermediate/gmake__/llex.o intermediate/gmake__/lstring.o intermediate/gmake__/ldump.o intermediate/gmake__/ldebug.o intermediate/gmake__/loadlib.o intermediate/gmake__/lopcodes.o intermediate/gmake__/linit.o intermediate/gmake__/ldo.o intermediate/gmake__/lapi.o intermediate/gmake__/liolib.o intermediate/gmake__/loslib.o intermediate/gmake__/lundump.o intermediate/gmake__/ltm.o intermediate/gmake__/ltablib.o -v -static -L. -s -rdynamic -lm -ldl
Using built-in specs.
Reading specs from $HOME/bin/musl-32-bit/lib/musl-gcc.specs
rename spec cpp_options to old_cpp_options
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.8/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 4.8.4-2ubuntu1~14.04' --with-bugurl=file:///usr/share/doc/gcc-4.8/README.Bugs --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.8 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.8 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --disable-libmudflap --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.8-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04)
COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/
LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib32/:/lib/../lib32/:/usr/lib/../lib32/:/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-m32' '-o' 'bin/release/premake4' '-v' '-static' '-L.' '-s' '-rdynamic' '-specs=$HOME/bin/musl-32-bit/lib/musl-gcc.specs' '-mtune=generic' '-march=i686'
/usr/lib/gcc/x86_64-linux-gnu/4.8/collect2 -dynamic-linker /lib/ld-musl-i386.so.1 -nostdlib -static -export-dynamic -z relro -o bin/release/premake4 -s $HOME/bin/musl-32-bit/lib/crt1.o $HOME/bin/musl-32-bit/lib/crti.o /usr/lib/gcc/x86_64-linux-gnu/4.8/crtbegin.o -L. -L$HOME/bin/musl-32-bit/lib -L /usr/lib/gcc/x86_64-linux-gnu/4.8/. intermediate/gmake__/premake.o intermediate/gmake__/os_uuid.o intermediate/gmake__/os_pathsearch.o intermediate/gmake__/os_match.o intermediate/gmake__/os_chdir.o intermediate/gmake__/os_mkdir.o intermediate/gmake__/os_stat.o intermediate/gmake__/os_getversion.o intermediate/gmake__/premake_main.o intermediate/gmake__/os_isdir.o intermediate/gmake__/string_endswith.o intermediate/gmake__/os_isfile.o intermediate/gmake__/scripts.o intermediate/gmake__/path_isabsolute.o intermediate/gmake__/os_rmdir.o intermediate/gmake__/os_getcwd.o intermediate/gmake__/os_is64bit.o intermediate/gmake__/os_copyfile.o intermediate/gmake__/lstate.o intermediate/gmake__/ltable.o intermediate/gmake__/lgc.o intermediate/gmake__/lobject.o intermediate/gmake__/lcode.o intermediate/gmake__/lmathlib.o intermediate/gmake__/lbaselib.o intermediate/gmake__/lmem.o intermediate/gmake__/lfunc.o intermediate/gmake__/lparser.o intermediate/gmake__/ldblib.o intermediate/gmake__/lzio.o intermediate/gmake__/lstrlib.o intermediate/gmake__/lvm.o intermediate/gmake__/lauxlib.o intermediate/gmake__/llex.o intermediate/gmake__/lstring.o intermediate/gmake__/ldump.o intermediate/gmake__/ldebug.o intermediate/gmake__/loadlib.o intermediate/gmake__/lopcodes.o intermediate/gmake__/linit.o intermediate/gmake__/ldo.o intermediate/gmake__/lapi.o intermediate/gmake__/liolib.o intermediate/gmake__/loslib.o intermediate/gmake__/lundump.o intermediate/gmake__/ltm.o intermediate/gmake__/ltablib.o -lm -ldl --start-group /usr/lib/gcc/x86_64-linux-gnu/4.8/libgcc.a /usr/lib/gcc/x86_64-linux-gnu/4.8/libgcc_eh.a -lc --end-group /usr/lib/gcc/x86_64-linux-gnu/4.8/crtend.o $HOME/bin/musl-32-bit/lib/crtn.o
/usr/bin/ld: skipping incompatible $HOME/bin/musl-32-bit/lib/libc.a when searching for -lc
/usr/bin/ld: cannot find -lc
collect2: error: ld returned 1 exit status
make[1]: *** [bin/release/premake4] Error 1
make: *** [Premake4] Error 2
The relevant line is:
/usr/bin/ld: skipping incompatible $HOME/bin/musl-32-bit/lib/libc.a when searching for -lc
Now the part I don't understand about this is, that when I ar x the libc.a (into a folder $HOME/bin/musl-32-bit/lib/libc) generated during the build step of musl-libc (see above), it proves that all of the objects included seem to be of the correct target architecture (all show ELF 32-bit LSB relocatable, Intel 80386, version 1 (SYSV), not stripped) as I can prove from coming up empty when issuing the following command:
find $HOME/bin/musl-32-bit/lib -name '*.o' -exec file {} +|grep -v 'ELF 32-bit LSB relocatable, Intel 80386, version 1 (SYSV), not stripped'
And in fact this gives no output. Similarly when looking inside the build directory using the same method, I cannot find any object file that doesn't match my expectation.
For good measure I decided to also task objdump to tell me more about the libc.a in question and came up with the same result:
objdump -a $HOME/bin/musl-32-bit/lib/libc.a|grep 'file format'|grep -v 'file format elf32-i386'
So my question is twofold:
what disqualifies a static library as "incompatible" when GCC is asked to link it?
what could be the particular issue I am seeing?
The first is what I am really interested in, but with the second I am asking to share your experience with trouble-shooting like this. Which verification steps have I missed, for example?
Please note that the "native" premake4 builds just fine with:
make -j 8 CC=$HOME/bin/musl-64-bit/bin/musl-gcc ARCH=-m64 LDFLAGS=-static verbose=1
From the output when adding the -v flag to LDFLAGS it appears as if the target always stays at x86_64-linux-gnu. I have yet to come up with a method to fix this, though.
In short, the musl-gcc wrapper script setup is not well-suited to use with -m32. I think what's happening is that the actual compiler is getting invoked in the default (64-bit) mode by musl-gcc, then the resulting object files are not compatible with the (intended, 32-bit) libc.
It may work if you put -m32 in the generated wrapper script. This will happen automatically with recent versions if you put the -m32 in $CC (i.e. CC="gcc -m32") rather than putting it in $CFLAGS.
Update: As noted in the discussion that was moved to chat, adding -Wl,-melf_i386 is probably also needed (due to flaws in the spec file used by the musl-gcc wrapper that don't account for -m32 support) but still does not seem to be sufficient.
The solution
It turns out the solution is rather simple.
We need to tell both the linker and the driver to use -m32 ... I was that far before. However, it turns out that the missing piece was to pass the linker option to the driver via CFLAGS like this -Wl,-melf_i386.
I am finally able to build and link the 32-bit executable on a multilib-enabled 64-bit host.
NB: Below information is left in place for those who want to learn how I investigated the issue.
Alright, so I investigated the issue a little further and the output gets more enlightening once you extract the object files. To reproduce what I am doing you may have to use Bash or a similar shell that allows for $(...) or you need to adjust the command lines accordingly.
First off it's important to have gcc-multilib and friends installed in order to target -m32 (i386-linux-gnu which happens to be an alias for i686-linux-gnu here).
The code and the make file
I had the following make file:
CC?=$(HOME)/bin/musl/bin64/musl-gcc
BLDARCH?=-m64
CFLAGS+=-v $(BLDARCH)
LDFLAGS+=-v -static $(BLDARCH)
all: helloworld
helloworld: helloworld.c
clean:
rm -f helloworld
rebuild: clean all
.PHONY: clean rebuild
.NOTPARALLEL: rebuild
and the following small helloworld.c:
#include <stdio.h>
int main(int argc, char** argv)
{
printf("Hello world!\n");
return 0;
}
and my 32-bit musl-libc was installed to $HOME/bin/musl/{bin,include,lib}32 respectively. The 64-bit one was installed to $HOME/bin/musl/{bin,include,lib}64 respectively.
Attempting to build with:
make CC=$HOME/bin/musl/bin32/musl-gcc BLDARCH=-m32 rebuild
always failed with the same meaningless lines:
/usr/bin/ld: skipping incompatible ~/bin/musl/lib32/libc.a when searching for -lc
/usr/bin/ld: cannot find -lc
collect2: error: ld returned 1 exit status
make: *** [helloworld] Error 1
Digging into the details
So after some contemplation I decided to re-do the steps done by the gcc driver manually.
This meant to run roughly (I replaced all occurrences of my home folder with a ~):
Compile: /usr/lib/gcc/x86_64-linux-gnu/4.8/cc1 -quiet -imultilib 32 -imultiarch i386-linux-gnu helloworld.c -nostdinc -isystem ~/bin/musl/include32 -isystem /usr/lib/gcc/x86_64-linux-gnu/4.8/include -quiet -dumpbase helloworld.c -m32 -mtune=generic -march=i686 -auxbase helloworld -version -fstack-protector -Wformat -Wformat-security -o /tmp/ccGmMuR1.s
Assemble: as -v -v --32 -o /tmp/ccgRGlqf.o /tmp/ccGmMuR1.s
Link: env COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/ LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.8/32/:/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib32/:/lib/../lib32/:/usr/lib/../lib32/:/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../:/lib/:/usr/lib/ COLLECT_GCC_OPTIONS="-v -v -static -m32 -o helloworld -specs=~/bin/musl/lib32/musl-gcc.specs -mtune=generic -march=i686" /usr/lib/gcc/x86_64-linux-gnu/4.8/collect2 -dynamic-linker /lib/ld-musl-i386.so.1 -nostdlib -static -z relro -o helloworld ~/bin/musl/lib32/crt1.o ~/bin/musl/lib32/crti.o /usr/lib/gcc/x86_64-linux-gnu/4.8/32/crtbegin.o -L~/bin/musl/lib32 -L /usr/lib/gcc/x86_64-linux-gnu/4.8/32/. /tmp/ccpL09mJ.o --start-group /usr/lib/gcc/x86_64-linux-gnu/4.8/32/libgcc.a /usr/lib/gcc/x86_64-linux-gnu/4.8/32/libgcc_eh.a -lc --end-group /usr/lib/gcc/x86_64-linux-gnu/4.8/32/crtend.o ~/bin/musl/lib32/crtn.o
Obviously this didn't change the error message a bit just yet:
/usr/bin/ld: skipping incompatible ~/bin/musl/lib32/libc.a when searching for -lc
/usr/bin/ld: cannot find -lc
collect2: error: ld returned 1 exit status
So I took out the -lc from the command line for collect2 and created a folder ~/bin/musl/lib32/archive into which I extracted the whole libc.a generated by my musl-libc build attempt. I then instructed collect2 where to find the object files like this:
Link: env COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/ LIBRARY_PATH=/usr/lib/gc
c/x86_64-linux-gnu/4.8/32/:/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib32/:/lib/../lib32/:/usr/lib/../lib32/:/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../:/lib/:/usr/lib/ COLLECT_GCC_OPTIONS="-v -v -
static -m32 -o helloworld -specs=~/bin/musl/lib32/musl-gcc.specs -mtune=generic -march=i686" /usr/lib/gcc/x86_64-linux-gnu/4.8/collect2 -dynamic-linker /lib/ld-musl-i386.so.1 -nostdlib -static -z relro -o helloworld $HOME/bin/musl/lib32/crt1.o ~/bin/musl/lib32/crti.o /usr/lib/gcc/x86_64-linux-gnu/4.8/32/crtbegin.o -L~/bin/musl/lib32 -L /usr/lib/gcc/x86_64-linux-gnu/4.8/32/. /tmp/ccpL09mJ.o --start-group /usr/lib/gcc/x86_64-linux-g
nu/4.8/32/libgcc.a /usr/lib/gcc/x86_64-linux-gnu/4.8/32/libgcc_eh.a --end-group /usr/lib/gcc/x86_64-linux-gnu/4.8/32/crtend.o ~/bin/musl/lib32/crtn.o $(find ~/bin/musl/lib32/archive -type f -name '*.o')
This gist is taking out -lc and appending $(find ~/bin/musl/lib32/archive -type f -name '*.o').
Which gave me a whole bunch of new, but more meaningful errors similar to the following ones:
/usr/bin/ld: Warning: size of symbol `__init_ssp' changed from 1 in ~/bin/musl/lib32/archive/__libc_start_main.o to 65 in ~/bin/musl/lib32/archive/__stack_chk_fail.o
/usr/bin/ld: Warning: size of symbol `__funcs_on_exit' changed from 126 in ~/bin/musl/lib32/archive/atexit.o to 1 in ~/bin/musl/lib32/archive/exit.o
# more of those
/usr/bin/ld: i386 architecture of input file `~/bin/musl/lib32/crt1.o' is incompatible with i386:x86-64 output
/usr/bin/ld: i386 architecture of input file `~/bin/musl/lib32/crti.o' is incompatible with i386:x86-64 output
# more of those
The plot thickens. Apparently the collect2 command gets the wrong idea about what to build. Which is odd considering the output for the governing environment variables:
COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/
LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.8/32/:/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib32/:/lib/../lib32/:/usr/lib/../lib32/:/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-v' '-v' '-static' '-m32' '-o' 'helloworld' '-specs=~/bin/musl/lib32/musl-gcc.specs' '-mtune=generic' '-march=i686'
... which I passed using env in my attempt to reproduce the conditions encountered by the collect2 wrapper when linking the libc.a.
In order to find out more about the internals of the GNU compilers one needs to read https://gcc.gnu.org/onlinedocs/gccint/
Unfortunately the part about collect2 doesn't help us here. But the lengthy output of /usr/lib/gcc/x86_64-linux-gnu/4.8/collect2 --help looks promising.
Just how could we smuggle our command line option to collect2?
For now I was going to settle for whatever I could run manually. So I attempted to tell the linker which output format I expected. Based on the list of supported targets I was interested in elf_i386.
Passing -melf_386 at the end of the previous line gave an interesting error output. Numerous referenced functions such as __vsyscall, __moddi3 and __divdi3 were undefined. That indicated they simply didn't exist in the object files from the static lib or in any of the startup .o files for that matter:
~/bin/musl/lib32/archive/aio.o: In function `cleanup':
aio.c:(.text+0x5ad): undefined reference to `__vsyscall'
aio.c:(.text+0x5bb): undefined reference to `__vsyscall'
aio.c:(.text+0x5e8): undefined reference to `__vsyscall'
aio.c:(.text+0x5f6): undefined reference to `__vsyscall'
aio.c:(.text+0x61d): undefined reference to `__vsyscall'
~/bin/musl/lib32/archive/aio.o:aio.c:(.text+0x62b): more undefined references to `__vsyscall' follow
~/bin/musl/lib32/archive/cpow.o: In function `cpow':
cpow.c:(.text+0x4f): undefined reference to `__mulxc3'
~/bin/musl/lib32/archive/cpowf.o: In function `cpowf':
cpowf.c:(.text+0x47): undefined reference to `__mulxc3'
~/bin/musl/lib32/archive/cpowl.o: In function `cpowl':
cpowl.c:(.text+0x4c): undefined reference to `__mulxc3'
~/bin/musl/lib32/archive/sysconf.o: In function `sysconf':
sysconf.c:(.text+0xcc): undefined reference to `__vsyscall'
~/bin/musl/lib32/archive/__getdents.o: In function `__getdents':
__getdents.c:(.text+0x13): undefined reference to `__vsyscall'
~/bin/musl/lib32/archive/opendir.o: In function `opendir':
opendir.c:(.text+0x37): undefined reference to `__vsyscall'
~/bin/musl/lib32/archive/readdir.o: In function `readdir':
readdir.c:(.text+0x1f): undefined reference to `__vsyscall'
~/bin/musl/lib32/archive/__init_tls.o: In function `__init_tls':
__init_tls.c:(.text+0x136): undefined reference to `__vsyscall6'
__init_tls.c:(.text+0x16e): undefined reference to `__vsyscall'
As I had already clarified in my question, the object files from the archive all stated that they were elf32-i386.
The functions __vsyscall and __vsyscall6 should end up in a file called syscall.o given the source file for i386 in musl-libc: src/internal/i386/syscall.s. Let's verify that first. Since there is also a file src/misc/syscall.c the name might be different though. Four files have syscall in the file name:
__syscall_cp.o
syscall_cp.o
syscall.o
syscall_ret.o
Querying those files using nm gave:
$ nm -s $(ls |grep syscall)
__syscall_cp.o:
00000000 t sccp
U __syscall
00000005 T __syscall_cp
00000000 W __syscall_cp_c
syscall_cp.o:
U __cancel
00000008 T __cp_begin
00000035 T __cp_cancel
00000030 T __cp_end
00000000 T __syscall_cp_asm
syscall.o:
00000000 T syscall
U __syscall_ret
U __vsyscall6
syscall_ret.o:
U __errno_location
00000000 T __syscall_ret
Symbols with the symbol type U are undefined and therefore expected by the linker to come from outside (external to each object file).
A final $ nm --defined-only *.o ../*.o|grep vsyscall was what was needed to verify that those symbols were indeed missing from the libc.a.
So the cross-built libc.a what was faulty after all. Back to the drawing board.
I hope this description helps others to figure out similar issues and look behind the scenes in GCC.
The saga continues
I was really surprised to see:
$ nm --defined-only ../libc.a |grep -B 2 vsyscall
syscall.o:
0000004b T __syscall
00000000 T __vsyscall
00000031 T __vsyscall6
but for the extracted object files the corresponding command (nm --defined-only *.o ../*.o|grep -B 2 vsyscall) would yield no output.
So inside the libc.a somehow nm sees the two symbols, but after extracting them they disappear? Odd.
Let's look for syscall.o in the libc.a:
$ nm ../libc.a |grep ^syscall
syscall.o:
syscall_ret.o:
syscall.o:
syscall_cp.o:
Whoa? So syscall.o exists twice inside the static library? Well that looks like it may just be the error cause we're looking for. And it certainly does explain why the symbols disappear. Likely the latter syscall.o overwrites the one first extracted when running ar x ....
Confirming:
$ nm ../libc.a |grep -A 4 ^syscall\.o
syscall.o:
0000004b T __syscall
U __sysinfo
00000000 T __vsyscall
00000031 T __vsyscall6
--
syscall.o:
00000000 T syscall
U __syscall_ret
U __vsyscall6
and looking into the musl-libc source tree after doing the 32-bit build:
$ find . -type f -name 'syscall.o' -exec nm {} +
./src/internal/syscall.o:
0000004b T __syscall
U __sysinfo
00000000 T __vsyscall
00000031 T __vsyscall6
./src/misc/syscall.o:
00000000 T syscall
U __syscall_ret
U __vsyscall6
Copying the former into the lib32/archive directory under a name that doesn't collide with existing names gives more errors on other functions, suggesting other object files may also exist as duplicates inside the generated libc.a.
Which ones are duplicates?
$ diff <(nm libc.a|grep ':$'|cut -f 1 -d :|sort) <(nm libc.a|grep ':$'|cut -f 1 -d :|sort -u)
--- /dev/fd/63 2015-10-05 23:58:53.683804823 +0000
+++ /dev/fd/62 2015-10-05 23:58:53.683804823 +0000
## -131,7 +131,6 ##
clogl.o
clog.o
clone.o
-clone.o
closedir.o
close.o
cnd_broadcast.o
## -1115,7 +1114,6 ##
__syscall_cp.o
syscall_cp.o
syscall.o
-syscall.o
syscall_ret.o
sysconf.o
sysinfo.o
This way we see clone.o and syscall.o are affected as duplicates. Which indicates that some object files are missing altogether from the libc.a given undefined references to the following symbols:
__divdi3
__moddi3
__mulxc3
__tls_get_new
__udivdi3
__umoddi3
These names happen to coincide with the ones from the Integer library routines listed for GCC. Which makes me think I am missing one library provided by GCC which to link. Like libgcc?! ...
I have package lib32gcc-4.8-dev which means I should have the required file:
$ apt-file list lib32gcc-4.8-dev|grep -E 'libgcc.*\.a'
lib32gcc-4.8-dev: /usr/lib/gcc/x86_64-linux-gnu/4.8/32/libgcc.a
lib32gcc-4.8-dev: /usr/lib/gcc/x86_64-linux-gnu/4.8/32/libgcc_eh.a
After the previous step I decided to set up a x86_32 version on Ubuntu 14.04 that was essentially at the same patch level.
I then compared the libgcc.a from the x86_32 machine with that from the x86_64 machine. They turned out to be almost identical except for the "symbol value" (in nm lingo) of a handful of functions.
Also, since the symbols were in the static library I attempted to link against the static library again. This worked only with -melf_i386 on the linker (collect2) command line.
After trying to use LDFLAGS and noticing that those were also passed to cc1, I set appended -Wl,-melf_i386 to the CFLAGS and now it worked. Brilliant.
As a side-note: I also swapped the /usr/lib/gcc/x86_64-linux-gnu/4.8/collect2 in the command line for ld of which collect2 is supposed to be a wrapper. The error output was identical.
I have a C++ library that is being cross-compiled by Gradle on a Mac for Windows, using mingw-w64. I get a "bad -rpath option" when the linker tries to link the binaries.
There are a complicated linker search paths in the debug output below that I have removed for clarity, but they are all ok. However, there is one part which seems to be wrong - it looks like something is being added to the search paths by mistake.
:linkWindows-x8664DebugTestSharedLibrary
Using built-in specs.
COLLECTGCC=/usr/local/bin/mingw-w64/mingw-w64-x8664/mingw/bin/g++
Target: x8664-w64-mingw32
Configured with: ../source/gcc-4.8.3/configure --build=x8664-apple-darwin13.2.0 --target=x8664-w64-mingw32 --disable-nls --disable-multilib --prefix=/usr/local/bin/mingw-w64/mingw-w64-x8664 --with-sysroot=/usr/local/bin/mingw-w64/mingw-w64-x8664 --with-mpc=/usr/local/bin/mingw-w64/pkgs/mpc/mpc-1.0.2-x8664 --with-mpfr=/usr/local/bin/mingw-w64/pkgs/mpfr/mpfr-3.1.2-x8664 --with-gmp=/usr/local/bin/mingw-w64/pkgs/gmp/gmp-6.0.0a-x8664 --with-host-libstdcxx='-lstdc++ ' --with-cloog=/usr/local/bin/mingw-w64/pkgs/cloog/cloog-0.18.0-x8664 --with-isl=/usr/local/bin/mingw-w64/pkgs/isl/isl-0.11.1-x8664 --enable-languages=c,c++ --enable-threads=win32 --enable-fully-dynamic-string --enable-lto
Thread model: win32
gcc version 4.8.3 (GCC)
COMPILERPATH=/usr/local/bin/mingw-w64/mingw-w64-x8664/x8664-w64-mingw32/bin/../lib/gcc/x8664-w64-mingw32/4.8.3/../../../../x8664-w64-mingw32/bin/
LIBRARYPATH=/usr/local/bin/mingw-w64/mingw-w64-x8664/x8664-w64-mingw32/bin/../lib/gcc/x8664-w64-mingw32/4.8.3/:/usr/local/bin/mingw-w64/mingw-w64-x8664/x8664-w64-mingw32/bin/../lib/gcc/:/usr/local/bin/mingw-w64/mingw-w64-x8664/x8664-w64-mingw32/bin/../lib/gcc/x8664-w64-mingw32/4.8.3/../../../../x8664-w64-mingw32/lib/../lib/:/usr/local/bin/mingw-w64/mingw-w64-x8664/mingw/lib/../lib/:/usr/local/bin/mingw-w64/mingw-w64-x8664/x8664-w64-mingw32/bin/../lib/gcc/x8664-w64-mingw32/4.8.3/../../../../x8664-w64-mingw32/lib/:/usr/local/bin/mingw-w64/mingw-w64-x8664/mingw/lib/
COLLECTGCCOPTIONS='-shared' '-o' '/Users/john/git/test/build/binaries/testSharedLibrary/windows-x8664Debug/libtest.dylib' '-L/Users/john/git/test/build/nar-dependencies/compiletestWindows-x8664Debug/lib' '-v' '-fno-use-linker-plugin' '-shared-libgcc' '-mtune=generic' '-march=x86-64'
/usr/local/bin/mingw-w64/mingw-w64-x8664/x8664-w64-mingw32/bin/../lib/gcc/x8664-w64-mingw32/4.8.3/../../../../x8664-w64-mingw32/bin/ld --sysroot=/usr/local/bin/mingw-w64/mingw-w64-x8664 -m i386pep --shared -Bdynamic -e DllMainCRTStartup --enable-auto-image-base -o /Users/john/git/test/build/binaries/testSharedLibrary/windows-x8664Debug/libtest.dylib /usr/local/bin/mingw-w64/mingw-w64-x8664/x8664-w64-mingw32/bin/../lib/gcc/x8664-w64-mingw32/4.8.3/../../../../x8664-w64-mingw32/lib/../lib/dllcrt2.o /usr/local/bin/mingw-w64/mingw-w64-x8664/x8664-w64-mingw32/bin/../lib/gcc/x8664-w64-mingw32/4.8.3/../../../../x8664-w64-mingw32/lib/../lib/crtbegin.o -LseriesOfSearchFoldersHereEachWithMinusLFlag **#/var/folders/75/fsklv28523d2gx1wth3rc7y0000101/T//ccOUlRAe** -lmingw32 -lgccs -lgcc -lmoldname -lmingwex -lmsvcrt -ladvapi32 -lshell32 -luser32 -lkernel32 -lmingw32 -lgccs -lgcc -lmoldname -lmingwex -lmsvcrt /usr/local/bin/mingw-w64/mingw-w64-x8664/x8664-w64-mingw32/bin/../lib/gcc/x8664-w64-mingw32/4.8.3/../../../../x8664-w64-mingw32/lib/../lib/crtend.o
Error message reads:
/usr/local/bin/mingw-w64/mingw-w64-x8664/x8664-w64-mingw32/bin/../lib/gcc/x8664-w64-mingw32/4.8.3/../../../../x8664-w64-mingw32/bin/ld: bad -rpath option
FAILED
This appears to be the offending bit that is causing the problem: #/var/folders/75/fsklv28523d2gx1wth3rc7y0000101/T//ccOUlRAe
What is that and why is it there?
Did you already solve it?
Most probably the paths to search for libraries is generated by Gradle Native as well. I have no clue about the directory structure. Which version of Gradle are you using? Did you read the user guide about native binaries? Are you able to share the gradle file? (As native support is incubating, it is likely to change quite a bit. It did for me, over the past versions)
I'm trying to generate code coverage files for a small C program compiled with clang on Debian Linux. Here's what I've done:
neuron#debian:~/temp$ ls
main.c test.c test.h
neuron#debian:~/temp$ clang *.c
neuron#debian:~/temp$ ./a.out
0
This is exactly as expected, I can compile and run things. Now trying to enable coverage.
neuron#debian:~/temp$ clang --coverage *.c
/usr/bin/ld: cannot find /usr/bin/../lib/libprofile_rt.a: No such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Trying to include the library for linking.
neuron#debian:~/temp$ clang --coverage -lprofile_rt *.c
/usr/bin/ld: cannot find -lprofile_rt
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Finding the library:
neuron#debian:~/temp$ find / -name \*profile_rt\* 2>/dev/null
/usr/lib/llvm-3.0/lib/libprofile_rt.so
/usr/lib/llvm-3.0/lib/libprofile_rt.a
neuron#debian:~/temp$ clang --coverage -lprofile_rt -L/usr/lib/llvm-3.0/lib *.c
/usr/bin/ld: cannot find /usr/bin/../lib/libprofile_rt.a: No such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Here's more verbose output of the last command: http://pastie.org/8468331. What concerns me there:
the linker uses tons of gcc libraries to link with (though this may be a result of llvm not having it's own binunitls);
profiling library is being searched for at /usr/bin/../lib/libprofile_rt.a instead of the path I provided.
If we pass the arguments to the linker the output is the same:
neuron#debian:~/temp$ clang --coverage -Wl,-L/usr/lib/llvm-3.0/lib *.c -lprofile_rt
/usr/bin/ld: cannot find /usr/bin/../lib/libprofile_rt.a: No such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation)
What do I do wrong?
Try changing the order of your link line from
clang --coverage -lprofile_rt -L/usr/lib/llvm-3.0/lib *.c
to
clang --coverage -L/usr/lib/llvm-3.0/lib *.c -lprofile_rt
Ok, doesn't seem like the linker is getting your -L for some reason. Maybe try
clang --coverage -Wl,L/usr/lib/llvm-3.0/lib *.c -lprofile_rt
The only way I've been able to resolve this issue is by creating a symbolic link to where LLVM/clang is looking for the library. I'm thinking this is a way for package maintainers to manage which feature-library is used by the host system.
ln -s /usr/lib/llvm-3.0/lib/libprofile_rt.a /usr/lib/libprofile_rt.a
Coverage, and other optional -f<***> profile features work as expected. I can verify this by adding the verbose -v switch.
Debian clang version 3.0-6.2 (tags/RELEASE_30/final) (based on LLVM 3.0)
Target: arm-unknown-linux-gnueabihf
Thread model: posix
"/usr/bin/clang" -cc1 -triple armv4t-unknown-linux-gnueabihf -S -disable-free -disable-llvm-verifier -main-file-name example.c -mrelocation-model static -mdisable-fp-elim -mconstructor-aliases -target-abi apcs-gnu -target-cpu arm1136jf-s -mfloat-abi hard -target-linker-version 2.22 -momit-leaf-frame-pointer -v -femit-coverage-notes -femit-coverage-data -resource-dir /usr/bin/../lib/clang/3.0 -fmodule-cache-path /var/tmp/clang-module-cache -internal-isystem /usr/local/include -internal-isystem /usr/bin/../lib/clang/3.0/include -internal-externc-isystem /usr/include/arm-linux-gnueabihf -internal-externc-isystem /usr/include -ferror-limit 19 -fmessage-length 130 -fno-signed-char -fgnu-runtime -fobjc-runtime-has-arc -fobjc-runtime-has-weak -fobjc-fragile-abi -fdiagnostics-show-option -fcolor-diagnostics -o /tmp/example-lLKOP1.s -x c example.c
clang -cc1 version 3.0 based upon llvm 3.0 hosted on arm-unknown-linux-gnueabihf
ignoring nonexistent directory "/usr/bin/../lib/clang/3.0/include"
ignoring nonexistent directory "/usr/bin/../lib/clang/3.0/include"
ignoring duplicate directory "/usr/local/include"
ignoring duplicate directory "/usr/include/arm-linux-gnueabihf"
ignoring duplicate directory "/usr/include/arm-linux-gnueabihf"
ignoring duplicate directory "/usr/include/arm-linux-gnueabihf"
ignoring duplicate directory "/usr/include"
#include "..." search starts here:
#include <...> search starts here:
/usr/local/include
/usr/include/arm-linux-gnueabihf
/usr/include
/usr/include/clang/3.0/include/
/usr/lib/gcc/arm-linux-gnueabihf/4.6/include/
/usr/lib/gcc/arm-linux-gnueabihf/4.6/include-fixed/
End of search list.
"/usr/bin/as" -o /tmp/example-WbJHFT.o /tmp/example-lLKOP1.s
"/usr/bin/ld" -X --hash-style=both --build-id --eh-frame-hdr -m armelf_linux_eabi -dynamic-linker /lib/ld-linux-armhf.so.3 -o example.o /usr/lib/gcc/arm-linux-gnueabihf/4.6/../../../arm-linux-gnueabihf/crt1.o /usr/lib/gcc/arm-linux-gnueabihf/4.6/../../../arm-linux-gnueabihf/crti.o /usr/lib/gcc/arm-linux-gnueabihf/4.6/crtbegin.o -L/usr/lib/gcc/arm-linux-gnueabihf/4.6/../../../arm-linux-gnueabihf -L/lib/arm-linux-gnueabihf -L/usr/lib/arm-linux-gnueabihf -L/usr/lib/gcc/arm-linux-gnueabihf/4.6 -L/usr/lib/gcc/arm-linux-gnueabihf/4.6/../../../arm-linux-gnueabihf -L/usr/lib/gcc/arm-linux-gnueabihf/4.6/../../.. -L/lib/arm-linux-gnueabihf -L/lib -L/usr/lib/arm-linux-gnueabihf -L/usr/lib /tmp/example-WbJHFT.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed \
/usr/lib/gcc/arm-linux-gnueabihf/4.6/crtend.o \
/usr/lib/gcc/arm-linux-gnueabihf/4.6/../../../arm-linux-gnueabihf/crtn.o \
/usr/bin/../lib/libprofile_rt.a
I recently compiled Clang 2.9 (from here) on an Ubuntu system. I'm having trouble compiling and linking a simple program. Does anyone know how to fix this? Here's the output I see:
a#ubuntu:~/Desktop$ clang add.c
/usr/bin/ld: 1: Syntax error: word unexpected (expecting ")")
clang: error: linker command failed with exit code 2 (use -v to see invocation)
uname -a output
Linux ubuntu 2.6.38-8-generic #42-Ubuntu SMP Mon Apr 11 03:31:50 UTC 2011 i686 i686 i386 GNU/Linux
gcc -v output:
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/i386-linux-gnu/gcc/i686-linux-gnu/4.5.2/lto-wrapper
Target: i686-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.5.2-8ubuntu4 --with-bugurl=file:///usr/share/doc/gcc-4.5/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.5 --enable-shared --enable-multiarch --with-multiarch-defaults=i386-linux-gnu --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib/i386-linux-gnu --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.5 --libdir=/usr/lib/i386-linux-gnu --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-plugin --enable-gold --enable-ld=default --with-plugin-ld=ld.gold --enable-objc-gc --enable-targets=all --disable-werror --with-arch-32=i686 --with-tune=generic --enable-checking=release --build=i686-linux-gnu --host=i686-linux-gnu --target=i686-linux-gnu
Thread model: posix
gcc version 4.5.2 (Ubuntu/Linaro 4.5.2-8ubuntu4)
clang verbose ouput
clang version 2.9 (tags/RELEASE_29/final)
Target: i386-pc-linux-gnu
Thread model: posix
"/usr/local/bin/clang" -cc1 -triple i386-pc-linux-gnu -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -main-file-name add.c -mrelocation-model static -mdisable-fp-elim -masm-verbose -mconstructor-aliases -target-cpu pentium4 -target-linker-version 2.21.0.20110327 -momit-leaf-frame-pointer -v -resource-dir /usr/local/bin/../lib/clang/2.9 -ferror-limit 19 -fmessage-length 138 -fgnu-runtime -fdiagnostics-show-option -fcolor-diagnostics -o /tmp/cc-JJruTv.o -x c add.c
clang -cc1 version 2.9 based upon llvm 2.9 hosted on i386-pc-linux-gnu
#include "..." search starts here:
#include <...> search starts here:
/usr/local/include
/usr/local/bin/../lib/clang/2.9/include
/usr/include
End of search list.
"/usr/bin/ld" --eh-frame-hdr -m elf_i386 -dynamic-linker /lib/ld-linux.so.2 -o a.out crt1.o crti.o crtbegin.o -L -L/../../.. /tmp/cc-JJruTv.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed crtend.o crtn.o
/usr/bin/ld: 1: Syntax error: word unexpected (expecting ")")
clang: error: linker command failed with exit code 2 (use -v to see invocation)
Sounds like a problem in your linker script. But the ld line above does not contain a specific linker script so the default one should be used (to see it simply type ld -v).
-L -L/../../..
that does look a bit weird. Don't see it on your original line though so not sure where it came from. Perhaps your installation is broken?
Do a gcc -v add.c and compare the lines invoking /usr/bin/ld. Perhaps add it to the question.