Linking AVR programs with Cargo - linker

I have a Rust project which I am currently compiling and linking by hand:
rustc --target=avr-atmel-none src/main.rs --emit=obj -o _build/main.rs.o -C opt-level=3
avr-gcc -Os -Wl,--gc-sections -mmcu=atmega328p -o _build/image.elf _build/main.rs.o
avr-objcopy -Oihex -R.eeprom _build/image.elf _build/image.hex
I would like to automate this with Cargo, so I started by setting avr-gcc as the linker, by adding the following to .cargo/config:
[build]
target = "avr-atmel-none"
[target.avr-atmel-none]
linker = "avr-gcc"
However, it seems cargo passes some extra arguments to the linker that avr-gcc cannot handle:
11:47:10 [cactus#galaxy interrupt-bug]$ cargo build --release
Compiling hello-avr v0.1.0 (file:///home/cactus/prog/rust/avr/interrupt-bug)
error: linking with `avr-gcc` failed: exit code: 1
|
= note: "avr-gcc" "-Wl,--as-needed" "-L" "/home/cactus/prog/rust/rust-avr/build/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/avr-atmel-none/lib" "/home/cactus/prog/rust/avr/interrupt-bug/target/avr-atmel-none/release/deps/hello_avr-8bce8eb24807f5a8.0.o" "-o" "/home/cactus/prog/rust/avr/interrupt-bug/target/avr-atmel-none/release/deps/hello_avr-8bce8eb24807f5a8" "-Wl,--gc-sections" "-pie" "-Wl,-O1" "-nodefaultlibs" "-L" "/home/cactus/prog/rust/avr/interrupt-bug/target/avr-atmel-none/release/deps" "-L" "/home/cactus/prog/rust/avr/interrupt-bug/target/release/deps" "-L" "/home/cactus/prog/rust/rust-avr/build/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/avr-atmel-none/lib"
= note: /usr/lib/gcc/avr/4.8.2/../../../avr/bin/ld: -pie not supported
collect2: error: ld returned 1 exit status
How do I remove these extra arguments from the avr-gcc invocation? Moreover, is there a way to integrate the third step, i.e. the avr-objcopy call, into the Cargo workflow?

A word of warning: development on avr-rust can politely be said to be cutting edge. It's very likely something that works one day may not the next, so answers like this are likely to go out of date quickly. We welcome all contributors to the project to help make it more usable.
You need to specify a target JSON file and the full set of linker arguments. Here's an example from an older project of mine (some of the exact values may now be incorrect):
{
"llvm-target": "avr-atmel-none",
"target-endian": "little",
"target-pointer-width": "16",
"os": "none",
"target-env": "gnu",
"target-vendor": "unknown",
"arch": "avr",
"data-layout": "e-p:16:16:16-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-n8",
"executables": true,
"linker": "avr-gcc",
"linker-flavor": "gcc",
"pre-link-args": {
"gcc": ["-mmcu=atmega328p", "-nostartfiles", "../interrupt_vector.S"]
},
"exe-suffix": ".elf",
"post-link-args": {
"gcc": ["-Wl,--no-gc-sections"]
},
"no-compiler-rt": true
}
For a full example see my example repository. This project used to work (see my blog series). I have recently updated it so it compiles against the master branch of avr-rust, but haven't tested the compiled code on a real device.
There's an open RFC for having Cargo post build scripts, but it doesn't seem likely to be merged. I continue to use a Makefile. xargo might be an alternative. There were also rumbles about a cargo subcommand that could be created.

Related

HiGHS fails to link with "undefined reference" errors

On Arch Linux x86_64, Rust 1.66.0 stable, g++ 12.2.0.
I have a problem linking the good-lp crate, and narrowed it down to its indirect dependency highs-sys, even outside my own code.
This happens on a clean checkout of the latest master (c785240), as well as v1.2.2, as well as v0.2.1 (which I used with great success for months through good-lp). So it isn't a new problem in highs-sys, and might very well be a problem with my toolchain.
Versions:
$ cargo --version
cargo 1.66.0 (d65d197ad 2022-11-15)
$ rustc --version
rustc 1.66.0 (69f9c33d7 2022-12-12)
$ cc --version
cc (GCC) 12.2.0
...
$ ld --version
GNU ld (GNU Binutils) 2.39.0
...
Cloning and building:
$ git clone --recursive https://github.com/rust-or/highs-sys
$ cd highs-sys
$ cargo build
...
Finished dev [unoptimized + debuginfo] target(s) in 3m 35s
$ cargo test --lib
Finished test [unoptimized + debuginfo] target(s) in 0.04s
Running unittests src/lib.rs (target/debug/deps/highs_sys-96b0b941fcc31775)
running 2 tests
test bindgen_test_layout___fsid_t ... ok
test bindgen_test_layout_imaxdiv_t ... ok
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
All fine so far. The problem happens when we try to use the generated library, even with its own integration tests:
$ cargo test --test test_highs_call
Compiling highs-sys v1.2.2 (/tmp/highs-sys)
error: linking with `cc` failed: exit status: 1
|
= note: "cc" "-m64" "/tmp/rustccbaKBt/symbols.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.110gvlokoswtomfn.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.13y9v90q7e3dvfo2.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.17wo7aophf8in4vn.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.1cdkl7c2apbn7crn.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.1dj1sbwseam0n9ft.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.1hgigplg6qv30idz.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.1ms9q0du418ic26n.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.1tcgu3qw79kuuw1h.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.1vs3h8jyr2rojt51.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.20xdzk038hc30h4j.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.25ph4lcxn1xxn6fn.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.27am5w6h2agsxcl7.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.2djhnnv2jm2ibcl3.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.2i1j9l2lu5t0k0vu.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.2meo5gq21h6h8bqe.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.2ol9s57flfu167qp.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.2s2g91x597h8e2ey.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.2v1cfn4oct7skc3q.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.2v9apdn2isgwdl5q.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.2wpeqt6tzxghqi56.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.2yk7p7w6ixraj427.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.2z4b238c1zbjnij6.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.2zlr9mnhg9hyf02g.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.32cfoccwtdvhq06x.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.33nztruk7k2tts8x.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.3cvdzi91jork7mds.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.3fb9xf8nxxzcec41.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.3is6rgf4g5g16vl3.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.3s3dhxd64ny5tive.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.3wn5g0h8q5u3bxky.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.4150s6y93i5ma35t.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.4335j389sygn8m99.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.44i1r6id1s31acf1.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.4injouqsnun7hcvt.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.4nb66ozge3nmeuwq.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.4p2li2uv9dxcrx2d.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.4pz1m3mlzi8imskl.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.4tg55omun1u77yh3.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.4thto7zkdscqxses.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.4vrsyb4og25lcxo8.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.4z0e0rsh1apo67mr.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.50cxv68n1h5055a5.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.515xy9o2bj3al2ry.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.59no2g8u5zmbleot.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.5b574862nrupsals.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.5fu18wwg3p5nmrsh.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.5gi5ablibidmtrpd.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.kjht10drvblu37w.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.ugtef95uy6fok7q.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.xcjfvort5vhydtl.rcgu.o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.3mrzemp57wngq71t.rcgu.o" "-Wl,--as-needed" "-L" "/tmp/highs-sys/target/debug/deps" "-L" "/tmp/highs-sys/target/debug/build/highs-sys-21b14ce07e65289c/out/lib" "-L" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libtest-5678a62cd2e949f2.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgetopts-cafb712826ee2a26.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunicode_width-bb4ac1585c9c1153.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_std-69fab17bbe87f87e.rlib" "/tmp/highs-sys/target/debug/deps/libhighs_sys-addd17046a23d2a8.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-a11e3ca400b3ed09.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-3e82a3fced649488.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-53a4330185981bcb.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-2a8b57667b4852b5.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-9370462deca12c5a.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-7da763b8d3620472.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-5bde27582a7f5af7.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-1204e05b2d47e3d7.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-43987de2766b6923.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-d6499a0705316aa5.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-c9a27c90d8fbf11e.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-8f159929cbfdfaf1.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-d2f1e8f3bb5cba95.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-9862f486269f442f.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-0434381f2f012ae2.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-68549403a59fd02e.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-4cefb2045f924a5b.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-272615fc4f10c50d.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-860619b93700e7eb.rlib" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-b73e5b4656934876.rlib" "-Wl,-Bdynamic" "-lstdc++" "-lgomp" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L" "/home/thomas/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro,-znow" "-nodefaultlibs"
= note: /usr/bin/ld: /tmp/highs-sys/target/debug/deps/test_highs_call-e669101ae4108d78.5gi5ablibidmtrpd.rcgu.o: in function `test_highs_call::highs_call':
/tmp/highs-sys/tests/test_highs_call.rs:88: undefined reference to `Highs_lpCall'
collect2: error: ld returned 1 exit status
= note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
= note: use the `-l` flag to specify native libraries to link
= note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)
error: could not compile `highs-sys` due to previous error
Similar output happens for the other integration test, test_highs_functions, but with undefined references to all Highs_* functions used in that file.
Verbose build output:
$ cargo build --verbose --test test_highs_call
...
Compiling highs-sys v1.2.2 (/tmp/highs-sys)
Running `rustc --crate-name test_highs_call --edition=2018 tests/test_highs_call.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 --test -C metadata=e669101ae4108d78 -C extra-filename=-e669101ae4108d78 --out-dir /tmp/highs-sys/target/debug/deps -C incremental=/tmp/highs-sys/target/debug/incremental -L dependency=/tmp/highs-sys/target/debug/deps --extern highs_sys=/tmp/highs-sys/target/debug/deps/libhighs_sys-addd17046a23d2a8.rlib -L native=/tmp/highs-sys/target/debug/build/highs-sys-21b14ce07e65289c/out/lib`
...
... and then the above error again
That rustc command, when run manually, also fails with the same error.
Running the cc command manually fails due to /tmp/rustccbaKBt/symbols.o having been removed, but if I remove that from the command line, the same error occurs again.
The symbol is mentioned as undefined (U) in this file:
$ nm target/debug/deps/test_highs_call-e669101ae4108d78.5gi5ablibidmtrpd.rcgu.o | grep Highs_lpCall
U Highs_lpCall
And it gets defined (T) in this one, which occurs later on the command line, as it should:
$ nm /tmp/highs-sys/target/debug/deps/libhighs_sys-addd17046a23d2a8.rlib | grep Highs_lpCall
nm: lib.rmeta: no symbols
00000000 T Highs_lpCall
However, all the symbols appear at address 00000000 which seems suspicious to my uninformed eye.
I'm honestly not sure if this is a problem with highs, but other libraries that link to C APIs are working fine for me.
Also tried with gcc11, which is still available in the repositories as a separate package:
$ export CC=/usr/bin/gcc-11
$ export CXX=/usr/bin/g++-11
$ export RUSTFLAGS="-C linker=/usr/bin/gcc-11"
This makes it also error out with "undefined reference" errors. Which is weird, because I've been using GCC 11 for quite a while.
But when I set these variables to use clang instead of gcc...
$ export CC=/usr/bin/clang
$ export CXX=/usr/bin/clang++
$ export RUSTFLAGS="-C link-arg=-fuse-ld=lld"
... then it links successfully and tests are green! But this isn't a proper solution.
Any idea what more I could try?
It turns out this was due to link-time optimization (LTO).
libhighs.a was compiled from C++ code by GCC using -flto -fno-fat-lto-objects, causing it to emit only GCC intermediate bytecode into the archive, but not actual machine code. These flags were added by CMake through its option INTERPROCEDURAL_OPTIMIZATION.
But because LTO was not enabled during linking, no machine code was being produced at link time either. Normally the GNU linker would take care of that, but I think the bytecode might have gotten unusable or lost when libhighs.a was linked into a .rlib crate (hypothesis, didn't check). And I don't want to use the GNU linker for Rust code anyway because then I would lose the option to use LTO on Rust code (which always uses LLVM bitcode).
So for now, I think the best solution is to disable LTO on the C++ HiGHS build. I filed an issue with highs-sys and chimed in on an issue with HiGHS to make LTO more optional.

How to install latest glibc (version 2.29) beside system installed one & compile a program?

Based on This Stackoverflow link I downloaded & installed glibc v2.29 in "/usr/local/glibc" path of Linux OS. Then based on this Stackoverflow link I tried to compile This Example, But I got following errors.
First Try Command:
gcc -Wall -g -o main main.c -Wl,--rpath=/usr/local/glibc/lib -Wl,--dynamic-linker=/usr/local/glibc/lib/ld-linux-x86-64.so.2
First Try Error Log:
main.c:1:10: fatal error: threads.h: No such file or directory
#include <threads.h>
^~~~~~~~~~~
compilation terminated.
Second Try Command:
In second try, I am using "-I" & "-L" GCC command options.
gcc -Wall -g -I/usr/local/glibc/include -o main main.c -L/usr/local/glibc/lib -Wl,--rpath=/usr/local/glibc/lib -Wl,--dynamic-linker=/usr/local/glibc/lib/ld-linux-x86-64.so.2
Second Try Error Log:
/tmp/ccCNYemW.o: In function `main':
/home/.../main.c:14: undefined reference to `thrd_create'
/home/.../main.c:16: undefined reference to `thrd_join'
collect2: error: ld returned 1 exit status
So I don't know where is the problem. Please Help me.
First of all, don't put an alternate libc (or alternate version of your libc) in a path searched by the normal include and library search (both link-time and runtime library search) for your main system one. This is a recipe for disaster. Installing a different glibc in /usr/local/ does avoid clobbering your system one, but now you just have two installed in places where the same tools can see and use them.
To do this right, you really need a full separate toolchain (gcc, binutils) in some completely separate path (like ~/my_glibc_root/... or /opt/alt_glibc_root/...). I'm not sure if there's a recommended way to do this. The glibc build procedures in Linux From Scratch might be a good place to look for ideas. In theory it can be done in a single stage; I do that with musl libc in musl-cross-make by careful use of intermediate make rules in the gcc build system. But applying the same idea to glibc probably requires some extra care.
Second Try Command: In second try, I am using "-I" & "-L" GCC command options.
gcc -Wall -g -I/usr/local/glibc/include -o main main.c -L/usr/local/glibc/lib -Wl,--rpath=/usr/local/glibc/lib -Wl,--dynamic-linker=/usr/local/glibc/lib/ld-linux-x86-64.so.2
This command is almost correct. The thrd_create and thrd_join functions are defined in libpthread, which you didn't link against.
Add -pthread to your compile command, and the link should succeed.
P.S. R's advice of not installing alternate GLIBC into /usr/local is also a good one.

DSO missing from command line although it is available

I am working with c++ code for a physics simulation, which uses a lot of external libraries (like GSL and cern`s ROOT). Trying to recompile project I encountered problems with linking. When running compilation of final file via:
g++ -fno-inline -O2 -fpic -o main.out ${ROOTINCS} main.o ext.o ${ROOTLIBS} $(objects2)
with :
objects2= many .o files made by us
ROOTLIBS=-L/usr/local/lib/root -lTree -lRIO -lNet -lHist -lMathCore -lCore -lGraf -lGraf3d -lGpad -lMatrix -lThread -lCint -lPhysics -lPostscript -lRint -lSpectrum -lg
ROOTINCS=-pthread -m64
I get annoying error:
/usr/bin/ld: /usr/local/lib/root/libHist.so: undefined reference to symbol 'gRandom'
/usr/local/lib/root/libMathCore.so: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
The problem is nm -C run on libMathCore states 'gRandom' is declared there. Also -lMathCore is present in my command line.
When I run ld to check if it understands the flag:
ld -L/usr/local/lib/root -lMathCore --verbose 2>/dev/null
it does not complain and tries to link properly.
According to https://stackoverflow.com/a/24675715/3602168 order of libraries is correct in my linking (libHist uses libMathCOre and therefore is stated first).
Compilation runs under g++ 4.8.2 on ubuntu 14.04, 64 bit
Converting comment to answer:
Have you tried moving $(objects2) before ${ROOTLIBS}? I think the issue may be that you have libraries specified before the object files that use them.

How to make Scons look for libstdc++ in nonstandard directory

I'm trying to use Scons to build a simple project on a server on which I have rights to install stuff only in specific locations (and not in /usr/ ). Since I'm not happy with default compiler the server is offering me, I installed g++4.8 and verified it works just fine. But when I try to use Scons to build a simple project, while it picks up correct g++ (I can get that by checking the version), it's looking for libstdc++ in /usr/ directories instead of the directory where g++4.8 installation resides. E.g. code compiles, but upon execution fails with:
./main: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by ./main)
Again - this doesn't happen when I call the compiler myself from the terminal.
Even when I add the lib path containing libraries for g++4.8 with LIBPATH option, I get the same error.
Here's my SConscript file:
Import('env')
COMPILER_FLAGS = '-Wall -fopenmp -O3 -std=c++11'
LINK_FLAGS = '-Wall -fopenmp -O3 -std=c++11'
LIB_PATH = 'myfolder/gcc-4.8.2/lib64'
PROGRAM = 'main'
SRC = ['main.cpp', 'Foo.cpp']
env.Append(CPPFLAGS = COMPILER_FLAGS)
env.Append(LINKFLAGS = LINK_FLAGS)
env.Program(target = PROGRAM, source = SRC, LIBPATH = LIB_PATH)
and SConstruct is just
import os
env = Environment(ENV = os.environ)
SConscript('./SConscript', exports=['env'], duplicate=0)
Edit:
I made sure location of my compiler comes in the path before default compiler. But even if I set it explicitly with Environment(CXX=...) it's the same story. Here's the build output:
/mypath/gcc-4.8.2/bin/g++ -o Foo.o -c -Wall -fopenmp -O3 -std=c++11 Foo.cpp
/mypath/gcc-4.8.2/bin/g++ -o main.o -c -Wall -fopenmp -O3 -std=c++11 main.cpp
/mypath/gcc-4.8.2/bin/g++ -o main -Wall -fopenmp -O3 -std=c++11 main.o Foo.o -L/mypath/gcc-4.8.2/lib64
scons: done building targets.
-bash-3.2$
-bash-3.2$
-bash-3.2$ ./main
./main: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by ./main)
./main: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.11' not found (required by ./main)
-bash-3.2$
Yet another edit:
ldd on both manual and scons compile reveal:
linux-vdso.so.1 => (0x00007fff513fd000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6(0x0000003e7f600000)
libm.so.6 => /lib64/libm.so.6 (0x0000003e79600000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000003e7de00000)
libc.so.6 => /lib64/libc.so.6 (0x0000003e79200000)
/lib64/ld-linux-x86-64.so.2 (0x0000003e78e00000)
So indeed even manual compile doesn't look for the libs in the right directory (or where I installed the compiler) and the problem isn't with the scons itself, but likely that I didn't configure something right, but then I'm really puzzled as to why the executable runs fine, while it doesn't for scons.
Ok, so my problem wasn't with scons, but with me not giving explicit paths to nonstandard locations of libstdc++ and friends. SO answer over here explains this in more detail:
Linking g++ 4.8 to libstdc++
You're misinterpreting the error. GCC always knows how to find its own libraries, including libstdc++. The problem is that after you've compiled the program the runtime linker (which is not part of GCC, it's part of your OS and comes from glibc) doesn't know how to find the newer libstdc++, so it finds the default system one, which is too old.
The problem and solution are described at in the Libstdc++ FAQ, "How do I insure that the dynamically linked library will be found?", and manual, "Finding Dynamic or Shared Libraries"
This doesn't sound right.
Can you show us what you do to override the compiler?
If you are only doing the above, I don't think your compiler will be overridden with the new version.
You need to do something like
env = Environment(CC='/path/to/gcc')
Or Environment(CXX='/path/to/g++') if you want to override the c++ compiler
Or is your path on your environment setup to have the directory of the custom compiler before the standard compilers directory?
It might help to clean and then run with scons with --debug=presub which will show you the command line used to build each target.
Also your environment is a dictionary, so try printing out different keys to make sure they match what you expect:
print env['CC']
print env['CXX']

Clang link-time optimization doesn't work properly on Fedora 18

I'm a newcomer to clang, so it's likely I'm doing something silly. But I've spent several hours looking for solutions, including searching here, where I haven't found questions addressing -flto with distro-provided packages. The detail of this description are specific to Fedora 18, but I'm having similar problems on Ubuntu 13.04, so the problem isn't specific to Fedora. It's either me or clang.
Problem: I'm trying to compile a simple hello-world program using clang++ -flto to get the benefits of link-time-optimization. Without -flto it works fine. With -flto it fails to link. Invoking as clang -flto -o hello hello.o -v to see the full linker command line, I get:
$ clang++ -flto -o hello hello.o -v
clang version 3.2 (tags/RELEASE_32/final)
Target: x86_64-redhat-linux-gnu
Thread model: posix
"/usr/bin/ld" --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o hello /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../lib64/crt1.o /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../lib64/crti.o /usr/lib/gcc/x86_64-redhat-linux/4.7.2/crtbegin.o -L/usr/lib/gcc/x86_64-redhat-linux/4.7.2 -L/usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../.. -L/lib -L/usr/lib -plugin /usr/bin/../lib/LLVMgold.so hello.o -lstdc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc /usr/lib/gcc/x86_64-redhat-linux/4.7.2/crtend.o /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../lib64/crtn.o
/usr/bin/ld: /usr/bin/../lib/LLVMgold.so: error loading plugin
/usr/bin/ld: /usr/bin/../lib/LLVMgold.so: error in plugin cleanup (ignored)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
There seem to be two problems:
clang++ invokes the linker as /usr/bin/ld, and that's not the gold linker. Fedora18 installs gold as /usr/bin/ld.gold. I've tried creating a symlink from /usr/local/bin/ld to /usr/bin/ld.gold, verified that which ld says /usr/local/bin/ld, but clang++ doesn't use that. It seems to be hardwired to /usr/bin/ld.
clang++ invoked the linker with -plugin /usr/bin/../lib/LLVMgold.so. That's wrong, as the Fedora distribution of clang places it at /usr/lib64/llvm/LLVMgold.so.
I have tried manually invoking that linker line above with the following tweaks:
Replace -plugin /usr/bin/../lib/LLVMgold.so with -plugin /usr/lib64/llvm/LLVMgold.so. This yields the error message hello.o: file not recognized: File format not recognized. So the non-gold linker seems to know about plugins but wont take the .o's which contain LLVM bitcode.
Replace /usr/bin/ld with /usr/bin/ld.gold. This works, generates an executable that runs as expected.
Both of the above with --plugin instead of -plugin. This change makes no difference.
So what's the best way for somebody who prefers to stick to the system-provided packages to use clang -flto? I'm hoping there is a config file, or undocumented options or environment variables that will let me override these. Or better, that I'm missing a package and a "yum install ..." will fix it.
I would prefer not to invoke the linker directly, as then my makefiles need to know system objects and libraries that they should be ignorant of (e.g. crt1.o, crtbegin.o, crtend.o). I could also build clang myself, but I'm not seeing anything in its configure script that lets me configure the path of the linker and plugin.
I'm running Fedora 18. The only non-distro packages on the computer are google chrome and VMware Tools (it's a guest inside VMWare Fusion). Versions of relevant Fedora packages (the whole computer is "yum updated" as of today, 29-Apr-2013):
$ yum list --noplugins installed binutils* clang* llvm* gcc*
Installed Packages
binutils.x86_64 2.23.51.0.1-6.fc18 #updates
binutils-devel.x86_64 2.23.51.0.1-6.fc18 #updates
clang.x86_64 3.2-2.fc18 #updates
clang-devel.x86_64 3.2-2.fc18 #updates
clang-doc.noarch 3.2-2.fc18 #updates
gcc.x86_64 4.7.2-8.fc18 #fedora
gcc-c++.x86_64 4.7.2-8.fc18 #fedora
llvm.x86_64 3.2-2.fc18 #updates
llvm-libs.x86_64 3.2-2.fc18 #updates
There is an utility alternatives in Fedora - it allows to subtitute one linker with another on system level:
$ sudo alternatives --display ld
ld - status is auto.
link currently points to /usr/bin/ld.bfd
/usr/bin/ld.bfd - priority 50
/usr/bin/ld.gold - priority 30
Current `best' version is /usr/bin/ld.bfd.
$ sudo alternatives --set ld /usr/bin/ld.gold
About LLVMgold.so location you can only report a bug in Fedora Bugzilla, since the path is built-in in clang sources:
lib/Driver/Tools.cpp: std::string Plugin = ToolChain.getDriver().Dir + "/../lib/LLVMgold.so";
Fedora guys may apply a patch to the Clang's source package, or create a symlink to LLVMgold.so.
There is no changes even in Fedora 20 yet.

Resources