Proper way of using link time opimization with source and assembly files? - c

I'm currently playing around with LTO for an embedded system (to see if it could reduce the size) and was having some issues getting everything to link properly using ld directly and was wondering what I was doing wrong. This is mostly playing with a toy program before I use this in a larger project.
The setup is basically I have 3 files:
test.c - data transform function
test_main.c - calls a function defined in start.S
start.S - calls a function in test.c and also contains _start
I compile the files using:
arm-none-eabi-as -mthumb -Wall start.S -o start.o
arm-none-eabi-gcc -Wall -Werror -march=armv7 -mtune=cortex-r7 -mthumb -fPIC -nostdlib -flto -static test.c -c test.o
arm-none-eabi-gcc -Wall -Werror -march=armv7 -mtune=cortex-r7 -mthumb -fPIC -nostdlib -flto -static test_main.c -c test_main.o
If I then try to link the program with ld I get:
arm-none-eabi-ld --static -fPIC --shared --fatal-warning test.o start.o test_main.o -test
arm-none-eabi-ld: test.o: plugin needed to handle lto object
arm-none-eabi-ld: test.o: plugin needed to handle lto object
arm-none-eabi-ld: test_main.o: plugin needed to handle lto object
arm-none-eabi-ld: test_main.o: plugin needed to handle lto object
If I use gcc though, it works:
arm-none-eabi-gcc -Wall -Werror -march=armv7 -mtune=cortex-r7 -mthumb -fPIC -nostdlib -flto -static test.o start.o test_main.o -o test.gcc
I have tried specifying the linker plugin directly but then I get a different error:
arm-none-eabi-ld --static -fPIC --shared --fatal-warning --plugin <path_to_correct>/liblto_plugin.so test.o start.o test_main.o -test
arm-none-eabi-ld: <path_to_correct>.../lto-plugin.c:741: all_symbols_read_handler: Assertion 'lto_wrapper_argv' failed.
Aborted (core dumped)
What flags, parameters, etc. am I missing in my ld call?

Related

How to let ld use PIC version of libc.a

To link gcc statically into a shared library, based on the answer in this question, the remaining problem is how to let the linker use the PIC version of libc.a instead of the non-PIC version. The problem is the same as that answer:
g++ -fPIC -Wall -O0 -fexceptions -g -c main.cpp -o main.o
ld -shared -static -o test.so main.o -lc
ld: //usr/lib/x86_64-linux-gnu/libc.a(malloc.o): relocation R_X86_64_TPOFF32 against `tcache' can not be used when making a shared object; recompile with -fPIC
This is what I'v tried:
sudo apt-get install libc-pic //then libc6-pic get installed successfully
ld -shared -static -o test.so main.o -lc //same as above, same error
ld -shared -static -o test.so main.o -lc-pic //not working: cannot find -lc-pic
ld -shared -static -o test.so main.o -lc6-pic //not working: cannot find -lc6-pic
Looking at the list of files in the package there is a libc_pic.a so the correct option seems to be -lc_pic.

scons linking problems depending on CC

I'm compiling a library that comes with a scons script. According to the documentation scons does everything automatically and stuff should just work.
However, the build succeeds or fails, depending on what I set CC to (even though I always invoke the same compiler):
/usr/bin/ld: context.os: relocation R_X86_64_32S against `a local symbol' can not be used when making a shared object; recompile with -fPIC
context.os was built with the following arguments.
env cc -o context.os -c ... -Wall -O2 -DNDEBUG -I. -I/usr/local/include ... context.c
So scons forgets the -fPIC argument. However if I set CC=cc it succeeds:
cc -o context.os -c ... -Wall -O2 -fPIC -DNDEBUG ...
The same if I set CC="ccache cc":
ccache cc -o context.os -c ... -Wall -O2 -fPIC -DNDEBUG ...
And of course, with -fPIC set, linking succeeds in both cases.
So, I wonder how does scons position independent code trigger work and why does it screw up in the CC="env cc" case?

using C and cuda create shared library got error at link stage

I was really struggled with this error when I try to build a shared library. My code utilize the Lapacke library and also CUDA. when I compile them, there are no errors(I compile them as)
gcc -m64 -Wall -fPIC -c xxx.c -o xxx.o $(INC)
where INC includes all directories
INC=-I. -I${JAVA_HOME}/include/ -I${JAVA_HOME}/include/linux/ I$/home/sniu/lapack-3.5.0/lapacke/include/ -I${CUDA_INSTALL_PATH}/include/ -I/home/sniu/CBLAS/include/
for cuda part, I wrote it as:
nvcc -m64 -arch=sm_20 -Xcompiler -fPIC $(INC) -c xxx.cu -o xxx.o
but I got the error message at the link stage:
gcc -m64 -D_REENTRANT -Wall -fPIC -g -shared -o libjniWrapper.so jniWrapper.o cholesky_inv.o wls_acc.o utils.o -L/home/sniu/lapack-3.5.0 -L/opt/cuda-toolkit/5.5.22/lib64 -lm -llapacke -llapack -lblas -lgfortran -lrt -lcudart -lcublas -ldl
/usr/bin/ld: /home/sniu/lapack-3.5.0/liblapacke.a(lapacke_dpotrf.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/home/sniu/lapack-3.5.0/liblapacke.a: could not read symbols: Bad value
collect2: error: ld returned 1 exit status
I am very sure libraries are there, I really confused why I got this error.
Any suggestions are appreciated, Thank you so much!

Member at n is not an ELF object

I am trying to build a library (Tesseract OCR) for Android. It seems to compile just fine, and I get a bunch of static libraries, but it fails during the linking phase.
The command is:
libtool: link: /opt/android-ndk-r8c/toolchains/arm-linux-androideabi-4.6/prebuilt/darwin-x86/bin/arm-linux-androideabi-g++ -fPIC -DPIC -shared -nostdlib /opt/android-ndk-r8c/platforms/android-8/arch-arm/usr/lib/crtbegin_so.o -Wl,--whole-archive ./.libs/libtesseract_api.a ../ccmain/.libs/libtesseract_main.a ../cube/.libs/libtesseract_cube.a ../neural_networks/runtime/.libs/libtesseract_neural.a ../textord/.libs/libtesseract_textord.a ../wordrec/.libs/libtesseract_wordrec.a ../classify/.libs/libtesseract_classify.a ../dict/.libs/libtesseract_dict.a ../ccstruct/.libs/libtesseract_ccstruct.a ../image/.libs/libtesseract_image.a ../cutil/.libs/libtesseract_cutil.a ../viewer/.libs/libtesseract_viewer.a ../ccutil/.libs/libtesseract_ccutil.a -Wl,--no-whole-archive -Wl,-rpath -Wl,/Users/xxx/dev/libs/leptonica/current/android-arm/release/lib -Wl,-rpath -Wl,/Users/xxx/dev/libs/leptonica/current/android-arm/release/lib -L./ -L../ -L../api -L../ccutil -L../viewer -L../cutil -L../image -L../ccstruct -L../dict -L../classify -L../wordrec -L../neural_networks/runtime -L../textord -L../cube -L../ccmain -L/opt/android-ndk-r8c/platforms/android-8/arch-arm/usr/lib -L/Users/xxx/dev/libs/leptonica/current/android-arm/release/lib /Users/xxx/dev/libs/leptonica/current/android-arm/release/lib/liblept.so -lz -L/opt/android-ndk-r8c/toolchains/arm-linux-androideabi-4.6/prebuilt/darwin-x86/bin/../lib/gcc/arm-linux-androideabi/4.6/armv7-a -L/opt/android-ndk-r8c/toolchains/arm-linux-androideabi-4.6/prebuilt/darwin-x86/bin/../lib/gcc/arm-linux-androideabi/4.6 -L/opt/android-ndk-r8c/toolchains/arm-linux-androideabi-4.6/prebuilt/darwin-x86/bin/../lib/gcc -L/opt/android-ndk-r8c/toolchains/arm-linux-androideabi-4.6/prebuilt/darwin-x86/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/lib -lstdc++ -lm -lc -ldl -lgcc /opt/android-ndk-r8c/platforms/android-8/arch-arm/usr/lib/crtend_so.o -Os -march=armv7-a --sysroot /opt/android-ndk-r8c/platforms/android-8/arch-arm -mfloat-abi=softfp -mfpu=vfpv3-d16 -marm -Wl,-rpath-link=/opt/android-ndk-r8c/platforms/android-8/arch-arm/usr/lib -Wl,-soname -Wl,libtesseract.so.3 -o .libs/libtesseract.so.3.0.1
And the output looks like:
/opt/android-ndk-r8c/toolchains/arm-linux-androideabi-4.6/prebuilt/darwin-x86/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld: error: ./.libs/libtesseract_api.a: member at 8 is not an ELF object
/opt/android-ndk-r8c/toolchains/arm-linux-androideabi-4.6/prebuilt/darwin-x86/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld: error: ./.libs/libtesseract_api.a: member at 96 is not an ELF object
/opt/android-ndk-r8c/toolchains/arm-linux-androideabi-4.6/prebuilt/darwin-x86/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld: error: ./.libs/libtesseract_api.a: member at 104400 is not an ELF object`
...
As far as I can tell, libtesseract_api.a and related files are valid. Any idea what's happening here? This isn't an error I've seen before.
Check if you have the right archive type by nm
.../arm-linux-androideabi-nm .libs/libtesseract_api.a
You should use the right format.
After adding the following to PATH, it link success :
$ANDROID_NDK/toolchain/bin

Force linker to export a resolved symbol when creating shared library

I am creating a shared library using c++ -shared (which is gcc running on x86_64). I can't manage to strip my problem down to a minimal test case, but the issue I'm having is that I am creating a .so out of a bunch of .o files. One of those .o files exports a symbol (nm shows 'D') that I want exported from the .so. Others of the .o's require that symbol (nm shows 'U'). As a result, the linker makes that symbol local in the resulting .so (nm shows 'd').
If I eliminate the requiring .o's from the command line, the .so does export the symbol. So it seems like the linker is deciding that since other .o's used to build the .so require the symbol, that it must only be needed by those .o's and it needn't be exported for real.
When I try to construct a minimal test case, it doesn't work this way; the symbol is always exported whether or not I add in a .o that requires it.
My full command line is:
c++ -fno-rtti -fno-exceptions -Wall -Wpointer-arith -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wcast-align -Wno-invalid-offsetof -Wno-variadic-macros -pedantic -Wno-long-long -gdwarf-2 -fno-strict-aliasing -pthread -pipe -DDEBUG -D_DEBUG -DDEBUG_sfink -DTRACING -gdwarf-2 -fPIC -shared -Wl,-z,defs -Wl,-h,libmozjs.so -o libmozjs.so jsapi.o jsarena.o jsarray.o jsatom.o jsbool.o jscntxt.o jsdate.o jsdbgapi.o jsdhash.o jsdtoa.o jsemit.o jsexn.o jsfun.o jsgc.o jsgcchunk.o jshash.o jsinterp.o jsinvoke.o jsiter.o jslock.o jslog2.o jsmath.o jsnativestack.o jsnum.o jsobj.o json.o jsopcode.o jsparse.o jsproxy.o jsprf.o jspropertycache.o jspropertytree.o jsregexp.o jsscan.o jsscope.o jsscript.o jsstr.o jstask.o jstypedarray.o jsutil.o jswrapper.o jsxdrapi.o jsxml.o prmjtime.o jsdtracef.o jstracer.o Assembler.o Allocator.o CodeAlloc.o Containers.o Fragmento.o LIR.o njconfig.o RegAlloc.o avmplus.o NativeX64.o jsbuiltins.o VMPI.o CTypes.o Library.o mozjs-dtrace.o -lpthread -Wl,-rpath-link,/bin -Wl,-rpath-link,/lib -Wl,--whole-archive ctypes/libffi/.libs/libffi.a -Wl,--no-whole-archive -L/home/sfink/moz-central/obj-dtrace/dist/lib -lplds4 -lplc4 -lnspr4 -lpthread -ldl -ldl -lm -lm -ldl
I get the same behavior with the core command that this invokes (I also manually pruned out some irrelevant flags):
/usr/bin/ld --no-add-needed --eh-frame-hdr --build-id -m elf_x86_64 --hash-style=gnu -shared -o libmozjs.so /usr/lib/gcc/x86_64-redhat-linux/4.4.4/../../../../lib64/crti.o /usr/lib/gcc/x86_64-redhat-linux/4.4.4/crtbeginS.o -L/usr/lib/gcc/x86_64-redhat-linux/4.4.4 -L/usr/lib/gcc/x86_64-redhat-linux/4.4.4 -L/usr/lib/gcc/x86_64-redhat-linux/4.4.4/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.4.4/../../.. jsapi.o jsarena.o jsarray.o jsatom.o jsbool.o jscntxt.o jsdate.o jsdbgapi.o jsdhash.o jsdtoa.o jsemit.o jsexn.o jsfun.o jsgc.o jsgcchunk.o jshash.o jsinterp.o jsinvoke.o jsiter.o jslock.o jslog2.o jsmath.o jsnativestack.o jsnum.o jsobj.o json.o jsopcode.o jsparse.o jsproxy.o jsprf.o jspropertycache.o jspropertytree.o jsregexp.o jsscan.o jsscope.o jsscript.o jsstr.o jstask.o jstypedarray.o jsutil.o jswrapper.o jsxdrapi.o jsxml.o prmjtime.o jsdtracef.o jstracer.o Assembler.o Allocator.o CodeAlloc.o Containers.o Fragmento.o LIR.o njconfig.o RegAlloc.o avmplus.o NativeX64.o jsbuiltins.o VMPI.o mozjs-dtrace.o -lnspr4 -lstdc++ -lm -lgcc_s -lpthread -lc -lgcc_s /usr/lib/gcc/x86_64-redhat-linux/4.4.4/crtendS.o /usr/lib/gcc/x86_64-redhat-linux/4.4.4/../../../../lib64/crtn.o
Update: probably more importantly, those symbols are in a different section, and have hidden visibility. Using objcopy to make them global doesn't help. I don't know what the rules are for which sections to use for what.

Resources