Where is the documentation of c lang env variables? - linker

TLDR I tried to read the clang llvm documentation to find a way to pass library path, I wasn't successful. Then I searched and found a solution. I am curious where I can find the documentation of my solution.
Details and why
I was trying to run ruby's bundle on my catalina and I faced this error:
ld: library not found for -lssl
I reinstalled openssl using brew install openssl and then brew link openssl I received the follwoing response from it:
Warning: Refusing to link macOS provided/shadowed software: openssl#1.1
If you need to have openssl#1.1 first in your PATH run:
echo 'export PATH="/usr/local/opt/openssl#1.1/bin:$PATH"' >> ~/.zshrc
For compilers to find openssl#1.1 you may need to set:
export LDFLAGS="-L/usr/local/opt/openssl#1.1/lib"
export CPPFLAGS="-I/usr/local/opt/openssl#1.1/include"
For pkg-config to find openssl#1.1 you may need to set:
export PKG_CONFIG_PATH="/usr/local/opt/openssl#1.1/lib/pkgconfig"
While I appreciated brew's effort to make it work non of those worked. I narrowed the linker issue to this to see what's going on:
clang -dynamic -bundle -o asdf -L/usr/local/Cellar/mysql/8.0.19/lib -lmysqlclient -lssl -lcrypt -v
Finally using LIBRARY_PATH I made passed the enviornment variable
export LIBRARY_PATH=/usr/local/opt/openssl#1.1/lib/

You were able to make the clang linkage:
clang -dynamic -bundle -o asdf -L/usr/local/Cellar/mysql/8.0.19/lib -lmysqlclient -lssl -lcrypt -v
succeed when it otherwise failed by setting:
export LIBRARY_PATH=/usr/local/opt/openssl#1.1/lib/
in the environment of the clang command.
That's evidence that clang is influenced by the LIBRARY_PATH environment
variable in the same way as GCC compilers.
LIBRARY_PATH is not mentioned in the ENVIRONMENT section of man clang,
or indeed at all, so this GCC-like behaviour can strictly be viewed as not mandatory.
It seems likely however that the omission is an oversight in the manual,
since clang generally strives to be an unobtrusive substitute for gcc. If
we run a Linux clang linkage in verbose mode:
$ export LIBRARY_PATH=/wheres/wally; clang -v main.o -L. -lfoo -lbar
clang version 10.0.0-4ubuntu1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/8
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/9
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/9
Candidate multilib: .;#m64
Selected multilib: .;#m64
"/usr/bin/ld" -z relro --hash-style=gnu --build-id --eh-frame-hdr \
-m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o a.out \
/usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/crt1.o \
/usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/crti.o \
/usr/bin/../lib/gcc/x86_64-linux-gnu/9/crtbegin.o \
-L. -L/usr/bin/../lib/gcc/x86_64-linux-gnu/9 \
-L/usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu \
-L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu \
-L/usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../.. \
-L/usr/lib/llvm-10/bin/../lib -L/lib -L/usr/lib \
-L/wheres/wally \ # < There's wally!
main.o -lfoo -lbar -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc \
--as-needed -lgcc_s --no-as-needed \
/usr/bin/../lib/gcc/x86_64-linux-gnu/9/crtend.o \
/usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/crtn.o
we observe clang inserting the expansion of -L${LIBRARY_PATH} into
the ld commandline after all of the other -Ldir options. And if we
track down LIBRARY_PATH in the clang source tree,
we'll see the code that makes it do so.
In the ENVIRONMENT section of man gcc,
LIBRARY_PATH is documented:
LIBRARY_PATH
The value of LIBRARY_PATH is a colon-separated list of directories, much like PATH .
When configured as a native compiler, GCC tries the directories thus specified when
searching for special linker files, if it can't find them using GCC_EXEC_PREFIX .
Linking using GCC also uses these directories when searching for ordinary libraries
for the -l option (but directories specified with -L come first).
and the same appears verbatim in GCC online documentation: 3.21 Environment Variables Affecting GCC
What clang does with LIBRARY_PATH conforms with the sentence I've emphasised.
It's worth knowing that while using LIBRARY_PATH for this purpose may have got
you out of a jam, it is not favoured in regular linkage practice, because it is
likely to be an invisible hand when you are studying the output of a
linkage: its effect is only revealed in verbose mode (by either gcc or clang). The
installer's recommendation:
export LDFLAGS="-L/usr/local/opt/openssl#1.1/lib"
is what you'd expect. I've no insight into why it didn't work for you.

Related

Can't compile a C program on a Mac after upgrading to Catalina 10.15

There's a previous question Can't compile C program on a Mac after upgrade to Mojave, and the answers to that have covered most of the variations on what goes wrong.
Now — as of Monday 2019-10-07 — you can upgrade to macOS Catalina 10.15. Once again, during the upgrade, the /usr/include directory has been blown away by the update, even though XCode 11.0 was installed before upgrading (from Mojave 10.14.6) to Catalina. Consequently, compilers built to expect that there is a /usr/include directory do not work any longer.
The main recommended step for the Mojave issues — using the command:
open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg
does not work out of the gate because the directory /Library/Developer/CommandLineTools/Packages/ does not exist (so there's not yet a .pkg file to open).
Is there a good (official) way to create and populate the directory /usr/include?
Before you proceed, make sure to install xcode command line tools.
xcode-select --install
Actually, you can do it! Actually all the C headers are found here in this folder:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/
We just need to create symlink for all the headers file into this folder:
/usr/local/include/
It worked for me! the following command line will take care of all the problems:
sudo ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/* /usr/local/include/
You will get some warning. Some of the headers already exists, like this:
ln: /usr/local/include//tcl.h: File exists
ln: /usr/local/include//tclDecls.h: File exists
ln: /usr/local/include//tclPlatDecls.h: File exists
ln: /usr/local/include//tclTomMath.h: File exists
ln: /usr/local/include//tclTomMathDecls.h: File exists
ln: /usr/local/include//tk.h: File exists
ln: /usr/local/include//tkDecls.h: File exists
ln: /usr/local/include//tkPlatDecls.h: File exists
totally ok to ignore. that's all.
For me adding the following path to CPATH solved the issue:
export CPATH=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
TL;DR
It appears that Apple considers /usr/include as something that has gone the way of the dodo — it is extinct — or maybe it's like Monty Python's Parrot.
Using the Apple-provided GCC (actually, that's Clang by any other name, as the version information shows) or Clang avoids problems. Both /usr/bin/gcc and /usr/bin/clang will find the system libraries four directory levels below:
/Applications/Xcode.app/Contents/Developer/Platforms/…
If you build your own GCC or other compiler, you will (probably) need to configure it to find the system libraries under the Xcode application directory.
Explorations
Immediately after the upgrade, I ran XCode 11.0. It wanted to install some extra components, so I let it do so. However, that did not reinstate /usr/include or the directory under /Library.
One of the other bits of advice in the previous question was to run:
xcode-select --install
When doing so, it claimed that it downloaded the command line utilities, and it ensured that /usr/bin/gcc and /usr/bin/clang etc were present. That's a useful step (though I didn't definitively check whether they were present before).
$ /usr/bin/gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 11.0.0 (clang-1100.0.33.8)
Target: x86_64-apple-darwin19.0.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
$
Using /usr/bin/gcc, it is now possible to compile programs:
$ make CC=/usr/bin/gcc al
co RCS/al.c,v al.c
RCS/al.c,v --> al.c
revision 1.7
done
/usr/bin/gcc -I/Users/jleffler/inc -g -O3 -std=c11 -pedantic -Wall -Wextra -Werror -Wshadow -Wmissing-prototypes -Wpointer-arith -Wold-style-definition -Wcast-qual -Wstrict-prototypes -DHAVE_MEMMEM -DHAVE_STRNDUP -DHAVE_STRNLEN -DHAVE_GETDELIM -o al al.c -L/Users/jleffler/lib/64 -ljl
$
However, /usr/include is still missing. There is a directory under /Library now:
$ ls /Library/Developer
CommandLineTools PrivateFrameworks
$ ls /Library/Developer/CommandLineTools
Library SDKs usr
$ ls /Library/Developer/CommandLineTools/SDKs
MacOSX.sdk MacOSX10.14.sdk MacOSX10.15.sdk
$ ls /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/
Entitlements.plist SDKSettings.json System
Library SDKSettings.plist usr
$
Neither the System nor the Library directory contain anything very promising.
When all else fails, read the manual
Next step — find and read the release notes:
Xcode 11 Release Notes
macOS Catalina 10.15 Release Notes
There's no information in there that relates to this. So, the probability is (AFAICS, after only an hour or two's effort) that Apple no longer support /usr/include — though it does still have a fully-loaded /usr/lib (no /lib though).
Time to check another compilation with GCC option -v added (in the makefile I used, setting UFLAGS adds the option to C compiler command line):
$ make UFLAGS=-v CC=/usr/bin/gcc ww
co RCS/ww.c,v ww.c
RCS/ww.c,v --> ww.c
revision 4.9
done
/usr/bin/gcc -I/Users/jleffler/inc -g -O3 -std=c11 -pedantic -Wall -Wextra -Werror -Wshadow -Wmissing-prototypes -Wpointer-arith -Wold-style-definition -Wcast-qual -Wstrict-prototypes -DHAVE_MEMMEM -DHAVE_STRNDUP -DHAVE_STRNLEN -DHAVE_GETDELIM -v -o ww ww.c -L/Users/jleffler/lib/64 -ljl
Apple clang version 11.0.0 (clang-1100.0.33.8)
Target: x86_64-apple-darwin19.0.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.15.0 -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -disable-free -disable-llvm-verifier -discard-value-names -main-file-name ww.c -mrelocation-model pic -pic-level 2 -mthread-model posix -mdisable-fp-elim -fno-strict-return -masm-verbose -munwind-tables -target-sdk-version=10.15 -target-cpu penryn -dwarf-column-info -debug-info-kind=standalone -dwarf-version=4 -debugger-tuning=lldb -ggnu-pubnames -target-linker-version 512.4 -v -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -I /Users/jleffler/inc -D HAVE_MEMMEM -D HAVE_STRNDUP -D HAVE_STRNLEN -D HAVE_GETDELIM -I/usr/local/include -O3 -Wall -Wextra -Werror -Wshadow -Wmissing-prototypes -Wpointer-arith -Wold-style-definition -Wcast-qual -Wstrict-prototypes -Wno-framework-include-private-from-public -Wno-atimport-in-framework-header -Wno-extra-semi-stmt -Wno-quoted-include-in-framework-header -pedantic -std=c11 -fdebug-compilation-dir /Users/jleffler/src/cmd -ferror-limit 19 -fmessage-length 110 -stack-protector 1 -fstack-check -mdarwin-stkchk-strong-link -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fobjc-runtime=macosx-10.15.0 -fmax-type-align=16 -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -o /var/folders/77/zx9nk6dn7_dg4xd4stvt42v00000gn/T/ww-4cb85b.o -x c ww.c
clang -cc1 version 11.0.0 (clang-1100.0.33.8) default target x86_64-apple-darwin19.0.0
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/local/include"
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
/Users/jleffler/inc
/usr/local/include
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.0/include
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)
End of search list.
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -lto_library /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libLTO.dylib -dynamic -arch x86_64 -macosx_version_min 10.15.0 -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -o ww -L/Users/jleffler/lib/64 /var/folders/77/zx9nk6dn7_dg4xd4stvt42v00000gn/T/ww-4cb85b.o -ljl -L/usr/local/lib -lSystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.0/lib/darwin/libclang_rt.osx.a
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/dsymutil" -o ww.dSYM ww
$
The key information in that blizzard of data is:
-isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
That's effectively the 'root' directory for the compilation, so there should be sub-directories under that for usr and usr/include:
$ ls /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
Entitlements.plist SDKSettings.json System
Library SDKSettings.plist usr
$ ls /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr
bin include lib libexec share
$ ls /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
AppleTextureEncoder.h dns_util.h memory.h simd
AssertMacros.h dtrace.h menu.h slapi-plugin.h
Availability.h editline miscfs spawn.h
AvailabilityInternal.h err.h module.modulemap sqlite3.h
AvailabilityMacros.h errno.h monetary.h sqlite3ext.h
AvailabilityVersions.h eti.h monitor.h stab.h
…lots more lines…
dirent.h mach-o security xcselect.h
disktab.h mach_debug semaphore.h xlocale
dispatch machine servers xlocale.h
dlfcn.h malloc setjmp.h xpc
dns.h math.h sgtty.h zconf.h
dns_sd.h membership.h signal.h zlib.h
$
This shows that the mile-long and totally unmemorable directory name does contain the standard C and POSIX headers, plus Apple-specific extras.
The previous /usr/local/ directory appears to be intact; the warning about usr/local/include not existing under the -isysrootdir is harmless (and not visible without the -v option).
Set the following implicit Make variables to point to where the headers are now located for Xcode Command Line Tools (Xcode CLI):
export CFLAGS+=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
export CCFLAGS+=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
export CXXFLAGS+=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
export CPPFLAGS+=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
The -isysroot option updates the location of the root files away from the system root directory /.
So, this ensures that the common /usr/* files are found in their new place.
That is, the files at /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk are now found. These files are:
Entitlements.plist
Library
SDKSettings.json
SDKSettings.plist
System
usr
On MacOS Catalina 10.15.4, with Xcode Version 11.5 (11E608c) I also needed to update the library path in my .zshrc (the MacOSX.sdk paths are new):
export CPATH='/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include:/opt/local/include'
export LIBRARY_PATH='/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib:/opt/local/lib'
I am a newbie with C++ compiler for R in OSX and I got the same issue that C++ could not find the header after OS was updated (missing math.h although it was there). I followed instructions from https://thecoatlessprofessor.com/programming/cpp/r-compiler-tools-for-rcpp-on-macos/ but nothing changed.
Finally, it worked for me after I reinstalled the Xcode CLI
xcode-select --install
and then change the flags to Var as #Coatless suggested:
export CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
export CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
export CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
export CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
If you have both the command line tools and XCode installed, make sure the SDK installed by the command line tools is actually being used:
#Check the current sdk
$ xcrun --show-sdk-path
#Change sdk
$ sudo xcode-select -s /Library/Developer/CommandLineTools #Using CommandLineTools SDK
$ sudo xcode-select -s /Applications/Xcode.app/Contents/Developer #Using XCode.app SDK
kudos to https://stackoverflow.com/a/61526989/596599 for this answer.
For me, it works well as follow:
1. xcode-select --install
2. sudo ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/* /usr/local/include/
3. export SDKROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
In my case I seemed to have llvm and gcc also installed using homebrew. When I removed those, and thus relied fully on the macOS clang, it could find the headers and the compiling worked again.
Short Answer
/Library/Developer/CommandLineTools/usr/bin/clang++ -o main main.cpp -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk
The Result
Explanation
In current macOS version, the c/c++ headers are searched inside /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/, instead of /usr/include. So you need to 'reset' the root directory using -isysroot option.
Hope this makes sense 🙂.
If using an external LLVM installation, add these to your ~/.bash_profile
LLVM_PATH="/usr/local/opt/llvm/" # or any other path
LLVM_VERSION="11.0.0"
export PATH="$LLVM_PATH:$PATH"
export SDKROOT=$(xcrun --sdk macosx --show-sdk-path)
export LD_LIBRARY_PATH="$LLVM_PATH/lib/:$LD_LIBRARY_PATH"
export DYLD_LIBRARY_PATH="$LLVM_PATH/lib/:$DYLD_LIBRARY_PATH"
export CPATH="$LLVM_PATH/lib/clang/$LLVM_VERSION/include/"
export LDFLAGS="-L$LLVM_PATH/lib"
export CPPFLAGS="-I$LLVM_PATH/include"
export CC="$LLVM_PATH/bin/clang"
export CXX="$LLVM_PATH/bin/clang++"
(adjust the clang version and the external llvm installation path.)
Then run source ~/.bash_profile
apue.h dependency was still missing in my /usr/local/include after following Komol Nath Roy answer in this question.
I downloaded the dependency manually from git and placed it in /usr/local/include
The solution was simpler than I thought. Install clang/llvm.
brew install llvm
Then we need to create symlinks ourselves.
for f in /usr/local/Cellar/llvm/9.0.0_1/bin/clang*; do ln -s ${f} /usr/local/bin/"${f##*/}"; done
And
ln -s /usr/local/Cellar/llvm/9.0.0_1/include/c++ /usr/local/include/c++
Depending upon your llvm version, modify the above commands.
Now, you can compile C++ programs without passing any custom flags.
clang++ hello.cpp
I tried 1) manually linking 2) brew install llvm, but they did not work.
Finally, this worked for me:
https://gitmemory.com/issue/pytorch/pytorch/31190/565153503
By setting the following env vars:
export CC=clang
export CXX=clang++
export MACOSX_DEPLOYMENT_TARGET=10.9
In my case, i did a millions things but i reckon following steps helped fix ruby installation.
xcode-select --install
Set these flags
export CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
export CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
export CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
export CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
sudo ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/* /usr/local/include/
export SDKROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
rbenv install 2.6.3 -v
For me the error was: xcrun[20873:1179298] Failed to open macho file at /Library/Developer/CommandLineTools/usr/bin/clang++ for reading: Too many levels of symbolic links
So I opened my terminal and went to the following folder (as mentioned by the error message): /Library/Developer/CommandLineTools/usr/bin/
Then I deleted the shortcut file named clang++
sudo rm clang++
Next, I made a copy of executable file named clang and renamed the copied file to clang++
sudo cp clang clang++
And finally it works.

gcc "relocation R_X86_64_PC32 against symbol `ff_M24A'" error when linking statically against ffmpeg on linux

I am trying to build a JNI shared library which statically links to ffmpeg.
But at the linking stage, gcc fails with the following error:
/usr/bin/ld: ./lib_lin64/libswscale.a(swscale.o): relocation R_X86_64_PC32 against symbol `ff_M24A' can not be used when making a shared object; recompile with -fPIC
I am using the following commands to compile my jni library:
gcc -I $JAVA_HOME/include -I $JAVA_HOME/include/linux -I ./include -fPIC -c *.c
gcc -shared -Wl,--no-undefined -o libnv_avc_dec.so *.o -Wl,-Bstatic -L./lib_lin64 -lavcodec -lavutil -lswresample -lswscale -Wl,-Bdynamic -lm
And I only use h264 decoding feature, so I am also building ffmpeg from source with the minimal required feature set. The ./configure command I use is:
./configure \
--enable-pic --prefix=ffmpeg-dist \
--disable-debug --enable-version3 --enable-gpl \
--disable-everything --enable-hwaccel=h264_vdpau --enable-hwaccel=h264_vaapi --enable-hwaccel=h264_qsv --enable-hwaccel=h264_mmal \
--enable-decoder=h264 --enable-decoder=h264_vdpau --enable-decoder=h264_crystalhd --enable-decoder=h264_mmal --enable-decoder=h264_qsv \
--disable-iconv --disable-securetransport --disable-xlib --disable-zlib --disable-lzma --disable-bzlib --disable-doc --disable-programs --disable-avformat --disable-avfilter --disable-postproc
So, as I understand, the linker tells me that ffmpeg should be compiled with -fPIC flag in order to make a shared library. But I believe that I've already done so by specifying the --enable-pic configure flag. And I am pretty much stuck here because I am not very familiar with autotools, nor with ffmpeg build process in particular.
If this is the issue of ffmpeg .a libs not being compiled with -fPIC flag, how can i force it? And if this is not the case, what am i doing wrong and how can i fix this error?
Environment details: Ubuntu 14.04.3 64-bit in Virtualbox, gcc 4.8.5 and 5.3 (both give the same results), ffmpeg v.2.8.5
If you're building a shared library but need to link with static ffmpeg libs add linker flags:
-Wl,-Bsymbolic
When using cmake:
set(CMAKE_SHARED_LINKER_FLAGS "-Wl,-Bsymbolic")

Linking issue on Debian8

I'm trying to recompile my software for debian 8, but i have run into this strange issue of libgssappi refusing to link with anything.
>~/torque_github$ gcc test.c -lgssapi
/usr/bin/ld: cannot find -lgssapi
collect2: error: ld returned 1 exit status
The library is present in the system, as seen here:
>~/torque_github$ /sbin/ldconfig -p | grep gssapi
libgssapi_krb5.so.2 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2
libgssapi.so.3 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libgssapi.so.3
On my Debian/Jessie/x86-64 system, /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so is provided (according to dpkg -S) by the libkrb5-dev package and /usr/lib/x86_64-linux-gnu/libgssapi.so.3 is provided by libgssapi3-heimdal package (and I don't have any libgssapi*dev package).
You probably should install both of them (with sudo aptitude install libkrb5-dev libgssapi3-heimdal command), and use pkg-config with krb5-gssapi to get compilation and linking flags.
gcc -Wall -g $(pkg-config --cflags krb5-gssapi) \
test.c \
$(pkg-config --libs krb5-gssapi) \
-o myprog
(you could have to change your test.c source code if some API has changed; perhaps you'll need to #include <krb5/krb5.h>)
You might even use gcc -v instead of gcc above.
Remember that order of arguments to gcc matters a big lot. Your initial question had a different order (and that is enough to make gcc fail)!

Clang link-time optimization doesn't work properly on Fedora 18

I'm a newcomer to clang, so it's likely I'm doing something silly. But I've spent several hours looking for solutions, including searching here, where I haven't found questions addressing -flto with distro-provided packages. The detail of this description are specific to Fedora 18, but I'm having similar problems on Ubuntu 13.04, so the problem isn't specific to Fedora. It's either me or clang.
Problem: I'm trying to compile a simple hello-world program using clang++ -flto to get the benefits of link-time-optimization. Without -flto it works fine. With -flto it fails to link. Invoking as clang -flto -o hello hello.o -v to see the full linker command line, I get:
$ clang++ -flto -o hello hello.o -v
clang version 3.2 (tags/RELEASE_32/final)
Target: x86_64-redhat-linux-gnu
Thread model: posix
"/usr/bin/ld" --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o hello /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../lib64/crt1.o /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../lib64/crti.o /usr/lib/gcc/x86_64-redhat-linux/4.7.2/crtbegin.o -L/usr/lib/gcc/x86_64-redhat-linux/4.7.2 -L/usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../.. -L/lib -L/usr/lib -plugin /usr/bin/../lib/LLVMgold.so hello.o -lstdc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc /usr/lib/gcc/x86_64-redhat-linux/4.7.2/crtend.o /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../lib64/crtn.o
/usr/bin/ld: /usr/bin/../lib/LLVMgold.so: error loading plugin
/usr/bin/ld: /usr/bin/../lib/LLVMgold.so: error in plugin cleanup (ignored)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
There seem to be two problems:
clang++ invokes the linker as /usr/bin/ld, and that's not the gold linker. Fedora18 installs gold as /usr/bin/ld.gold. I've tried creating a symlink from /usr/local/bin/ld to /usr/bin/ld.gold, verified that which ld says /usr/local/bin/ld, but clang++ doesn't use that. It seems to be hardwired to /usr/bin/ld.
clang++ invoked the linker with -plugin /usr/bin/../lib/LLVMgold.so. That's wrong, as the Fedora distribution of clang places it at /usr/lib64/llvm/LLVMgold.so.
I have tried manually invoking that linker line above with the following tweaks:
Replace -plugin /usr/bin/../lib/LLVMgold.so with -plugin /usr/lib64/llvm/LLVMgold.so. This yields the error message hello.o: file not recognized: File format not recognized. So the non-gold linker seems to know about plugins but wont take the .o's which contain LLVM bitcode.
Replace /usr/bin/ld with /usr/bin/ld.gold. This works, generates an executable that runs as expected.
Both of the above with --plugin instead of -plugin. This change makes no difference.
So what's the best way for somebody who prefers to stick to the system-provided packages to use clang -flto? I'm hoping there is a config file, or undocumented options or environment variables that will let me override these. Or better, that I'm missing a package and a "yum install ..." will fix it.
I would prefer not to invoke the linker directly, as then my makefiles need to know system objects and libraries that they should be ignorant of (e.g. crt1.o, crtbegin.o, crtend.o). I could also build clang myself, but I'm not seeing anything in its configure script that lets me configure the path of the linker and plugin.
I'm running Fedora 18. The only non-distro packages on the computer are google chrome and VMware Tools (it's a guest inside VMWare Fusion). Versions of relevant Fedora packages (the whole computer is "yum updated" as of today, 29-Apr-2013):
$ yum list --noplugins installed binutils* clang* llvm* gcc*
Installed Packages
binutils.x86_64 2.23.51.0.1-6.fc18 #updates
binutils-devel.x86_64 2.23.51.0.1-6.fc18 #updates
clang.x86_64 3.2-2.fc18 #updates
clang-devel.x86_64 3.2-2.fc18 #updates
clang-doc.noarch 3.2-2.fc18 #updates
gcc.x86_64 4.7.2-8.fc18 #fedora
gcc-c++.x86_64 4.7.2-8.fc18 #fedora
llvm.x86_64 3.2-2.fc18 #updates
llvm-libs.x86_64 3.2-2.fc18 #updates
There is an utility alternatives in Fedora - it allows to subtitute one linker with another on system level:
$ sudo alternatives --display ld
ld - status is auto.
link currently points to /usr/bin/ld.bfd
/usr/bin/ld.bfd - priority 50
/usr/bin/ld.gold - priority 30
Current `best' version is /usr/bin/ld.bfd.
$ sudo alternatives --set ld /usr/bin/ld.gold
About LLVMgold.so location you can only report a bug in Fedora Bugzilla, since the path is built-in in clang sources:
lib/Driver/Tools.cpp: std::string Plugin = ToolChain.getDriver().Dir + "/../lib/LLVMgold.so";
Fedora guys may apply a patch to the Clang's source package, or create a symlink to LLVMgold.so.
There is no changes even in Fedora 20 yet.

Cannot compile using ALSA

I am trying to create an C application on Debian GNU/Linux which uses the PortAudio interface. To do this I must compile my program with gcc -lrt -lasound -ljack -lpthread -o YOUR_BINARY main.c libportaudio.a from this docs.
For this I installed libasound2-dev, and I checked where the files are using apt-file search libasound.so, this is the output:
lib32asound2: /usr/lib32/libasound.so.2
lib32asound2: /usr/lib32/libasound.so.2.0.0
lib32asound2-dev: /usr/lib32/libasound.so
libasound2: /usr/lib/x86_64-linux-gnu/libasound.so.2
libasound2: /usr/lib/x86_64-linux-gnu/libasound.so.2.0.0
libasound2-dev: /usr/lib/x86_64-linux-gnu/libasound.so
So the libasound should be installed correctly, but when I compile my program with this makefile:
DMXTest: main.c libdmx.a
gcc -static -Wall main.c -L. -ldmx -lusb -lrt -lasound -ljack -lfftw3 -g -o main libportaudio.a
I get the following error: /usr/bin/ld: cannot find -lasound.
How can I link this library correctly?
You don't have libasound.a for -static, you will need that, or you can almost certainly just remove -static from the Makefile (likely in LDFLAGS or CFLAGS).
There's is a related Debian bug 522544, and a related Ubuntu bug #993959.
You may be able to build your own libasound from source, though as it also uses other libraries (notably libpthread.so, librt.so and libdl.so) I suspect it may remove some functionality when you build it statically, though it's supported with ./configure --enable-static at build time
(or try --enable-shared=no --enable-static=yes).
FWIW, the use of static binaries is "discouraged" by the glibc maintainers, though I don't agree...
To compile my code i used the following command
gcc -o rec_mic rec_mic.c -lasound
and it works perfectly, without create my own static library.

Resources